Drone

A Stacks project for an owned vehicle autonomy platform. Build custom TypeScript behaviors, track a selected person with a Pi AI Camera, and use ArduPilot or PX4 as optional drivers while we implement our own runtime, estimation, controls and mission engine.

Status: experimental, simulation and bench development. Physical flight commands are disabled. The MAVLink command adapters are for local ArduCopter/PX4 quadcopter SITL only. No hardware flight test or full SITL flight has been performed for this release. This is an initial framework, not a finished autonomous aircraft.

Start with the documentation guide for setup, workflow guides, implementation status and the dependency roadmap. Buddy serves the documentation at /docs.

Run it

Requires Bun >= 1.4.2. The owner has a Raspberry Pi 5 with 8 GB RAM.

bun install --frozen-lockfile
cp .env.example .env
./buddy key:generate
./buddy dev

Open the frontend URL printed by Buddy. Run simulation, change following distance and altitude, then inject target loss or low battery. The browser simulation is local to each tab and has no connection to an aircraft. It deliberately pauses time when the tab is hidden.

bun run simulate        # deterministic terminal demo, no hardware
./buddy drone:simulate  # the same library through a native Stacks command
bun run telemetry       # receive-only MAVLink endpoint on 127.0.0.1:14551
bun run sitl --enable-sitl # 30-second local simulator experiment; read docs/sitl.md first
./buddy lint
./buddy typecheck
./buddy test

Owned incident collection

The public-safety foundation executes a bounded incident sector through the owned survey runtime, collecting scoped visible and numeric synthetic thermal evidence. Link/battery observations enter the actual supervisor; incident revisions stop the accepted plan and require a new pilot decision.

./buddy public-safety:plan examples/public-safety.json
./buddy public-safety:simulate --output /tmp/incident-collected
./buddy public-safety:simulate --scenario link-loss
./buddy public-safety:replay /tmp/incident-collected/recording.json --at-ms 600

Native incident roles, source-bound annotations/review, hold decisions and managed datasets now connect to separately accepted local purge, disabled by default. Recipient-confirmed incident handover adds atomic coordination transfer. Open /incidents for the authorized read and command workspace, including current source context, observation/review forms and recipient acknowledgement. The same workspace runs owned synthetic collection with live progress, cooperative stop and deliberate source admission. The full workflow remains in progress under #23: sector/team assignments and operational/hardware qualification are still required. See public-safety collection and remaining scope. Physical outputs are inhibited.

Owned vehicle missions

Open /missions to select quadcopter, fixed-wing, VTOL, ground, surface or subsea simulation. One workspace shows plans, energy budgets, estimated routes, decision playback, payload requests and correlated modeled acknowledgements. Host loss shows the last host snapshot separately from independent board expiry.

./buddy air:simulate --scenario crosswind --seconds 120
./buddy ground:simulate --scenario logistics --seconds 120
./buddy subsea:simulate --scenario nominal --inspection /tmp/modeled-subsea-inspection.json
./buddy surface:simulate --scenario nominal --seconds 120 --survey /tmp/modeled-harbour-survey.json

These profiles use owned raw sensors, estimation, mission policy, controls and authority. Physical outputs remain inhibited. Read the shared contract, air model, ground model and surface corrections and subsea pressure/DVL inspection.

./buddy vtol:hover separately exercises VTOL development: vertical takeoff, a nearby hover waypoint and contact-gated landing. The TypeScript library also includes a bounded modeled front/cruise/back experiment with stable-phase pause and resume. ./buddy vtol:simulate runs its nominal route through vertical landing and reports the qualified mode observations. Named scenarios expose phase-bound sensor/manual/host loss, cruise pause/resume, declared front/back abort and recovery, pusher open-circuit timeout and global endurance expiry. Add --electrical to either command for fictional propulsion/battery load and delayed energy measurements. Plans can explicitly declare bounded hover-and-land transition recovery. Complete recovery/abort envelopes and measured hybrid energy qualification remain open. The VTOL workspace selects the same 27 named cases, fault phases and optional electrical model, with captured mode and battery playback. The 2 m/s headwind and tailwind cases complete their routes. Separate east/west crosswind plans complete with explicitly wind-facing back-hover headings. Their original fixed north-facing limit cases retain blocked envelope outcomes, without confirmed hover or landing. Reports and playback show the requested heading and distinguish declared world wind from observed exposure checks; controller inputs contain no wind oracle. These are bounded fictional plans, not arbitrary-wind support.

The bounded back-capture planner is still in development. Its declared fixture-IMU estimator, conservative uncertainty checks and landing flare complete the calm 330 m example in focused modeled tests. Its wind and anchor declarations retain distinct supported and limited outcomes.

./buddy vtol:simulate --scenario planned-back-capture retains the cruise leg and explicitly requests a 330 m hover anchor. It includes the fictional electrical model, forecasts a bounded wing-borne approach before braking and retains its sampled model traces separately from actual captured hover/contact evidence. Source, wind and resource changes invalidate the accepted forecast. The six planned- wind cases also expose startup and observed phase onset, with distinct completed routes, planning limits and unreached wind exposure. Read the planner assumptions, remaining DAG budget and limits.

Simulated carried load

Characterize a centered or forward point mass, a prescribed separation and native link loss through our existing multirotor sensor, estimator, control and authority loop.

./buddy cargo:load-bench --scenario forward-separated
./buddy cargo:load-bench --scenario link-loss

Link loss is deliberately unqualified and exits 1. This is a bounded physics prerequisite for #25; delivery, physical release, custody and endurance qualification remain open.

Agriculture sensing

Open /agriculture to run owned crop scouting, inspect raw red/NIR calibration and source pixels, view a clipped field/parcel sampling layer and record explicitly selected livestock declarations.

./buddy agriculture:plan examples/agriculture.json
./buddy agriculture:simulate --output /tmp/new-field-visit
./buddy agriculture:process /tmp/new-field-visit/recording.json

The numeric model is checked against pinned vendor panel-image extracts. Physical calibration, crop interpretation and animal identity remain unverified. Read semantics and evidence.

Prescribed application

Open /application to inspect an inert-water sprayer or inert-granule spreader on the owned ground runtime. Separate implement leases, quantized instruments, exclusion guards, independent shutdown probes and exact modeled dose/overlap accounting are implemented. Dry mode and no grant are the defaults; metered simulation requires both explicit selections.

./buddy application:plan examples/application-sprayer.json
./buddy application:simulate --implement sprayer --mode apply --authorize-implement --output /tmp/new-application
./buddy application:process /tmp/new-application/recording.json
./buddy application:replay /tmp/new-application/recording.json

Material, calibration, weather and unqualified local requirements travel with private source-bound records. Physical calibration, field dose/drift, local authorization and HIL remain open in #22. Read application semantics and limits.

Synthetic environmental sampling

Run an owned two-height airborne mission with a calibrated digital cell index, capture-time estimated position, conditional uncertainty and unsigned local provenance.

./buddy environment:simulate --output /tmp/new-environment-dataset
./buddy environment:simulate --profile sensor-loss

Expected faults preserve their native aborted status even when their demonstration qualifies. Read dataset semantics and limits. Physical samples, wildlife workflows and operational qualification remain in #27.

Simulated camera reframe

Coordinate one selected synthetic POI, an owned hover mission, a lagged gimbal and three modeled still captures with explicit loss/ACK handling.

./buddy cinema:simulate --scenario body-disturbance
./buddy cinema:simulate --scenario target-loss

Read camera path semantics and limits. This partial angular reframe keeps physical gimbals, translational dolly/orbit paths, lens/video control and handoff open in #26.

Camera tracking

Open /camera after configuring a unique DRONE_CAMERA_TOKEN in your ignored .env. Choose replay for a hardware-free test, or Pi AI Camera on the Pi. Explicit selection confirms a visual lock; ambiguity, stale inference and lost detections invalidate it until reselection. See camera setup and limits. Image-space tracking is not metric navigation.

Calibrated target fusion

Open /vision for recorded explicit visual/GNSS pairing, registered-depth localization and owned physics follow scenarios. The live Pi camera stays image-only until measured calibration, synchronized depth and surveyed capture pose are available.

./buddy vision:validate examples/vision-calibration.json
./buddy vision:simulate --scenario follow
./buddy vision:simulate --scenario depth-loss

Read the measurement, pairing and uncertainty contract. The included calibration, depth and target source are synthetic; physical calibration is unverified.

Aerial survey collection

Open /mapping to inspect clipped lanes, terrain-relative footprints, acknowledged exposures and recorded collection faults on the owned runtime. The export includes PNG imagery, ODM-style geo.txt, capture pose/calibration metadata and a quality report. Collection is synthetic; reconstruction, physical accuracy and measured RTK/PPK/GCP evidence remain unverified.

./buddy mapping:plan --config examples/survey.json
./buddy mapping:simulate --scenario nominal --output /tmp/new-survey-dataset
./buddy mapping:simulate --scenario crosshatch

Read the survey collection and export contract.

Stockpile volume processing

Open /volumes to inspect known-surface integration, cut/fill, base assumptions, gaps and uncertainty. The native engine accepts numeric cell grids and indexed TINs with explicit reference/provenance; it keeps acquisition, reconstruction and measurement separate.

./buddy volumes:demo --scenario pyramid
./buddy volumes:process examples/stockpile/request.json
./buddy volumes:import examples/stockpile/surface.asc --format asc --metadata examples/stockpile/metadata.json --measurement examples/stockpile/measurement.json

Read the numeric import, integration and audit contract. Examples are synthetic; physical surveyed accuracy, raw-point meshing and richer format adapters remain unverified.

Asset-relative inspection

Open /inspection for a bridge-face route with capture-bound visible/synthetic thermal images, camera quality, explicit operator observations and repeat-visit geometry comparison.

./buddy inspection:plan examples/inspection.json
./buddy inspection:simulate --scenario nominal --visit before --output /tmp/new-inspection

The inspection contract separates owned simulation, declared review and physical defect evidence. Measured asset registration and visible/thermal hardware acceptance remain open.

Camera and payload adapters

Issue #8 adds an owned Pi AI Camera still adapter, bounded command/ACK and clock synchronization, image manifests, estimator-pose association and typed bus contracts. Physical camera calibration and execution remain unvalidated; other payload device drivers remain planned.

./buddy payloads:bench
./buddy payloads:bench --scenario lost-ack
./buddy payloads:capture --driver replay --output /tmp/new-drone-capture

See payload acquisition, Pi commands and acceptance gates. A standalone physical capture has no live pose feed and reports unavailable association. No output claims georeferencing.

Owned runtime and CLI

The first native kernel now builds for a host harness and a Pico 2 development board. It provides synthetic timestamped samples, bounded scheduling/protocol state and a neutral-only GPIO profile. Board timing, stack and watchdog-reset measurements remain open in issue #2.

./buddy runtime:bench --ticks 1000
./buddy runtime:bench --fault deadline
./buddy runtime:build --target pico2 --sdk /path/to/pico-sdk
bun run runtime:cli
./storage/runtime-build/cli/drone --help

Read the runtime build and bench guide and wire contract. The standalone CLI uses the same native Stacks command definitions as Buddy. Runtime commands do not flash hardware; physical camera acquisition requires an explicit --driver imx500. Keep your Pi 5 for Stacks mission/camera logic; the Pico 2 is a separate bench board.

Runtime diagnostics and replay

Issue #9 adds bounded private journals of the native host runtime, explicit loss accounting, verified crash-prefix recovery, provenance and offline replay through CLI and /diagnostics.

./buddy diagnostics:record --ticks 1000 --fault deadline --output /tmp/new-runtime-recording
./buddy diagnostics:replay /tmp/new-runtime-recording/journal.ndjson
./buddy diagnostics:inspect /tmp/new-runtime-recording/journal.ndjson

The viewer opens files locally in the browser. The v1 format excludes imagery, location and operator tokens. See diagnostics, durability and remaining acceptance. The signed runtime image workflow verifies offline TUF candidates through updates:stage and updates:inspect, with explicit root pins, compatibility and externally supplied release floors. updates:enroll and updates:trust preserve the initial root/policy record. The independently pinned store retains verified metadata, clock/release floors and historical recovery; updates:prepare authenticates actual image bytes under that protected state. updates:reserve, updates:candidates and updates:retire add durable candidate identity, recorded-byte status and explicit retirement while retaining files/history. updates:trial, updates:slots, updates:rollback and updates:abandon add retained host slots, actual image/boot/session health and fresh-policy rollback for the owned neutral host build. Physical logging/commissioning, key repair, complete storage-fault coverage and board image/install/boot/rollback qualification remain pending.

Sensor acquisition and replay

Issue #3 adds a separate Pico 2 sensor image for ICM-42688-P, BMP280 and u-blox NAV-PVT, plus SI conversion, calibration profiles, independent freshness/quality checks and simulator replay. The retained dataset is synthetic. Physical capture, calibration and timing evidence remain open.

./buddy sensors:replay tests/fixtures/sensors/bench-v1.json --at-ms 500
./buddy sensors:inspect /tmp/sensors-uart.bin
./buddy runtime:build --target pico2 --sensors --sdk /path/to/pico-sdk

See the sensor wiring, calibration and clock guide. Sensor replay is diagnostic; the owned estimator below consumes it separately from flight control telemetry.

Owned estimator

Issue #4 adds gyro/gravity tilt, GNSS position/velocity in a fixed NED frame, separate pressure-relative height and explicit health/reset reporting. Absolute heading remains unavailable.

./buddy estimator:replay tests/fixtures/estimation/motion-v1.json --at-ms 6040

Read the estimator guide and remaining work. Replay uses every sensor event; this Pi-side bench estimator is separate from ideal simulator control telemetry. Physical estimator validation and a tightly coupled inertial EKF remain open.

Simulated owned multirotor controls

Issue #5 adds bounded position, velocity, attitude and rate loops with quad-X allocation in a rigid-body physics harness. It uses raw modeled sensors and our estimator, with an explicitly assumed initial heading. Physical commands remain disabled.

./buddy controls:simulate --scenario step --seconds 20
./buddy controls:simulate --scenario wind --seconds 20

See the control equations, simulation assumptions and remaining work.

Simulated authority and failsafes

Issue #6 adds explicit arming/engagement, RC takeover, scoped command leases, latched faults and an independent modeled output watchdog on the owned control harness.

./buddy authority:simulate --scenario nominal --seconds 20
./buddy authority:simulate --scenario pi-loss --seconds 8

See the state machine, failure policies and hardware completion gates. Physical kill/RC wiring and HIL validation remain open. Zero simulated thrust is not a landing.

Owned mission engine

Issue #10 adds versioned mission graphs on the owned sensor, estimator, control and authority physics stack, with validation, pause/resume, checkpoint verification, conditional branches and a native stx scenario viewer at /missions. Physical outputs remain inhibited.

./buddy missions:validate examples/local-mission.json
./buddy missions:simulate --scenario recovery --output /tmp/new-mission-recording
./buddy missions:replay /tmp/new-mission-recording/mission.json --at-ms 6000
./buddy missions:checkpoint /tmp/new-mission-recording/mission.json --at-ms 4980 --output /tmp/new-mission-checkpoint
./buddy missions:recover /tmp/new-mission-checkpoint/mission.json /tmp/fresh-mission-observation.json --output /tmp/new-mission-recovery
./buddy missions:resume /tmp/new-mission-recovery/mission.json /tmp/next-mission-observation.json --output /tmp/new-mission-resume

Read the mission semantics, evidence and remaining work. Camera tasks emit simulated events. Recovery requires retained supervised authority and explicit operator resume. The recovery command saves a paused, verified simulation trace using an explicit fresh observation file. It does not restore whole-runtime state or connect to hardware.

Owned range-backed navigation

Issue #11 adds range-based obstacle routing, terrain clearance, RTL and selected-site landing on the owned mission physics stack. Perception latency is measured on the virtual clock; physical sensor and braking calibration remain open. Metric follow/orbit goal interfaces keep selection explicit.

./buddy navigation:simulate --scenario avoid-land
./buddy navigation:simulate --scenario replan

See navigation assumptions, evidence and remaining work.

The retained patrol experiment keeps one vehicle across two explicitly approved missions, with optional selected-pad contact and partial-charge verification and isolation. A finite native recurrence ledger records due slots and explicit local reviews. The finite patrol command defaults to a neutral preview and accepts explicit decision files with separate charge opt-in:

./buddy patrol:simulate
./buddy patrol:simulate --decisions examples/patrol-decisions.json --unsigned-local-review --enable-charge

Optional recording and replay retains bounded input and summary files. Replay reruns the same model. Recorded decisions require fresh local review, and charge decisions also require a fresh charge opt-in. Neutral replay needs neither:

./buddy patrol:simulate --output /tmp/patrol-neutral
./buddy patrol:replay /tmp/patrol-neutral

The authenticated patrol review API records owner-only intent, approval, denial and one durable consumption acknowledgment through native Stacks Auth, Gates and SQLite. These records do not launch a vehicle or grant charging authority. Production scheduling, runtime handoff, recovery and physical docking remain open under issue #24.

External-driver protocol services

Issue #7 adds shared acknowledged commands, parameter read/write, mission upload/download/resume and optional strict MAVLink 2 signing behind both driver factories. Physical writes remain disabled.

./buddy drivers:bench --driver ardupilot --drop-first-ack
./buddy drivers:bench --driver px4 --signed --drop-first-ack

See the protocol contract and remaining firmware SITL gates. These commands use scripted UDP peers, with no autopilot firmware or flight simulated.

Antenna tracking

Issue #16 adds owned selected-vehicle pointing, calibration, travel/cable limits, manual control and loss handling with a lagged mount simulation. Physical mount and RF measurements remain open.

./buddy antenna:bench --scenario nominal
./buddy antenna:bench --scenario target-loss

Read antenna interfaces, modeled evidence and remaining acceptance.

Behavior-level parity audit

Issue #28 adds source-backed behavior cases, executable evidence selectors, provenance/license checks and bounded upstream refresh. It reports unclassified and unsupported work explicitly; a valid audit is not full feature parity.

bun run parity:check
bun run parity:check --run-evidence

See the coverage matrix and upstream review commands.

Owned autopilot and roadmap

The dependency-linked roadmap and feature matrix cover the requested air, ground, surface, subsea, payload and industry workflows. A pinned inventory accounts for 274 top-level upstream module/driver/library entries. The first native bench firmware is implemented; hardware validation and detailed feature parity remain open, with explicit issues and evidence gates. See the ownership decision and driver contract.

Where your code goes

Path	Responsibility
resources/functions/drone/	Public drone library: types, behaviors, target sources, safety, controllers, simulation
resources/functions/drone/mavlink.ts	Small verified MAVLink v1/v2 common-message codec
resources/functions/drone/udp.ts	Bun UDP adapter, imported explicitly on the server only
resources/functions/drone/camera/	TypeScript tracker, session, API and a small Picamera2 acquisition helper
resources/functions/drone/controls/	Simulated cascaded multirotor control, quad-X allocation, physics and modeled sensors
resources/functions/drone/estimation/	Owned tilt, GNSS local-position filter, pressure height, health and replay
resources/functions/drone/sensors/	Typed sensor stream, calibration, clocks, health and capture/replay
resources/functions/drone/drivers/	Explicit ArduPilot/PX4 factories and mode profiles
resources/components/	Reusable stx simulator, operating-envelope and hardware components
resources/views/index.stx	Mission workspace
app/Commands/Simulate.ts	Native Buddy command
app/Actions/DroneStatus.ts	Read-only capabilities Action
routes/api.ts	Typed GET /api/drone/capabilities route
examples/custom-behavior.ts	Example extension point
tests/	Behavioral, controller-lifecycle and protocol tests

The application uses published Stacks packages, stx templates, and ts-css utility styling. The scaffold's retained storage/framework/defaults/ files are framework defaults, not drone logic. No database is needed for the transient simulator. Add a native Stacks model and model-derived migration when adding persisted aircraft profiles or flight records.

Write a behavior

import { defineBehavior } from './resources/functions/drone'

const stayNorth = defineBehavior({
  name: 'stay-north',
  velocity: ({ telemetry, target }) => ({
    north: (target.position.north + 12 - telemetry.position.north) * 0.5,
    east: (target.position.east - telemetry.position.east) * 0.5,
    down: (-12 - telemetry.position.down) * 0.5,
  }),
})

Use createSimulator({ behavior: stayNorth }) to test its motion, or pass it to createController(adapter, stayNorth, { targetId: 'pilot' }), feed the selected target, and explicitly renew a short autonomy lease. See the API. Every requested velocity passes through decide(). Custom behavior is trusted synchronous code; run untrusted code outside the flight process.

Build the aircraft

Buy the parts list around the Pi you already own, then follow the Pi bring-up guide. Budget approximately $1,100-1,350 before tax and shipping for a complete starter bench and airframe, with local Remote ID requirements budgeted separately. Prices are estimates except where explicitly marked as observed listings.

The current development split is the Stacks application on the Pi plus your chosen external autopilot on a dedicated Pixhawk. The accepted direction is to replace those firmware layers with our own implementation through the roadmap. You own the follow logic, behavior extensions, simulation, UI and integration code. This does not claim that flight-controller firmware, radio firmware, camera firmware, or board designs were authored here. Owned estimation, controls, authority and mission policy run in simulation; the native board kernel remains neutral-only. Custom PCB design is not implemented.

Current boundaries

• Implemented: kinematic follow simulation, configurable envelope, custom behavior hooks, monotonic freshness and lease checks, target identity lock, GPS-fix normalization with jump rejection, verified MAVLink subset, local UDP receive/transmit tests, receive-only Pi telemetry.
• Hardware control is receive-only. No arming, takeoff, mode switching, motor output, or RTL command is implemented.
• SITL commands require the separate explicit runners; bun run sitl:px4 --enable-sitl selects PX4. Loopback is an accidental-use guard, not a security boundary against another local process or a router forwarding to a physical vehicle.
• Implemented camera path: Picamera2 IMX500 acquisition/inference extraction, bounded JPEG preview, TypeScript selected-person association, explicit loss/reselection and an optional visual motion guard. Calibrated range/pose-backed selected-point localization and fusion run in simulation; Pi camera, depth, heading and physical field qualification remain pending. No automatic person re-identification or phone pairing is claimed.
• Owned multi-vehicle physics, raw-sensor estimation, supervised controls, missions, navigation, diagnostic journals and domain workflows have explicit simulation evidence. Native board/HIL, physical actuator control and whole-runtime restart acceptance remain separate roadmap gates.
• Software envelope prediction assumes immediate ideal velocity response and a stationary target over its horizon. It does not model braking, wind, sensor error, moving obstacles, or aircraft dynamics.
• A camera detecting a person does not establish who that person is or their GPS distance. First integrate an explicitly paired GNSS/phone target feed, then add calibrated visual/range fusion. The current 2D image observation stays separate.

Read architecture and failure behavior, validation evidence, and the flight-development checklist before extending control.

MIT. Maintained by Chris Breuer.