Skip to content

feat(firmware): OTA tooling that verifies the node actually took the image - #1836

Open
clonea1 wants to merge 1 commit into
ruvnet:mainfrom
clonea1:contrib/ota-tooling
Open

feat(firmware): OTA tooling that verifies the node actually took the image#1836
clonea1 wants to merge 1 commit into
ruvnet:mainfrom
clonea1:contrib/ota-tooling

Conversation

@clonea1

@clonea1 clonea1 commented Sep 4, 2026

Copy link
Copy Markdown

Two scripts for pushing firmware over the air to a fleet, both built around one
idea: "the OTA endpoint returned 200" is not evidence that anything changed.

ota_push.py pushes to one node and then polls /ota/status until the node
reports the EXPECTED version from a DIFFERENT partition than it started on. A
version string that did not move means the update did not take, however healthy
the response looked -- and a node that reboots the instant it finishes writing
can drop the HTTP response entirely, so the upload result alone cannot be
trusted either way. It also refuses to push a file that does not start with
0xE9 rather than discovering that after the write.

fleet_ota.sh rolls an image across the fleet ONE node at a time and stops at
the first node that fails to verify. Sequential and fail-fast is deliberate: a
parallel push puts every node at risk simultaneously, and a node that does not
come back needs physical access. On failure the nodes it has not reached are
left alone. --first lets you start with a board you can recover, so a bad image
lands where it is cheapest to fix.

Node addresses are discovered from /api/v1/nodes rather than hard-coded, so
this works on any fleet with no edits; --nodes overrides for a fleet the server
has not seen. The version defaults to the contents of version.txt so the check
cannot drift from what was built.

The PSK is taken from a FILE named by RUVIEW_OTA_PSK_FILE, never from argv or
an environment variable holding the key itself -- both are visible in a process
listing, and that key can replace firmware. Both scripts refuse to run without
it rather than falling back to something permissive.


Verified against a live nine-node fleet: discovery returns all nine from
/api/v1/nodes with nothing hard-coded, both guards fire (missing
RUVIEW_OTA_PSK_FILE, missing image), and bash -n is clean. The roll itself
was used to move nine boards through two firmware versions today, one node at a
time, with the verify step catching each before the next was touched.

No hard-coded addresses, paths, or versions remain in either file.

…image

Two scripts for pushing firmware over the air to a fleet, both built around one
idea: "the OTA endpoint returned 200" is not evidence that anything changed.

ota_push.py pushes to one node and then polls /ota/status until the node
reports the EXPECTED version from a DIFFERENT partition than it started on. A
version string that did not move means the update did not take, however healthy
the response looked -- and a node that reboots the instant it finishes writing
can drop the HTTP response entirely, so the upload result alone cannot be
trusted either way. It also refuses to push a file that does not start with
0xE9 rather than discovering that after the write.

fleet_ota.sh rolls an image across the fleet ONE node at a time and stops at
the first node that fails to verify. Sequential and fail-fast is deliberate: a
parallel push puts every node at risk simultaneously, and a node that does not
come back needs physical access. On failure the nodes it has not reached are
left alone. --first lets you start with a board you can recover, so a bad image
lands where it is cheapest to fix.

Node addresses are discovered from /api/v1/nodes rather than hard-coded, so
this works on any fleet with no edits; --nodes overrides for a fleet the server
has not seen. The version defaults to the contents of version.txt so the check
cannot drift from what was built.

The PSK is taken from a FILE named by RUVIEW_OTA_PSK_FILE, never from argv or
an environment variable holding the key itself -- both are visible in a process
listing, and that key can replace firmware. Both scripts refuse to run without
it rather than falling back to something permissive.

Co-Authored-By: claude-flow <ruv@ruv.net>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant