Propagate the full frame integrity verdict to every consumer

[bhelm]

Problem

Both platforms' verifiers compute a combined integrity result — header checksum and payload
CRC32 — but the parsers discard it, return a hardcoded ok = true, and forward only the payload CRC
as a tri-state. Every gate therefore rejects only a provably wrong payload CRC, so two classes pass:
frames with a broken header but valid payload (the header result is never read), and
unverifiable frames too short to compute a payload CRC at all — "unknown" counts as passing. A
structural gap compounds it: the reassembler's only lower bound is four bytes, yielding WHOOP 4.0
frames of 8–10 bytes whose payload CRC is never computed, and at the WHOOP 5.0/MG parser bound the
first byte of the CRC trailer is read as the inner packet type. So a noisy stream or a nearby,
interfering or impersonating peer can drive live state and, at worst, forge history metadata that
advances the trim cursor and tells the strap to discard data that was never provably stored.
Reported as bhelm#5, rated P0.

Change

Swift protocol core. ParsedFrame.ok now carries the verifier's full verdict — header checksum
valid and payload CRC provably valid, unverifiable means rejected — plus a non-optional
rejection reason. The reason sits on the parse result, not the verifier result, because consumers
only ever call the parser: the frame is deliberately parsed exactly once (now pinned by a debug
assertion), so a verifier-only reason would force every reporting consumer to verify twice. It
decodes with a default, so hand-built deserialization of ParsedFrame does not turn strict and
break. Per-family minimum total lengths (WHOOP 4.0 eleven bytes, deliberately admitting its real zero-payload metadata frames; 5.0/MG thirteen bytes, requiring at least one payload byte), the exact
declared length and rejection of surplus trailing bytes are enforced in the verifier, not per
parser — five Swift and three Kotlin call sites invoke the verifier directly and would otherwise
stay unprotected. The reassembler gets the same bound and resynchronises to the next frame start
instead of emitting a short frame. Every read of a named inner field is clamped to the minimum of
the CRC-trailer start and the actual frame size, so a frame at the bound cannot fake a metadata
field out of its own trailer and a truncated frame cannot read past the buffer; a field counts as
present when start + length does not exceed that bound. The eight-byte history-end acknowledgement
block is explicitly exempt — it reaches into the trailer by construction and is echoed back to the
strap unchanged; clipping it would alter the most consequential write the system performs. The
payload-only frame builder now computes a real header checksum instead of a zero placeholder.

Swift consumers, gates and diagnostics. The six state-driving gates — router, historical
classifier, stream extraction, history extraction, clock correlation, data-range reply — require the
full verdict instead of "not explicitly wrong". The data-range reply is in scope because it sets the
two bounds the offload plausibility-checks records against: a forged reply narrows the window until
genuine records fail and an empty section is acked anyway, which is the P0's end state by another
route. Evidence-preserving readers keep their direction: the history-path reader deciding which
raw frames are archived before the trim ack treats a negative verdict as a reason to archive, and
mechanically tightening it would have deleted the only durable copy of exactly the frames the strap
is about to release. Diagnostic surfaces that read ok as parsability moved to a real parsability
signal, so a CRC-failing frame keeps its decoded packet type.

Kotlin twin. The same core and the same gates, including the second, inline-verifying ECG
payload path — which the Swift side inherits from the central verifier for free and Kotlin did not.

Shared oracle. A Swift-generated file of frame bytes with verdict, reason and history
classification per frame, byte-identical in both test trees; both suites check every row and all
three fields, so a one-sided change fails on the other side.

Python capture tools. Two points only: the same family minimums in the verify helper, and the
feature evaluation now filters on the full verdict instead of the payload CRC alone, where a
missing value counted as passing.

Docs. PROTOCOL_IMPLEMENTATION.md, LIBRARY.md, CONTRIBUTING.md, ANDROID.md, PRIVACY_SECURITY.md,
BLE_REVERSE_ENGINEERING.md state the new bounds; two showed the loose gate as the example.

No new connection behaviour: rejected frames cause no disconnect, no reset, no reconnect change.
Holding the trim ack for a section the gate rejects wholesale was considered and rejected — that
hold path is unbounded, so a strap producing persistently rejectable records would stall the offload
with no user recourse. The class is made measurable instead: counters per rejection reason, plus
exactly one named counter for the previously-passing class (header-or-length wrong while the
payload CRC verifies). A single reason bucket cannot express that conjunction, and the length bucket
also collects harmless post-dropped-notification aborts. Nothing leaves the device.

Size, and how to read the diff

62 files, +6,641 / −549. Where the lines are:

Part	Files	Added	Removed
Tests, Kotlin	6	1,544	4
Tests, Swift (WhoopProtocol)	15	1,308	10
Shared oracle (one identical JSON per test tree)	2	996	0
Product code, Swift (package + Strand app)	15	919	230
Product code, Kotlin	11	864	221
App-target tests (StrandTests)	3	551	0
Docs	6	299	56
Python capture tools (+ their tests)	4	160	28

Two thirds of the added lines are tests and test data. Of the product code, by role:

Role	Added	Removed
Core: verifier bounds, ParsedFrame verdict + reason, clamped inner reads	773	305
Gates and consumers	535	118
Diagnostics: per-reason counters, capture/summary reason fields	398	8
CLI tools (whoop-decode, whoop-re, Python capture)	95	28

Roughly half of the product-code additions are comments explaining a bound or a decision at the
place where it is made; the code itself is smaller than the total suggests.

What is one atomic change, and why. The parser used to return a constant ok = true on both
platforms (Interpreter.swift:188, Framing.kt:295 on main). Five of the six gates already
read ok, so the core alone — verdict plus verifier bounds plus clamped reads — already tightens
them. Two seams did not, and they are the reason core and gates cannot land separately:

• the data-range reply had no verdict gate at all, on either platform; a forged reply would
  still have narrowed the offload window;
• Kotlin's v26 PPG path decoded the record directly, past the historical decoder, so the
  central verdict never reached it.

Both platforms have to land together because the oracle is what proves parity: both suites
check the same byte-identical file, field by field. Landing one side first would either fail the
other side's suite against the new expectations or defer the parity proof to a later PR. It
appears twice because each test tree must be self-contained; cmp in Verification pins the
copies identical. Its 498 lines are 45 readable-JSON cases (26 WHOOP 4.0, 19 WHOOP 5.0/MG; 14
positive, 31 negative), not a binary blob.

What is deliberately in scope but separable. The diagnostics layer (counters per reason,
the one named counter for the previously-passing class, reason fields in captures) does not
decide any frame. It is in this PR because the hardware abort criterion below reads it; without
it the strap run has nothing to observe. The Python capture alignment (+160) is an independent
third implementation of the same rule and only fixes the two points named above. Four of the six
docs contained normative statements about crcOK that became false with this change; the other
two are additive. If you would rather review the P0 alone, the diagnostics package and the Python
alignment can be split off into a follow-up — say so and I will do it.

Suggested reading order: Framing.swift / Framing.kt (verifier bounds, ParsedFrame
verdict and reason) → Interpreter.swift, PostHooks.swift / HistoricalStreams.kt (clamped
inner reads, the ack-block exemption, the v26 gate) → the two new gates (DataRange.swift /
DataRange.kt, wired in BLEManager.swift / WhoopBleClient.kt) → the existing gates that now
receive the full verdict (FrameRouter.swift, HistoricalMeta.swift, Streams.*,
HistoricalStreams.swift, ClockCorrelation.swift) → the oracle JSON and the two oracle tests →
FrameDiagnostics.swift / FrameRejectTally.kt → docs.

Verification

Run by me on Linux, on the integrated tree:

• swift build && swift test in Packages/WhoopProtocol → 757 tests, 1 skipped, 0 failures.
• ./gradlew testFullDebugUnitTest --no-build-cache --rerun-tasks → 5683 tests, 6 skipped, 0
  failures, 0 errors.
• python3 -m unittest discover Tools/linux-capture → 234 tests, OK.
• python3 Tools/doc_comment_lint.py → OK (24 baselined sites, baseline unchanged);
  python3 Tools/i18n_audit.py --ci main → exit 0.
• cmp of the two oracle copies → identical; grep -rn "crcOK != false\|crcOK == false" docs/
  → no hits, so no doc still shows the old gate.
• Against the real recorded corpora in the tree (115 frames): no frame the new bounds reject.
  The smallest real WHOOP 4.0 history frame sits exactly on the bound; the smallest real 5.0 frame
  is 124 bytes.
• App targets, on CI rather than here: the App build (macOS + iOS) workflow on this repository
  at bd1ec3ad9 and the same six workflows on my fork — xcodegen generate, Strand build and
  xcodebuild test → 1,650 StrandTests, 1 skipped, 0 failures, including the three new
  files (FrameIntegrityGateTests 18 cases, BackfillMetaForgeryTests 9,
  AppTargetFrameCorpusIntegrityTests 3); NOOPiOS build green.
• Hardware (2026-09-12): WHOOP 5.0/MG on Android 16 with the 11.1.1 staging build (467): sustained live traffic and repeated history offloads completed. All history chunks were acknowledged after persistence; the largest observed offload handled 1,490 frames and persisted 4,058 rows. One live type-18 frame with a bad header checksum was rejected as intended. There was no payloadCRCOKButEnvelopeRejected event, no critical valid-payload-CRC warning, and no rejected frame in an acknowledged offload section. This is the only device family available to me for physical testing. No macOS box of my own; the app-target evidence above is CI.

Acceptance

Items 1–8 were closed by the CI run cited under Verification (this repository and the fork). The available WHOOP 5.0/MG hardware run is now complete; other device families are unavailable to me and are recorded as residual coverage rather than a draft blocker.

• Both app targets compile: xcodegen generate plus xcodebuild for Strand and NOOPiOS
  (this diff touches Strand/BLE and Strand/Collect, which no default CI compiles).
• StrandTests: real recorded frames stay valid — the frame literals in three app-target tests.
• StrandTests: router rejects a broken-header frame (end to end).
• StrandTests: router rejects an under-length frame (end to end).
• StrandTests: router rejects a truncated frame (end to end).
• StrandTests: history metadata cannot be forged across offload and ack.
• StrandTests: clock correlation takes no invalid frame as an anchor.
• StrandTests: the data-range reply does not narrow the offload window unchecked.
• Hardware available to the author: WHOOP 5.0/MG live session and repeated history offloads with Test Centre enabled. No abort criterion fired: the named previously-passing-class counter stayed clear, no critical valid-payload-CRC warning appeared, and no rejected frame belonged to a subsequently acknowledged offload section. A single bad-header live frame was rejected as intended. WHOOP 4.0 and any other unavailable family remain untested on physical hardware.

Behaviour change / rollout notes

Frames that previously passed are now rejected: broken header with valid payload, unverifiable,
under-length per family, wrong declared length, surplus trailing bytes. Clamped inner reads mean a
frame near the bound no longer yields metadata mined from its own trailer. One visible decision
changes with it: the unbonded-offload probe no longer counts a broken-header frame as evidence that
the strap delivers notifications — intended, and documented in the probe. No schema change, no
stored-value change, no migration, no backup-whitelist key; diagnostic and capture formats gain a
reason field additively and keep the packet type for rejected frames. Older captures still
parse; violating frames are marked invalid rather than silently processed.

Rollback: a single revert restores the code. It does not restore a trim cursor already
advanced at runtime — the offload acks a section as soon as persisting did not fail, and a section
whose records the new gate rejects wholesale does not fail, it is empty. The raw-frame archive
catches that for the history data type only, not for event, metadata and protocol frames. That
combination is the one way this change could lose user data, and it is why the hardware abort
criterion is worded as above.

Scope limits and follow-ups

• R-01 (standing): reply paths branching on a raw byte comparison with no storage or offload
  effect stay unhardened — including two probe outputs that pronounce a finding about the strap and
  one that persists its payload. The one such path with offload effect (the data-range reply) was
  pulled into scope instead. The promise is "the six state-driving gates are hardened", not "every
  frame consumer".
• R-02 (standing): the Python capture tools remain an independent third and fourth
  implementation of the same rule; only the two points above are aligned. Full unification would be
  its own change with its own oracle.
• R-03 (residual hardware coverage): the available WHOOP 5.0/MG passed live and history testing. WHOOP 4.0 and any other unavailable family have no physical run from the author; a firmware variant that frames differently would first surface at the wrist.
• Not claimed: authenticity. CRC is not a signature — a peer forming the envelope correctly is
  not excluded. The trust boundary is the BLE seam.
• Unchanged: the checksum algorithms (CRC-8, CRC-16-Modbus, CRC-32), the bonding and handshake
  paths; no new write commands to hardware, no foreign frame bytes or firmware literals.
• Cross-platform parity tooling is not part of this PR; parity here rests on the shared oracle.

Branch: fix/issue-5-frame-integrity. Contribution offered under the repository's PolyForm
Noncommercial 1.0.0 license.

[bhelm]

CI update: the first macOS run failed on three synchronous tests in BackfillMetaForgeryTests calling the @MainActor Backfiller.endData(from:family:); fixed by marking them @MainActor (test file only). Second run is green: Strand build + xcodebuild test (the seven new/updated StrandTests cases in the checklist above all pass, 30 cases across the three new files), NOOPiOS build green — https://github.com/ryanbr/noop/actions/runs/34173194509/job/101897370594. That closes every checklist item except the hardware run per device family, which still needs a real strap.

[bhelm]

Rebased onto current main (1cca838fd) — the StrandAnalytics failure was the missing import fixed by #1986. Description updated: the eight app-target/StrandTests checklist items are closed by CI (1,650 StrandTests, 0 failures), and there is now a section on where the 6.6k lines sit and which parts are separable if you would rather review the P0 alone. Next: the hardware run on WHOOP 5.0/MG on Android with my own strap; 4.0 and the Apple side stay open.

[ryanbr]

Reviewed. #1977 and #1978 are merged, thanks.

Two things on this one.

The draft gate. The body still opens with "Draft: please do not merge before the hardware run",
but the PR is no longer marked as a draft, so it reads as ready to anyone glancing at the queue.
Did the hardware run happen on both device families, or was the draft flag cleared by accident? I
would rather leave it open than merge past your own gate, so just say which and I will act on it.

One comment to correct before it goes in. FrameLimits carries this on both platforms,
identically:

WHOOP 4.0: [SOF][len u16][crc8][type][seq][cmd] + >=1 payload byte + [crc32 u32] = 11 bytes.

That composition sums to 12. The constant 11 is the right value, and I would not want it changed:
it preserves the old 7 <= length bound exactly, and your own oracle's w4_meta_history_start_11
and w4_meta_history_complete_11 are real captured frames declaring len=7 with zero payload
bytes, so 12 would reject genuine WHOOP 4.0 metadata. But WHOOP 5.0's 13 does enforce the
>= 1 payload byte the sentence describes, tightening the old effective minimum of 12. So one
sentence describes two different rules and misstates the one where being wrong loses live data. The
PR body repeats it, so both want the wording. Worth being explicit that the 4.0 bound admits a
zero-payload frame on purpose.

The rest holds up on the parts I went through. The reassembler guards avail < 4 before reading
the length word and resyncs the same way the ceiling path does; payloadLimit clamps on both ends;
the post-hook readers take limit as a required argument, so a field added later cannot skip the
decision. The oracles are byte-identical between the two test trees, and
w5_meta14_fakes_history_complete pins the actual attack.

The end_data exemption is the part I looked hardest at and you have it right. On the real 25-byte
4.0 HISTORY_END in your oracle the ack slice frame[17..25] is four inner bytes plus the frame's
own CRC32 trailer, so clipping it to the trailer start would have altered the bytes echoed on every
4.0 trim ack. Still bounded by the frame size on both platforms.

Understood on the residual risk, and I agree the hardware run is the right gate for it: the header
checksum has never gated anything in production, so nine recorded frames passing is good evidence
but not a fleet. The always-on first-sighting counter for the previously-passing class is the right
instrument to read it with.

[bhelm]

Thanks — you are right. I corrected the identical Swift and Kotlin comments: the 11-byte WHOOP 4.0 minimum deliberately admits real zero-payload metadata frames and preserves the old length >= 7 rule, while the 13-byte WHOOP 5.0/MG minimum requires at least one payload byte. The constants and behavior are unchanged, and the PR body now makes the distinction explicit. The available hardware run has also happened on WHOOP 5.0/MG (live plus repeated history offloads, with no abort criterion firing); I do not have WHOOP 4.0 hardware, and the PR records that remaining coverage explicitly.

[ryanbr]

Reviewed at 4b6d82a. This is careful work and the write-up is the most useful part of it: you named the failure mode that would make this change dangerous rather than leaving me to find it. I am not merging it today, but nothing below is a rejection of the approach.

What I verified rather than took on trust

The bounds are structurally derived, and the description undersells that. I read them expecting values fitted to the captures, which is what "the smallest real WHOOP 4.0 frame sits exactly on the bound" reads like, and that had me worried there was no margin. There is no margin because there cannot be: 11 is [SOF][len u16][crc8][type][seq][cmd] plus the CRC32 trailer, so a shorter 4.0 frame is missing a header field or its trailer. That is a much stronger argument than "nothing in the corpus trips it", and I would lead with it.

The two oracle copies are byte-identical, and the file holds what you say: 45 cases, 26 WHOOP 4.0 and 19 WHOOP 5.0/MG, 14 accepting and 31 rejecting. The self-describing coverage block is a good idea and I want it kept.

The empty-section ack path is as you describe it. I followed it in Backfiller.kt rather than assume. An all-rejected section decodes to nothing, the insert of nothing succeeds, and the ack advances. What makes it survivable is that rejected frames reach rejectedSink and are archived durably before the ack, and a failed archive write holds the ack. Your decision to keep the evidence-preserving readers pointing the way they already pointed is the single most important call in this PR, and it is right.

What I want before it goes in

1. It conflicts with main, in one file. Interpreter.swift, against the WHOOP 5 console-sequence change that landed earlier today. There are 70 commits on main since you branched. The conflict is small, but it is in a file this PR rewrites heavily, so I would rather you resolved it than I did.

2. One declared rejection reason has no oracle case, and may not be reachable.

payloadCRCUnverifiable exists on both platforms, is documented as deliberately fail-closed, has a Kotlin twin at the null branch and its own name in the JSONL writer. It has zero cases in the oracle.

Tracing it in the 4.0 verifier, the fixed order is minimum length, then exact length, then header checksum, then the CRC32 switch. By the time the switch is reached the frame has cleared the length rules, and I could not construct an input that arrives there with an uncomputable CRC32. If that is right, the "unverifiable counts as passing" class from your problem statement is actually closed by the length bound rather than by this reason, and the reason is dead on that path.

I do not want to guess which. Either add a case that pins it, or say where it is still reachable, or remove it. What I want to avoid is a fail-closed branch that the parity oracle does not pin, because the Kotlin null branch can then drift from the Swift one and both suites stay green.

3. The two families are justified differently, and only one is structural.

4.0 admits its real zero-payload metadata frames deliberately, which is why the bound is 11 and not 12. 5.0/MG requires at least one payload byte, and the support for that is empirical: the smallest observed frame is 124 bytes. That is an enormous margin, but it is a different kind of claim from the 4.0 one.

Is a zero-payload 5.0/MG frame structurally impossible, or merely unobserved? If it is the latter, the 13-byte bound rejects a frame class we have not seen, and I would rather know that is a deliberate bet than discover it from a wrist.

4. The WHOOP 4.0 hardware run, which I can do.

R-03 is honest and it is also the residual I care about most, because the 4.0 bound is the exactly-on-the-boundary one and the failure mode for a wrong rejection is a section that acks empty. I have a 4.0. Rebase onto main and I will run live traffic and repeated history offloads against your abort criterion, and report the counters here.

That closes the gap you cannot close, and it is a better use of the hardware than another 5.0 run.

On splitting

You offered to split the diagnostics and the Python alignment into a follow-up. I would rather you did not. The counters are what the hardware criterion reads, and pulling them out means the 4.0 run has nothing to observe. Your argument for keeping them is sound and I am accepting it.

Not in question

The trim-ack hold you considered and rejected: I agree, and for the reason you give. An unbounded hold turns a strap emitting rejectable records into an offload that never advances and that the user cannot clear. Measuring the class is the right call while the class is still theoretical.

No authenticity claim, no checksum algorithm changes, no new writes to hardware. Good.

[ryanbr]

Status update, and a heads-up about what moved under you.

Main gained two console-decode changes today, and both touch lines this PR rewrites:

• fix: decode WHOOP 5 console sequence as a wrapping u8 #2192 (by @Trillient) split the WHOOP 5 console header: byte 9 is a wrapping u8 console_sequence, byte 10 a separate raw header byte. It used to be read as one u16 named record_index.
• whoop5: decode the Kotlin console sequence as a wrapping u8 #2196 is mine, carrying the same change to the Kotlin twin, which fix: decode WHOOP 5 console sequence as a wrapping u8 #2192 had left behind.

I rebased your branch locally to check it is still tractable, and it is. Two conflicts, one per platform, both the same shape and both in the console decoder:

Interpreter.swift          decodeWhoop5ConsoleLogs
Framing.kt                 decodeConsoleLogsWhoop5

The resolution is to keep the new field split and add your limit argument to each read, so on the Kotlin side:

private fun decodeConsoleLogsWhoop5(frame: ByteArray, limit: Int, parsed: MutableMap<String, Any?>) {
    frame.u8(9, limit)?.let { parsed["console_sequence"] = it }
    frame.u8(10, limit)?.let { parsed["console_header_byte_10"] = it }
    frame.u32(12, limit)?.let { parsed["unix"] = it.toInt() }
    frame.u16(16, limit)?.let { parsed["subsec"] = it }
    val payEnd = limit

The Swift side is the same idea. Your payEnd = limit change is kept as you wrote it.

With that resolution the Kotlin protocol package is 611 tests, 0 failures on the rebased branch, including your two new frame-integrity suites. So the collision is cosmetic rather than semantic: nothing about your clamping and the new field split disagree.

Rebase when convenient and that is the merge conflict dealt with. I am not pushing to your branch.

What is still open from my review, unchanged:

1. The payloadCRCUnverifiable question. That one is yours to answer, since it is a design call about whether the reason is reachable at all.
2. The WHOOP 4.0 hardware run. Still the thing I care about most and still mine to do. Once you have rebased I will take the branch to a 4.0 and report the counters against your abort criterion.

Nothing here changes my view of the change itself.

[bhelm]

Addressed the open review items and force-with-lease updated the PR branch after rebasing onto current main.

• Resolved both console-decoder conflicts by keeping the split console_sequence / raw header byte fields and applying the frame limit to every read on Swift and Kotlin.
• Removed payloadCRCUnverifiable on both platforms. After the minimum-length and exact-length checks have passed, the CRC32 range is always computable, so that reason was unreachable. The enum/wire names, diagnostics, oracle coverage, tests, and docs now contain only the reachable mismatch case.
• Clarified that the 13-byte WHOOP 5.0/MG minimum is a deliberate empirical compatibility policy, not a structural impossibility proof. The text now reflects the smaller real frames already in the repository (20-byte command responses, 24-byte battery frames, and 32-byte realtime frames) instead of claiming 124 bytes as the smallest capture.
• Removed stale references to an oracle generator that is not shipped in the branch; the two checked-in oracle copies remain byte-identical.

Targeted verification is green: 26 Swift frame-integrity tests, the Kotlin frame-integrity build/tests, 47 Python framing tests, oracle byte-identity, documentation lint, and focused stale-text checks.

The remaining gate is the WHOOP 4.0 hardware run you offered. The branch is ready for that run; the documented abort criterion and counters are unchanged.