Skip to content

HTTP negotiation, codec registry, cross-language interop - #14

Merged
eliahilse merged 1 commit into
mainfrom
http
Aug 20, 2026
Merged

HTTP negotiation, codec registry, cross-language interop#14
eliahilse merged 1 commit into
mainfrom
http

Conversation

@eliahilse

Copy link
Copy Markdown
Owner

Closes the gap between "three conformant implementations" and "you can serve a request."

What was missing

The site advertises transparent JSON fallback and negotiated per request. Both were spec prose with no implementation. And spec/plan-columnar-v3.md §6.4 recommends a fingerprint-keyed registry that did not exist — without it profiles are operationally unusable: retraining changes the fingerprint, so a rolling deploy means every request between mismatched peers falls back to JSON, and the only safe move is never to retrain.

The protocol — spec/negotiation-v1.md

  • Client sends Hyperfly-Accept: <fp>[, <fp>], most preferred first. Server serves the first it holds, else JSON + Hyperfly-Offer.
  • Client preference decides, which is what lets a client migrate itself during rotation without the server tracking who holds what.
  • Vary: Hyperfly-Accept on every response — the same URL yields either representation, so a shared cache would otherwise serve one peer's binary to a peer that cannot read it.
  • .well-known/hyperfly/{fingerprint} serves the canonical artifact, immutable because it is content-addressed. A client derives the codec from parsed content and verifies the fingerprint it computes, so a server cannot induce it to hash bytes it has not understood.
  • Request bodies get 415 rather than a guess: a body already sent has no safe fallback.
  • Hostile Hyperfly-Accept is bounded (32 entries) and malformed entries are dropped, not fatal.

The proof

apps/interop — a Bun server and a Python client over real HTTP, starting from a client that holds nothing:

1. no artifacts -> json, 2447 B, offered 14f65f1f
2. fetched artifact, derived fingerprint matches: 14f65f1f
3. binary -> 258 B (9.5x smaller than the JSON)
4. binary value equals the JSON value the server sent first

The golden vectors prove three implementations agree on bytes; this proves two of them agree on a conversation. CI runs it on every push, in a job with both runtimes.

Also: hyperfly/http subpath export with a Fetch adapter (respond / discovery / readBody) covering Hono, Workers, Bun.serve and Next route handlers, plus the Python equivalents.

TS 194 tests, Python 138, Rust 9, all green.

🤖 Generated with Claude Code

Three conformant implementations could not, until now, serve a single request.
The site advertised transparent JSON fallback and per-request negotiation; both
were described in the spec and absent from the code.

spec/negotiation-v1.md defines the exchange. A client advertises the
fingerprints it holds in Hyperfly-Accept, most preferred first; a server serves
the first one it can and otherwise answers JSON with a Hyperfly-Offer naming an
artifact the client could fetch. Steady state costs no extra round trip, a cold
client bootstraps in one, and every response carries Vary: Hyperfly-Accept
because the same URL yields either representation and a shared cache would
otherwise hand one peer's binary to a peer that cannot read it.

CodecRegistry is the piece that makes profiles operable rather than
theoretical. Retraining changes the fingerprint, so a deployment holding one
codec per route turns every rollout into a cutover in which in-flight clients
fall back to JSON until the fleet converges — the incentive being never to
retrain, which quietly defeats the feature. Holding the outgoing codec beside
the incoming one makes rotation a transition, and letting the client's
preference decide lets it migrate itself without the server tracking who holds
what.

Artifacts are served from .well-known and are content-addressed, so a hit is
immutable and cacheable forever while a miss is a 404 rather than an error. A
client derives its codec from the parsed artifact and verifies the fingerprint
it computes equals the one it requested, so a server cannot induce it to hash
bytes it has not understood. Request bodies get 415 rather than a guess,
because a body already sent has no safe fallback.

apps/interop is the demonstration the golden vectors imply but cannot give: a
Bun server and a Python client walking the whole protocol over real HTTP. The
client starts empty, is offered an artifact, fetches it, checks it, and then
reads 258 bytes where it first received 2447 — with both representations
carrying an identical value. CI now runs it on every push.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_hf1
@eliahilse
eliahilse merged commit b91d3d3 into main Aug 20, 2026
7 checks passed
@eliahilse
eliahilse deleted the http branch August 20, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant