Resolve dotted and array paths in request receiver topic_identifier - #34
Open
leggetter wants to merge 3 commits into
Open
Resolve dotted and array paths in request receiver topic_identifier#34leggetter wants to merge 3 commits into
leggetter wants to merge 3 commits into
Conversation
The receiver looked up topic_identifier with a flat lookup:
topic = output.headers[identifier] || output.body?.[identifier];
23 providers declare an identifier that nests the event type, such as
data.type, events[].eventType or entry[].changes[].field. None of them
resolve through a flat lookup, so capturing any of those providers with
yarn dev:receiver named every sample untitled-<hash>.
Measured against one real sample per affected provider, replayed through
the receiver in an isolated tree:
before 0 of 22 named correctly, 22 untitled
after 19 of 22 named correctly
Supported forms are a plain key, a dotted path, and [] array segments
including nested ones. A literal key that exists is matched before the
value is treated as a path, so a header whose real name contains a dot
still resolves. Only a scalar can name a file; anything else falls back
to untitled- rather than writing [object Object].
The three that still do not resolve are stale configs rather than
resolver limitations, and are left alone deliberately since each needs a
decision about the right value:
mailgun declares event-data.event, sample has event at top level
twitter declares data.event_type, sample has direct_message_events
at top level
xero topic is a composite of eventCategory/eventType, so no
single identifier can produce CREDITNOTE/CREATE
compile.ts is unaffected: it reads the topic key already stored in each
sample file and never consults topic_identifier. No published output
changes.
README documents the supported path forms.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01N3a91JneFov5QaEUrcXXog
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
leggetter
marked this pull request as ready for review
August 28, 2026 15:57
The resolver has six branches that all fail silently: a wrong answer becomes a filename, not an exception. The PR's measured before/after came from a throwaway script, so nothing in the repo would catch a regression. requestReceiver.ts calls app.listen() at import, so a test importing it would hang. The two pure functions move to topic.ts unchanged, with their original comment verbatim, and requestReceiver.ts imports from there. 14 tests over plain keys, dotted paths, single and nested array segments, an array segment under a dotted path, literal keys that contain a dot, paths landing on objects, empty arrays, broken paths, null sources, and numeric coercion. Verified the tests can fail: changing current[0] to current[1] in the array branch turns 3 of them red. `tsc --noEmit` reports the same two pre-existing errors in scripts/ as before, and `yarn compile` produces an identical tree, so the extraction changes no published output. Uses node:test with the ts-node already in devDependencies, so the repo gains a `yarn test` script and no new dependency. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3a91JneFov5QaEUrcXXog
The repo had no workflows, so `yarn test` only ever ran if someone remembered to. A test nobody runs is a comment. Triggers on pull requests and pushes to main. Node 22 with the yarn cache, `yarn install --frozen-lockfile` so the lockfile is authoritative, then `yarn test`. Verified by running the same sequence against a clean export of this tree: install from the lockfile, then 14 of 14 passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01N3a91JneFov5QaEUrcXXog
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The receiver resolved
topic_identifierwith a flat lookup:23 providers declare an identifier that nests the event type -
data.type,events[].eventType,entry[].changes[].fieldand so on. None resolve through a flat lookup, so capturing any of them withyarn dev:receivernamed every sampleuntitled-<hash>.Raised by the Copilot reviewer on #33. The finding was correct about the mechanism but attributed it to that PR; it is pre-existing and affects 22 providers besides the one #33 adds.
Measured, not asserted
One real sample per affected provider, replayed through the receiver into an isolated tree containing only
index.jsonfiles so every write is observable:main)Passing includes the array forms:
okta(data.events[].eventType),whatsapp(entry[].changes[].field),klaviyo(data[].topic),microsoft-graph(value[].changeType),fastspring(events[].type).Three that still don't resolve, deliberately left alone
These are stale configs, not resolver limitations. Each needs a decision about the right value, which doesn't belong in a mechanical fix:
mailgunevent-data.eventeventat top leveltwitterdata.event_typedirect_message_eventsat top levelxeroevents[].eventType→ resolves toCREATECREDITNOTE/CREATE, a composite ofeventCategory/eventTypexerois the interesting one: its topic combines two fields, so no single identifier can ever produce it. That's a schema question, not a lookup bug.These three are the reason this PR doesn't close the issue entirely.
Supported forms
eventbody.event, or the header of that namedata.typebody.data.typeevents[].eventTypebody.eventsentry[].changes[].fieldA literal key that exists is matched before the value is treated as a path, so a header whose real name contains a dot still resolves. Only a scalar can name a file; anything else falls back to
untitled-rather than writing[object Object].Scope
compile.tsis unaffected - it reads thetopickey already stored in each sample file and never consultstopic_identifier. No published output changes;yarn compileproduces an identical tree.No new type errors:
tsc --noEmit -p tsconfig.jsonreports the same two pre-existing errors inscripts/on this branch as onmain.README documents the supported path forms.
Relationship to #33
Independent. #33 is correct as it stands - its samples were written directly rather than through the receiver, and each already carries a correct
topickey. This PR is why #33 needs no change.🤖 Generated with Claude Code
https://claude.ai/code/session_01N3a91JneFov5QaEUrcXXog
Generated by Claude Code