feat: follow an athlete past the fragment you pointed at by ralyodio · Pull Request #35 · profullstack/reeleel

ralyodio · 2026-08-10T12:47:17Z

Follow-up to #34. That PR made scoring honest; this one attacks the limit it exposed — the athlete is only known for the sliver of the game the user happened to click.

The problem

After #34 and a thorough re-detection, the production project had Joe correctly bound across 6 fragments — all of them inside 219.5s–251.2s of a 300s game. Re-identification matches on box overlap, so it can only ever confirm a binding where one already exists. It cannot discover the same child anywhere else. Result: the 5 suggested moments all fired on activity_near_goal, high_motion — scene-wide signals with nothing to do with Joe.

What this adds

A reeleel-cv appearance command: a coarse HSV histogram of the torso (not the whole box — legs and background dilute it), from one pass over the 540p proxy. ~6s for 600 frames and 7,280 boxes. Boxes arrive on stdin; there are thousands.

Colour is a veto, never an identifier. That is the entire design, and it's the finding that changed the implementation mid-flight. Measured on the production game:

rule	tracks	recovered coverage
colour only (≥0.55)	661 of 1152	2,306s — in a 300-second video
continuity only	56	120.3s
continuity + colour	14	51.2s (from 6 tracks / 31.7s)

Teammates wear the same shirt, so a colour signature identifies a team — and the children it wrongly volunteers are precisely the ones standing next to yours. Continuity alone links whoever is nearby. So the identity claim rests on continuity (a fragment beginning where and when another ended, within 2s and a distance a child could actually run, scaled to frame width), and colour can only rule a link out.

The gap limit is measured, not chosen: at 4s the accepted links reached 894px of a 1920-wide frame for eight more seconds of coverage.

Matching is iterative — coverage compounds, because the fragment continuing the athlete's new last track wasn't adjacent to anything before it was accepted — and the reference signature re-averages as it grows, so it can follow a lighting change down the court.

Nothing is assigned

Proposals are pre-selected in the existing picker with the evidence behind each: the gap, the distance, and the colour agreement, on the tile and in the tooltip. A human confirms. A confident wrong answer here puts another family's child in a highlight reel, so it is deliberately a suggestion.

Tests

29 new tests (552 total, 5 skipped). The ones that matter are the guards:

two teams' colours separate below 0.1; the same jersey through a lighting change stays above 0.8
white vs black shirts separate (achromatic pixels go to lightness bins, not a meaningless hue)
a link is refused across too long a silence, across a jump no child could run, and between two fragments on screen together
reach scales with the frame, not a hardcoded pixel count

Lint and full build clean.

Honest limitation

51.2s of 300s is a 62% improvement, not a solved problem. Chains stop where the tracker loses the athlete for more than two seconds. Going further needs either jersey-number OCR (the numbers are a few pixels at 540p) or a learned re-ID embedding — both materially larger, and both riskier in the same direction.

🤖 Generated with Claude Code

Identifying a child only ever labels them where the user happened to look, and re-identification could only confirm a binding where one already existed. On a real game that left an athlete known for 31.7s of 300s across six fragments, all inside the one 32-second window originally clicked. Every signal that follows the athlete was dark for the other 90%, and the moments that survived were scene-wide ones with nothing to do with them. A jersey is the one thing about a child a detector can see that stays the same all afternoon, so the worker gains an `appearance` command: a coarse HSV histogram of the torso, from one pass over the 540p proxy. Boxes arrive on stdin because there are thousands of them. Colour is a veto, not an identifier, and that is the whole design. Measured against the production game, three ways: colour only (>= 0.55) 661 of 1152 tracks, 2306s of "athlete" in a 300s video continuity only 56 tracks, 120.3s both 14 tracks, 51.2s (from 6 tracks, 31.7s) Teammates wear the same shirt, so colour alone selects a *team* — and the children it wrongly volunteers are exactly the ones standing next to yours. Continuity alone links whoever happens to be nearby. The identity claim therefore rests on continuity — a fragment that begins where and when another ended, within 2s and a distance a child could actually run — with colour able only to rule a link out. At a 4s gap the accepted links reached 894px of a 1920-wide frame for eight more seconds of coverage, which is where that trade stops being worth taking. Nothing is assigned. Matches are pre-selected in the picker with the evidence behind each one — the gap, the distance, the colour agreement — and a human confirms, because the cost of a confident wrong answer is another family's child in your highlight reel. Also: `run` can write to a child's stdin, which the box list needs. Verified against production via scripts/appearance-probe.mjs and scripts/stitch-probe.mjs, both read-only; the numbers above are theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

PR #35 was squash-merged as fea3483, which is this branch's own 2833a9d with a new hash and no ancestry link. Git therefore saw the four files that commit introduced as add/add conflicts against themselves. There is no content disagreement to settle: origin/master's tree is byte-identical to 2833a9d, so every conflict resolves to this branch's side and the merge result is exactly the tree of c8efaef. Verified with `git diff c8efaef` being empty after resolution. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

…footage The reason #35 shipped returning zero matches is not that the judgement was wrong — it was that nothing could exercise it. Every decision lived in a module that also reaches a database, a subprocess and a filesystem, so importing any of it drags in a native driver, so the only way to try it against real footage was to re-implement it in a probe. A probe that agrees with a re-implementation proves nothing about what ships, and the bug was in the plumbing between the two halves, where neither was looking. `stitch.ts` now holds the whole decision and touches nothing: the link rule, the colour veto, the candidate filter and the iterative chain. `appearance.ts` keeps only fetch, call and return. The probe imports the same `chooseAthleteTracks` the app calls rather than a copy of it. Run against the production project, read-only, with the shipped code: reference 6 track(s), candidates 2598, 10090 boxes, 600 frames in 5s +218.0-218.9s gap 0.60s dist 120px colour 0.770 +251.5-253.8s gap 1.07s dist 379px colour 0.786 +255.7-256.3s gap 1.93s dist 152px colour 0.867 +256.3-259.5s gap 0.03s dist 60px colour 0.861 +260.7-261.0s gap 1.27s dist 85px colour 0.769 +253.9-254.4s gap 1.33s dist 50px colour 0.724 +262.0-266.9s gap 0.97s dist 405px colour 0.734 +266.9-273.9s gap 0.07s dist 52px colour 0.787 before: 6 track(s), 31.7s after : 14 track(s), 51.2s Ten tests over the isolated logic, including the two that matter: a matching shirt that continues nothing is refused, and a fragment on screen beside the athlete is refused however well its colour agrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

) * feat: follow an athlete past the fragment you pointed at Identifying a child only ever labels them where the user happened to look, and re-identification could only confirm a binding where one already existed. On a real game that left an athlete known for 31.7s of 300s across six fragments, all inside the one 32-second window originally clicked. Every signal that follows the athlete was dark for the other 90%, and the moments that survived were scene-wide ones with nothing to do with them. A jersey is the one thing about a child a detector can see that stays the same all afternoon, so the worker gains an `appearance` command: a coarse HSV histogram of the torso, from one pass over the 540p proxy. Boxes arrive on stdin because there are thousands of them. Colour is a veto, not an identifier, and that is the whole design. Measured against the production game, three ways: colour only (>= 0.55) 661 of 1152 tracks, 2306s of "athlete" in a 300s video continuity only 56 tracks, 120.3s both 14 tracks, 51.2s (from 6 tracks, 31.7s) Teammates wear the same shirt, so colour alone selects a *team* — and the children it wrongly volunteers are exactly the ones standing next to yours. Continuity alone links whoever happens to be nearby. The identity claim therefore rests on continuity — a fragment that begins where and when another ended, within 2s and a distance a child could actually run — with colour able only to rule a link out. At a 4s gap the accepted links reached 894px of a 1920-wide frame for eight more seconds of coverage, which is where that trade stops being worth taking. Nothing is assigned. Matches are pre-selected in the picker with the evidence behind each one — the gap, the distance, the colour agreement — and a human confirms, because the cost of a confident wrong answer is another family's child in your highlight reel. Also: `run` can write to a child's stdin, which the box list needs. Verified against production via scripts/appearance-probe.mjs and scripts/stitch-probe.mjs, both read-only; the numbers above are theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: measure a torso against the frame it was actually cropped from #35 shipped appearance matching and it returned zero matches on the very footage it was designed against — where a read-only probe of the same algorithm had found eight. Two faults, both in the seam between the parts, both invisible to every unit test because each half was right on its own. The coordinate space. Tracks are stored in source-video pixels, and appearance deliberately reads the 540p proxy because a shirt's colour survives that and decodes in a fraction of the time. The worker took the box space from the file it had opened, so 1920-wide boxes were measured against a 960-wide frame at a scale of 1: every torso rect landed off the right-hand edge, was clamped to nothing or to background, and the signatures came back empty or meaningless. The space now travels with the boxes, in the same JSON payload, because it is a property of the coordinates and not of whatever file happens to be open. The candidate floor. Proposals reused the picker's 1.5s minimum, which exists so a human has a crop long enough to recognise. Stitching wants the opposite: short fragments are the connective tissue, and five of the eight links that recovered a real athlete were under 1.5s. Lowered to 0.25s, where continuity and colour carry the claim instead of a person's eyes. `decodePlanFor` is now a pure function with the scaling rule stated once, and signatures.integration.test.ts builds a two-colour video with ffmpeg and checks that a box quoted in a larger space still lands on the right half of it — skipped where ffmpeg is absent, as the model test already is. Verified against real pixels: left vs right 0.0000, left vs left 1.0000, both crops non-empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: put the matching decision where it can be run against real footage The reason #35 shipped returning zero matches is not that the judgement was wrong — it was that nothing could exercise it. Every decision lived in a module that also reaches a database, a subprocess and a filesystem, so importing any of it drags in a native driver, so the only way to try it against real footage was to re-implement it in a probe. A probe that agrees with a re-implementation proves nothing about what ships, and the bug was in the plumbing between the two halves, where neither was looking. `stitch.ts` now holds the whole decision and touches nothing: the link rule, the colour veto, the candidate filter and the iterative chain. `appearance.ts` keeps only fetch, call and return. The probe imports the same `chooseAthleteTracks` the app calls rather than a copy of it. Run against the production project, read-only, with the shipped code: reference 6 track(s), candidates 2598, 10090 boxes, 600 frames in 5s +218.0-218.9s gap 0.60s dist 120px colour 0.770 +251.5-253.8s gap 1.07s dist 379px colour 0.786 +255.7-256.3s gap 1.93s dist 152px colour 0.867 +256.3-259.5s gap 0.03s dist 60px colour 0.861 +260.7-261.0s gap 1.27s dist 85px colour 0.769 +253.9-254.4s gap 1.33s dist 50px colour 0.724 +262.0-266.9s gap 0.97s dist 405px colour 0.734 +266.9-273.9s gap 0.07s dist 52px colour 0.787 before: 6 track(s), 31.7s after : 14 track(s), 51.2s Ten tests over the isolated logic, including the two that matter: a matching shirt that continues nothing is refused, and a fragment on screen beside the athlete is refused however well its colour agrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

* feat: follow an athlete past the fragment you pointed at Identifying a child only ever labels them where the user happened to look, and re-identification could only confirm a binding where one already existed. On a real game that left an athlete known for 31.7s of 300s across six fragments, all inside the one 32-second window originally clicked. Every signal that follows the athlete was dark for the other 90%, and the moments that survived were scene-wide ones with nothing to do with them. A jersey is the one thing about a child a detector can see that stays the same all afternoon, so the worker gains an `appearance` command: a coarse HSV histogram of the torso, from one pass over the 540p proxy. Boxes arrive on stdin because there are thousands of them. Colour is a veto, not an identifier, and that is the whole design. Measured against the production game, three ways: colour only (>= 0.55) 661 of 1152 tracks, 2306s of "athlete" in a 300s video continuity only 56 tracks, 120.3s both 14 tracks, 51.2s (from 6 tracks, 31.7s) Teammates wear the same shirt, so colour alone selects a *team* — and the children it wrongly volunteers are exactly the ones standing next to yours. Continuity alone links whoever happens to be nearby. The identity claim therefore rests on continuity — a fragment that begins where and when another ended, within 2s and a distance a child could actually run — with colour able only to rule a link out. At a 4s gap the accepted links reached 894px of a 1920-wide frame for eight more seconds of coverage, which is where that trade stops being worth taking. Nothing is assigned. Matches are pre-selected in the picker with the evidence behind each one — the gap, the distance, the colour agreement — and a human confirms, because the cost of a confident wrong answer is another family's child in your highlight reel. Also: `run` can write to a child's stdin, which the box list needs. Verified against production via scripts/appearance-probe.mjs and scripts/stitch-probe.mjs, both read-only; the numbers above are theirs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix: measure a torso against the frame it was actually cropped from #35 shipped appearance matching and it returned zero matches on the very footage it was designed against — where a read-only probe of the same algorithm had found eight. Two faults, both in the seam between the parts, both invisible to every unit test because each half was right on its own. The coordinate space. Tracks are stored in source-video pixels, and appearance deliberately reads the 540p proxy because a shirt's colour survives that and decodes in a fraction of the time. The worker took the box space from the file it had opened, so 1920-wide boxes were measured against a 960-wide frame at a scale of 1: every torso rect landed off the right-hand edge, was clamped to nothing or to background, and the signatures came back empty or meaningless. The space now travels with the boxes, in the same JSON payload, because it is a property of the coordinates and not of whatever file happens to be open. The candidate floor. Proposals reused the picker's 1.5s minimum, which exists so a human has a crop long enough to recognise. Stitching wants the opposite: short fragments are the connective tissue, and five of the eight links that recovered a real athlete were under 1.5s. Lowered to 0.25s, where continuity and colour carry the claim instead of a person's eyes. `decodePlanFor` is now a pure function with the scaling rule stated once, and signatures.integration.test.ts builds a two-colour video with ffmpeg and checks that a box quoted in a larger space still lands on the right half of it — skipped where ffmpeg is absent, as the model test already is. Verified against real pixels: left vs right 0.0000, left vs left 1.0000, both crops non-empty. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * refactor: put the matching decision where it can be run against real footage The reason #35 shipped returning zero matches is not that the judgement was wrong — it was that nothing could exercise it. Every decision lived in a module that also reaches a database, a subprocess and a filesystem, so importing any of it drags in a native driver, so the only way to try it against real footage was to re-implement it in a probe. A probe that agrees with a re-implementation proves nothing about what ships, and the bug was in the plumbing between the two halves, where neither was looking. `stitch.ts` now holds the whole decision and touches nothing: the link rule, the colour veto, the candidate filter and the iterative chain. `appearance.ts` keeps only fetch, call and return. The probe imports the same `chooseAthleteTracks` the app calls rather than a copy of it. Run against the production project, read-only, with the shipped code: reference 6 track(s), candidates 2598, 10090 boxes, 600 frames in 5s +218.0-218.9s gap 0.60s dist 120px colour 0.770 +251.5-253.8s gap 1.07s dist 379px colour 0.786 +255.7-256.3s gap 1.93s dist 152px colour 0.867 +256.3-259.5s gap 0.03s dist 60px colour 0.861 +260.7-261.0s gap 1.27s dist 85px colour 0.769 +253.9-254.4s gap 1.33s dist 50px colour 0.724 +262.0-266.9s gap 0.97s dist 405px colour 0.734 +266.9-273.9s gap 0.07s dist 52px colour 0.787 before: 6 track(s), 31.7s after : 14 track(s), 51.2s Ten tests over the isolated logic, including the two that matter: a matching shirt that continues nothing is refused, and a fragment on screen beside the athlete is refused however well its colour agrees. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: print track ids from the stitch probe The spans told you what was matched and not what to act on. The ids are what gets assigned, so they belong in the output — including a comma-joined line of the whole set, reference plus proposals, which is exactly the shape the identify endpoint takes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>

ralyodio merged commit fea3483 into master Aug 10, 2026
4 checks passed

ralyodio mentioned this pull request Aug 10, 2026

fix: measure a torso against the frame it was actually cropped from #36

Merged

ralyodio mentioned this pull request Aug 10, 2026

feat: #14 in white, not just a name #42

Merged

Navigation Menu

Uh oh!

feat: follow an athlete past the fragment you pointed at - #35

feat: follow an athlete past the fragment you pointed at#35
ralyodio merged 1 commit into
masterfrom
feat/athlete-appearance

ralyodio commented Aug 10, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

ralyodio commented Aug 10, 2026

The problem

What this adds

Nothing is assigned

Tests

Honest limitation

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant