Skip to content

feat(gre): surface observed Performance on the dashboard - #3

Open
f15cubing wants to merge 2 commits into
agent/exam-preset-fixfrom
agent/gre-dashboard-performance
Open

f15cubing wants to merge 2 commits into
agent/exam-preset-fixfrom
agent/gre-dashboard-performance

Conversation

@f15cubing

Copy link
Copy Markdown
Owner

What

Replaces the stale "Arrives Thursday (MCQ surface)" placeholder in the GRE dashboard's Performance slot with a live, observed Performance score.

Exam Mode already persists per-item attempts to a profile side-file (gre_exam_results.jsonl, never the collection). The read-only greDashboardData handler now pools those attempts and renders rights-only accuracy as a Wilson range with n, through the existing CalibrationStrip.

Honesty ceilings (PRD D2 / AGENTS.md)

  • Always a range, never a bare point (Wilson CI + n).
  • Give-up state, never a fabricated 0 when there are no attempts (not_available with a "take a timed exam" prompt).
  • The ScoreSlot guard still collapses any range-less / unknown state to not_available — a fabricated number can never render.
  • Three scores stay separate — Performance is not blended into Memory/Readiness.
  • Deliberately the observed low-n surface, not the calibrated logistic+Platt model in scoring/performance.py (that needs a multi-student attempt corpus; fitting it on one learner's handful of attempts would look confident exactly when it knows least).

Changes

  • qt/aqt/gre/dashboard_data.py — pure load_exam_attempts() + observed_performance(); build_view_model() gains an exam_attempts param.
  • qt/aqt/mediasrv.pygreDashboardData reads the side-file (best-effort, read-only, never raises into the response) and passes it through.
  • ts/routes/gre-dashboard/ScoreSlot.svelte — renders CalibrationStrip for a new observed state; +page.svelte wires the props.
  • Tests: 9 new unit tests (dashboard_data suite now 20; 23/23 with the mediasrv suite).

Verification

  • ./ninja check_svelte green.
  • ./ninja check_pytest_aqt green; the two dashboard test files 23/23 directly.
  • Read-only: no OpChanges, no collection mutation (endpoint only reads the mastery RPC + the side-file).

Notes

  • Lane: fast (Qt-UI + pure read-only view-model). Not self-merged.
  • Base: stacked on agent/exam-preset-fix (= the commit the outer repo currently pins, d11b424), so the diff is a single commit; retarget to main once that merges.
  • Pre-existing format debt: dashboard_data.py/mediasrv.py at the base already fail ruff format --check; this PR keeps its additions clean rather than reformatting unrelated lines.
  • Follow-ups: carry observed Performance into the synced gre_scorecard (Android panel still not_available); live GUI smoke of the slot with a real timed mock.

Made with Cursor

Felipe Caicedo added 2 commits July 5, 2026 13:18
A read-only desktop explainer (Tools ▸ "How this app differs from FSRS"): we
build on FSRS rather than replacing it — interleaving, a timed exam mode, three
separated scores, and the give-up rule. The interleaving section is interactive
and runs the REAL vendored interleave.py on a fixed example queue.

Fast lane (Qt-UI-only): new gre-method SvelteKit route (reuses the dashboard
tokens + CalibrationStrip) + read-only greMethodInterleave endpoint + GreMethod
QDialog + Tools-menu hook + vendored interleave.py (drift-guarded). No
rslib/proto/scheduler; the endpoint never touches col and returns no OpChanges.
Replace the stale "Arrives Thursday" placeholder with a live, observed
Performance score. The read-only greDashboardData handler now pools the
per-item attempts Exam Mode already persists to the gre_exam_results.jsonl
profile side-file (never the collection) and renders rights-only accuracy as a
Wilson range with n, via the existing CalibrationStrip.

Honesty ceilings held: always a range, never a bare point; a give-up
not_available state (never a fabricated 0) with no attempts; the ScoreSlot
guard still collapses any range-less state to not_available. This is the
observed low-n surface, deliberately NOT the calibrated logistic+Platt model in
scoring/performance.py (which needs a multi-student attempt corpus).

- dashboard_data.py: pure load_exam_attempts() + observed_performance();
  build_view_model() gains an exam_attempts param
- mediasrv.py: greDashboardData reads the side-file (best-effort, read-only)
- ScoreSlot.svelte: renders CalibrationStrip for an "observed" state
- 9 new unit tests (dashboard_data suite 20; 23/23 with mediasrv); check_svelte
  + check_pytest_aqt green

Read-only; no OpChanges; three scores stay separate. Pre-existing ruff-format
debt in dashboard_data.py/mediasrv.py at the base is left untouched.
@f15cubing
f15cubing force-pushed the agent/gre-dashboard-performance branch from b7cf7c2 to 6d05314 Compare July 5, 2026 18:37
f15cubing pushed a commit to f15cubing/speedrun that referenced this pull request Jul 5, 2026
…oard

Bumps the anki submodule pin to f15cubing/anki@b7cf7c2 (observed Performance on
the GRE dashboard) and updates the codebase docs in the same change:

- qt.md: new "Observed Performance" section (Exam Mode attempts side-file ->
  pooled Wilson range via CalibrationStrip), updated dashboard_data/ScoreSlot/
  mediasrv notes, tests count 11->20, footer SHA bump; scoring-adapter honesty
  note clarified (synced card Performance still not_available -> follow-up)
- models/performance.md: "Live surface" section (observed accuracy vs the
  calibrated, eval-only logistic+Platt model)
- INDEX.md: dashboard row bumped + observed_performance code paths
- STATUS.md: in-flight entry

Pairs with f15cubing/anki#3. Read-only; three scores stay separate.
f15cubing added a commit to f15cubing/speedrun that referenced this pull request Jul 5, 2026
…oard (#54)

Bumps the anki submodule pin to f15cubing/anki@b7cf7c2 (observed Performance on
the GRE dashboard) and updates the codebase docs in the same change:

- qt.md: new "Observed Performance" section (Exam Mode attempts side-file ->
  pooled Wilson range via CalibrationStrip), updated dashboard_data/ScoreSlot/
  mediasrv notes, tests count 11->20, footer SHA bump; scoring-adapter honesty
  note clarified (synced card Performance still not_available -> follow-up)
- models/performance.md: "Live surface" section (observed accuracy vs the
  calibrated, eval-only logistic+Platt model)
- INDEX.md: dashboard row bumped + observed_performance code paths
- STATUS.md: in-flight entry

Pairs with f15cubing/anki#3. Read-only; three scores stay separate.

Co-authored-by: Felipe Caicedo <felipecaicedo@Felipes-MacBook-Pro.local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant