Skip to content

feat(gre): desktop scoring adapter — synced gre_scorecard on dashboard open (Task 6) - #2

Open
f15cubing wants to merge 67 commits into
mainfrom
agent/scoring-desktop
Open

f15cubing wants to merge 67 commits into
mainfrom
agent/scoring-desktop

Conversation

@f15cubing

Copy link
Copy Markdown
Owner

What & why

Desktop-authoritative scoring adapter (Task 6). On GRE dashboard open, compute the three separate scores and write a synced gre_scorecard to col.conf so AnkiDroid can render it read-only (Task 7). Memory = FSRS-mastery Wilson range (None at n=0); Performance = not_available (no desktop attempt bank yet); Readiness = gated off with reasons + evidence panel — deliberately NOT derived from Memory (firewall / no-blend ceiling), never a bare number. Extends the W2 dashboard + the pure-stdlib outer-repo scoring/ package (imported via a sys.path bridge to the repo root).

Area(s) touched

  • qt/aqt/gre/scoring_adapter.py (new) — compute_and_write_scorecard(col)
  • qt/aqt/gre_dashboard.py_write_scorecard(mw) on-open hook (guarded; never blocks the dialog)
  • qt/tests/test_scoring_adapter.py (new) — 3 tests

engine/Rust PR? no — Qt-UI Python only; engine lane solely because it lands in the anki submodule. Docs ship in the companion f15cubing/speedrun PR.

Docs updated

Companion speedrun PR: docs/codebase/qt.md (§ scoring adapter), docs/codebase/INDEX.md, scoring/scoring.md; new Last-verified f15cubing/anki@ed74d1f.

Test evidence

Env: PYTHONPATH=pylib:out/pylib:out/qt ANKI_TEST_MODE=1 out/pyenv/bin/pytest -p no:cacheprovider qt/tests

  • test_scoring_adapter.py3 passed (RED→GREEN: schema/persist/gated on a fresh collection · evidence panel present when gated · dashboard-open hook writes the card)
  • All GRE aqt tests (test_gre_* + scoring_adapter) → 32 passed (no regression)
  • ruff check + ruff format --check + mypy → clean
  • Headless round-trip smoke: scorecard persists across close/reopen (no corruption); Readiness gated with reasons [<200 graded reviews, <50% topic coverage, no exam attempts yet]

Engine/Rust extra gate

N/A for the Rust-test items — no Rust/proto/FFI/scheduler change (Qt-UI Python only).

  • Undo: preserved — col.set_config(..., undoable=False) creates no undo entry.
  • No corruption: no schema change; round-trip open/close verified.
  • Read-only invariant: the mastery read path is unchanged and still returns plain rows (never OpChanges); the adapter's only write is the gre_scorecard config value.
  • Files touched: 3 (2 new + 1 hook line). Merge difficulty: low (isolated to qt/aqt/gre).

Made with Cursor

abdnh and others added 30 commits May 6, 2026 15:36
…fe_path

_builtin_data previously did no path validation, allowing directory
traversal attacks via crafted bundled file requests. Extract
ensure_safe_path as a reusable guard and apply it to both
_handle_local_file_request and _builtin_data. UnsafePathException is
caught at the top-level request handler.
---------

Co-authored-by: Andrew Sanchez <andrewsanchez@users.noreply.github.com>
The generator expression used 'host' as the loop variable, shadowing
the outer 'host' from request.headers. Rename to 'h' for clarity.
The 'release' job does not exist in this workflow, so the condition
would always fail when draft-release is true. Remove the stale clause.
Move git config into the commit step since it still needs it.
For patch releases run from non-main branches (e.g. 25.09.3).
…4436)

* fix(build): respect .python-version when generating pyenv

* refactor
This will be addressed later on `main`.
This reverts commit 82bfc32cdcd6eab6dcb885e61ef91485651e40d7.

This doesn't work. Fetch requests in iframes are also ResourceTypeXhr
Serve collection media with a restrictive CSP that sandboxes active documents, blocks script execution, and prevents network access. This means SVG/HTML loaded via iframe or object can still render where the browser allows it, but cannot call back into the editor page or Anki's local API.

Keep add-on web exports exempt from the sandbox, since they share the LocalFileRequest path but are trusted extension assets. The editor page CSP is intentionally left limited to script-src so existing user content such as remote iframe embeds, dictionary sites, YouTube embeds, and SVG object tags is not newly blocked by the editor itself.

Add tests for the untrusted media CSP, local API connect blocking, the add-on exemption, and the editor CSP compatibility expectations.
andrewsanchez and others added 25 commits May 8, 2026 14:19
…gate)

Add StatsService.MasteryQuery returning, per requested topic tag,
{ total_cards, reviewed_count, mastered_count, avg_recall } computed by a
single FSRS-retrievability SQL pass aggregated in Rust. Hierarchical tag
match (topic + ::* descendants); mastered = memory state AND R >= 0.9.

New cards short-circuit to NULL retrievability, which also sidesteps a
debug-only integer overflow in extract_fsrs_retrievability for unreviewed
cards on a young collection. Pure SELECT: no transact, no OpChanges.

Tests: empty/no-match zeros, aggregation+hierarchy, read-only invariant
(undo step + card count unchanged, quick_check_corrupt clean).
…-only test

Harden mastery_query_is_read_only to also assert the due/new/review queue counts
(via Collection::counts) are byte-identical after the read, matching the design
spec §4 read-only invariant (previously only the undo step + card count +
quick_check_corrupt were asserted).
- ScoreSlot: add optional reasons[] prop; render as bulleted list under
  body when present and non-empty (Readiness slot only; Performance passes none)
- ScoreSlot: allowlist state class to insufficient_evidence / not_available,
  map unknown strings to not_available to avoid malformed markup
- MemoryPanel: guard mean_r with optional-chaining fallback (?.toFixed(2) ?? "—")
- test_gre_dashboard_data.py: move `from dataclasses import dataclass as _dc`
  from mid-file (E402/I001) to the top-level import block; fix isort order;
  drop unused `tax` variable (F841) in test_query_topics.
- gre_dashboard.py: fix ruff I001 import block sort order.
GreDashboard uses AnkiWebViewKind.GRE_DASHBOARD, which was missing from
the api-access allowlist in _profileForPage, so the AuthInterceptor never
injected the Bearer token and mediasrv rejected the read-only data POST
with 403 'Unexpected API access'. Add the kind to the trusted-page list
(our own SvelteKit route, no third-party JS), matching graphs/deck-options.
- Add qt/aqt/gre/data/gre-study-deck.apkg (2.3 MB, 5407 cards, seed 42)
- Add qt/aqt/gre/deck_autoimport.py: GRE_DECK_VERSION="2026-07-02",
  _import_bundled(col), _run_if_needed(col), maybe_import_gre_deck(mw).
  Import policy: merge_notetypes=True, update_notes/notetypes=IF_NEWER,
  with_scheduling=False (preserves FSRS history), with_deck_configs=False.
- Register hook in qt/aqt/main.py via gui_hooks.collection_did_load so
  the importer fires once per profile open, skipping no-op re-imports.
- Add qt/tests/test_gre_autoimport.py: 3 tests (first import adds >5000
  cards + sets version; second run at same version is a no-op; missing
  version triggers import). All pass with built pylib backend.

No Rust/proto/scheduler/undo changes. UI/import glue only.
Give the GRE readiness dashboard a distinctive, intentional identity in
place of the default-Anki look: a calibrated "instrument" system where
uncertainty is the visual language. Every metric renders through one
signature component — CalibrationStrip — a shaded 95% band + point tick
on a 0..max axis, used at all three scales (exam headline, per-bucket,
per-leaf). n=0 draws an explicit dotted "not yet" rail rather than a
fabricated position.

- tokens.css: 6-token palette + type roles (system tabular mono for all
  numerals), light/dark via Anki's .night-mode; no bundled fonts.
- CalibrationStrip.svelte + lib.ts (pure geometry) + lib.test.ts (8 vitest).
- Restyle +page/MemoryPanel/CoverageMap/ScoreSlot; amber (not red) give-up
  state; single best-next leaf ringed; restrained page-load reveal
  (prefers-reduced-motion respected).

Pure presentation: no view-model/dashboard_data.py change (that stays the
Thursday scoring layer's surface); still read-only; three scores stay
separate; the state guard still forbids a fabricated readiness number.
…ored)

Vendor the faithful GRE Math Subject Test exam-mode core into the fork so it
runs in (and ships with) the app — the app cannot read the outer eval/bank/ at
runtime, so the logic + items are vendored (the dashboard_data.py + taxonomy.json
pattern).

- exam.py: deterministic, blueprint-matched (ETS 50/25/25) form assembly at the
  official pace (2.58 min/item; presets full/half/third/mini = 66/33/22/11) +
  rights-only scoring (no penalty for wrong/omitted) with per-leaf/bucket
  breakdown + Wilson CI + the attempts record for the scoring seam.
  Headless-importable (no aqt deps), mirrors dashboard_data.py.
- exam_items.json: vendored copy of the authored eval bank (80 items),
  drift-guarded against eval/bank/items.yaml by the outer test.
- test_gre_exam.py: pace/blueprint/assembly determinism/insufficient/firewall/
  rights-only/Wilson/attempts. check:pytest:aqt green.

The SvelteKit exam shell + mediasrv endpoints + dialog + mastery gate are B-2.
…hell

The interactive exam shell on the B-1 core, faithful to the computer-delivered
GRE Math Subject Test: one global countdown, one item at a time, five A–E
single-select options, Mark + free Back/Next, a Review-screen navigator grid, no
calculator, no pause, auto-submit at 0:00, and no per-item feedback (results only
after submit). Presets full/half/third/mini at the official 2.58 min/item pace.

- ts/routes/gre-exam/: +page (session state machine + timer), ItemView,
  Countdown, Navigator, Results (reuses the dashboard CalibrationStrip +
  tokens.css); lib.ts (formatClock/tally) + lib.test.ts.
- mediasrv: read-only greExamForm (assembles a blueprint-matched form; never
  sends the keys to the client) + greExamSubmit (rights-only server-side scoring;
  persists an attempts side-file for the scoring layer; reveals keys after
  submit). Route registered in is_sveltekit_page.
- webview: AnkiWebViewKind.GRE_EXAM + api-access allowlist. gre_exam.py QDialog +
  Tools-menu action. Mastery-gate structure present (threshold 0.0 for now).

Read-only: no collection/OpChanges writes (attempts go to a profile side-file).
Firewalled: draws only from the vendored eval items (partition p0).
Verified: check:svelte + check:pytest:aqt green; vitest lib green; headless GUI
smoke of setup/exam/review/results.
Desktop: first-run auto-import of the bundled GRE study deck
…nki-deckauto-integration

# Conflicts:
#	qt/aqt/main.py
The exam webview rendered items as plain ASCII. Now item math is delimited
LaTeX (\(...\) / \[...\]) typeset by MathJax (SVG output — self-contained,
offline, no external font files), matching the study-deck cards.

- add ts/routes/gre-exam/mathjax.ts: sets the MathJax config then dynamically
  imports the engine (code-split, offline chunk) and exposes typesetMath()
- ItemView / Results typeset their content; +page keys ItemView by item id so
  each item gets a fresh DOM (no Svelte/MathJax churn on re-select)
- exam_items.json re-vendored from the LaTeX-migrated eval bank

No engine/Rust/proto change; read-only exam surface unchanged otherwise.

@f15cubing f15cubing left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Engine-lane review — Task 6 desktop scoring adapter

Reviewed by the engine-lane reviewer (different agent than the builder). Recommendation: APPROVE — clear to merge once the merge-order/pin steps in note 2 are handled.

Note: submitted as a review comment rather than a formal --approve because GitHub's review API refuses approve/request-changes on a PR authored by the same account (Can not approve your own pull request). The approval recommendation stands. Not merging, per instructions.

What I actually reviewed: the real change is the tip commit ed74d1fe6 (parent 277f12e) = 3 files, +204:

  • qt/aqt/gre/scoring_adapter.py (new, +118)
  • qt/aqt/gre_dashboard.py (+18, guarded on-open hook)
  • qt/tests/test_scoring_adapter.py (new, +68, 3 tests)

The gh pr diff spans ~60 files only because main is far behind this stacked branch (see note 2) — not scope creep in this task.

Base gate — PASS

  • Scope — PASS. Tip commit is exactly the 3 expected files; no unrelated edits.
  • Tests — PASS. Re-ran in the pre-built worktree: pytest qt/tests/test_scoring_adapter.py test_gre_dashboard_data.py test_gre_dashboard_mediasrv.py -> 17 passed (0.52s). Builder's posted green (3 adapter + 32 GRE aqt, ruff/format/mypy clean, headless round-trip smoke) trusted + spot-checked per building-and-testing.
  • Docs — PASS. Updated in the companion f15cubing/speedrun#28 (qt.md § scoring adapter, INDEX.md, scoring.md; Last-verified -> ed74d1f).
  • PR body — PASS. All template sections present.

Not a Rust change — CONFIRMED

Tip commit touches no rslib/, .proto, or pylib FFI — Qt-UI Python only; engine-lane solely because it lands in the anki submodule. Rust extra-gate items (>=3 Rust tests, etc.) correctly N/A.

Ceiling spot-check (read the code) — PASS

  1. Undo preserved. Only collection write is col.set_config("gre_scorecard", card); set_config's default is undoable=False (pylib docstring: creates no undo entry, preserves undo history). No other mutation.
  2. Read-only / no corruption. col.mastery_query read path is untouched by this commit and still returns plain rows, never OpChanges. Adapter only READs mastery + the W2 view-model and WRITEs the single config value; no schema change.
  3. Three scores never blended; Readiness never a bare number. scorecard.build() nests memory/performance/readiness as separate objects. Readiness is shown=False, estimate=None with non-empty reasons (give_up() -> <200 graded reviews, <50% topic coverage, plus no exam attempts yet at n=0) and the full evidence panel (coverage_pct, confidence, reasons, best_next_topic). It is derived from review/coverage proxies, not from Memory — the adapter never feeds vm["memory"] into readiness and deliberately avoids rd.project(). Firewall intact.
  4. No ETS leak. Adapter reads taxonomy + FSRS mastery rows + the W2 view-model only; never imports exam_items.json / eval/bank / any ETS anchor table.
  5. AGPL headers present on both new files.

Non-blocking notes

  1. (nit) The adapter relies on the default undoable=False instead of passing it explicitly as the PR body states. Correct today; passing it explicitly would match the body and be robust to any future default change.
  2. (merge hygiene) PR is CONFLICTING and diffs ~60 files vs main because the branch is stacked on the not-yet-on-main W1 mastery / W2 dashboard / exam-mode / autoimport work. At merge, ensure that prior work is already on main so a squash of this PR contains only Task 6, then reconcile ankitects#28's pin to the squash-merged SHA (already called out in both PR bodies).

@f15cubing
f15cubing force-pushed the agent/scoring-desktop branch from ed74d1f to 7c4836c Compare July 3, 2026 15:13
f15cubing added a commit to f15cubing/speedrun that referenced this pull request Jul 3, 2026
#28)

Restacked scoring on top of Exam-Mode LaTeX (a631ec3). On dashboard open, writes the synced three-score gre_scorecard to col.conf (Memory range; Performance not-available; Readiness gated with evidence panel). Different-agent review APPROVED (f15cubing/anki#2).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants