Skip to content

Parse once in /v1/explain/all-levels - #7

Merged
ethanzhoucool merged 2 commits into
mainfrom
claude/all-levels-parse-once
Nov 3, 2026
Merged

ethanzhoucool merged 2 commits into
mainfrom
claude/all-levels-parse-once

Conversation

@ethanzhoucool

Copy link
Copy Markdown
Owner

Summary

POST /v1/explain/all-levels is the endpoint the web UI calls on every run. Its docstring gives the reason it exists: "The parse and analysis passes are the expensive part and are level-independent." But it called explain() once per level, so every request parsed and analysed the snippet three times.

That also caused a correctness bug. Each pass created a fresh IR with new node ids. The response's shared analysis block came from the ELI5 pass, so the BEGINNER and DEVELOPER lines/annotations/concepts carried nodeIds that matched nothing in analysis. For def add(a, b): ... on main:

eli5      analysis.functions[0].nodeId = 2   line 1 nodeId = 2
beginner  analysis.functions[0].nodeId = 2   line 1 nodeId = 10
developer analysis.functions[0].nodeId = 2   line 1 nodeId = 18

Changes

  • core.py: new Explanation.at_level(level). It re-runs only the narration (annotate, summarize, line_view) over the existing parse and analysis, for another audience. LLM enrichment is level-specific and is not carried over (all-levels never enriched anyway).
  • api/main.py: explain_all_levels calls explain() once and uses at_level() for the other two levels. The response shape and key order are unchanged.

Testing

  • pytest on Python 3.13, 3.12 and 3.11: 152 passed, 3 skipped. ruff is clean, and mypy reports the same 26 errors as before.

  • Same output: I dumped the full all-levels JSON on main and on this branch for the 5 bundled web samples, a detection case, a syntax-error case and a 60 KB slice of stdlib code. With nodeId/nodeIds stripped, the two dumps are byte-for-byte identical (1.5 MB each).

  • Speed (endpoint through TestClient, Python 3.13, median):

    input main this PR
    Python web sample (1.2 KB, 30 runs) 17.9 ms 12.5 ms
    195 KB of stdlib Python (5 runs) 1112 ms 558 ms
  • New tests. All fail on main:

    • test_all_levels_parses_once counts adapter parse calls;
    • test_all_levels_node_ids_match_the_shared_analysis;
    • test_at_level_matches_a_fresh_explanation, one per level: summary and line text/kind match a fresh explain() at that level.

Generated by Claude Code

claude and others added 2 commits November 2, 2026 07:01
The endpoint's docstring says parsing and analysis are level-independent,
which is the reason it exists, but it called explain() once per level and
so parsed and analysed the snippet three times. It is the endpoint the web
UI calls on every run.

Re-running the whole pipeline also gave each level its own node ids. The
response's `analysis` came from the ELI5 pass, so BEGINNER and DEVELOPER
lines carried node ids that matched nothing in it.

Explanation.at_level() re-narrates an existing parse and analysis for
another audience, and the endpoint uses it for the other two levels.
The response is unchanged apart from the now-consistent node ids, and it
is ~30% faster on the web sample and 2x on a 195KB file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01UigC7feaY8NkAy9oay3GDW
@ethanzhoucool
ethanzhoucool marked this pull request as ready for review November 3, 2026 17:17
@ethanzhoucool
ethanzhoucool merged commit c3a468f into main Nov 3, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants