Skip to content

Parallel execution-tree fetch, request hooks and client-side rate limiting (v0.1.34) - #79

Merged
Volv-G merged 1 commit into
masterfrom
parallel-execution-tree-fetch
Oct 12, 2026
Merged

Volv-G merged 1 commit into
masterfrom
parallel-execution-tree-fetch

Conversation

@Volv-G

@Volv-G Volv-G commented Oct 12, 2026

Copy link
Copy Markdown
Collaborator

get_execution_details used to fetch a run's execution tree with one sequential request per execution. It now fetches level by level in parallel, without changing its output, and the client paces its own requests.

  • Parallel tree fetch: new tangle_cli.execution_tree.fetch_execution_tree fetches breadth-first, one batch per level, with up to execution_tree_max_workers (8) threads. It runs inline without threads (e.g. Pyodide). get_execution_details uses it, and the enriched output is unchanged.
  • Request hooks: no-op prefetch(paths) / persist(paths) on TangleApiClient, as a generic caching extension point.
  • Client rate limiting: max_requests_per_second (default 8, None disables, off under emscripten) paces every physical send across threads.
    • A 429 or 503 pauses the shared limiter; the other workers resume with jitter.
    • A 503's Retry-After is now honoured, including for log-stream opens.
    • Limiter waits never carry a send past the retry deadline.
    • Auth refresh and header snapshots are serialized across workers.
  • Version: 0.1.34. The README documents the fetcher, hooks and limiter.

Tests

  • Run-details goldens generated from the previous sequential traversal (byte-identical output; each execution fetched once).
  • Fetcher, hook and threading tests.
  • Deterministic fake-clock rate-limiter tests, plus client-integration tests.
  • Full suite passes locally.

…iting (v0.1.34)

- get_execution_details fetches the tree breadth-first (one batch per
  level, execution_tree_max_workers threads, inline without threads) via
  the new tangle_cli.execution_tree.fetch_execution_tree. Enriched output
  is byte-identical (goldens from the previous sequential traversal).
- No-op prefetch(paths)/persist(paths) client hooks as a caching
  extension point.
- Client-wide rate limiting (max_requests_per_second, default 8; off under
  emscripten): every physical send is paced across threads; 429 and 503
  pause the shared limiter (others resume with jitter); 503 Retry-After is
  honoured, including for log-stream opens; limiter waits never carry a
  send past the retry deadline. Auth refresh and header snapshots are
  serialized for parallel workers.
@Volv-G
Volv-G requested a review from Ark-kun as a code owner October 12, 2026 18:48
@Volv-G
Volv-G merged commit 4cfd0d3 into master Oct 12, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant