Repository navigation
feat: fail early when an export cannot fit its index - #120
Merged
Merged
Conversation
The index value and byte limits were only checked when index.json was written, after every state had been prepared. Planning now projects the fewest values and bytes the index can hold and fails with export_invalid before any notebook runs, from the spec alone and again from the resolved plan. Preparation measures each committed state with the counters ExportIndex.to_bytes() uses and stops as soon as the remaining states cannot fit, keeping prepared states reusable. Optional outputs count at their cheapest unavailable entry. ExportPlan reports the bound as index_budget and lists optional_outputs, and the CLI plan summary prints it.
|
marimo-export uv pip install "marimo-export @ https://github.com/marimo-team/marimo-export/releases/download/preview/marimo_export-0.2.3.dev13-py3-none-any.whl"
pnpm add "https://github.com/marimo-team/marimo-export/releases/download/preview/marimo-team-marimo-export-0.2.3-dev.13.tgz" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The index value and byte limits were only enforced when
index.jsonwas written, after every state had been prepared. A large state space could run for many minutes before it failed on a limit it was always going to exceed.The producer now projects the fewest values and bytes the index can hold and checks that bound three times:
export_invalidin about a second, with a message that suggests how many states to keep and the samedetailskeys asExportIndex.to_bytes().The projection measures real index records with the counters
to_bytes()uses. Each unprepared output counts at the cheapest entry it can write, so optional outputs count at their unavailable entry.ExportPlanreports the bound asindex_budgetand listsoptional_outputs, andmarimo-export planprints it on anIndexline.