Repository navigation
Conversation
…cannot open (zarr-developers#4604) * feat(zarr-metadata): add a reader that reports what a reader cannot open `read_openable_node_metadata_v3` reads a v3 `zarr.json` as `read_node_metadata_v3` does and adds a problem of the new kind `unclaimed_declaration` for each declaration a reader must understand that nothing in scope claims: an unknown codec, data type or chunk grid, or an unknown top-level member without `"must_understand": false`. Its reading has no problems exactly when a reader that understands the scope can open the node. The other readers keep accepting unclaimed declarations, so the format stays open; their docstrings and the parsing guide now point to `unclaimed_declarations` and the new reader. Prompted by refkv's report that an unknown v3 member parses without error. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-metadata): number the changelog fragment for #440 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-metadata): number the unopenable-declarations fragment after upstream PR 4604 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
… each spec claim (zarr-developers#4605) * docs(zarr-metadata): split the landing page into guide pages and link each spec claim Shrink index.md to a short pitch, one example and a reader-by-reader list of where to go. Move the rest into guide pages: models and readers, what the readers check, consolidated metadata, repairing known writer bugs, writing an extension, JSON Schema and Pydantic. Each claim backed by a Zarr spec links to the spec section that makes it; the consolidated metadata page says what the specs do and do not settle and links the zarr-specs PRs and issues and the original zarr-python implementation. Trim the README so it no longer mirrors the landing page, and say "context" where the pages said "scope". Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): number the guide-pages fragment after its fork PR Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): say "member" for a JSON name/value pair in the guide pages The rename branch keeps "member" for a name/value pair of a metadata object, so the new pages follow: "a top-level member that the spec does not define is an extra field". "Extra field" and "extension field" stay, after the names extra_fields and ZarrV3ExtensionField. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): say "member" for a JSON name/value pair on every page Sweep the parsing-validation and context pages, which said "field" for a name/value pair of a metadata object, and state the one exception on the models page: "extra field" follows the spec's "additional fields" and the names extra_fields, ExtraFieldDefinition and must_understand_fields; "field" otherwise means a field of a Python class, or a part of a structured data type. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): say "member" in the two must_understand_fields sentences The models page said "each unclaimed field" and "any such field" where the landing page and every neighboring sentence say "member". Roborev flagged the mismatch. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): say why the additional-fields link points at the latest spec v3.1.0 has no "Additional fields" heading; PR 373 added it after the release, so the link cannot be pinned. Say so beside the link. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): split the additional-fields sentence and date its claim Roborev found the three-clause sentence hard to parse and its "no tagged release yet" unanchored in time. Two sentences now, anchored to v3.1.0, with a note to pin the link once a release carries the heading. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): fix six wording and example issues roborev found on the guide pages The pydantic example now builds the manifest with model_validate, so it type-checks as the next sentence claims. The consolidated page says three other readers take a context, not two. The equality section quotes the v2 spec on a missing .zattrs in two sentences. The two numbered Sphinx anchors are replaced by line ranges in the spec source at the v3.1.0 tag. The landing page says once that the library does no chunk or store I/O. The extensions page example and the zarr_metadata.definition docstring example are pinned to each other by a test. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(zarr-metadata): make the example-drift test say what it could not find Roborev asked for diagnostics: the test now asserts there is exactly one extensions-session block and exactly one marker line in the docstring, each with a message, and compares line lists so a mismatch points at the differing line. Assisted-by: ClaudeCode:claude-fable-5-1 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * docs(zarr-metadata): build the extension example with union and say "context" throughout On main a Context has no update method, so the acme.lz4 example on the extensions page now adds its definition with union, as the definition module docstring does. Two leftover "scope"s in the parsing page say "context", and the models page says "member" for an extra field. The example-drift test imports at module level. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-metadata): number the guide-pages fragment after upstream PR 4605 Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-metadata): fence the module docstring examples as python so the API pages highlight them The examples in the definition, json_schema, pydantic and typed_json module docstrings were indented code blocks. An indented block names no language and pymdownx.highlight does not guess one, so the API reference rendered them as plain text. Each is now a ```python fence, and the example-drift test reads the definition docstring's fence. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(zarr-metadata): write the guide pages in short sentences and lists Long comma-chained sentences on the guide pages, README and landing page are split into short sentences, and enumerations become bulleted or numbered lists. models, validation-rules, consolidated-metadata, repair, extensions, json-schema and pydantic are restructured; context and parsing-validation keep their narrative voice and only have their long sentences split. No link, claim or code block changes. Assisted-by: ClaudeCode:claude-opus-5-5 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
`to_key_value` takes a writer instead of an `indent` keyword. A
`JSONWriter` owns how a document becomes JSON bytes: it holds the
settings (`indent`, `sort_keys`, `ensure_ascii`, `trailing_newline`)
and its `encode` method does the writing, so the separators, the bare
non-finite tokens and the surrogate escapes are no longer split off in
a private helper. `to_key_value` also accepts a mapping of the settings,
`JSONWriterSchema`, so callers need not import the class:
`to_key_value({"indent": 2})`.
The defaults are those of `json.dumps`, so a bare `to_key_value()` writes
the same bytes as before. A setting `json.dumps` would misread raises,
and so does a mapping key that is not a setting.
Rebased onto main as one commit, replacing the JSONWriterConfig design.
Assisted-by: ClaudeCode:claude-opus-5-5
d-v-b
force-pushed
the
feat/zarr-metadata-json-writer-config
branch
from
November 9, 2026 12:31
8771582 to
95857ac
Compare
…s#4610 Assisted-by: ClaudeCode:claude-opus-5-5
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤖 AI text below 🤖
Agent summary
Breaking:
to_key_valuetakes a writer instead of anindentkeyword. AJSONWriterowns how a document becomes JSON bytes: it holds the settings (indent,sort_keys,ensure_ascii,trailing_newline), and itsencodemethod does the writing.to_key_valuealso accepts a plain mapping of the settings, soto_key_value(indent=2)becomesto_key_value({"indent": 2}), with no import.JSONWriteris a final, frozen dataclass inzarr_metadata.model._json_writer, exported fromzarr_metadataandzarr_metadata.model.encodeholds every choice about the bytes: the separators of Python'sjsonmodule, non-finite numbers written as bare tokens (as zarr-python writes attributes), and lone surrogates written as escapes whenensure_asciiis off. The privatedump_store_jsonhelper is gone, so nothing else writes store JSON.JSONWriterSchemais the closedTypedDictof the settings, every one optional.JSONWriterLike = JSONWriter | JSONWriterSchemais what each model'sto_key_value(writer=...)accepts.JSONWriter.from_dictturns either into a writer (a writer passes through), andto_dictgoes back.json.dumps, which the models already used, so a bareto_key_value()writes the same bytes as before.json.dumpswould accept but misread raises when the writer is built:indent=True(read as 1), a negative indent (read as 0), or a string indent that is not JSON whitespace (written verbatim, giving invalid JSON). A mapping key that is not a setting raisesTypeError, so a misspelled setting is not ignored.docs/models.mdgains a "Writing store bytes" section with a tested example.tests/model/test_json_writer.pychecks, for all five model types and a range of writers given both asJSONWriterand as mappings, that each key's bytes are whatjson.dumpswrites under the same settings, and that the store reads back to an equal model. Each refused value has its own test.The design moved from an earlier
JSONWriterConfigsettings object, which a separate private function read, to one object that both holds the settings and encodes, and accepts a mapping so callers need not import it. The agent rebased ontomainas one commit and had a separate agent review the redesign. That review found no correctness bugs, and its two actionable points (subclasses passing throughfrom_dict, and gaps in the success cases) are fixed here.Follow-up elsewhere
The ArrayIO branch (
claude/arrayio-lazy-metadata-48d8e5, fork drafts #419/#420) callsto_key_value(indent=zarr_config.get("json_indent"))instore_metadata. Once this lands, that call becomesto_key_value({"indent": zarr_config.get("json_indent")}). This PR does not touch that branch.🤖 Generated with Claude Code