Skip to content

feat(storage,policy): object-store primitives for the cluster port (RFC-006 PR 1/3)#186

Merged
aaltshuler merged 2 commits into
mainfrom
feat/object-store-primitives
Jun 11, 2026
Merged

feat(storage,policy): object-store primitives for the cluster port (RFC-006 PR 1/3)#186
aaltshuler merged 2 commits into
mainfrom
feat/object-store-primitives

Conversation

@aaltshuler

Copy link
Copy Markdown
Collaborator

First of three PRs implementing RFC-006's scoped execution (object-storage support for clusters; file:// retained, no deprecation — per #185 discussion). Purely additive primitives; no behavior changes for existing callers.

StorageAdapter gains three primitives (Local + S3)

  • read_text_versioned — content + opaque backend version token (S3: ETag; local: content sha256)
  • write_text_if_match — token-conditioned replace. S3 = conditional put (PutMode::Update/If-Match), verified against RustFS beta.8 through the real object_store 0.12.5 path (no extra builder config needed — the plan's open question, resolved empirically); local = compare + temp/rename, the same single-machine semantics the cluster ledger has today. CAS-lost is Ok(None), never silent.
  • delete_prefix — recursive + idempotent (S3 non-atomicity documented for crash-retry callers).

Gated S3 coverage (both green against local RustFS)

  • s3_adapter_conditional_writes_contract — pins the conditional-write behavior the cluster ledger will depend on; turns red if a RustFS/backend bump regresses it
  • s3_schema_apply_migrates_live_graph — closes the previously-untested schema-apply-on-S3 path before the cluster's schema executor leans on it (9s live run)

Policy: from-source twins

PolicyConfig::from_source + PolicyEngine::load_{graph,server}_from_source; path loaders delegate. Needed for policy bundles living in a non-filesystem catalog. Kind-alignment validation stays loud through the new path.

Test plan

  • Engine lib: 125 passed (2 new local-primitive tests); policy: 17 passed (1 new)
  • Both gated S3 tests green against RustFS beta.8 (:9100)
  • cargo test --workspace --locked — green (exit 0 in-log)

Next: PR 2 (storage: root — ClusterStore port + URI graph roots), PR 3 (bucket-backed serving + RustFS e2e).

🤖 Generated with Claude Code

aaltshuler and others added 2 commits June 11, 2026 05:09
…on StorageAdapter

Three primitives the cluster's object-storage port (RFC-006) needs, on the
engine's existing adapter rather than a parallel store:

- read_text_versioned: content + an opaque backend version token (S3: the
  ETag from GET; local: content sha256 — ETags don't exist on a filesystem).
- write_text_if_match: replace only when the token still matches. S3 maps to
  a conditional put (PutMode::Update / If-Match) — verified against RustFS
  beta.8 through the real object_store 0.12.5 path, no extra builder config
  needed; local compares content then swaps via temp+rename, the same
  single-machine semantics callers had before this trait (safe under their
  own lock protocol, not a cross-process barrier by itself). CAS-lost is
  Ok(None), never silent.
- delete_prefix: recursive + idempotent (local remove_dir_all; S3 list +
  delete, with the non-atomicity documented for crash-retry callers).

Gated S3 coverage: s3_adapter_conditional_writes_contract pins the
conditional-write behavior the cluster ledger will depend on (red if a
backend bump regresses it), and s3_schema_apply_migrates_live_graph closes
the previously-untested schema-apply-on-S3 path before the cluster's schema
executor leans on it. Engine gains the sha2 workspace dep.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PolicyConfig::from_source + PolicyEngine::load_graph_from_source /
load_server_from_source — the path-based loaders delegate to them. Needed by
callers whose policy bundles don't live on the local filesystem (the cluster
catalog on object storage); kind-alignment validation stays loud through the
new path.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aaltshuler has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant