feat(cluster,server): bucket-backed serving — config-free --cluster s3:// + RustFS e2e (RFC-006 PR 3/3)#194
Merged
Merged
Conversation
…RI boot
Two serving changes that complete RFC-006's read side:
ServingPolicy carries the policy bundle CONTENT (digest-verified at
snapshot read) instead of a blob path — the catalog may live on object
storage, and the server must not re-read mutable state after the
snapshot. The server grows a PolicySource enum: File for omnigraph.yaml
deployments (unchanged), Inline for cluster boots, wired through
PolicyEngine::load_{graph,server}_from_source.
read_serving_snapshot_from_storage(uri) reads the applied revision
straight from a storage root, and --cluster accepts a scheme-qualified
URI (s3://bucket/prefix): config-free serving — a serving box needs only
the URI and credentials; the ledger and catalog on the bucket ARE the
deployment artifact. Bare paths keep the config-directory behavior.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…docs s3_cluster.rs runs the full control-plane lifecycle against a real bucket (CI: containerized RustFS; locally the RustFS binary): import → lock released (pins the drop-time release regression caught on the first live smoke) → apply (graph roots + catalog on the bucket, nothing local) → serving snapshots from both the config dir and the bare URI → schema evolution → approved delete (prefix removal) → empty-cluster refusal. The server suite gains the config-free boot test: --cluster s3://… with zero local files serves a stored query over HTTP. CI: the rustfs job runs both suites; the classify filter covers the cluster store/serve modules and the new test files. The server smoke drops its name filter — every test in the s3 target is bucket-gated, and a filter matching nothing passes vacuously (which silently ran zero tests for a while). Docs: deployment.md gains the Bucket-no-volume shape as the preferred cloud deployment; cluster.md/server.md document --cluster <uri>; testing.md maps the new suite. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
aaltshuler has reached the 50-review limit for trial accounts. To continue receiving code reviews, upgrade your plan.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completes the object-storage arc (#186 primitives → #190 storage root → this). Two commits:
Inline policy content + config-free URI boot
ServingPolicycarries verified content, not a path: the snapshot read already digest-verifies every catalog blob — it now keeps the text. The server grows aPolicySourceenum (Filefor omnigraph.yaml deployments — unchanged;Inlinefor cluster boots), wired throughPolicyEngine::load_*_from_source. This was the last gap blocking policies on S3-rooted clusters.--cluster s3://bucket/prefix— config-free serving.read_serving_snapshot_from_storage(uri)reads the applied revision straight from the storage root; a serving box needs only the URI and credentials. Bare paths keep the config-directory behavior; the routing is one scheme check.Gated RustFS e2e + CI + docs
omnigraph-cluster/tests/s3_cluster.rs: the full control-plane lifecycle on a real bucket — import → lock-release regression (pins the drop-time fix caught on the first live smoke) → apply (graph + catalog on the bucket, nothing local) → snapshots via config dir and bare URI → schema evolution → approved delete (prefix removal) → empty-cluster refusal. Passes live against RustFS in 6.3s.--cluster s3://…boot with zero local files serves a stored query over HTTP. Passes live (8.2s for the pair).cargo testfilter matching nothing passes vacuously, which had this step silently running zero tests (found while fixing the refactor(server): modularize the test monolith and lib.rs — pure code movement #192 rename).deployment.mdflips the preferred cloud shape to Bucket, no volume; volume stays as the file-rooted alternative.Verification
Full
cargo test --workspace --lockedgreen (57 suites); both gated suites green against live RustFS beta.8; the byte-compat suite (98 cluster tests) untouched.🤖 Generated with Claude Code