docs(influxdb3): document object store consistency requirements for MinIO#7442
Conversation
…inIO Add an "Object store requirements" section to the shared MinIO setup guide covering the read-after-write and conditional PUT semantics the catalog depends on. For Enterprise, add a firm warning against NFS-backed MinIO and against multiple independent MinIO instances sharing a single backing volume. For Core, add a lighter note recommending locally-attached POSIX storage. Also document `influxdb3 debug object-store-check` (introduced in 3.10.0) as the way to validate a MinIO deployment against these requirements.
Vale Style Check Results
Warnings (4)
✅ Check passed |
github-actions
Bot
added
product:shared
|
There was a problem hiding this comment.
Pull request overview
This PR updates the shared InfluxDB 3 MinIO object-storage setup guide to document the object-store consistency and conditional-write semantics that the catalog depends on, and to provide product-specific operational guidance for Core vs Enterprise deployments.
Changes:
- Adds an “Object store requirements” section describing required read-after-write, list-after-write, and conditional PUT semantics.
- Adds product-scoped callouts: an Enterprise warning about unsupported MinIO topologies and a Core note with lighter deployment guidance.
- Documents
influxdb3 debug object-store-check(3.10.0+) as a way to validate a MinIO deployment against the requirements.
| influxdb3 debug object-store-check \ | ||
| --object-store s3 \ | ||
| --bucket BUCKET_NAME \ | ||
| --aws-endpoint MINIO_URL \ | ||
| --aws-access-key-id AWS_ACCESS_KEY_ID \ | ||
| --aws-secret-access-key AWS_SECRET_ACCESS_KEY \ | ||
| --aws-default-region AWS_DEFAULT_REGION \ | ||
| --check-prefix oscheck |
There was a problem hiding this comment.
Good catch — aligned the example with the canonical MinIO convention used elsewhere on this page: MINIO_USERNAME/MINIO_PASSWORD, http://localhost:9000, and --aws-allow-http. Also dropped --aws-default-region since the existing influxdb3 serve MinIO example doesn't set it. Fixed in facd90b.
|
@lilic @hiltontj — question on scope for this PR and follow-up docs: I compared the requirements described in this PR against the checks in
For this setup page, keeping the requirements at a high level and pointing at the tool feels right. But there's a bigger question: Do we want a separate, in-depth "InfluxDB 3 object store requirements" reference page in docs-v2 that enumerates the semantics we depend on, or do we want the Arguments for a dedicated docs page:
Arguments for keeping the tool as the source of truth:
My weak preference: tool as source of truth, plus a short reference page that says "these are the classes of semantics we test; run the tool for the authoritative enumeration." No specific check IDs in the docs. Curious what you both think. |
Per Copilot review: use MINIO_USERNAME/MINIO_PASSWORD, http://localhost:9000, and --aws-allow-http to match the canonical serve example on the same page. Drop --aws-default-region for the same reason.
Applies the InfluxDataDocs.Branding substitution across the shared MinIO guide and the two consumer stubs — both in new prose from this PR and in pre-existing lines that Vale flagged.
The three pagination/error semantics tested by object-store-check (HEAD-missing NotFound, strict start-after, multi-page listing) are not current concerns for vanilla MinIO. Historical MinIO bugs in these areas were confined to the deprecated S3 gateway and are fixed. Keep the pagination/error checks in mind for a future backend-neutral requirements reference, but do not imply on this MinIO setup page that MinIO is likely to violate them.
|
Reverted the "Strict pagination and error semantics" bullet in 3c3a89a. Rationale after checking each of the three checks against MinIO's actual bug history:
So enumerating these on the MinIO setup page implied a risk that doesn't apply to MinIO specifically — it was really just reflecting what the The three bullets that remain (read-after-write consistency, list-after-write consistency, conditional PUT) do have real MinIO-specific risk paths (NFS backend, shared-mount topology, catalog split-brain), so they stay. This reinforces the question I asked above about a backend-neutral requirements page — those three pagination/error checks are still valuable ops knowledge, they just don't belong on a MinIO-specific page. |
GitHub URLs of the form <repo>?tab=<name>#<anchor> resolve via client-side hydration; the anchor is not present in the initial HTML the fragment checker fetches. This produces a false positive on any docs page that cites a GitHub URL copied from the tab bar (readme-ov-file, packages, issues, etc.), while the link is fully functional in a browser. The exclusion is scoped to <org>/<repo>?tab=X#Y so it does not affect GitHub URLs where the fragment is real (blob/main/file.md#L42, etc.).
My first thought before seeing your comments was "we need a neutral object store requirements page". I'll take a stab at it and get your review, but won't hold up your PR for it. |
|
Filed #7444 to track two follow-ups from this PR: moving the general object-store consistency requirements to the shared object-storage overview page, and dropping the Vale rule that partially capitalizes "object store." No change requested here for that — just linking for visibility. (Style-only fixes — capitalization, hyphenation, em dash, wording — are split out separately in #7445, based on this branch, so they don't mix with the technical review.) |
Style-only cleanup for #7442's new "Object store requirements" content: - Revert "the Object store" to sentence case ("the object store") throughout; partial capitalization of a generic term doesn't disambiguate anything and is tracked for removal in #7444 - Fix Google.LyHyphens: "locally-attached" -> "locally attached" - Fix Google.EmDash: remove spaces around the em dash - Avoid "above" as an in-page reference; use "preceding" instead No content changes -- SME review of #7442's technical claims is unaffected.
There was a problem hiding this comment.
Only thing maybe we can callout a small warning saying this tool should be used with caution as it does do requests against the given object store so they should always use with caution.
or do we want the object-store-check tool (and its --help-all) to be the authoritative source?
this would be my preference, as the original PR just added some initial checks but as new requirements come in we might need other things.
Co-authored-by: Lili Cosic <cosiclili@gmail.com>
jstirnaman
left a comment
jstirnaman
left a comment
There was a problem hiding this comment.
Addressing lilic's review-body nit about a caution callout on object-store-check — left as a suggestion inline. Her other suggestion (line wording) is already applied in f166871.
Generated by Claude Code
| validates an object store against the preceding semantic requirements. | ||
| Run it against your MinIO endpoint before putting the deployment into | ||
| production, and again after any change to the MinIO topology or backing | ||
| storage: |
There was a problem hiding this comment.
Per @lilic's comment above about calling out that this tool should be used with caution — suggested addition before the code block:
| storage: | |
| storage: | |
| > [!Caution] | |
| > `object-store-check` sends live read, write, and delete requests to the | |
| > object store you point it at. The synthetic checks are isolated to a | |
| > disposable location (`<check-prefix>/oscheck-<uuid>/`) and don't touch your | |
| > real catalog data, but every request still counts against the target | |
| > endpoint's real request quota. Use credentials scoped to a non-production | |
| > bucket or prefix where possible. |
There was a problem hiding this comment.
Not sure if this needs caution as opposed to note as I'm assuming it's a generally one-time thing, but not something I'm concerned on if you think caution makes sense. Otherwise lgtm
There was a problem hiding this comment.
No opinion on the caution label, but I like warning about what the tool is going to do (though if the tools bakes in the warnings we might not need it here). I agree that usually this is a one-time check because the infra is often a "set it and forget it" thing. However, the recommendation is cautious and hints at the exceptional cases (though they aren't named):
- we might add additional qualifications and checks to the utility at a later point
- it's possible for changes to the object store configuration to break compatibility
Style-only cleanup for influxdata#7442's new "Object store requirements" content: - Revert "the Object store" to sentence case ("the object store") throughout; partial capitalization of a generic term doesn't disambiguate anything and is tracked for removal in influxdata#7444 - Fix Google.LyHyphens: "locally-attached" -> "locally attached" - Fix Google.EmDash: remove spaces around the em dash - Avoid "above" as an in-page reference; use "preceding" instead No content changes -- SME review of influxdata#7442's technical claims is unaffected.
Summary
> [!Warning]block calling out two topologies that violate those semantics and are unsupported for production:> [!Note]recommending locally-attached POSIX storage and pointing at MinIO's own deployment guidance.influxdb3 debug object-store-check(introduced in 3.10.0, PR influxdb_pro#3532) as the way to validate a MinIO deployment against the requirements.Why
Investigating a recent customer incident on Enterprise 3.9.2 surfaced that the customer's MinIO setup — two MinIO instances backed by shared NFS — is exactly the topology MinIO itself warns against, and it is a plausible contributor to catalog-related panics and stale-view startup warnings we observed. Nothing in the public docs currently tells operators what object-store semantics {{% product-name %}} depends on or that certain MinIO topologies violate them.
The
object-store-checksubcommand was built to diagnose exactly this class of issue but has no public docs reference. This PR is the first public mention.Reviewer note
Adding @lilic as reviewer since she authored the
object-store-checksubcommand and can verify the flag usage and requirement wording are accurate.Test plan
show-inconditional rendering displays the Enterprise warning only on the Enterprise MinIO page and the Core note only on the Core MinIO page> [!Warning],> [!Note]) renders as intended