Skip to content

[fm] Guard support bundle creation with SitrepGuardedInsert.#10535

Open
mergeconflict wants to merge 1 commit into
mergeconflict/fm-sitrepguardedinsert-alertsfrom
mergeconflict/fm-sitrepguardedinsert-support-bundles
Open

[fm] Guard support bundle creation with SitrepGuardedInsert.#10535
mergeconflict wants to merge 1 commit into
mergeconflict/fm-sitrepguardedinsert-alertsfrom
mergeconflict/fm-sitrepguardedinsert-support-bundles

Conversation

@mergeconflict

@mergeconflict mergeconflict commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Wire the support bundle resource through SitrepGuardedInsert:

  • impl SitrepGuardedResource for SupportBundle;
  • schema: support_bundle_generation on fm_sitrep and the rendezvous_support_bundle_created marker table (migration fm-bundle-resource-deletion);
  • support_bundle_create's FM path routes through the combinator, surfacing a stale sitrep as Error::Conflict;
  • SitrepBuilder tracks support_bundle_generation, bumping it when the outstanding support-bundle-request set changes; the closed-case carry-forward filter drops fully-satisfied closed cases and keeps those with unsatisfied support bundle requests;
  • fm_rendezvous reads the expected generation and aborts the support bundle loop on a stale mismatch; fm_analysis loads existing markers to drive carry-forward; omdb displays the new status fields and generation.

support_bundle_generation and alert_generation are tracked and guarded independently, so a stale generation for one resource aborts only that resource's rendezvous loop.

Context: #10248, builds on #10564 and #10533. Note, garbage collection for rendezvous_support_bundle_created rows is not in this PR, that should be coming later this week if all goes well.

@mergeconflict mergeconflict self-assigned this Jun 2, 2026
@mergeconflict mergeconflict added the fault-management Everything related to the fault-management initiative (RFD480 and others) label Jun 2, 2026
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from e15c2c1 to e5d67e2 Compare June 2, 2026 19:32
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from 1bc2f60 to 62cc3dd Compare June 2, 2026 19:32
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from e5d67e2 to 809d7d3 Compare June 2, 2026 19:50
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from 62cc3dd to 3d5af68 Compare June 2, 2026 19:50
@AlejandroME AlejandroME added this to the 21 milestone Jun 4, 2026
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from 809d7d3 to 7128ff2 Compare June 4, 2026 20:56
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch 2 times, most recently from a1beef9 to e97748d Compare June 5, 2026 16:38
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from 7128ff2 to 1574845 Compare June 5, 2026 16:38
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from e97748d to f3832d4 Compare June 8, 2026 16:31
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from 1574845 to 4d0f1dd Compare June 8, 2026 16:31
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from f3832d4 to cbc506b Compare June 8, 2026 16:41
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch 2 times, most recently from 85edf6b to a3ab6c2 Compare June 8, 2026 16:52
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch 2 times, most recently from 59680b0 to b6f5bfe Compare June 8, 2026 17:57
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from a3ab6c2 to 26cbbe1 Compare June 8, 2026 17:57
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from b6f5bfe to a7d6f65 Compare June 8, 2026 20:08
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from 26cbbe1 to da05cf5 Compare June 8, 2026 20:08
The support-bundle mirror of the preceding alert change: `impl
SitrepGuardedResource for SupportBundle`, the `support_bundle_generation`
column and `rendezvous_support_bundle_created` marker table (migration
fm-bundle-resource-deletion), `support_bundle_create`'s FM path routed
through the combinator inside its transaction, support-bundle generation
tracking and carry-forward, the fm_rendezvous bundle loop, the fm_analysis
bundle-marker lookup, and the omdb display.

`support_bundle_generation` and `alert_generation` are tracked and guarded
independently, so a stale generation for one resource aborts only that
resource's rendezvous loop.
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-alerts branch from a7d6f65 to aee00ec Compare June 8, 2026 20:12
@mergeconflict mergeconflict force-pushed the mergeconflict/fm-sitrepguardedinsert-support-bundles branch from da05cf5 to 064090e Compare June 8, 2026 20:12
Comment on lines +975 to +976
// need a sitrep present before calling. Insert one at generation 0; the
// FM call below uses the same expected generation.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// need a sitrep present before calling. Insert one at generation 0; the
// FM call below uses the same expected generation.
// need a sitrep present before calling. Insert one at generation 1; the
// FM call below uses the same expected generation.

Comment on lines +318 to +320
} else if !case.alerts_requested.is_empty()
|| !case.support_bundles_requested.is_empty()
{

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this just be an unconditional else ?

Comment thread schema/crdb/dbinit.sql
-- support_bundle_generation on the sitrep being executed.
-- Taken together, these two conditions ensure that no sitrep will ever attempt
-- to resurrect a deleted bundle, meaning the marker is no longer needed.
CREATE TABLE IF NOT EXISTS omicron.public.rendezvous_support_bundle_created (

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Notably we aren't back-filling any tables here based on existing bundles. Is this because we expect no FM-generated bundles to exist when this PR is merged?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

fault-management Everything related to the fault-management initiative (RFD480 and others)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants