Skip to content

Widen SCM service-deletion wait budgets in NSIS stress tests - #70239

Open
twangboy wants to merge 1 commit into
saltstack:3006.xfrom
twangboy:fix_nsis_2
Open

Widen SCM service-deletion wait budgets in NSIS stress tests#70239
twangboy wants to merge 1 commit into
saltstack:3006.xfrom
twangboy:fix_nsis_2

Conversation

@twangboy

@twangboy twangboy commented Sep 4, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

CreateService for salt-minion can race the SCM's async cleanup after a prior uninstall's SimpleSC::RemoveService, since the SCM doesn't purge the service registry key until every handle is closed. The existing 15s wait / 20s CreateService retry budget was too tight: PR #70238 stress run 33911838747 (iteration 88) showed this window climbing past 35s under load before exhausting the retries and aborting.

  • Raise the install-side pending-deletion wait and the uninstall-side post-removal wait from 15s to 60s (30->120 polls at 500ms) in Salt-Minion-Setup.nsi.
  • Raise the CreateService retry loop from 10 to 20 attempts (2s apart, 40s total).
  • Raise SCM_WAIT_SECS in conftest.py's clean_env() from 15s to 60s to match.
  • Log elapsed wait time on the success path (not just on timeout) in all three wait loops, so future stress-test logs show the SCM latency trend across iterations instead of only surfacing it once a run blows the budget and fails.

Merge requirements satisfied?

[NOTICE] Bug fixes or features added to Salt require tests.

Commits signed with GPG?

Yes

CreateService for salt-minion can race the SCM's async cleanup after
a prior uninstall's SimpleSC::RemoveService, since the SCM doesn't
purge the service registry key until every handle is closed. The
existing 15s wait / 20s CreateService retry budget was too tight:
PR saltstack#70238 stress run 33911838747 (iteration 88) showed this window
climbing past 35s under load before exhausting the retries and
aborting.

- Raise the install-side pending-deletion wait and the
  uninstall-side post-removal wait from 15s to 60s (30->120 polls
  at 500ms) in Salt-Minion-Setup.nsi.
- Raise the CreateService retry loop from 10 to 20 attempts (2s
  apart, 40s total).
- Raise SCM_WAIT_SECS in conftest.py's clean_env() from 15s to 60s
  to match.
- Log elapsed wait time on the success path (not just on timeout)
  in all three wait loops, so future stress-test logs show the SCM
  latency trend across iterations instead of only surfacing it once
  a run blows the budget and fails.
@twangboy twangboy added this to the Sulphur v3006.28 milestone Sep 4, 2026
@twangboy twangboy self-assigned this Sep 4, 2026
@twangboy
twangboy requested a review from a team as a code owner September 4, 2026 20:40
@twangboy twangboy added the test:full Run the full test suite label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant