Skip to content

Add "When workflows run" section to Test Engine documentation #3107

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jameshill wants to merge 1 commit into from
Closed

Add "When workflows run" section to Test Engine documentation #3107

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions pages/test_engine/workflows.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,16 @@ Repeated occurrences of the test meeting the alarm/recover conditions do not ret

<%= image "alarm-and-recovery-dark.png", class: 'dark-only', width: 1424 / 2, height: 1280 / 2, alt: "A diagram showing three tests with different transition count events", align: :center %>

## When workflows run

Workflow monitors are **event-driven**, not scheduled. They evaluate each time new test execution data is ingested into Test Engine—there is no cron or periodic schedule involved. When a test run completes and its results are uploaded, the relevant monitors evaluate the incoming data against their configured conditions and thresholds.

This means:

- **Alarm and recover events are generated in response to test executions.** If no tests are running, no workflow events are produced.
- **Threshold changes take effect on the next test execution.** For example, if you adjust a [probabilistic flakiness](/docs/test-engine/workflows/monitors#probabilistic-flakiness) threshold or a [transition count](/docs/test-engine/workflows/monitors#transition-count) alarm value, the updated threshold is applied the next time test data is ingested for a matching test—not at a scheduled interval.
- **The frequency of workflow evaluation matches the frequency of your test runs.** Workflows for a test suite that runs on every commit will evaluate more often than one that runs nightly.

## Rate limit

Each workflow monitor has a rate limit of 500 events per minute across alarm and recover events. If a workflow exceeds this limit within a one minute window, no new alarm or recover events will trigger their configured [actions](/docs/test-engine/workflows/actions) for the remainder of that minute. Event processing resumes in the following minute when usage falls below the limit. To avoid hitting the limit, you can refine your workflow using [tag filters](/docs/test-engine/workflows/monitors#tag-filters) or adjust monitor thresholds.
Expand Down