Skip to content

fix: describe baseline sample scans instead of reporting them as failed - #1249

Merged
crypt0rr merged 1 commit into
mainfrom
fix/baseline-sample-comparison-state
Oct 7, 2026
Merged

crypt0rr merged 1 commit into
mainfrom
fix/baseline-sample-comparison-state

Conversation

@crypt0rr

@crypt0rr crypt0rr commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Summary

Successful baseline-learning scans are no longer described as "did not complete successfully", and a scan's comparison result no longer changes over time (#1224).

Schema 65

  • Adds scans.comparison TEXT NOT NULL DEFAULT '', written when a managed scan is finalized:

    Value Meaning
    compared The job had a baseline when the scan finished, even if no changes were found.
    baseline_sample The job had no baseline yet.
    baseline_established The sample that completed the baseline.
    not_compared A failed, timed-out or canceled scan, or a result kept without a comparison: the security scope changed during the scan, the cycle was discarded or expired, the unit was paused, or notification destinations were unreadable.
    '' A row written before the upgrade (legacy).
  • The column is added only when missing, as schemas 59 and 60 were, and its values are validated in Go. A CHECK constraint is deliberately not used, because changing one later would mean rebuilding the scans table.

  • Backup and restore copy the database file, and a test confirms the value survives both.

API (jobScan and jobScanChanges share one resolver)

Scan comparison_state comparison_source
Not success or incomplete not_compared none
Recorded compared compared, with the stored scan-time changes scan_time
Baseline sample baseline_sample, no changes none
Established the baseline baseline_established none
Recorded not_compared not_compared none
Legacy '' the previous behaviour; only these rows still reach current_baseline_legacy

Scan objects also gain an optional comparison field, left out for legacy rows.

Console copy

  • Baseline sample: "This scan was a baseline sample; no comparison was performed."
  • Established: "This scan established the baseline."
  • Incomplete scan while learning: "…no comparison was performed. Incomplete scans do not count as baseline samples."
  • Kept without a comparison: "This scan was not compared with the baseline."
  • Failed: the existing "did not complete successfully" copy.

Docs

  • The jobs guide gains a "Scan comparison" section, linked from Your first scan.
  • database-compatibility.md gets a schema 65 entry ("introduced in v0.26.0").
  • api-compatibility.md gains a comparison-states table.
  • SECURITY.md is updated.

Compatibility

  • Schema 65. An older binary refuses the upgraded database, so rolling back means restoring the pre-upgrade ./data backup, as for other schema changes.
  • Pre-upgrade samples. Learning samples recorded before the upgrade cannot be told apart from genuine legacy rows. They keep the legacy live diff; only new scans get the stable states.
  • API. New comparison_state values baseline_sample and baseline_established, plus an optional comparison field on scans.

Validation

  • gofmt, go vet ./...
  • go test -race:
    • internal/engine, internal/web, internal/app, internal/model;
    • the migration, schema, scan, restore, backup, retention, tenant, finalize, history, export, legacy and public tests in internal/store.
  • ./scripts/check-schema-docs.sh: schema documentation matches version 65.
  • npm run lint, and npm run test:coverage with its gates.
  • The CI diff-coverage check against main gave 100% for frontend lines and 96.1% for Go (minimum 80%).
  • npm --prefix docs run build
  • After rebasing onto main: Vitest (45 files, 485 tests), and the engine, web, store, app and schema checks again.

New tests:

  • Schema 65 migration test.
  • The value read back through every scan read path, and after backup and restore.
  • Engine test with Samples: 2: sample, established, compared, failed, reset, and an incomplete scan while learning.
  • Web test following the issue's reproduction: establish the baseline, accept an incident, a legacy row, a failed scan, a reset.
  • Resolver table test.
  • Seven console copy variants.

Fixes #1224

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Deploying edgewatch with  Cloudflare Pages  Cloudflare Pages

Latest commit: 5f70149
Status: ✅  Deploy successful!
Preview URL: https://3b86dbe1.edgewatch-cpd.pages.dev
Branch Preview URL: https://fix-baseline-sample-comparis.edgewatch-cpd.pages.dev

View logs

@crypt0rr
crypt0rr force-pushed the fix/baseline-sample-comparison-state branch from 82d0c78 to b49f520 Compare October 7, 2026 17:21
Managed scans record their comparison outcome when they are finalized,
in a new scans.comparison column added by schema 65: compared,
baseline_sample, baseline_established, or not_compared. The job scan
detail and changes endpoints report comparison_state from that outcome,
so a successful scan that ran while the job was learning its baseline
is a baseline sample, the sample that completed the baseline says it
established it, and neither is later diffed against a baseline that
did not exist when it ran. Failed, timed-out and canceled scans, and
results kept without a comparison, are not_compared. Only rows recorded
before schema 65 keep the current_baseline_legacy fallback.

The console describes each state, keeping the failure copy for scans
that did not complete. The job guide explains scan comparison states,
and the API, database compatibility and security notes cover the new
values and schema 65.

Fixes #1224
@crypt0rr
crypt0rr force-pushed the fix/baseline-sample-comparison-state branch from b49f520 to 5f70149 Compare October 7, 2026 17:31
@crypt0rr
crypt0rr merged commit fe1ba34 into main Oct 7, 2026
15 checks passed
@crypt0rr
crypt0rr deleted the fix/baseline-sample-comparison-state branch October 7, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] Successful baseline-learning scans are described as "did not complete successfully"

1 participant