Include resolved assessment context for reproducible and comparable reports #148

Motivation

A score for five curated pages and a selected check set is not directly comparable to a full-check score for another product or to a discovery audit. Results also depend on the target environment, sampling strategy, configuration overrides, thresholds, and AFDocs/scoring versions.

The Documentation at Scale guide tells teams to retain this context manually. Audit what reports already retain, then fill the gaps so consumers can understand and reproduce the intended assessment without collecting unrelated machine details.

Design Scope

• Define a versioned assessment-context schema using existing report/config types where possible. Consider original/effective target and product scope, resolved checks and exclusions, sampling strategy, requested/actual pages and tags, relevant options/thresholds, version information, and network/environment classification.
• Distinguish requested selection from actual tested evidence, including replacements, skipped pages, and discovery/coverage completeness from Expose structured discovery completeness and fallback evidence in reports #141.
• Record effective values after CLI/config resolution. Do not dump raw configuration, credentials, process environments, or local absolute paths into shareable reports.
• Define redaction, potentially sensitive URL handling, size bounds, and any stable fingerprints used for comparisons. Explain what can and cannot be reproduced when source data, site content, or random sampling changes.
• Coordinate with the pre-run summary in Show resolved page selection and check request scopes before a run #143 while keeping intended work distinct from actual observations.

Acceptance Criteria

• Consumers can distinguish curated, discovered, fallback, and partial assessments and explain material differences in selected checks, scope, options, and versions.
• CLI and programmatic JSON reports expose consistent context with a documented backward-compatible schema strategy.
• Tests cover CLI overrides, skipped/excluded checks, tagged and untagged pages, deterministic versus random selection, canonical/preview targets, and incomplete discovery.
• Sensitive fields are explicitly reviewed and redacted or omitted; existing safe network-context semantics are preserved.
• Retained context or fingerprints have documented ordering/stability behavior and are bounded for large curated lists.
• Documentation shows a like-for-like comparison and explains why tags are grouping labels, not stratified or statistically representative sampling.
• No implicit score-formula change, automatic host-wide averaging, or cross-product dashboard is included. Define the meaning of aggregation before implementing it later.

References

#141 and #143 provide complementary evidence and planning context. Related: #25, #23, #73, #104, #120 and PR #140. Design notes: working-notes/page-discovery-notes.md, Documentation at scale section.