Enable ADM validation across regression tests + fix validation logging#479
Merged
Conversation
…logs The winston logger was never configured with format.splat(), so printf-style %s placeholders were never interpolated -- 'Bad request: %s' logged literally and the payload (including ADM validation details) was dropped. Add splat() to a shared base format applied in both format modes. Also collapse requestValidation's two log lines (a bare message plus a raw JSON.stringify(err) blob) into a single labeled 'Request failed validation: %s' line, consistent with the other error handlers.
Small single-object/data-driven test fixtures were inconsistently split into separate .json files and loaded via readJson/require. Inline them so the payload under test is visible at rest in the spec, matching the dominant convention (~55 specs already inline their fixtures). Large STIX collection bundles remain external. - groups.query.json -> baseGroup const in groups.query.spec.js - teams.invalid.json -> inline array in teams-invalid.spec.js - user-accounts.invalid.json -> inline array in user-accounts-invalid.spec.js - delete teams.valid.json and user-accounts.valid.json (orphaned, no consumers)
Enable config.validateRequests.withAttackDataModel across all six technique specs and make the seeded payloads ADM-compliant (valid kill_chain_name and x_mitre_platforms enums; drop non-compliant, non-asserted fields such as x_mitre_impact_type and x_mitre_data_sources). - techniques.spec.js / .query / .revoke / -pagination: ADM-compliant fixtures - techniques.convert: flag flip only (fixture already compliant) - techniques.tactics: flag flip; bundle is imported via the fidelity-tolerant import path, which records per-object ADM issues rather than rejecting them - techniques.query.json inlined into the spec (single-object fixture) - pagination: add a validateWithAdm option so the suite pins ADM state itself instead of inheriting the shared config singleton from a prior spec
seansica
changed the title
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## next #479 +/- ##
==========================================
+ Coverage 68.87% 69.15% +0.27%
==========================================
Files 220 220
Lines 30829 30832 +3
Branches 2665 2676 +11
==========================================
+ Hits 21234 21321 +87
+ Misses 9557 9474 -83
+ Partials 38 37 -1 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Enable config.validateRequests.withAttackDataModel across the analytics specs and make the seeded payloads ADM-compliant: - x_mitre_platforms 'windows' -> 'Windows' (enum is case-sensitive) - drop 'description' from the detection-strategy fixture (not in its ADM schema) - replace placeholder non-UUID data-component STIX ids with valid type--uuidv4 values (kept consistent across reference and assertion); the non-existent-ref test uses a valid-format-but-absent UUID so ADM (400) does not preempt the intended 404 - omit an empty x_mitre_log_source_references array (schema requires it to be non-empty when present) - analytics-pagination: pin ADM via the validateWithAdm option
Enable config.validateRequests.withAttackDataModel and replace placeholder sector strings with valid enum values (x_mitre_sectors and x_mitre_related_assets[].related_asset_sectors must be one of Electric, Water and Wastewater, Manufacturing, Rail, Maritime, General).
Flag flip only; the campaign and marking-definition fixtures were already ADM-compliant.
- add required x_mitre_version to the x-mitre-collection fixture (every ATT&CK domain object requires it; the full collection schema rejected its absence) - bundled malware fixture x_mitre_platforms 'platform-1' -> 'Android'
… enabled Flag flip plus the pagination validateWithAdm option; fixtures were already ADM-compliant (work-in-progress / partial schema).
- x_mitre_collection_layers placeholders ['duis','laboris'] -> ['Host','Network'] (must be from the supported collection-layers enum) - data-sources-pagination: pin ADM via the validateWithAdm option
- replace invalid-v4 analytic STIX ids (version nibble must be 4) with valid UUIDv4 values, kept consistent across the analytic definitions and the detection strategy's x_mitre_analytic_refs - x_mitre_platforms 'windows' -> 'Windows' on the seeded analytics - detection-strategies-pagination: pin ADM via the validateWithAdm option
Enable ADM request validation in the groups CRUD, query, and input-validation specs. Pin the pagination harness to ADM validation and add enterprise-attack domains to the full-schema query fixture.
Enable ADM request validation in the identities CRUD spec. Add an external reference to the seeded identity so full-schema update validation passes.
…lidation enabled Enable ADM request validation in the marking-definitions CRUD spec. No fixture changes were required; the existing WIP marking-definition payload is ADM-compliant.
Enable ADM request validation in the matrices CRUD spec. Use enterprise-attack as the seeded matrix domain so the server-composed matrix external reference is ADM-compliant.
Enable ADM request validation in the mitigations CRUD spec. Pin the pagination harness to ADM validation; no fixture field changes were required.
Enable ADM request validation in the notes CRUD spec. No fixture changes were required; note objects do not currently have an ADM schema wired into request validation.
… enabled Enable ADM request validation in the recent-activity API spec. No request fixture changes were required; the suite seeds existing STIX bundle fixtures through the collection bundle importer.
Enable ADM request validation in the references API spec for consistency. No fixture changes were required; references are system metadata records rather than STIX object request payloads.
…bled Enable ADM request validation in the relationships CRUD and pagination specs. Pass validateWithAdm to the pagination harness. Remove the pagination helper's generated name from relationship fixtures because ADM does not allow name on relationship objects. Pin OpenAPI validation in the pagination spec so recursive runs initialize route validation consistently.
Enable ADM request validation in the reports API spec. No fixture changes were required; the WIP software and relationship setup payloads are ADM-compliant.
Enable ADM request validation in the session API spec for consistency. No fixture changes were required; session requests are not STIX object payloads.
Enable ADM request validation in the software CRUD and pagination specs. Pass validateWithAdm to the pagination harness. Replace the synthetic platform fixture with Android so ADM platform validation passes. Pin OpenAPI validation in the pagination spec so recursive runs initialize route validation consistently.
Enable ADM request validation in both stix-bundles specs. Add valid v4 fixture IDs and required ADM metadata for the new-spec bundle. Normalize the legacy bundle's placeholder ATT&CK IDs, enum values, aliases, citations, and required version/modifier fields.
…validation enabled Enable ADM request validation in both system-configuration specs. Use a valid ATT&CK tactic external reference in the organization-identity fixture.
Enable ADM request validation in the tactics CRUD and tactics-techniques specs. Use valid ATT&CK tactic and technique external IDs, tactic shortnames, technique phase names, and subtechnique flags in the fixtures. Remove the unsupported marking-definition domain field from the bundle fixture.
Enable ADM request validation in the teams specs for consistency. No fixture changes were required; teams use non-STIX payloads.
…bled Enable ADM request validation in the user-accounts specs for consistency. No fixture changes were required; user-accounts use non-STIX payloads.
…nabled Enable ADM request validation in the attack-objects API spec and pin ADM validation in pagination. Normalize attack-object import fixtures to use valid ATT&CK IDs, required technique data sources, subtechnique flags, full-schema metadata, and valid software platforms. Remove unsupported marking-definition domains from the import bundles.
…dation enabled Enable ADM request validation in the collection-bundles basic and streaming specs. Normalize reusable bundle fixtures with valid ATT&CK external references, domains, platforms, data source formats, collection metadata, and group alias ordering. Update import error-count assertions for the intentionally malformed missing-spec-version fixture now that ADM records that validation error.
…dation enabled Enable ADM request validation in the collection-indexes spec for consistency. No fixture changes were required; collection indexes use non-STIX payloads.
|
🎉 This PR is included in version 4.17.1 🎉 The release is available on:
Your semantic-release bot 📦🚀 |
|
🎉 This PR is included in version 4.17.1 🎉 The release is available on GitHub release Your semantic-release bot 📦🚀 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Migrates the regression test suites to run with ATT&CK Data Model (ADM) request validation enabled, and fixes the server-side logging that made ADM validation failures hard to read. Also a small test-fixture consistency cleanup.
Why
ADM validation was integrated into the REST API but the regression tests were written against looser, pre-ADM payloads, so they ran with
validateRequests.withAttackDataModel = false. This left the strict validation path untested. When ADM failures did occur, the logs were unreadable.