Skip to content

docs(trace): correct OCSF walkthrough for tool results - #247

Merged
Lrd0036 merged 3 commits into
GenAI-Security-Project:integrationfrom
Lrd0036:lrd/docs-ocsf-walkthrough
Nov 6, 2026
Merged

Lrd0036 merged 3 commits into
GenAI-Security-Project:integrationfrom
Lrd0036:lrd/docs-ocsf-walkthrough

Conversation

@Lrd0036

@Lrd0036 Lrd0036 commented Oct 10, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

Replace the invented OCSF event and unsupported logger API with a Process
Activity example for a command tool's subprocess exiting on its own. Add guards
that check the published JSON against that scenario and the ACS schemas.

Which issue does this implement

Part of #234.

Base branch

  • integration, because this changes documentation and tests.

Type of change

  • Documentation
  • Tooling or CI (documentation guards in tests/)

Changes

Replace the walkthrough's invented agent event and unsupported logger API with
one OCSF 1.9.0 Process Activity example for a completed command tool. The event
models printf exiting on its own: both actor.process and process identify
PID 4243, with the Python runtime recorded as their parent process.

Copy schema-defined ACS fields into unmapped.acs, including tool.name,
request_id, request_id_ref, and UUID session/request identifiers. Include the
result's required outputs field and link directly to the class table and hook.

Add a guard that reads the published JSON, checks the reflexive exit semantics,
and validates its copied ACS metadata and payload against the repository schemas
with UUID format checking. It rejects the old aliases and malformed IDs. The
additional name restrictions apply to this example's claim to copy declared
fields; the wire schemas remain extensible.

Set metadata.correlation_uid to the step's <session_id>:<request_id>, which
the mapping requires on every step event since #275. The guard fails if the field
is dropped, reversed, built from request_id_ref, or uses another separator.

Why

The previous walkthrough used class 3001 as an invented "AI Agent Activity",
seconds timestamps, and the agent identity as an identity provider. Those errors
survived the documentation build and existing guards.

OCSF 1.9.0 follows Bar's October 8 version decision in #121.
#121 is closed, and the mapping and events.md now name OCSF 1.9.0.

I tested this

  • The branch merges the current integration commit, bba1b84
    (checked 2026-11-04).
  • The full local pytest suite passes. It was run as
    uv run --locked pytest -q; -q changes output verbosity.
  • uv run --locked mkdocs build --strict passes on my machine.

Validation

  • uv run --locked pytest -q: 1220 passed, 1 skipped. The local filesystem
    cannot hold both case-differing names for the schema path collision test.
  • uv run --locked mkdocs build --strict: passed.
  • Before correlation_uid was added, the published JSON returned 0 errors
    and 48 warnings
    from the official hosted OCSF v2 validator after its schema
    version was checked as 1.9.0. All warnings concern omitted recommended
    attributes. The hosted server build was not pinned. metadata.correlation_uid
    is a native OCSF 1.9.0 metadata attribute; the validator has not been re-run.
  • Desktop (1024 px) and mobile (375 px) rendered-page checks passed on the new
    head. The correlation_uid line scrolls inside the code block, and the page
    does not scroll horizontally.
  • git diff --check: passed.

The CI guard is scoped to this scenario; it is not a complete OCSF validator.
No live exporter, agent/Guardian execution, or SIEM ingestion/query was exercised.

Risk and security

No material runtime change identified. This changes documentation and its guards.
The example contains illustrative process facts and sample identifiers, with no
attached provenance. It makes no new requirement for arbitrary tools to expose
OS process details.

Architecture and knowledge

Only the walkthrough and its documentation guard change. The normative class
assignments, hook schemas, and envelope schema are unchanged.

Follow-up

This completes the docs/acs.md walkthrough slice. The class, finding_info,
metadata, producer, and SIEM-query repairs in implementation_examples.md
remain part of #234.

#234 suggested using extend_ocsf.md as a base; #235
subsequently found that all three guide examples fail official OCSF validation,
including the tool-call example's missing device and use of a tool as
process. This example therefore identifies the actual exiting subprocess and
host, while leaving guide repairs, stale ACS aliases, and the approved
version/name updates under #235 and #121.

The host-runtime fields illustrate a command tool that really spawns a process;
this does not settle #235's general field-source or class-fit questions for
arbitrary ACS tools. Agent identity remains in unmapped.acs.agent_id while
the mapping decision on the AI Operation profile's native ai_agent field is
pending in #235.

The other walkthrough tabs still illustrate an undefined acs/trace/emit
method. Their protocol examples remain outside this slice.

Checklist

  • Commits are signed off with git commit -s (required by the DCO).
  • Prose follows STYLE.md.
  • No secrets, tokens, or internal URLs in the diff.

Security

  • This change has no security impact.

No runtime code or normative security requirements change; the risk assessment
and the limits of the example are recorded above.

Part of GenAI-Security-Project#234.

Replace the unsupported OCSFLogger snippet and invented event class with
an OCSF 1.9.0 Process Activity event for a subprocess self-exit. Copy
schema-defined ACS fields with UUIDs, result correlation, and outputs so
the walkthrough does not teach an incompatible payload.

Follow the OCSF 1.9.0 version decision in GenAI-Security-Project#121. Add focused guards that
read the published example, check reflexive process semantics and
millisecond timestamps, and validate copied ACS fields against the
existing schemas without changing the wire contract.

Validation on this source snapshot: 911 tests passed, 1 existing
filesystem skip; strict MkDocs build passed. The hosted OCSF validator
reported 0 errors and 48 missing-recommended-attribute warnings. Desktop
and mobile rendering and both section links were checked.

The integration-guide repairs and producer/query examples remain
follow-up work under GenAI-Security-Project#234 and GenAI-Security-Project#235.

Signed-off-by: Lrd0036 <lrd0036@auburn.edu>

@drk-ram drk-ram left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head cffa5a870dd89c5bafbad5c3cb34b46ce4025e76. Approving. The replacement example is accurate OCSF, and it is tied to ACS's own schemas by a test, which the original walkthrough never was.

What I verified

  • Class and activity. I read the OCSF 1.9.0 process_activity class from schema.ocsf.io. Class 1007, category 1, activity_id 2 = Terminate and type_uid 100702 are correct. actor, process and device are required on that class, and the example carries all three. exit_code and time as epoch milliseconds match the 1.9.0 definitions. process.terminated_time and cmd_line exist on the Process object.
  • Mapping. specification/v0.1.0/trace/ocsf-mapping.json maps steps/toolCallResult to class 1007, as the test asserts, so the example follows the project's own mapping.
  • Reflexive exit. For a process that exits on its own, actor.process and process identify the same process. The PR says so explicitly and the guard enforces it.
  • Copied ACS fields. unmapped.acs is checked against the request-envelope.json Metadata definition and the tool-call-result.json payload schema, with UUID format checking. The old aliases (tool.id, execution_id, session_id: "sess_123") are rejected by parametrized tests.
  • Error handling in the guard. declared_objects_only constrains this example's copied fields without changing the extensible wire schemas, and the docstring says so.

Validation I ran on a clean checkout of this head (Windows host)

  • uv run --locked pytest -q tests/test_ocsf_examples.py: 10 passed.
  • uv run --locked mkdocs build --strict: passed.
  • Merge simulation against #271 (92c3128105e3): no conflicts.

I also checked that #234 now carries status:accepted, so the triage caveat in the description no longer applies.

Non-blocking observations

  1. unmapped.acs.outputs copies the tool's raw output ("hello") into the event, and cmd_line copies the full command line. For a toy printf this is harmless. Tool outputs and command lines are where secrets and personal data turn up in practice, and SIEM pipelines retain them far longer than the agent session does. The "No security impact" box is fair for the diff, but the walkthrough is the page implementers copy from. One sentence suggesting that real deployments copy a digest or a reference instead of raw outputs, and apply redaction to cmd_line, would be worthwhile.
  2. The OCSF event does not carry the chain_hash or the ACS signature, so a SIEM analyst cannot tie the event back to the signed, chained ACS step. Carrying the chain head in unmapped.acs would let the trace correlate with the §8.6 commitment. That fits better in #235 or the follow-up than here.
  3. The final hunk also adds a missing trailing newline to docs/acs.md. It is harmless and worth keeping.

The scoping language is good: the guard is not a complete OCSF validator, the hosted validator run is disclosed, and the generic-tool limits are stated.

@bar-capsule bar-capsule left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the class, activity and timestamps check out, and the guard is a real improvement on the old example.

One blocker: since #275, the mapping requires every step event to set metadata.correlation_uid to <session_id>:<request_id> (ocsf-mapping.json step_to_class, and events.md). The example doesn't have it. Please:

  • add "correlation_uid": "7673ad6b-b5b3-4eec-8f46-66c83eef0b52:1287ce09-fcda-4588-a5e9-3fa83bfe5291" to metadata
  • have the guard assert it equals <session_id>:<request_id>, so the test fails if it's dropped
  • merge current integration, so CI runs against today's mapping

Signed-off-by: Lrd0036 <lrd0036@auburn.edu>
Since GenAI-Security-Project#275 the mapping requires every step event to set
metadata.correlation_uid to its own <session_id>:<request_id>, so a SIEM
can join the step to its decision event on native fields. The example
now carries it, and the guard fails if it is dropped, reversed, built
from request_id_ref, or uses another separator.

Signed-off-by: Lrd0036 <lrd0036@auburn.edu>
@Lrd0036

Lrd0036 commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator Author

@bar-capsule thanks. All three are in:

  • metadata.correlation_uid is set to 7673ad6b-b5b3-4eec-8f46-66c83eef0b52:1287ce09-fcda-4588-a5e9-3fa83bfe5291, and a sentence under the example says why it's there.
  • The guard asserts it equals <session_id>:<request_id> from unmapped.acs. Mutation cases show the check fails when the field is dropped, reversed, built from request_id_ref, or uses another separator.
  • Merged integration at bba1b84. 1220 passed, 1 skipped, and mkdocs build --strict is clean.

@drk-ram drk-ram left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed at head 147f7d4f6453011761b657af21a8abaf5dc9353e. Approving. This head resolves Bar's blocker from the 4 November review, and my earlier approval at cffa5a8 stands on the content that has not changed.

What changed since cffa5a8. Two commits: a merge of integration at bba1b84 (the branch now sits exactly on that commit, so CI runs against today's mapping), and 147f7d4, which touches docs/acs.md and tests/test_ocsf_examples.py only.

What I checked

  • metadata.correlation_uid is 7673ad6b-b5b3-4eec-8f46-66c83eef0b52:1287ce09-fcda-4588-a5e9-3fa83bfe5291, which is the example's own unmapped.acs.session_id and unmapped.acs.request_id joined with a colon, in that order. That is the pair ocsf-mapping.json (step_to_class) and events.md require on every step event.
  • The new sentence under the example is accurate. The decision event's unmapped.acs carries request_id, decision, chain_hash and the other response fields, and metadata.correlation_uid is the gated step's pair (events.md, line 115, and join_fields in the mapping), so a SIEM can join the step event to its decision event on one native field.
  • The guard asserts equality with <session_id>:<request_id> taken from unmapped.acs, not a literal copy of the string. I ran the four mutation cases (dropped, reversed, built from request_id_ref, a / separator) and each raises. I also deleted the field from the page text itself: 5 tests fail, so the page cannot lose it silently.
  • This also answers my earlier note 2 (that a SIEM could not tie the step back to its chain commitment). The step event now carries the join key, and the decision event it joins to carries chain_hash, so I have nothing further on it for this PR.

Validation at this head (Windows host)

  • uv run pytest -q tests/test_ocsf_examples.py: 15 passed (it was 10 at cffa5a8).
  • With the trace and doc guards, test_ocsf_examples.py, test_trace_mapping_defaults.py, test_hook_taxonomy.py and test_doc_links.py: 40 passed.
  • Full uv run pytest -q: 1200 passed, 1 skipped, 10 failed, 20 errors. The 30 are in the same five files that fail identically on integration at bba1b84 (test_closing_choice.py, test_fetch_roadmap.py, test_overrides.py, test_pr_intake_workflow.py, test_publish_schemas.py), none of them touched here.
  • uv run --no-dev mkdocs build --strict -d _site/docs: exit 0.
  • Merge simulation (git merge-tree) against #271 (de524c6), #498 (d3966f3) and #496 (034f52a): no conflicts.

One earlier note still stands, unchanged and non-blocking. unmapped.acs.outputs and cmd_line still carry the tool's raw output and the full command line. That is harmless for printf hello, but this is the page implementers copy from, so a sentence suggesting a digest or reference for outputs and redaction for cmd_line would help. It can follow in a later slice.

Approving.

@bar-capsule bar-capsule left a comment •

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. correlation_uid and its guard are in, and the example now joins its decision event on one field.

@drk-ram 's first note is still worth one sentence, in this PR or a follow-up: real deployments should copy a digest or reference instead of raw outputs, and redact cmd_line.

@Lrd0036
Lrd0036 merged commit db457a8 into GenAI-Security-Project:integration Nov 6, 2026
8 checks passed
@github-project-automation github-project-automation Bot moved this from In progress to Done in ACS Project Tracker Nov 6, 2026
@Lrd0036
Lrd0036 deleted the lrd/docs-ocsf-walkthrough branch November 6, 2026 04:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

3 participants