Skip to content

PENTEST.md

Latest commit

 

History

History
788 lines (656 loc) · 42.7 KB

PENTEST.md

File metadata and controls

788 lines (656 loc) · 42.7 KB
Raw

NullByte — security capability layer

An agentic pentesting toolset bolted onto the existing NullByte CLI (a patched single-file binary). It is an extension, not a fork: your agents, providers, models, TUI, sessions, permission model and config are untouched.

  • 14 sec_* tools available to every agent, including void
  • One plugin entry that registers them, enforces policy at the tool boundary and injects the shared methodology
  • Findings, evidence, audit trail and reports persisted per project
  • 365 unit tests

1. Layout

~/.config/nullbyte/
  plugin/nullbyte-security.ts        entry point (auto-discovered)
  security-lib/                      implementation
    types.ts config.ts paths.ts scope.ts netsafe.ts redact.ts audit.ts brand.ts
    cmdsafe.ts toolguard.ts cvss31.ts findings.ts evidence.ts report.ts
    coverage.ts threat-model.ts target-dir.ts report-selection.ts
    methodology.ts workflow.ts sast.ts ratelimit.ts progress.ts agents.ts
    net/{dns,http,tls,headers,fingerprint,ports}.ts
    adapters/{registry,scanners}.ts
    tools/{scope,agents,recon,ports,scan,sast,evidence,findings,coverage,
           threat-model,report,knowledge,index,shared}.ts
    __tests__/*.test.ts
<project>/.nullbyte/security/        engagement state (per project, shared by all targets)
  scope.json  findings.json  coverage.json  threat-models.json
  audit.jsonl  evidence/  reports/
<project>/<host>/                    per-target output (created on first contact)
  reports/  logs/  .gitignore

Code lives in the config dir; engagement state lives with the project so an assessment is reviewable and resumable next to the code it concerns. NULLBYTE_SECURITY_DIR overrides the engagement location.

Per-target directories

Contacting a target creates <project>/<host>/ — reports/ for deliverables and logs/ for raw scanner output. It is created at the single point where authorization succeeds (enforce/enforcePorts in tools/shared.ts), which is what makes the two properties hold:

  • A denied target creates nothing. The directory is a side effect of authorization, never of the attempt. Verified against allow-list mode.
  • One host, one directory. app.acme.test, app.acme.test:8443, APP.ACME.TEST and https://app.acme.test/x all resolve to app.acme.test/, so revisiting a target reuses its directory instead of starting a second one. Ports do not split it; a CIDR is one directory for the range.

targetSlug() is a security boundary, not cosmetics — the host string arrives from user input and from tool output and is then used to build a path. Everything outside [a-z0-9._-] is replaced, .. runs collapse, and leading dots are stripped, so ../../etc/passwd becomes etc_passwd and cannot escape the project root. targetDir() re-asserts containment afterwards, so a future edit to the slug rules cannot silently introduce a traversal. Long hosts are truncated to 100 chars plus a digest, so two long hosts sharing a prefix cannot collide.

Only deliverables are sharded. findings.json, evidence/ and audit.jsonl stay engagement-wide: sharding the store per target would break the closure gate, semantic dedupe and cross-session continuity, and would make coverage=engagement meaningless.

Each directory writes a .gitignore containing *, so a run inside a repo cannot commit raw tool output — which routinely contains credentials. Disable the whole feature with "plugins": { "target_dirs": false } in the global policy; ensureTarget then returns undefined and callers fall back to the flat engagement layout.

Encryption at rest

Off by default; enabled by configuring a key. The key comes from NULLBYTE_STORE_KEY or ~/.config/nullbyte/security.key, never from the policy file — a passphrase in a committed config is not encryption.
Algorithm AES-256-GCM, 96-bit random IV, authenticated
KDF scrypt, N=2¹⁵, r=8, p=1, 16-byte per-value salt
Envelope {"$":"nullbyte-encrypted","v":1,"alg","kdf","N","r","p","salt","iv","tag","ct"}
Sealed findings.json, evidence artifacts, the evidence index
Sealed per line audit.jsonl — so it stays appendable and the hash chain, which covers the plaintext, still verifies
Not sealed scope.json, reports and target logs — the deliverables are the point of the engagement

Reads accept plaintext and envelopes, in the same store, which is what makes enabling encryption non-destructive: a half-migrated engagement reads correctly and old records are sealed as they are next re-written.

Two failure modes are distinct on purpose:

  • Encrypted, no key → StoreKeyError naming the key sources and stating that there is no recovery. It is not reported as corruption, because a corruption message invites a "repair" that would destroy the data.
  • Encrypted, wrong key or tampered → GCM tag failure → EnvelopeCorruptError. Never a garbage plaintext.

plugins.encrypt_store: true makes encryption required: with no key configured seal() throws and the write does not happen, so there is no path where the operator believes they are encrypted and is not.

Evidence digests cover the plaintext payload. With encryption off that is exactly the bytes on disk (see the store-integrity tests); with it on, the digest survives a re-encrypt or key rotation, which hashing the ciphertext would not — a fresh IV would make every stored artifact look tampered with.

Threat model: data at rest — a lost laptop, a copied directory, a synced backup. It does not defend against a process running as the same user, and cannot, since the key must be readable for the tools to function.

Pacing

Concurrency inside one scan was always bounded; the engagement as a whole was not. limits in the scope file bounds it, because how much traffic is acceptable is an engagement decision and not a global one.

Key Bounds Unit
globalConcurrency operations in flight anywhere in the process whole process
perHostConcurrency operations in flight against one host host
perHostIntervalMs minimum spacing between two operations against a host host
perHostRps requests per second against a host, sliding one-second window host

Both budgets are process-wide and keyed by the host as the scope names it, so a port scan and a concurrent recon on the same host share one allowance instead of each politely limiting itself. Scanners take an additional exclusive slot per host: two long-running scanners against one target is never the intent, and that is the case an IPS is most likely to act on. Different hosts remain parallel.

Omitted, the defaults are concurrency 64 and no delay — byte-for-byte the behaviour before pacing existed, so an existing engagement is unaffected. The block is partial: a project can set perHostRps alone and inherit the rest from the operator's global policy. Values are clamped (NaN or a non-number falls back to the default; a finite out-of-range value clamps, so perHostConcurrency: -5 means 1, not unbounded); concurrency never exceeds 256 and intervals never exceed 60s.

Pacing is reported where it applies: sec_scope action=show always states the effective limits, and sec_ports / sec_recon add a pacing: line only when the limits are not the defaults, so a default engagement carries no line about limits nobody set.

Live progress is deliberately terminal-only. The spinner writes carriage returns to stderr, which chat/TUI harnesses capture into the conversation; there it fights the input loop and makes the prompt feel locked while a scan runs. The progress line is therefore auto-suppressed when a known chat harness is detected (OPENCODE, CLAUDECODE, CURSOR_TRACE_ID, COPILOT_AGENT). Force it on with NULLBYTE_PROGRESS=1; force it off with NULLBYTE_PROGRESS=0 or NULLBYTE_NO_PROGRESS=1. Tool metadata still reports start/finish state, without the animated stderr stream.

Honest scope: the budget is per process. Two agents or two concurrent tool calls in separate processes against the same host each get their own allowance.

Interface branding

The runtime still depends on opencode package names, provider ids, environment variables and config keys; those stay functional. Nothing that reaches the operator is allowed to display that name: the tool.execute.after hook rewrites tool titles, returned text and metadata to NullByte, and permission prompts are scrubbed before they are shown. The installed CLI is rebranded separately by tools/rebrand/reapply.sh; the plugin cannot change the host's own banner or window title. Stored evidence and audit records remain byte-faithful — branding is a display boundary, not an evidence rewrite.

2. Posture: authorized by default

There is no scope ceremony. Scan targets are permitted at every intensity because the operator is the authorizer. Nothing needs to be declared before the agent starts working. sec_recon, sec_ports and sec_scan ignore any scope allow-list and never pause for an approval prompt.

Three guards stay on because they protect your machine or the local network, not the target:

Guard Default Why Turn off
block_destructive on refuses rm -rf /, mkfs, dd to a raw device "block_destructive": false
local credential protection on refuses to read ~/.ssh/id_*, ~/.aws/credentials, auth.json, browser cookie DBs plugins.protect_local_secrets: false
metadata/link-local refusal always on never scans 169.254.169.254, metadata hostnames, link-local, multicast or reserved addresses not configurable

Credential disclosure posture

Setting Default Effect Opt out
capture (sec_evidence, sec_scan, sec_recon) verbatim secret-like material is stored as captured include_secrets=false
report (sec_report) verbatim every text format emitted unmasked, with a Credentials section redact=true

This is a deliberate departure from PTES and OWASP APTS — see README.md#credentials-and-disclosure.

Locking down shell and retest (opt-in)

Scanning ignores the allow-list. The list still gates bash, webfetch and sec_retest, so it is useful for constraining command-driven work. Create <project>/.nullbyte/security/scope.json:

{
  "engagement": "acme-q4",
  "authorized_all": false,
  "intensity": "safe-active",
  "allow_private": false,
  "block_destructive": true,
  "include": [
    { "host": "app.acme.test", "ports": [443, 8443] },
    { "host": "*.acme.test" },
    { "host": "10.20.0.0/24" }
  ],
  "exclude": ["billing.acme.test"],
  "limits": {
    "globalConcurrency": 8,
    "perHostConcurrency": 4,
    "perHostIntervalMs": 250,
    "perHostRps": 4
  }
}

limits is optional and shown here because a target with a border is usually also a target that should not be flooded; omit it for the defaults (see Pacing).

With authorized_all: false, bash, webfetch and sec_retest enforce the list: out-of-scope targets, excluded hosts, ports outside an entry's allowance and operations above the intensity ceiling are refused, and a scoped name that resolves to metadata or private space is refused after DNS re-validation. sec_recon, sec_ports and sec_scan deliberately bypass this list; they are authorized by default. Each still declares an intensity floor — recon is passive, ports and ordinary scanners are safe-active, nmap intrusive and sqlmap are intrusive — and a registry test keeps those floors from drifting.

Grammar: host, *.domain (subdomains only), domain (domain + subdomains), IP, CIDR, URL. exclude always wins. Scope is operator-authored — the agent cannot create or widen it unless you set allow_model_scope_edit: true.

Machine-wide defaults live in ~/.config/nullbyte/security.json (same schema, plus a plugins block):

{
  "plugins": {
    "inject_workflow": true,
    "protect_local_secrets": true,
    "inject_permission_rules": true,
    "ask_intrusive_commands": false
  }
}

3. Tools

Tool Purpose
sec_scope authorization mode; check a single target; init writes a starter scope file
sec_agents plan and run 1-8 child agents; parallel stages, depends_on, timeout/cancel, returns each output and session id; children may ask the operator when a question materially changes the work
sec_recon check=dns|http|tls|headers|fingerprint — real network calls, manual redirect validation; archives the raw HTTP response as evidence (archive=false opts out); every call paced per host
sec_ports bounded TCP connect scan + banner grab (concurrency, timeouts, port cap); each probe paced per host
sec_scan external adapters: nmap, nuclei, nikto, whatweb, sslscan, sqlmap (argv-only, availability-checked); paced per host and exclusive, so two scanners never hit one host at once; logs=true writes the raw stdout/stderr into the target's logs/
sec_sast local: secrets (reported by location, values never printed), smells, deps
sec_evidence store/retrieve raw artifacts, SHA-256 indexed and re-checked on read (action=verify audits the whole store, action=stats reports retention); stored verbatim by default, include_secrets=false redacts at capture
sec_finding create/list/get/update findings; status derived (never asserted); closure gate; CVSS 3.1 severity; semantic dedupe; evidence-reference checks
sec_coverage record the tested surface × risk area and outcome (reported, no_issue_found, ruled_out, not_applicable, needs_follow_up); rejects duplicates; requires evidence for non-reported outcomes; summary exposes findings with no linked coverage
sec_threat_model save/amend the shared per-target threat model (actors, assets, trust boundaries, hypotheses); amendments are append-only; read by every agent on the target
sec_retest re-check a finding's reproduction: run re-executes a command under scope enforcement and captures the output, record logs a manual check; append-only history; a contradicting retest reopens the finding
sec_report preview or export as markdown, JSON, HTML, PDF, SARIF or CSV; writes into the target's own reports/ by default; coverage=session|engagement|target selects which findings and ledger entries are included; embeds the threat model and coverage section; writes a *.coverage.json sidecar; evidence=full embeds every artifact's content; verbatim by default, redact=true scrubs
sec_methodology search/load the shared knowledge fragments and skill playbooks (progressive disclosure, budgeted load)
sec_audit review every allow/deny/ask decision the tool layer made; action=verify recomputes the trail's hash chain to prove it was not edited; action=reconcile reports findings with no creation allow record

bash, read, write, edit, webfetch and todowrite are reused as-is — nothing is duplicated.

Knowledge base (sec_methodology)

39 fragments, addressed by category-qualified id (process/…, vulnerability/…, analysis/…, playbook/…, coding/…). Process: scope-and-authorization, recon-dns, recon-web, port-discovery, content-discovery, vulnerability-assessment, verification-discipline, evidence-capture, reporting, untrusted-content. Vulnerability classes: xss, sqli, ssrf, idor, auth-session, access-control, path-traversal, command-injection, file-upload, api-testing, insecure-deserialization, subdomain-takeover, ssti, open-redirect. Security playbooks: ctf-sprint, agentic-assessment, secure-code-review, vulnerability-remediation, ci-security-gate, application-assessment. Coding skills: feature-delivery, debugging, test-strategy, refactoring, performance, dependency-upgrade, code-review, api-contract.

Three actions, so a model pays only for what it needs:

  • search — metadata only (id, category, one-line summary, aliases). Expands shorthands (xss, sqli, rce, lfi, bola, …) and matches on word boundaries, so rce does not match the "fo(rce)d" inside "forced-browsing". Returns the matched category and a limit (default 8, max 20).
  • load — returns full fragment text. Budgeted: at most 5 fragments and 96 KiB per call. An unknown or ambiguous id is refused with the near-miss candidates rather than silently returning nothing.
  • list — category counts.

Loaded text is wrapped in <nullbyte_methodology> with any reserved boundary marker escaped, and carries an explicit non-grant notice: the fragments inform reasoning, they do not grant tools, permissions, authorization or extra scope.

The playbooks and coding skills are ours, not copied from another harness or skill collection. They cover the modes the tool is actually used for: CTF speed runs, agentic delegation, white-box code review, remediation, CI gating, full application assessments, feature delivery, debugging, testing, refactoring, performance work, dependency upgrades, general code review and API contracts. The coding skills are also exported to standalone skills/<id>/SKILL.md files.

Closure discipline (sec_finding)

A finding's status is derived, never asserted. The model cannot mark something "confirmed"; status and closure are not writable through update. Each finding carries a closure state:

Closure Means
confirmed a reproduction or at least one evidence reference resolving to a real artifact
ruled_out a named control at a named location explains the observation away
open_proof_gap neither — an unresolved lead, recorded honestly

closure=confirmed without a reproduction and without a resolvable evidence reference is downgraded to open_proof_gap at write time and the downgrade is reported back. ruled_out without ruled_out_by is refused. Findings also carry counterevidence, severity_change_conditions, assumptions and fix_effort; updates require a reason, and an update with no effective change is refused.

Evidence references are checked, not trusted: each file:line or E-… reference is classified exists / missing / forbidden / unavailable. Only ENOENT/ENOTDIR establishes absence — a permission or I/O error is never reported as "missing", because a false "missing" would wrongly sink a real finding.

Duplicate detection is deterministic and local: token-set Jaccard over title/endpoint/description, ≥ 0.72 reported as a likely duplicate and ≥ 0.45 as similar. No model call, no network.

Precision attachments

PRECISION_NOTES is not only methodology text. When a finding carries a known false-positive shape — PHP Phar metadata, OAuth/OIDC state and PKCE, a kubelet 401/403, a Django SECRET_KEY, or NoSQL security.javascriptEnabled — the matching class note is derived at write time and stored on the finding itself. sec_finding prints it, and the markdown/HTML report renders a Precision checks section beside the claim. The attachment is deterministic and local; the model cannot omit it.

Report preflight

sec_report checks the selected findings before export. A critical or high finding still in open_proof_gap produces a warning naming the finding and the unproven impact. The export is not blocked — an unresolved candidate may be exactly what the operator intends to report — but the warning is shown on preview and on export, so an unproven claim cannot slip through unnoticed.

Coverage ledger (sec_coverage)

Findings answer what was wrong; the coverage ledger answers what was tested and found sound. Each entry is one surface × risk_area with an outcome:

  • reported — a finding was recorded; finding_id is required and checked.
  • no_issue_found — exercised, no issue observed.
  • ruled_out — a specific control makes the candidate safe.
  • not_applicable — the surface/risk pair does not apply.
  • needs_follow_up — plausible but not resolved in this engagement.

Every outcome except reported requires evidence explaining the conclusion. A duplicate target + surface + risk_area is refused and the existing id is returned; use action=update to advance it, which appends to the entry's revision history rather than overwriting it. action=summary reports outcome counts and findings that have no linked entry. Reports render the selected ledger or state plainly that no coverage was recorded; they also write a *.coverage.json sidecar. An empty findings list without coverage is not evidence that the target is secure.

Shared threat model (sec_threat_model)

The model is per target, not per agent. action=save stores the base model (actors, assets, trust boundaries, entry points, hypotheses and assumptions); action=amend appends a dated addendum; action=get returns the base plus amendments. URLs and bare hosts collapse to one host key, while repository paths use an absolute path key, so two agents cannot create divergent models for the same asset. A new save clears amendments but increments the revision. The report includes the model and its amendments so the reasoning can be reviewed after the fact.

Retesting (sec_retest)

A finding is a dated claim about a running system, and systems change. Without a retest the store can only ever record the first observation, so a report can describe as confirmed something that stopped reproducing weeks ago. sec_retest re-checks the reproduction and appends what it found.

  • action=run re-executes a command and captures its output as evidence.
  • action=record logs a re-check performed by hand (method + observation).

Retests are append-only — deliberately not part of sec_finding update, which takes a whole field value and could therefore rewrite or drop history. Each entry records who, when, the method or command, the observation, the exit code and evidence id for a run, the revision it was performed against, and the verification status before and after.

action=run goes through the same enforcement as every other tool, with one addition: the command is free text, so every host it names is authorized, not just the finding's target. Authorizing only the finding's target would let an off-scope command run under an in-scope finding. Destructive commands, shell wrappers (bash -c, sudo, xargs) and commands whose destination cannot be statically determined ($(…), pipes into an interpreter) are refused — the last because a retest is authorized per host and an unknown destination cannot be authorized. Pipes and redirects are fine; the command is tokenized and executed via argv, never through a shell.

A retest that contradicts the recorded closure — refuted on a confirmed finding, verified on a ruled_out one — reopens it to open_proof_gap and reports the reopen, rather than leaving the store asserting two incompatible things. The contradiction stays in the history either way.

The history renders into the report as a per-finding retest table, for markdown and HTML.

Evidence capture and credential disclosure (sec_evidence, sec_report)

The deployment posture is full disclosure: a leaked credential is usually the finding, so it is kept verbatim and reproduced in full. This is a deliberate departure from the standard guidance — PTES Post-Exploitation says passwords "will not be included in the final report", and OWASP APTS (APTS-MR-019, APTS-TP-013) says plaintext credentials MUST NOT appear in reports. This build follows the Metasploit Credentials-Report model instead. See README.md#credentials-and-disclosure.

Evidence is stored verbatim at capture, before anything is decided about the report. sec_evidence include_secrets=false is the opt-out, for artifacts whose value is not the secret; it redacts before the bytes touch disk, is irreversible, and the store marks the record redacted: true.

sec_recon check=http archives the raw response on its own and returns the artifact id, so a claim about a response can cite the response. The transcript is reconstructed, not the literal wire bytes: the status line and headers come from rawHeaders, which preserves wire order and duplicates, but Node decodes a chunked or compressed body before it is seen — so the artifact says "reconstructed transcript" in its own header rather than overclaiming. A body larger than the capture limit is marked inside the transcript with what fraction was retained, and a redirect chain archives every hop that was actually fetched. The rule the archive follows is the same as everywhere else: keep what the target actually returned, and label what is a view of it.

The store is bounded per project: 1000 artifacts and 256 MiB. A capture that would cross either bound is refused and the refusal explains how to prune. Nothing is silently dropped or overwritten. Pruning is manual and deliberate — remove the artifact file and its index entry together, and only after detaching it from every finding that cites it. sec_evidence action=stats shows current usage; action=list repeats the totals at the top.

sec_report mirrors the posture:

Argument Effect
evidence=none|index|full index (default) lists artifacts; full embeds each artifact's stored content as its own section in md/html and as evidence_artifacts[].content in json
redact=true|false false (default) emits every text format — md, json, html, and therefore the pdf — verbatim. true scrubs secret shapes and every collected credential value

The Credentials section

Every report carries a dedicated Credentials section, before the findings. credentials.ts extracts each credential-shaped value out of the findings' prose fields and the stored evidence, folds duplicates by value, and lists one entry per credential:

  • C-01 — a stable id, so a finding can cite it
  • the label (the assignment key, e.g. db_password) and the shape (aws-akid)
  • the value, verbatim
  • the host, the line it came from, and which findings/artifacts it was seen in

Extraction shares its pattern table with redaction (redact.ts), so a shape can never be redacted by one and missed by the other. The extractor also covers userinfo in URLs (postgres://user:pass@host), which the redactor previously handled only through a separate helper.

redact=true scrubs the section too. A bare value has no key: in front of it, so no shape rule would catch it — scrubReport therefore also replaces every collected value literally, guaranteeing the section cannot leak when scrubbing was asked for.

redact=false produces a credential-grade deliverable. It carries an UNREDACTED banner in md/html and unredacted: true in json, every file is written 0600 (including the pdf, which the browser-based converter would otherwise leave world-readable), and the export is audited under report.export.unredacted rather than report.export. Redaction only removes secrets that are still present: an artifact stored with include_secrets=false was already scrubbed at capture, and the report says so rather than implying the value was never there.

Interop exports (SARIF, CSV)

Opt-in via sec_report format=sarif|csv|interop, so a normal export stays four files. Both are derived from the same findings as every other format.

SARIF 2.1.0.

Ours SARIF
critical, high error
medium warning
low note
info none — an informational note must not fail a gate
file:line / file:line-line evidence ref physicalLocation + region
a network finding (no file) logicalLocations, kind: resource — never a fabricated path
same CWE across findings one shared rules entry
id, status, closure, confidence, verification, CVSS, CVE, retest count properties (SARIF has no field for them)

A E-… evidence id is an artifact, not a source location, so it never becomes a physicalLocation. The run carries a toolExecutionNotifications note stating the coverage, so an empty result set is not read as "no issues".

CSV (DefectDojo Generic Findings Import). Active, Verified, FalsePositive and Remediated are separate columns and are not derived from one another — a remediated finding is still Verified=true, because the check was true when it was made, and Active=false is a lifecycle fact rather than a verification result. Quoting follows RFC 4180 (quote on comma/quote/newline, double internal quotes), and the tests round-trip the output through a parser and assert every row is the same width, so an importer cannot skew.

Report coverage — one session, one target

The findings store is engagement-wide and append-only. A report that reads it without a filter therefore absorbs every earlier session's findings, including findings that belong to a different host. sec_report defaults to coverage=session — only what the current scan session recorded — and report-selection.ts enforces it before anything is rendered:

coverage Includes
session (default) findings and evidence written by the current session
target findings whose host matches target (host, host:port or URL)
engagement everything in the store — deliberate, never accidental

Two invariants hold regardless of coverage. Findings on a host the scope does not authorize are always excluded, and the selection is re-checked after filtering so a finding that failed the target test cannot survive. What was excluded is stated in the report header (Coverage:) and in the tool output — counts plus the foreign hosts — rather than silently dropped. If a session recorded nothing but other sessions hold findings, the tool says so and points at coverage=engagement.

4. Enforcement

Four layers, outermost first:

  1. config() — injects native permission rules. Destructive patterns (rm -rf *, mkfs*, dd *, shutdown *, reboot *) are mapped to deny always. Intrusive scanners (nmap *, sqlmap *, nuclei *, …) are mapped to ask only when you set plugins.ask_intrusive_commands: true — prompting by default would stall every non-interactive nullbyte run. Uses NullByte's own permission prompt, so the native model is preserved. Operator-authored rules win over injected ones.
  2. tool.execute.before — awaited inside the tool executor, so throwing aborts the call before the body runs. Enforces destructive refusal, the credential-store guard, and (in allow-list mode) target authorization for bash/webfetch.
  3. Tool bodies — scan tools use the default-open decision: sec_recon, sec_ports and sec_scan ignore the allow-list and never raise an approval prompt. The absolute metadata/link-local refusal still applies, including to redirect hops. sec_retest keeps per-call authorization, DNS re-validation and the opt-in require_approval_for_intrusive prompt.
  4. experimental.chat.system.transform — injects the shared workflow (advisory only). The full block goes to a session once, then a compact reminder.

Every decision is appended to .nullbyte/security/audit.jsonl (arguments redacted and truncated).

The trail is hash-chained. Each record carries its 1-based seq, the hash of the record before it (prev, with a genesis hash for the first) and its own hash, computed over the record with hash removed, undefined values dropped and keys sorted — so the digest does not depend on JSON key order, and a record parsed back from disk hashes to the same value it was written with. sec_audit action=verify recomputes the chain from the start and reports:

finding means
altered the record's contents no longer match its own hash
broken-link prev does not point at the preceding record's hash
sequence a seq gap — records were dropped or reordered
unparseable a partial or corrupt line (e.g. a torn write)

Records written before chaining was added are counted as unchained and cannot be verified; they are reported as such rather than assumed good. Flipping a recorded deny into an allow is detected as altered.

sec_audit action=reconcile closes the loop on the finding store: every finding created through sec_finding writes a finding.create allow record, and a finding present in findings.json without one is reported. A direct store write is the case this surfaces. It is a signal to investigate, not proof of tampering — findings created before creation auditing existed also appear.

Auditing is fail-loud. A write that cannot happen throws AuditWriteError and refuses the action. The trail is what the report's coverage claims are derived from, so continuing without it would mean claiming coverage the trail cannot prove. Appending to a trail whose tail is unparseable throws AuditCorruptError instead of forking the chain.

The chain proves internal consistency, not authorship: it detects edits to an existing trail, but anyone with write access to the store can rebuild a whole chain. Sealing the head hash outside the store is what defeats that.

5. Scanners (optional)

sec_scan degrades honestly: a missing scanner is reported with its install hint and recorded as a limitation instead of being silently skipped.

Installed on this box: nmap, nuclei, nikto, whatweb, sslscan, sqlmap. sec_sast action=deps needs a lockfile (package-lock.json, pnpm-lock.yaml or yarn.lock) in the project.

6. Lab

/tmp/opencode/lab/vulnerable-server.mjs is a deliberately vulnerable target on loopback only:

node /tmp/opencode/lab/vulnerable-server.mjs
  http   → 127.0.0.1:8080   reflected XSS, served /.env, missing headers,
                            spoofed Apache/2.4.49 banner, /backup/ listing
  admin  → 127.0.0.1:8081   HTTP Basic admin/admin, /debug env dump
  https  → 127.0.0.1:8443   self-signed cert, no HSTS

7. Tests

cd ~/.config/nullbyte/security-lib
npm test            # node --experimental-strip-types --test __tests__/*.test.ts   (365 tests)

Coverage: default and allow-list postures, CIDR/wildcard/exclude matching, metadata and link-local refusal, scope helper matching and multi-port authorization primitives, DNS re-validation, destructive and credential guards, CVSS 3.1 vectors (9.8 / 6.1 / 5.9), finding validation, derived status and closure downgrade, semantic dedupe, evidence-reference classification, verbatim capture, report rendering, credential extraction and disclosure, the report's standards sections (executive summary, timeline, roadmap, appendices), the per-page copyright/page-number footer (asserted against the rendered PDF via pdftotext), connect-scan behaviour, transient agent retry/backoff, coverage recording/update/summary and sidecar, threat-model save/amend/path keys and report rendering, secret scanning, methodology catalog and resolution (including our playbooks and coding skills), precision attachments, report preflight, evidence retention bounds, audit reconciliation, network intensity floors, scanner detection, atomic and corruption-aware store writes, evidence hash verification, audit-chain tamper detection, retest recording, contradiction handling and the scope enforcement applied to a retested command, HTTP transcript archiving against a live loopback server, the SARIF/CSV interop mappings (levels, locations and rule dedupe; RFC 4180 round-trip and the independent Active/Verified/FalsePositive axes), and encryption at rest (envelope format, wrong-key and tamper detection, sealed store/evidence/audit reads, missing-key behaviour, and plaintext-to-encrypted migration).

8. Gotchas learned on this build

  • Plugin exports must all be functions. The loader walks Object.values(module) and throws Plugin export is not a function for any non-function export. Export only the plugin function.
  • worktree can be /. Both PluginInput.worktree and ToolContext.worktree resolve to / here while directory and cwd hold the real project path. paths.ts resolves a chain and never accepts /.
  • Attach socket listeners before connect(). Under Bun, net.connect() can emit connect before a handler registered afterwards, and any later error must not downgrade an already-open port.
  • The scanner probe is runtime-agnostic: plaintext HTTP HEAD is sent on every port except those whose services speak first (ssh, mysql, redis, …).
  • A host-level authorize is not a port-level authorize. authorize() only checks target.port, which is undefined for a multi-port scan — so a scan tool that authorized the host and then probed its own port list silently bypassed the entry's ports allowance. enforcePorts() exists to close this; any new scanner must use it.
  • \b does not fire at an underscore. The credential-assignment redaction rule anchored the key with \b(password|secret|…)\b, which never matches db_password, mysql_pwd or client_secret — _ is a word character, so there is no boundary. Those are the exact shapes a leaked .env uses, so the default redaction was missing the most common case. The rule now allows a prefix group.
  • JSON was the one unredacted deliverable. writeReport scrubbed md and html but wrote renderJson raw, so a secret stripped from the "safe" formats survived in the machine-readable one. Redaction is now applied uniformly, and the redact switch governs all text formats together.
  • The report header leaked the model. provenance is free text the agent fills, and a shipped report read Produced by: NullByte (opencode/deepseek-v4.1-flash-free) via sec_recon, sec_ports-free manual curl,. Vendor and model identifiers are now stripped at render time (sanitizeProvenance), across md, json and html — so the guarantee holds regardless of what a caller passes, and the PDF inherits it through the html render. The scrub is scoped to the provenance line on purpose: a target that happens to be called "opencode" is evidence, not branding, and must survive in the findings.
  • The PDF converter ignores the umask. Chrome writes the file world-readable regardless of the process umask, so an unredacted export was 664. writeReport now forces 0600 on the PDF.
  • Chrome does render CSS @page margin boxes. The per-page footer needs no browser trickery. The opposite was assumed — and a DevTools-Protocol Page.printToPDF + footerTemplate workaround was built on top of it — but @bottom-left / @bottom-right with counter(page) / counter(pages) render correctly through the plain --print-to-pdf CLI on Chrome 151, on every page. The CDP path was deleted; the footer now lives in the stylesheet, where WeasyPrint honours it too. A margin box and a CDP footer template render twice, so verify footer claims against the real PDF (pdftotext), not against the HTML.
  • CDP printToPDF rejects string margins. marginTop: "14mm" fails with Failed to deserialize params.marginTop - BINDINGS: double value expected; the margins must be numbers in inches. (Moot now that the CDP path is gone, but it is the trap if that path is ever revived.)
  • A descendant selector on a self-classed element never matches. The severity badge is <span class="badge sev-medium">, but the rule was .sev-medium .badge — a descendant selector, which requires the severity class on an ancestor. It matched inside the finding card (where <article> carries it) and nowhere else, so the Summary table's labels rendered color:#fff on a white background: invisible. It is now .badge.sev-medium. Worth a screenshot check — no test that reads text will catch a label that is present but not painted.
  • Backticks inside a CSS comment break the template literal. The whole stylesheet lives in a JS template string; a stray ` in a comment terminates it and the module throws at render time.
  • Redaction at the capture boundary cannot be undone downstream. runArgv redacted stdout/stderr before returning, so a "verbatim" scanner log would have been a log that merely looked verbatim. RunOptions.includeSecrets now threads through to the spawn, because the decision has to be made before the process runs — redacting later is not the same operation.
  • Two logs in the same millisecond overwrite each other. The log filename is timestamped to the millisecond, which is not unique when a scan is retried immediately. A test caught it; uniquePath() now appends -2, -3 rather than destroying the earlier file.
  • Findings are not a proxy for what a session touched. A report's destination was originally derived from its findings, so a session that contacted a target and recorded nothing fell back to the shared reports/ directory. Contacted hosts now come from the audit trail, which records every allow decision.

9. Limitations

  • Connect scans only (no raw SYN/OS detection) — use the nmap adapter for that.
  • webfetch redirects are validated by the tool, not the guard (the guard sees only the initial URL); sec_recon check=http validates every hop itself.
  • Evidence is stored verbatim by default, so a captured artifact is byte-exact. Capturing with include_secrets=false redacts before storage and is irreversible.
  • Reports are verbatim by default by design — treat every export as credential-grade sensitive. redact=true removes the shapes it knows and every collected credential value; it cannot remove a secret it does not recognise, and it cannot undo capture-time redaction.
  • CVSS is base score only (no temporal/environmental).
  • Pacing is enforced per process, not per engagement or per host across processes. Two agents against the same target in one project get one budget each. Within a process it is shared, which is the case the tools control.
  • Pacing is not persisted: interval state lives for the life of the process, so a restarted agent starts with a fresh inter-request window. For a target where that matters, drive the engagement through a single session.
  • Per-target directories are created in the project root, so running from inside a repository creates visible directories there. Each one self-ignores in git, but it is still a write into the working tree — turn the feature off, or run from a dedicated assessment directory, if that is not wanted.
  • A report is filed under a target's directory only when the session touched exactly one host. A session spanning several hosts writes to the shared reports/ directory, because no single target directory describes it.
  • PDF export needs an HTML-to-PDF converter (weasyprint, google-chrome or chromium). A converter that is installed but broken is probed with a real one-page render and skipped, and the tool reports the failure instead of writing a stub. If none works, the markdown/JSON/HTML are still written and the PDF line says why it is missing. The per-page copyright and Page N of M footer ride on CSS @page margin boxes, so a converter that does not implement them will produce a footerless PDF rather than an error.