Jev end to end: the policy author and publish skills, for failproofai 1.0.9

[chhhee10]

Why

failproofai-policy-author here predates Jev, FailproofAI's semantic evaluator, and has drifted from the copy the product maintains. Anyone installing it today gets no guidance on Jev checks, reviewable blocks, packs as the way to enable builtins, or failproofai guarding the agent's own session. The product copy itself was also out of date for failproofai 1.0.9, which no longer ships Jev's 16 checks in the npm package.

failproofai-policy-publish has no Jev guidance at all, although a pack is the only way a Jev check reaches a machine.

This PR covers Jev end to end, from writing a check to publishing the pack that carries it, all corrected for 1.0.9. It replaces #23, which did the same for 1.0.8-beta.0. Everything this repo changed on its own is kept.

What changed

Jev in 1.0.9: no pack, no policy

• Jev asks only the checks that installed packs declare. FailproofAI's 16 are now the FailproofAI/jev-policies pack, not built into the CLI.
• A reviewedBy counts only for checks installed on the machine. A policy naming a check the machine doesn't have stays hard.
• The 16 names are reserved, so another pack can't declare them. A name that two packs declare differently is asked for neither.
• Budget: publish limits a pack from anyone other than FailproofAI to 9,101 characters of questions, and the dry run fails if it's over. The old text said the limit was the whole 27,591.
• Publishing:
  • --min-cli-version defaults to 1.0.8-beta.0 for packs with checks;
  • the dry-run output wording is new;
  • publish with no repository is a dry run.
• Installing: --cli <agent> scopes a pack to one agent; re-adding with a new version upgrades it; policies remove uninstalls it.

Reviewable clearing and consent, as 1.0.9 decides them

• A reviewable block clears only when every reviewer answered "no concern", was overridden by the user's request, or was a deny the user's task softened. A warning nobody consented to keeps the block, and one from a deny-mode check cancels every clear on that call (combine.ts, jev-review.ts). The product copy said a warning counted as a clear.
• The consent rules decideV1 checks in code:
  • op_requested clears a shell command only when every target it names appears in what the user typed;
  • task_step does not soften a command the user named only in part;
  • a command the scan cannot read whole is never cleared or softened. That covers any $ expansion, globs, brace expansion, heredocs, eval and sh -c, so no request clears rm -rf build/*.

Publishing Jev packs (failproofai-policy-publish, from #23, rewritten for 1.0.9)

• New Packs that carry Jev checks section:
  • the 1.0.9 dry-run lines;
  • what publish refuses;
  • the budget and minCliVersion;
  • the rollback reminder for packs of checks alone;
  • what policies show and policies add print for a pack's checks.
• references/publishing.md covers:
  • every refusal message as 1.0.9 prints it;
  • the budget arithmetic;
  • reserved and contested names, and the FailproofAI/ namespace;
  • minCliVersion;
  • installing a dry run locally (the only way to exercise a check before release);
  • observe and --cli packs.
• Discovery also finds files that call semanticPolicies.add.

Hermes

• instruct on the native plugin blocks the first attempt in each model response and lets the retry through. The old text said it was only a stderr note.
• user_said is always empty on Hermes, so Jev probes must judge the command itself.
• Hermes tools that Jev can't classify get every check asked.

Brought in from the product copy

• Jev: when to write a semantic check, a reviewable block, or a hard regex; how to write probes; the two-tier pack pattern.
• When failproofai guards your own session: draft in policy-drafts/ and hand the operator the commands.
• Builtins are enabled through the FailproofAI pack. enabledPolicies stops counting once any pack is installed.
• 40 builtins, with the builtins reference regenerated.

Kept from this repo

• Routing to failproofai-policy-publish and fp-cloud-cli. The product copy routes to a failproofai-policy-deploy skill that doesn't exist here.
• The policy backtest sections in SKILL.md and references/cloud.md. The feature is still live in the dashboard.
• The canonicalNames fix in fleet-tool-coverage.mjs.

Generator

• sync-builtins.mjs reads the Jev checks from the jev-policies reference copy. Before this, on 1.0.9 it silently dropped the whole section.
• It also records each builtin's authority and reviewedBy in policy-events.json.

README: both policy rows mention Jev.

How it was merged

This repo's first import of the skill (acad059) is byte-identical to the product copy at the time. Each file was merged three ways from that base: this repo's main on one side, the product copy corrected for 1.0.9 on the other. The 25 conflicts were resolved by hand, following the rules above. The generated files were regenerated rather than merged.

The publish-side content comes from #23. #23 was written against 1.0.8-beta.0, so every message and number in it was re-checked against the 1.0.9 tag and rewritten where it had changed. The dry-run line, the policies add output, the list of commands consent can't clear, and the "built-in checks" framing had all changed.

Verification

• python3 scripts/validate-skills.py: 6 skills, 0 errors, 0 warnings.

• sync-builtins.mjs --check and sync-harnesses.mjs --check both pass against the failproofai 1.0.9 source. The second commit changes no generated file.

• node --check passes on every script.

• Every claim about 1.0.9 was checked against the released source at tag 1.0.9 (ee5dc022):

  • pack-cli.ts, pack-manifest.ts and pack-store.ts;
  • effective-reviewers.ts and precondition-names.ts;
  • semantic/pack-policies.ts, semantic/decide.ts, semantic/facts.ts, semantic/combine.ts and semantic/jev-review.ts;
  • semantic/intent.ts and hermes-plugin/ledger.py.

• The Jev workflow was run end to end on 1.0.9 with real Hermes agents:

  1. author the pack;
  2. publish --dry-run;
  3. install locally with --cli hermes;
  4. observe;
  5. enforce: 28/28 correct decisions;
  6. remove.

  That run predates the second commit, which only adds text checked against source.

Supersedes #23.

🤖 Generated with Claude Code

[SiddarthAA]

lgtm @chhhee10!