Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 70 additions & 0 deletions .codex/skills/agentic-content-review/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
---
name: agentic-content-review
description: Agentic VectorLint-style content review using active markdown rules, one-rule-per-subagent delegation, parser-clean markdown findings, exact quote grounding, and density scoring. Use when reviewing docs, marketing copy, specs, PR descriptions, implementation artifacts, or other prose/content files against workspace rules in .vlint/rules or skill-bundled default rules.
---

# Agentic Content Review

Use this skill to review prose and other content files against active markdown rules with an agent-native workflow.

## Review Model

- Delegate one reviewer subagent per active rule per source file.
- Use path-only delegation: provide source and rule file paths, and let the reviewer read the files directly.
- Have each reviewer read the source file and exactly one rule file itself.
- Optimize the reviewer pass for high recall, then use the main-agent audit to remove unsupported findings.
- Require each reviewer to run parser and scorer checks before returning.

## Core Definitions

- `parser-clean` means all required finding fields are present; source and rule files exist; the Evidence quote is found in the source file; the Rule quote is found in the rule file; Context supports violation is `true` or `false`; Confidence is numeric from `0.0` to `1.0`; Line is numeric and plausible; and score computation completes without errors.
- `same-anchor groups` means findings that share source file, rule path, Evidence quote, Rule quote, and Line. Treat them as semantic-review warnings only, never as duplicate decisions.
- Density scoring is strict by default, based on finding count divided by word count, clamped to a `0-10` range. Recompute the final score after main-agent edits.

## Reviewer Sub-Agent Setup

If your toolset supports sub-agent definitions, check whether a sub-agent definition named `content-reviewer` exists in your workspace's sub-agent directory. If it does not exist, announce that you are creating it and create it using the content of `references/reviewer-directive.md` as its directive. Do not spin up a sub-agent at this step — only create the definition.

If your toolset does not support sub-agent definitions, announce it and proceed using your toolset's inline sub-agent capability with `references/reviewer-directive.md` as the prompt.

---

## Workflow

1. Resolve active rules from workspace `.vlint/rules/*/rule-index.yml` files, or from bundled defaults at `.codex/skills/agentic-content-review/rules/default/rule-index.yml` when no workspace indexes exist or the caller explicitly asks for defaults.
2. Pair each source file with every active rule.
3. Dispatch one reviewer subagent for each file-and-rule pair.
4. Collect markdown findings with exact Evidence quotes and Rule quotes.
5. Run parser and scorer validation on every reviewer response.
6. Treat same-anchor groups as semantic review warnings, not duplicate decisions.
7. Consolidate into a scored report with findings, warnings, and a short recommendation.

## Output Rules
- Use the empty-findings form from `references/finding-template.md` when the reviewer finds no issues.
- Keep findings structurally consistent with `references/finding-template.md`.
- Ground every finding in exact Evidence and Rule quotes.
- Keep final claims grounded in exact Evidence and Rule quotes plus the main-agent semantic audit.
- Use `references/rule-index-format.md` to resolve active rules from YAML pack indexes.
- Use `references/reviewer-directive.md` to prompt reviewer subagents.
- Use `references/subagent-adapters.md` to normalize subagent output.


## Commands

List active rules:

```bash
npx tsx .codex/skills/agentic-content-review/scripts/list-active-rules.ts <workspace-root>
```

Use bundled defaults even when workspace rules exist:

```bash
npx tsx .codex/skills/agentic-content-review/scripts/list-active-rules.ts <workspace-root> --include-defaults
```

Parse and score review markdown:

```bash
npx tsx .codex/skills/agentic-content-review/scripts/parse-review-and-score.ts <review.md> --write-log
```
7 changes: 7 additions & 0 deletions .codex/skills/agentic-content-review/agents/openai.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
interface:
display_name: "Agentic Content Review"
short_description: "Per-rule agent review with scoring"
default_prompt: "Use $agentic-content-review to review this content with active .vlint rules and return a scored report."

policy:
allow_implicit_invocation: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# Finding Template

Use this exact field set for each markdown finding. Replace placeholder text with actual values.

```md
### Finding
- Rule path: `relative/path/to/rule.md`
- Source file: `relative/or/absolute/path.md`
- Line: `source line number where the evidence quote begins`
- Evidence quote: `exact source text supporting the finding`
- Rule quote: `exact text copied from the rule file`
- Flag reasoning: `why the evidence, rule, context, and plausible non-violation make this worth flagging`
- Issue: `one sentence describing the problem`
- Plausible non-violation: `one sentence describing the best benign interpretation`
- Context supports violation: `true or false`
- Suggestion: `one short sentence describing how to fix the issue`
- Confidence: `number from 0.0 to 1.0 reflecting certainty this is a genuine violation`
```

## Field Meanings

- `Flag reasoning`: concise explanation of why the finding should be surfaced after considering the evidence quote, rule quote, surrounding context, and plausible non-violation.
- `Plausible non-violation`: the strongest benign interpretation to consider after identifying the issue.
- `Context supports violation`: `true` when the surrounding source/workspace context strengthens the violation claim after considering the plausible non-violation; `false` when the context weakens or undermines the claim.
- `Suggestion`: one short sentence describing how to fix the issue. This maps to VectorLint's `suggestion` field.
- `Confidence`: number from `0.0` to `1.0` reflecting certainty that this is a genuine violation after considering rule support, evidence, context, and plausible non-violation.

## Empty Findings

If a reviewer has no findings, return:

```md
## Findings
No findings.
```

This empty-findings form is the only accepted response shape that does not contain `### Finding` blocks.
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Reviewer Directive

Use this directive for each reviewer subagent.

## Role

You are a VectorLint-style content reviewer assigned to one active rule and one source file.

## VectorLint Directive

Your goal is to flag every instance that matches the assigned rule. A report with zero findings is correct only when the content contains no matching violations. A report with findings is correct when the content contains those violations. Accuracy means flagging what is present and only what is present.

The assigned rule is the only criterion for flagging findings. Flag every instance that matches the rule's pattern. Leave out text that does not match the assigned rule. Use the rule file as the source of rule support; cite the assigned rule text, not this directive, as the reason for a finding.

Interpret source structure from the file type and content structure. In structured formats such as MDX, cards, tabs, and list items can be independent content units rather than continuous prose. In plain markdown or text files, prose is usually continuous unless headings, horizontal rules, or other structure separate it. When applying proximity-based rules in structured formats, evaluate structural elements independently unless the rule explicitly requires cross-element evaluation.

Each finding has two jobs:

1. Identify a candidate violation.
2. Record the grounding and gate-check information that lets the parser and main agent decide whether it should be surfaced.

For each candidate, make these checks visible through the template fields:

- Rule support: the assigned rule quote supports the finding.
- Evidence exactness: the evidence quote is copied exactly from the assigned source file.
- Context support: surrounding source/workspace context supports the violation after considering acceptable uses.
- Plausible non-violation: the strongest benign reading is stated directly.
- Suggestion: one short sentence describes how to fix the issue.

## Inputs

- Read the source file from its path.
- Read the assigned rule file from its path; this is the only rule being evaluated in this reviewer pass.
- Start from the assigned source file and rule file, then read any additional workspace context that materially improves judgment: linked files, imported files, neighboring documentation, definitions, examples, or project conventions.
- Use the assigned source file for every `Evidence quote` and the assigned rule file for every `Rule quote`.

## Task

- Inspect the full source file with high recall, using additional context to distinguish genuine violations from acceptable uses.
- Find every rule-matching issue you can justify from the text.
- Ground each finding in an exact Evidence quote copied from the source file.
- Ground each finding in an exact Rule quote copied from the rule file.
- Use the source file's actual line numbers. The `Line` value is where the evidence quote begins.
- Ensure the Evidence quote, Rule quote, and Line are all parser-clean before returning.
- Provide `Suggestion` as one short sentence describing how to fix the issue.
- Return parser-safe markdown rather than freeform commentary.
- Run the parser/scorer checks before returning.

## Confidence Calibration

Assign `Confidence` after every other finding field is written.

- `0.75-1.0`: the violation is demonstrable from the text and rule without assumptions about intent or missing context.
- `0.50-0.74`: the text fits the rule pattern, but one reasoning step is needed to connect the text to the rule.
- `0.25-0.49`: the text has multiple plausible interpretations, including both violating and acceptable readings.
- `0.00-0.24`: the violation claim depends mostly on assumptions not present in the text or rule.

## Return Format

- Return findings using the exact fields from `finding-template.md`.
- Generate each finding in the template order: anchors first, flag reasoning before the issue label, plausible non-violation after the issue, suggestion before confidence, and confidence last.
- If no findings exist, return the empty findings form from the template.
- The empty findings form is the only accepted response shape that does not contain `### Finding` blocks.
- Use the template fields only; leave out severity and any other unlisted fields.
- Preserve separate findings for separate issues, even when findings share the same anchor.
- Keep the output parser-clean before returning.
- If multiple findings share the same anchor, keep them distinct and let consolidation mark the group as a semantic warning.
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Rule Index Format

Use YAML rule indexes stored at `.vlint/rules/<rule-pack-name>/rule-index.yml`.
Bundled defaults live at `.codex/skills/agentic-content-review/rules/default/rule-index.yml`.

## Shape

```yaml
pack: brand
active: true
rules:
- id: voice
name: Brand Voice
path: voice.md
active: true
description: Check whether content matches brand tone and voice.
```

## Selection Rules

- Dispatch reviewers only for rules marked `active: true`.
- Resolve the rule file relative to the pack directory before creating the subagent prompt.
- Use bundled defaults only when no workspace `.vlint/rules/*/rule-index.yml` exists or the caller explicitly asks for defaults.
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# Subagent Adapters

Use an adapter layer to normalize reviewer prompts and responses.

## Prompt Contract

Provide each reviewer with:

- `source_file`
- `rule_id`
- `rule_file`
- any minimal scope hint from the rule index

Use path-only delegation: provide file paths rather than copied content blocks, partial excerpts, or summaries of the source text.

## Dispatch Rules

- Create one subagent per active rule per source file.
- Read the assigned source file and assigned rule file inside the subagent.
- Let the subagent read additional workspace context when it materially improves rule judgment.
- Keep the reviewer prompt short and specific.
- Create a fresh subagent for each file-and-rule pair.

## Core Checks

- `parser-clean` means required fields are present; source and rule files exist; the Evidence quote is found in the source file; the Rule quote is found in the rule file; Context supports violation is `true` or `false`; Confidence is numeric from `0.0` to `1.0`; Line is numeric and plausible; and score computation completes without errors.
- `same-anchor groups` means findings sharing source file, rule path, Evidence quote, Rule quote, and Line. Treat them as semantic-review warnings only.
- Density scoring is strict by default, based on finding count divided by word count, clamped to `0-10`. Recompute the final score after main-agent edits.

## Response Normalization

- Parse the reviewer output against `finding-template.md`.
- Accept only output that matches `finding-template.md`; route severity fields or unsupported sections back through parser-clean repair.
- Re-run once if the output is not parser-clean.
- Preserve same-anchor groups as semantic review warnings during consolidation.
- Aggregate findings into a scored report after normalization.

## Host Behavior

- Treat reviewer output as structured review data, not prose.
- Keep the consolidation layer responsible for semantic review of same-anchor groups and scoring density.
- Keep the reviewer responsible for recall and rule-specific judgment.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# AI Patterns

Flag generic AI-like transition phrases that add no content, including "in today's fast-paced world", "delve into", "unlock the power of", "it is worth noting", and "seamlessly" when used as empty praise.

Flag broad, polished-but-empty statements that could apply to almost any product or topic.

Treat common phrases as acceptable when they are specific and supported. Flag them only when the surrounding sentence is generic, unsupported, or non-specific.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Directness

Flag filler phrases that do not add meaning, including "it is important to note", "it should be noted", "in order to", and similar setup phrases.

Flag hedging that weakens a concrete claim without adding accuracy, including "may help", "can potentially", and "might be able to" when the surrounding context supports a direct statement.

Treat cautious wording as acceptable when it is necessary for technical accuracy, uncertainty, legal caution, or honest limitation.
18 changes: 18 additions & 0 deletions .codex/skills/agentic-content-review/rules/default/rule-index.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
pack: default
active: true
rules:
- id: directness
name: Directness
path: directness.md
active: true
description: Flag filler phrases, hedging, and indirect phrasing that weakens clarity.
- id: unsupported-claims
name: Unsupported Claims
path: unsupported-claims.md
active: true
description: Flag claims that need evidence, qualification, or source support.
- id: ai-patterns
name: AI Patterns
path: ai-patterns.md
active: true
description: Flag generic AI-like phrasing and empty transitional language.
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Unsupported Claims

Flag claims that present superiority, certainty, performance, safety, or business impact without evidence in the surrounding content.

Flag phrases such as "the best", "the fastest", "guaranteed", "proven", and "trusted by teams" when the content does not include support, qualification, or a source.

Treat clearly scoped opinion, quoted customer language, and claims immediately supported by evidence in the same section as acceptable.
Loading
Loading