From 745fbeba3368f85b76fc89367749a853f61daad5 Mon Sep 17 00:00:00 2001 From: Dave Wilding Date: Mon, 7 Sep 2026 08:42:44 +0800 Subject: [PATCH] Trigger probe workflow automatically when an issue is opened Add 'issues: [opened]' as a trigger alongside the existing workflow_dispatch. The workflow now runs automatically when someone opens an issue, and can still be manually dispatched to re-run on an existing issue. The ISSUE_NUMBER env var and concurrency group use github.event.issue.number for the issues trigger, falling back to github.event.inputs.issue_number for workflow_dispatch. A job-level if condition guards against running when neither is available. Updated README and AGENT_DESIGN.md to reflect the automatic trigger. --- .github/workflows/probe-issue.yaml | 7 +++++-- AGENT_DESIGN.md | 8 ++++---- README.md | 10 +++++----- 3 files changed, 14 insertions(+), 11 deletions(-) diff --git a/.github/workflows/probe-issue.yaml b/.github/workflows/probe-issue.yaml index 025e7a1..3a8f477 100644 --- a/.github/workflows/probe-issue.yaml +++ b/.github/workflows/probe-issue.yaml @@ -1,6 +1,8 @@ name: Probe issue on: + issues: + types: [opened] workflow_dispatch: inputs: issue_number: @@ -14,14 +16,15 @@ permissions: pull-requests: write concurrency: - group: probe-issue-${{ github.event.inputs.issue_number }} + group: probe-issue-${{ github.event.issue.number || github.event.inputs.issue_number }} cancel-in-progress: false jobs: probe: runs-on: ubuntu-latest + if: github.event_name == 'workflow_dispatch' || github.event.issue != null env: - ISSUE_NUMBER: ${{ github.event.inputs.issue_number }} + ISSUE_NUMBER: ${{ github.event.issue.number || github.event.inputs.issue_number }} OPENCODE_VERSION: '1.18.16' steps: - name: Checkout diff --git a/AGENT_DESIGN.md b/AGENT_DESIGN.md index 04caf70..77882bd 100644 --- a/AGENT_DESIGN.md +++ b/AGENT_DESIGN.md @@ -2,7 +2,7 @@ ## Goal -A manually dispatched GitHub Action in `basic-charms` that reads an issue describing public charm-dev documentation, writes tests that verify how things actually behave, and opens a PR. The agent is skeptical of the docs: it forms its own understanding of how the code behaves, writes a test asserting that understanding, and deduces what the CI result means for the doc. The user reviews the PR, inspects the CI results, and reads the agent's reasoning to determine whether the doc was validated or refuted. The PR is an artifact for review, not for merging. +A GitHub Action in `basic-charms` that reads an issue describing public charm-dev documentation, writes tests that verify how things actually behave, and opens a PR. The agent is skeptical of the docs: it forms its own understanding of how the code behaves, writes a test asserting that understanding, and deduces what the CI result means for the doc. The user reviews the PR, inspects the CI results, and reads the agent's reasoning to determine whether the doc was validated or refuted. The PR is an artifact for review, not for merging. ## Files (all under `.github/`) @@ -45,7 +45,7 @@ The `fetch_url` tool is the other exception: it lets the agent fetch content fro ## Workflow flow -1. `workflow_dispatch` with `issue_number` (required). +1. Triggered automatically when an issue is opened (`issues: [opened]`), or manually via `workflow_dispatch` with `issue_number` (required) to re-run on an existing issue. 2. Checkout with `persist-credentials: false`, `fetch-depth: 0`. No git credentials in `.git/config` during the agent run. 3. `git config core.hooksPath /dev/null` — defense in depth, inert hooks. 4. Setup Node 24, install `opencode-ai` (version pinned in the `OPENCODE_VERSION` env var in the workflow), set up uv. @@ -147,7 +147,7 @@ Doc-fetch allowlist: only `canonical.com`, `ubuntu.com`, `raw.githubusercontent. Agent staging and cleanup: agent definition and tool files copied to `.opencode/` before the run, removed before diff collection. -Manual dispatch only: no automatic triggers. The user explicitly chooses to run this. +Triggered automatically on issue open, or manually via `workflow_dispatch` to re-run on an existing issue. Repository setting: the repo must have "Allow GitHub Actions to create and approve pull requests" enabled (Settings → Actions → General → Workflow permissions). This is the gate that lets the `GITHUB_TOKEN` create PRs. The workflow declares `pull-requests: write` in its `permissions:` block, but that alone is not enough — the repo-level flag must also be on. The default workflow permission should remain `read` (least privilege); each workflow declares its own `permissions:` block. @@ -223,7 +223,7 @@ OpenCode vulnerability allowing code execution despite `bash: deny`: low, outsid ## Dry-run mode (not implemented) -The workflow has no dry-run mode. The workflow is manually dispatched (a human already chose to run it), the agent can return `BLOCKED` when it cannot proceed, and an unwanted PR is cheap to close and delete. A dry-run mode would add complexity across the input, env vars, conditional steps, and issue-comment branches for a mode whose main use is during initial development of the agent script. +The workflow has no dry-run mode. The workflow is triggered automatically on issue open (or manually for re-runs), the agent can return `BLOCKED` when it cannot proceed, and an unwanted PR is cheap to close and delete. A dry-run mode would add complexity across the input, env vars, conditional steps, and issue-comment branches for a mode whose main use is during initial development of the agent script. If dry-run is wanted later, implement it as follows: diff --git a/README.md b/README.md index c13d099..6204c52 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,9 @@ To perform an adversarial test: 2. Create an issue in `basic-charms` with a link to the documentation page and a description of the claim you want to test. This could be a quote from the page or an explanation of something the page implies. -3. Run the [Probe issue](https://github.com/dwilding/basic-charms/actions/workflows/probe-issue.yaml) workflow, entering the issue number in the **Run workflow** UI. + The [Probe issue](https://github.com/dwilding/basic-charms/actions/workflows/probe-issue.yaml) workflow runs automatically when an issue is opened. You can also trigger it manually from the Actions page (enter the issue number in the **Run workflow** UI) to re-run the probe on an existing issue. -4. Wait for a PR to be created. +3. Wait for a PR to be created. The PR will modify one or more of the basic charms (and possibly their unit tests or integration tests) to test the documentation claim you described in the issue. @@ -22,11 +22,11 @@ To perform an adversarial test: > I believe `` is true. I added a test asserting it, which is expected to pass. If CI passes, the doc is correct. -5. Review the PR to make sure the agent's changes are meaningful and trustworthy. Once you're satisfied, approve the PR's workflow runs to trigger CI. +4. Review the PR to make sure the agent's changes are meaningful and trustworthy. Once you're satisfied, approve the PR's workflow runs to trigger CI. -6. After the CI checks have completed, use the PR description to draw a conclusion about the documentation. +5. After the CI checks have completed, use the PR description to draw a conclusion about the documentation. -7. Decide how to fix the documentation — if needed. +6. Decide how to fix the documentation — if needed. The agentic workflow that creates the PR is explained in [AGENT_DESIGN.md](AGENT_DESIGN.md). It's a **highly experimental** workflow based on ideas explored in [SecondSkoll/generic-agentic-workflows](https://github.com/SecondSkoll/generic-agentic-workflows). It uses OpenCode, OpenRouter, and GLM-5.2.