feat(outside-voice): add llm CLI fallback between Codex and Claude subagent

[diogolealassis]

What

When Codex is unavailable or errors, plan-review and second-opinion skills currently fall back to the Claude subagent. That fallback works, but loses cross-model independence — the subagent is the same model family as the primary review. This PR adds a middle step: try the llm CLI (datasette/llm) with a configured model before falling back to Claude.

New chain: Codex → llm → Claude subagent

When llm is installed and the configured model is registered (e.g. via llm install llm-xai for Grok), the outside voice goes through a genuinely different model family before falling through to Claude. When llm isn't available, behavior is unchanged.

Why this matters today

Codex CLI currently fails at startup for users whose ~/.codex/config.toml has an xai provider with wire_api = "chat" — which is the default per xAI's setup docs but deprecated per openai/codex#7782:

Error loading config.toml: `wire_api = "chat"` is no longer supported.
in `model_providers.xai.wire_api`

The error fires at CLI startup BEFORE the provider is selected, so calls targeted at OpenAI also fail. Users in this state get the Claude subagent fallback on every outside-voice step. They lose the cross-model value the skill is designed to deliver.

Hit twice in a single 2026-05-20 session on /plan-eng-review and /plan-ceo-review. The fix on the user side is one line (wire_api = "responses"), but for users who never had OpenAI auth in the first place (they use Codex/llm because of Grok/Anthropic preference), restoring Codex doesn't actually help — they need a non-Codex cross-model path.

How

Two functions in scripts/resolvers/review.ts produce the outside-voice section that gets templated into 4 skill files:

• generateCodexPlanReview — used by /plan-*-review skills
• generateCodexSecondOpinion — used by /office-hours

Both now produce a Codex → llm → Claude subagent chain. The llm step is gated on:

1. New outside_voice_llm_model config key (default grok-4-fast)
2. command -v llm succeeds
3. llm models lists the configured model

If any check fails, the chain falls through cleanly to the existing Claude subagent path. No behavior change for users who don't have llm installed.

The default grok-4-fast rather than grok-4-latest is deliberate — the latter's reasoning mode can hang 2-5 minutes on big prompts (the llm stream is empty during hidden reasoning, looks like a hang). grok-4-fast is the right shape for routine outside-voice work.

Configuration

gstack-config set outside_voice_llm_model grok-4-fast    # default
gstack-config set outside_voice_llm_model ""             # disable, skip llm step
gstack-config set outside_voice_llm_model gpt-4o         # any model `llm models` lists

Empty string skips the llm step entirely (Codex → subagent direct).

Files changed

bin/gstack-config                |  12 ++++
office-hours/SKILL.md            |  64 +++++++++++++++++++++--
plan-ceo-review/SKILL.md         |  47 +++++++++++++++++-
plan-devex-review/SKILL.md       |  47 +++++++++++++++++-
plan-eng-review/SKILL.md         |  47 +++++++++++++++++-
scripts/resolvers/review.ts      | 111 ++++++++++++++++++++++++++++++++++++++--
6 files changed, 314 insertions(+), 14 deletions(-)

The SKILL.md changes are mechanically regenerated by bun run gen:skill-docs — the only hand-edits are in bin/gstack-config and scripts/resolvers/review.ts.

Verification

• bun run scripts/skill-check.ts passes, all freshness checks green
• Spot-checked generated SKILL.md files — llm fallback section present, bash blocks syntactically correct, outside_voice_llm_model referenced consistently
• Manually traced the 4-state matrix (codex/llm available × works/errors) through the generated bash to confirm fallthrough behavior
• Sequencing fix in generateCodexSecondOpinion: $CODEX_PROMPT_FILE deletion moved from "after Codex" to "after chain end" since llm step reuses the prompt file
• Did NOT test end-to-end against a real llm invocation with my Grok key — the resolver functions produce templated prose instructions for the agent, not executable code per se. Happy to do this if you want a smoke-test screenshot.

What this doesn't do

• Doesn't change generateAdversarialStep (the /ship + /review adversarial flow). Different shape (always-on, not configurable). Could benefit from the same fallback but out of scope for this PR; happy to do as a follow-up.
• Doesn't bump VERSION or add a CHANGELOG.md entry — leaving those to your release workflow.
• Doesn't add a first-run prompt for outside_voice_llm_model via AskUserQuestion. The default grok-4-fast is sensible enough that most users won't need to touch it. If you want a first-run prompt I can add it.
• Doesn't add tests. The resolver functions are pure string-returning, and the generated bash is exercised at agent-runtime not at gstack-build-time. Open to suggestions on how you'd want this tested.

Generated by

A real /plan-eng-review + outside-voice flow on a downstream user project (diogolealassis/gws-platform) hit the Codex failure twice in one session and surfaced the gap. Existing project memory codex-cli-broken-for-xai and the newly logged codex-cli-global-config-blocks-all-providers document the trigger. This PR closes the loop on the gstack side.

Happy to break this into smaller commits or split out the generateCodexSecondOpinion cleanup-sequencing fix if you'd prefer.

🤖 Generated with Claude Code

[jbetala7]

One config-surface gap: this adds outside_voice_llm_model to lookup_default() and the header, but not to the list / defaults key loops in bin/gstack-config.

That means gstack-config get outside_voice_llm_model returns the new default, while gstack-config defaults and the active-values section of gstack-config list omit the key. Users then cannot discover or audit the new fallback model through the normal config surface, and the generated docs drift from the CLI output.

Please add outside_voice_llm_model to both loops and pin it with a focused config regression, similar to the explain_level default/list/defaults issue tracked by #1607/#1608.

[ljodea]

Heads-up: a fresh take on Grok outside-voice landed in #2016.

What #2016 adds: a standalone /grok skill mirroring /codex (review / challenge / consult via Grok Build CLI + gstack-grok-probe). That covers explicit, on-demand second opinions — not the automatic plan-review chain this PR targets.

What #2016 does not do: wire Grok into generateCodexPlanReview / generateCodexSecondOpinion. When Codex fails in /plan-*-review or /office-hours, the chain is still Codex → Claude subagent on current main.

Suggested path forward:

1. Merge v1.59.1.0 feat: add /grok skill for xAI second opinion from Claude Code #2016 first (Grok Build CLI is the right integration surface post–Grok Build; datasette llm was the pre-Build workaround).
2. Follow-up PR that revives this PR's Codex → Grok → Claude subagent chain using grok --prompt-file + gstack-grok-probe patterns from v1.59.1.0 feat: add /grok skill for xAI second opinion from Claude Code #2016, instead of llm -m grok-4-fast.
3. Config key could be outside_voice_grok (enabled/disabled) rather than outside_voice_llm_model, or keep a model-effort knob if Grok CLI exposes one.

The @jbetala7 config-surface feedback (add key to list / defaults loops, not just lookup_default) still applies to whichever config key ships.

Happy to help port the fallback chain once #2016 lands — the prompt-reuse + $CODEX_PROMPT_FILE cleanup sequencing fix in this PR is still valuable.