diff --git a/README.md b/README.md index 4f65ea4..98f8016 100644 --- a/README.md +++ b/README.md @@ -151,12 +151,12 @@ Use `@main` while the project is pre-release. After the first stable release, pi | Mode | Provider examples | Credentials | Best for | |---|---|---|---| -| Local CLI | `codex-cli`, `claude-cli` | Existing CLI login | Local development or self-hosted runners | +| Local CLI | `codex-cli`, `claude-cli`, `grok-cli`, `opencode-cli` | Existing CLI login | Local development or self-hosted runners | | Hosted API | `openai`, `anthropic`, `gemini`, `mistral`, `groq` | Provider API key | Managed CI | | Local model | `ollama` | Usually none | Privacy and predictable cost | | Gateway | `openrouter` or a custom `--base-url` | Gateway-specific | Central routing and policy | -Provider names other than the two local CLIs resolve to factories exported by [`@agentskit/adapters`](https://www.npmjs.com/package/@agentskit/adapters). Run `npx --yes github:AgentsKit-io/code-review-cli --list-providers` for common choices. +`grok` is the xAI API provider; `grok-cli` is the separate Grok Build CLI entry. `opencode-cli` is the OpenCode CLI entry. API providers are discovered from factories exported by [`@agentskit/adapters`](https://www.npmjs.com/package/@agentskit/adapters). Run `npx --yes github:AgentsKit-io/code-review-cli --list-providers` to see IDs, support levels, transports, and model requirements. Credentials resolve in this order: @@ -224,6 +224,8 @@ Run these commands from the repository you want to review: |---|---|---|---| | `codex-cli` | Codex CLI logged in | Optional | `npx --yes github:AgentsKit-io/code-review-cli --provider codex-cli` | | `claude-cli` | Claude CLI logged in | Optional | `npx --yes github:AgentsKit-io/code-review-cli --provider claude-cli` | +| `grok-cli` | Grok Build CLI; experimental | Optional | `... --provider grok-cli` | +| `opencode-cli` | OpenCode CLI; experimental | Optional | `... --provider opencode-cli` | | `openai` | `OPENAI_API_KEY` | Required | `... --provider openai --model gpt-4o` | | `anthropic` | `ANTHROPIC_API_KEY` | Required | `... --provider anthropic --model ` | | `gemini` | `GEMINI_API_KEY` | Required | `... --provider gemini --model ` | @@ -257,10 +259,23 @@ In shortened examples, replace `...` with `npx --yes github:AgentsKit-io/code-re | `--no-fail` | Keep findings advisory | | `--conventions ` | Inject project conventions | | `--api` | Back-compatible alias for `--provider anthropic` | +| `doctor --provider ` | Offline provider diagnostics; no model request | +| `doctor --live` | Explicit provider smoke-test mode | +| `doctor --json` | Stable machine-readable diagnostics | +| `--mode ` | `isolated` (default) or explicit local-only `trusted-local` | | `--help` | Full command help | When no conventions path is supplied, the CLI looks for `CONVENTIONS.md`, `CONTRIBUTING.md`, `.cursorrules`, or `AGENTS.md`. +### Doctor + +Run `doctor` before a review to check a registered provider’s executable, version, transport, model requirement, configuration mode, and credential presence. It is offline by default: API credentials are checked only for presence and values are never printed; local CLI login is represented as login-managed until a provider-specific live check is available. Unknown local CLI versions warn locally and fail when `CI=true`. Exit `0` means healthy, `1` means a failed diagnostic, and `2` means invalid CLI usage. + +```sh +npx --yes github:AgentsKit-io/code-review-cli doctor --provider codex-cli +npx --yes github:AgentsKit-io/code-review-cli doctor --provider openai --model gpt-4o --json +``` + ## Cost and privacy A full review runs seven lenses across selected files and then verifies candidate findings. Control usage with `--max-files`, `--votes`, `--concurrency`, paths, and workflow triggers. For sensitive code, use a local model or an approved private gateway; provider data policies still apply to hosted APIs. diff --git a/docs/OPERATIONS.md b/docs/OPERATIONS.md index 7d9de2b..bb1baf0 100644 --- a/docs/OPERATIONS.md +++ b/docs/OPERATIONS.md @@ -6,7 +6,7 @@ This guide is the repository-native reference for running AgentsKit Code Review | Provider class | Examples | Secret or login | Network boundary | |---|---|---|---| -| Logged-in local CLI | `codex-cli`, `claude-cli` | Existing local login | Provider CLI policy | +| Logged-in local CLI | `codex-cli`, `claude-cli`, `grok-cli`, `opencode-cli` | Existing local login | Provider CLI policy | | Hosted API | `openai`, `anthropic`, `gemini`, `mistral`, `groq` | Repository/org secret | Selected code reaches provider | | Local model | `ollama` | Usually none | Host or runner network only | | Gateway | `openrouter`, custom `--base-url` | Gateway secret | Gateway policy and routing | @@ -15,6 +15,19 @@ Credential precedence is `--api-key`, `LLM_API_KEY`, then `_API_KEY`. Do not run hosted review on code whose policy forbids external processing. A local model reduces external disclosure but does not remove the need to secure the runner, logs, cache, and generated SARIF. +## Provider registry and doctor + +Provider IDs are versioned registry entries. `grok` is the xAI API adapter, while `grok-cli` is the experimental Grok Build CLI; `opencode-cli` is the experimental OpenCode CLI. `--list-providers` prints registry metadata and dynamically discovered API factories, including each support level (`stable`, `experimental`, or `unsupported`), transport, and model requirement. + +Use the offline doctor before execution: + +```sh +npx --yes github:AgentsKit-io/code-review-cli doctor --provider codex-cli +npx --yes github:AgentsKit-io/code-review-cli doctor --provider openai --model gpt-4o --json +``` + +It checks the named executable and version, transport, model requirement, configuration mode, and credential presence without making a model request. API keys are represented only as `configured` or `missing`; they are never printed. Local CLI credentials are represented as login-managed because login storage is provider-specific. `doctor --live` is the explicit provider smoke-test path. Unknown local CLI versions warn during local runs and fail in CI. Doctor exits `0` when checks pass, `1` when a provider check fails, and `2` for invalid usage. + ## pre-commit integration The root `.pre-commit-hooks.yaml` exposes `agentskit-review` as a Node hook. It uses `pass_filenames: false` because the CLI reviews a Git diff, explicit paths, a pull request, or stdin rather than interpreting positional filenames. It is confined to the `manual` stage by default so cloning the hook does not silently add model calls to every commit. diff --git a/llms-full.txt b/llms-full.txt index e6a7017..0b5c137 100644 --- a/llms-full.txt +++ b/llms-full.txt @@ -169,12 +169,12 @@ Use `@main` while the project is pre-release. After the first stable release, pi | Mode | Provider examples | Credentials | Best for | |---|---|---|---| -| Local CLI | `codex-cli`, `claude-cli` | Existing CLI login | Local development or self-hosted runners | +| Local CLI | `codex-cli`, `claude-cli`, `grok-cli`, `opencode-cli` | Existing CLI login | Local development or self-hosted runners | | Hosted API | `openai`, `anthropic`, `gemini`, `mistral`, `groq` | Provider API key | Managed CI | | Local model | `ollama` | Usually none | Privacy and predictable cost | | Gateway | `openrouter` or a custom `--base-url` | Gateway-specific | Central routing and policy | -Provider names other than the two local CLIs resolve to factories exported by [`@agentskit/adapters`](https://www.npmjs.com/package/@agentskit/adapters). Run `npx --yes github:AgentsKit-io/code-review-cli --list-providers` for common choices. +`grok` is the xAI API provider; `grok-cli` is the separate Grok Build CLI entry. `opencode-cli` is the OpenCode CLI entry. API providers are discovered from factories exported by [`@agentskit/adapters`](https://www.npmjs.com/package/@agentskit/adapters). Run `npx --yes github:AgentsKit-io/code-review-cli --list-providers` to see IDs, support levels, transports, and model requirements. Credentials resolve in this order: @@ -242,6 +242,8 @@ Run these commands from the repository you want to review: |---|---|---|---| | `codex-cli` | Codex CLI logged in | Optional | `npx --yes github:AgentsKit-io/code-review-cli --provider codex-cli` | | `claude-cli` | Claude CLI logged in | Optional | `npx --yes github:AgentsKit-io/code-review-cli --provider claude-cli` | +| `grok-cli` | Grok Build CLI; experimental | Optional | `... --provider grok-cli` | +| `opencode-cli` | OpenCode CLI; experimental | Optional | `... --provider opencode-cli` | | `openai` | `OPENAI_API_KEY` | Required | `... --provider openai --model gpt-4o` | | `anthropic` | `ANTHROPIC_API_KEY` | Required | `... --provider anthropic --model ` | | `gemini` | `GEMINI_API_KEY` | Required | `... --provider gemini --model ` | @@ -275,10 +277,23 @@ In shortened examples, replace `...` with `npx --yes github:AgentsKit-io/code-re | `--no-fail` | Keep findings advisory | | `--conventions ` | Inject project conventions | | `--api` | Back-compatible alias for `--provider anthropic` | +| `doctor --provider ` | Offline provider diagnostics; no model request | +| `doctor --live` | Explicit provider smoke-test mode | +| `doctor --json` | Stable machine-readable diagnostics | +| `--mode ` | `isolated` (default) or explicit local-only `trusted-local` | | `--help` | Full command help | When no conventions path is supplied, the CLI looks for `CONVENTIONS.md`, `CONTRIBUTING.md`, `.cursorrules`, or `AGENTS.md`. +### Doctor + +Run `doctor` before a review to check a registered provider’s executable, version, transport, model requirement, configuration mode, and credential presence. It is offline by default: API credentials are checked only for presence and values are never printed; local CLI login is represented as login-managed until a provider-specific live check is available. Unknown local CLI versions warn locally and fail when `CI=true`. Exit `0` means healthy, `1` means a failed diagnostic, and `2` means invalid CLI usage. + +```sh +npx --yes github:AgentsKit-io/code-review-cli doctor --provider codex-cli +npx --yes github:AgentsKit-io/code-review-cli doctor --provider openai --model gpt-4o --json +``` + ## Cost and privacy A full review runs seven lenses across selected files and then verifies candidate findings. Control usage with `--max-files`, `--votes`, `--concurrency`, paths, and workflow triggers. For sensitive code, use a local model or an approved private gateway; provider data policies still apply to hosted APIs. @@ -352,7 +367,7 @@ This guide is the repository-native reference for running AgentsKit Code Review | Provider class | Examples | Secret or login | Network boundary | |---|---|---|---| -| Logged-in local CLI | `codex-cli`, `claude-cli` | Existing local login | Provider CLI policy | +| Logged-in local CLI | `codex-cli`, `claude-cli`, `grok-cli`, `opencode-cli` | Existing local login | Provider CLI policy | | Hosted API | `openai`, `anthropic`, `gemini`, `mistral`, `groq` | Repository/org secret | Selected code reaches provider | | Local model | `ollama` | Usually none | Host or runner network only | | Gateway | `openrouter`, custom `--base-url` | Gateway secret | Gateway policy and routing | @@ -361,6 +376,19 @@ Credential precedence is `--api-key`, `LLM_API_KEY`, then `_API_KEY`. Do not run hosted review on code whose policy forbids external processing. A local model reduces external disclosure but does not remove the need to secure the runner, logs, cache, and generated SARIF. +## Provider registry and doctor + +Provider IDs are versioned registry entries. `grok` is the xAI API adapter, while `grok-cli` is the experimental Grok Build CLI; `opencode-cli` is the experimental OpenCode CLI. `--list-providers` prints registry metadata and dynamically discovered API factories, including each support level (`stable`, `experimental`, or `unsupported`), transport, and model requirement. + +Use the offline doctor before execution: + +```sh +npx --yes github:AgentsKit-io/code-review-cli doctor --provider codex-cli +npx --yes github:AgentsKit-io/code-review-cli doctor --provider openai --model gpt-4o --json +``` + +It checks the named executable and version, transport, model requirement, configuration mode, and credential presence without making a model request. API keys are represented only as `configured` or `missing`; they are never printed. Local CLI credentials are represented as login-managed because login storage is provider-specific. `doctor --live` is the explicit provider smoke-test path. Unknown local CLI versions warn during local runs and fail in CI. Doctor exits `0` when checks pass, `1` when a provider check fails, and `2` for invalid usage. + ## pre-commit integration The root `.pre-commit-hooks.yaml` exposes `agentskit-review` as a Node hook. It uses `pass_filenames: false` because the CLI reviews a Git diff, explicit paths, a pull request, or stdin rather than interpreting positional filenames. It is confined to the `manual` stage by default so cloning the hook does not silently add model calls to every commit. diff --git a/readme-standard-v1.json b/readme-standard-v1.json index d462452..f2e3d03 100644 --- a/readme-standard-v1.json +++ b/readme-standard-v1.json @@ -221,7 +221,7 @@ "docs/OPERATIONS.md", "test/cli-smoke.test.mjs" ], - "sourceHash": "sha256:d6f8cf7be73f4b674d2b4cfb3da901471107d28baa91cc14031df19753857f6e" + "sourceHash": "sha256:9e43fecef7502fabffa1bc976a9febebdf14d7522431cd18e8f7bb4553078e68" }, "exceptions": [] } diff --git a/src/cli.ts b/src/cli.ts index be78b01..8f10414 100644 --- a/src/cli.ts +++ b/src/cli.ts @@ -13,7 +13,6 @@ * * Exit code: 1 when a finding at/above --block survives (unless --no-fail) — wire to CI. */ -import * as adapters from '@agentskit/adapters' import type { AdapterFactory } from '@agentskit/core' import { createProgressObserver } from '@agentskit/ink' import { readFileSync } from 'node:fs' @@ -23,6 +22,7 @@ import { claudeCode } from './claude-code-adapter.js' import { codexCli } from './codex-adapter.js' import { ollamaReview } from './ollama-adapter.js' import type { SourceConfig } from '../agents/code-review/sources.js' +import { diagnoseProvider, factoryFor, providerEntry, providerRegistry, resolveProviderId, type DoctorReport, type ProviderEntry } from './provider-registry.js' const HELP = `AgentsKit Code Review — deep, low-noise review with your model @@ -62,14 +62,20 @@ Provider options: --model Model id (required for API/local-server providers) --api-key Or use LLM_API_KEY / _API_KEY --base-url Custom endpoint or local gateway + --transport Provider transport (doctor validates it) --api Back-compatible alias for --provider anthropic +Diagnostics: + doctor Check one provider without making a model request + --live Explicitly enable a provider smoke test for doctor + --json Emit machine-readable doctor output + --mode Configuration mode: isolated or trusted-local + --ci Apply CI version and trust checks + --help Show this help --list-providers List common providers ` -const PROVIDERS = 'codex-cli\nclaude-cli\nanthropic\nopenai\ngemini\ngrok\nollama\ndeepseek\nmistral\ngroq\nopenrouter\ntogether' - function flag(name: string): string | undefined { const i = process.argv.indexOf(`--${name}`) return i >= 0 ? process.argv[i + 1] : undefined @@ -111,10 +117,15 @@ async function main() { return } if (has('list-providers')) { - console.log(PROVIDERS) + console.log(providerRegistry().map(formatProvider).join('\n')) + return + } + if (process.argv.includes('doctor') || has('doctor')) { + await runDoctor() return } const source = await resolveSource() + await preflightProvider() const adapter = buildAdapter() const reporters: Reporter[] = [markdownReporter()] @@ -153,8 +164,9 @@ async function main() { * `{ apiKey, model, baseUrl? }`. `--api` is a back-compat alias for `--provider anthropic`. */ function buildAdapter(): AdapterFactory { - const provider = flag('provider') ?? (has('api') ? 'anthropic' : undefined) - if (!provider) throw new Error('choose a provider with --provider (run --list-providers for common options)') + const requestedProvider = flag('provider') ?? (has('api') ? 'anthropic' : undefined) + const provider = requestedProvider && resolveProviderId(requestedProvider) + if (!provider) throw new Error(requestedProvider ? `unknown --provider "${requestedProvider}" (run --list-providers for common options)` : 'choose a provider with --provider (run --list-providers for common options)') const model = flag('model') ?? (has('api') ? 'claude-opus-4-8' : undefined) if (provider === 'claude-cli') return claudeCode({ model }) if (provider === 'codex-cli') return codexCli({ model }) @@ -163,14 +175,60 @@ function buildAdapter(): AdapterFactory { return ollamaReview({ model, ...(flag('base-url') ? { baseUrl: flag('base-url') } : {}) }) } - const make = (adapters as Record)[provider] - if (typeof make !== 'function') { - throw new Error(`unknown --provider "${provider}". Use "claude-cli" or any @agentskit/adapters factory (anthropic, openai, gemini, grok, ollama, deepseek, mistral, groq, openrouter, together, …).`) - } + const entry = providerEntry(provider) + const make = entry && factoryFor(entry) + if (!entry || !make) throw new Error(`provider "${provider}" is registered but has no local adapter yet`) if (!model) throw new Error(`--model is required for provider "${provider}"`) const apiKey = flag('api-key') ?? process.env.LLM_API_KEY ?? process.env[`${provider.toUpperCase()}_API_KEY`] ?? '' const baseUrl = flag('base-url') - return (make as (c: Record) => AdapterFactory)({ apiKey, model, ...(baseUrl ? { baseUrl } : {}) }) + return make({ apiKey, model, ...(baseUrl ? { baseUrl } : {}) }) +} + +async function preflightProvider(): Promise { + const requested = flag('provider') ?? (has('api') ? 'anthropic' : undefined) + const id = requested && resolveProviderId(requested) + const entry = id && providerEntry(id) + if (!entry || entry.kind === 'api') return + const report = await diagnoseProvider({ + provider: entry.id, + model: flag('model'), + transport: flag('transport'), + mode: flag('mode'), + apiKey: flag('api-key'), + ci: has('ci'), + }) + const version = report.checks.find((check) => check.name === 'version') + if (version?.status === 'warn') console.error(`warning: ${entry.id} ${version.detail}`) + if (!report.ok) throw new Error(`${entry.id} provider preflight failed: ${report.checks.filter((check) => check.status === 'fail').map((check) => `${check.name}: ${check.detail}`).join('; ')}`) +} + +async function runDoctor(): Promise { + const requested = flag('provider') ?? (has('api') ? 'anthropic' : undefined) + if (!requested) throw new Error('doctor needs --provider ') + const report = await diagnoseProvider({ + provider: requested, + model: flag('model'), + transport: flag('transport'), + mode: flag('mode'), + live: has('live'), + ci: has('ci'), + apiKey: flag('api-key'), + }) + if (has('json')) console.log(JSON.stringify(report)) + else console.log(formatDoctor(report)) + if (!report.ok) process.exitCode = report.checks.some((check) => check.name === 'provider' && check.status === 'fail') ? 2 : 1 +} + +function formatProvider(entry: ProviderEntry): string { + const aliases = entry.aliases.length ? `\taliases=${entry.aliases.join(',')}` : '' + return `${entry.id}\tkind=${entry.kind}\tsupport=${entry.support}\ttransport=${entry.defaultTransport}\tmodel=${entry.model}${aliases}` +} + +function formatDoctor(report: DoctorReport): string { + const lines = [`Provider doctor: ${report.provider} (${report.support})`] + for (const check of report.checks) lines.push(` ${check.name}: ${check.status.toUpperCase()} — ${check.detail}`) + lines.push(`Result: ${report.ok ? 'PASS' : 'FAIL'}`) + return lines.join('\n') } /** Best-effort: feed a conventions doc to every lens if one exists. */ diff --git a/src/local-cli-process.ts b/src/local-cli-process.ts index 4a8b8fa..09cf3fb 100644 --- a/src/local-cli-process.ts +++ b/src/local-cli-process.ts @@ -22,9 +22,9 @@ function terminateProcessTree(child: ChildProcess): void { export function runLocalCli( command: string, args: string[], - options: { readonly cwd?: string } = {}, + options: { readonly cwd?: string; readonly timeoutMs?: number } = {}, ): Promise<{ readonly stdout: string; readonly stderr: string }> { - const timeoutMs = localCliTimeoutMs() + const timeoutMs = options.timeoutMs ?? localCliTimeoutMs() return new Promise((resolve, reject) => { const child = spawn(command, args, { diff --git a/src/provider-registry.ts b/src/provider-registry.ts new file mode 100644 index 0000000..54749d9 --- /dev/null +++ b/src/provider-registry.ts @@ -0,0 +1,211 @@ +import * as adapters from '@agentskit/adapters' +import type { AdapterFactory } from '@agentskit/core' +import { runLocalCli } from './local-cli-process.js' + +export const PROVIDER_REGISTRY_VERSION = 1 as const + +export type ProviderKind = 'api' | 'cli' | 'local-server' +export type SupportLevel = 'stable' | 'experimental' | 'unsupported' +export type Transport = 'api' | 'acp' | 'headless' | 'http' +export type ModelRequirement = 'required' | 'optional' | 'none' + +export interface ProviderEntry { + readonly id: string + readonly aliases: readonly string[] + readonly kind: ProviderKind + readonly support: SupportLevel + readonly description: string + readonly factoryName?: string + readonly executable?: string + readonly versionArgs?: readonly string[] + readonly minimumVersion?: string + readonly transports: readonly Transport[] + readonly defaultTransport: Transport + readonly model: ModelRequirement + readonly dataBoundary: 'local' | 'remote' | 'unknown' + readonly credentialEnv: readonly string[] + readonly credentialMode: 'api-key' | 'login' | 'none' + readonly capabilities: Readonly> +} + +const API_METADATA: Record> = { + anthropic: api('Anthropic', ['ANTHROPIC_API_KEY']), + openai: api('OpenAI', ['OPENAI_API_KEY']), + gemini: api('Google Gemini', ['GEMINI_API_KEY']), + grok: api('xAI Grok API', ['GROK_API_KEY']), + deepseek: api('DeepSeek', ['DEEPSEEK_API_KEY']), + mistral: api('Mistral', ['MISTRAL_API_KEY']), + groq: api('Groq', ['GROQ_API_KEY']), + openrouter: api('OpenRouter', ['OPENROUTER_API_KEY']), + together: api('Together AI', ['TOGETHER_API_KEY']), +} + +const LOCAL_PROVIDERS: readonly ProviderEntry[] = [ + cli('codex-cli', 'OpenAI Codex CLI', 'codex', 'stable'), + cli('claude-cli', 'Claude Code CLI', 'claude', 'stable'), + { + ...cli('grok-cli', 'Grok Build CLI', 'grok', 'experimental'), + transports: ['acp', 'headless'], + defaultTransport: 'acp', + }, + { + ...cli('opencode-cli', 'OpenCode CLI', 'opencode', 'experimental'), + transports: ['acp', 'headless'], + defaultTransport: 'acp', + }, + localServer('ollama', 'Ollama local model server', 'stable'), +] + +const FACTORY_EXCLUSIONS = /(?:Adapter|Embedder)$|^(?:bail|chunkText|create|fetch|inMemorySink|mock|recording|replay|simulate|langchain|langgraph)/i +const KNOWN_API_SUPPORT = new Set(Object.keys(API_METADATA)) + +function api(description: string, credentialEnv: readonly string[]): Omit { + return { + kind: 'api', support: 'stable', description, transports: ['api'], defaultTransport: 'api', model: 'required', + dataBoundary: 'remote', credentialEnv, credentialMode: 'api-key', + capabilities: { streaming: true, tools: true, structuredOutput: true }, + } +} + +function cli(id: string, description: string, executable: string, support: SupportLevel): ProviderEntry { + return { + id, aliases: [], kind: 'cli', support, description, executable, versionArgs: ['--version'], minimumVersion: '0.1.0', + transports: ['headless'], defaultTransport: 'headless', model: 'optional', dataBoundary: 'local', + credentialEnv: [], credentialMode: 'login', capabilities: { streaming: false, tools: true, structuredOutput: true }, + } +} + +function localServer(id: string, description: string, support: SupportLevel): ProviderEntry { + return { + id, aliases: [], kind: 'local-server', support, description, + transports: ['http'], defaultTransport: 'http', model: 'required', dataBoundary: 'local', + credentialEnv: [], credentialMode: 'none', capabilities: { streaming: true, tools: true, structuredOutput: true }, + } +} + +export function discoverApiFactories(source: Record = adapters): string[] { + return Object.keys(source).filter((name) => typeof source[name] === 'function' && !FACTORY_EXCLUSIONS.test(name)).sort() +} + +export function providerRegistry(source: Record = adapters): ProviderEntry[] { + const entries = new Map() + for (const entry of LOCAL_PROVIDERS) entries.set(entry.id, entry) + for (const id of discoverApiFactories(source)) { + if (entries.has(id)) continue + const metadata = API_METADATA[id] ?? { + ...api(`${id} API adapter`, [`${id.toUpperCase()}_API_KEY`]), + support: KNOWN_API_SUPPORT.has(id) ? 'stable' : 'experimental', + } + entries.set(id, { id, aliases: id === 'anthropic' ? ['api'] : [], factoryName: id, ...metadata }) + } + for (const [id, metadata] of Object.entries(API_METADATA)) { + if (!entries.has(id) && typeof source[id] === 'function') entries.set(id, { id, aliases: id === 'anthropic' ? ['api'] : [], factoryName: id, ...metadata }) + } + return [...entries.values()].sort((a, b) => a.id.localeCompare(b.id)) +} + +export function resolveProviderId(id: string, entries: readonly ProviderEntry[] = providerRegistry()): string | undefined { + const normalized = id.trim().toLowerCase() + return entries.find((entry) => entry.id.toLowerCase() === normalized || entry.aliases.some((alias) => alias.toLowerCase() === normalized))?.id +} + +export function providerEntry(id: string, entries: readonly ProviderEntry[] = providerRegistry()): ProviderEntry | undefined { + const resolved = resolveProviderId(id, entries) + return entries.find((entry) => entry.id === resolved) +} + +export interface DoctorCheck { + readonly name: string + readonly status: 'pass' | 'warn' | 'fail' | 'skip' + readonly detail: string +} + +export interface DoctorReport { + readonly registryVersion: 1 + readonly schemaVersion: 1 + readonly provider: string + readonly support: SupportLevel + readonly live: boolean + readonly ok: boolean + readonly checks: readonly DoctorCheck[] +} + +export interface DoctorOptions { + readonly provider: string + readonly model?: string + readonly transport?: string + readonly mode?: string + readonly live?: boolean + readonly ci?: boolean + readonly apiKey?: string + readonly env?: NodeJS.ProcessEnv + readonly entries?: readonly ProviderEntry[] +} + +const versionPattern = /(?:^|[^\d])v?(\d+)\.(\d+)\.(\d+)(?:[-+][0-9A-Za-z.-]+)?/m +const VERSION_CHECK_TIMEOUT_MS = 5_000 + +export function parseVersion(output: string): string | undefined { + const match = output.match(versionPattern) + return match ? `${match[1]}.${match[2]}.${match[3]}` : undefined +} + +function compareVersions(left: string, right: string): number { + const a = left.split('.').map(Number) + const b = right.split('.').map(Number) + return a[0]! - b[0]! || a[1]! - b[1]! || a[2]! - b[2]! +} + +function checkVersion(entry: ProviderEntry, output: string, ci: boolean): DoctorCheck { + const version = parseVersion(output) + if (!version) return { name: 'version', status: ci ? 'fail' : 'warn', detail: ci ? 'unknown version (CI requires a recognized version)' : 'unknown version (local run allowed)' } + if (entry.minimumVersion && compareVersions(version, entry.minimumVersion) < 0) return { name: 'version', status: 'fail', detail: `unsupported version ${version}` } + return { name: 'version', status: 'pass', detail: version } +} + +export async function diagnoseProvider(options: DoctorOptions): Promise { + const entries = options.entries ?? providerRegistry() + const entry = providerEntry(options.provider, entries) + if (!entry) return { registryVersion: PROVIDER_REGISTRY_VERSION, schemaVersion: 1, provider: options.provider, support: 'unsupported', live: Boolean(options.live), ok: false, checks: [{ name: 'provider', status: 'fail', detail: 'unsupported provider' }] } + + const env = options.env ?? process.env + const ci = Boolean(options.ci || env.CI === 'true' || env.CI === '1') + const checks: DoctorCheck[] = [ + { name: 'support', status: entry.support === 'experimental' ? 'warn' : entry.support === 'unsupported' ? 'fail' : 'pass', detail: entry.support }, + { name: 'transport', status: options.transport && !entry.transports.includes(options.transport as Transport) ? 'fail' : 'pass', detail: options.transport ?? entry.defaultTransport }, + { name: 'model', status: entry.model === 'required' && !options.model ? 'fail' : 'pass', detail: entry.model === 'required' ? (options.model ? 'configured' : 'required') : entry.model }, + ] + + const mode = options.mode ?? env.AGENTSKIT_REVIEW_MODE ?? 'isolated' + checks.push({ name: 'configuration', status: mode === 'trusted-local' && (ci || entry.kind === 'api') ? 'fail' : ['isolated', 'trusted-local'].includes(mode) ? 'pass' : 'fail', detail: mode }) + + const credentialPresent = Boolean(options.apiKey || env.LLM_API_KEY || entry.credentialEnv.some((name) => Boolean(env[name]))) + checks.push({ + name: 'credentials', + status: entry.credentialMode === 'api-key' ? (credentialPresent ? 'pass' : 'fail') : 'pass', + detail: entry.credentialMode === 'api-key' ? (credentialPresent ? 'configured' : 'missing') : entry.credentialMode === 'login' ? 'login-managed (not inspected offline)' : 'not required', + }) + + if (entry.executable) { + try { + const result = await runLocalCli(entry.executable, [...(entry.versionArgs ?? ['--version'])], { timeoutMs: VERSION_CHECK_TIMEOUT_MS }) + checks.push({ name: 'executable', status: 'pass', detail: entry.executable }) + checks.push(checkVersion(entry, `${result.stdout}\n${result.stderr}`, ci)) + } catch (error) { + const e = error as { code?: string; message?: string } + checks.push({ name: 'executable', status: 'fail', detail: e.code === 'ENOENT' ? 'not found' : e.code === 'ETIMEDOUT' ? 'timed out' : 'unavailable' }) + if (e.code === 'ETIMEDOUT') checks.push({ name: 'version', status: 'fail', detail: e.message?.startsWith(entry.executable) ? e.message : 'version check timed out' }) + } + } else { + checks.push({ name: 'executable', status: 'skip', detail: 'API provider' }) + checks.push({ name: 'version', status: 'skip', detail: 'API provider' }) + } + + if (options.live) checks.push({ name: 'live', status: 'skip', detail: 'provider smoke test is opt-in and provider-specific' }) + return { registryVersion: PROVIDER_REGISTRY_VERSION, schemaVersion: 1, provider: entry.id, support: entry.support, live: Boolean(options.live), ok: !checks.some((check) => check.status === 'fail'), checks } +} + +export function factoryFor(entry: ProviderEntry): ((config: Record) => AdapterFactory) | undefined { + const factory = entry.factoryName ? (adapters as Record)[entry.factoryName] : undefined + return typeof factory === 'function' ? factory as (config: Record) => AdapterFactory : undefined +} diff --git a/test/cli-smoke.test.mjs b/test/cli-smoke.test.mjs index 7e75f0a..828f012 100644 --- a/test/cli-smoke.test.mjs +++ b/test/cli-smoke.test.mjs @@ -142,4 +142,112 @@ test('the built CLI exposes provider and usage discovery without credentials', ( assert.equal(providers.status, 0, providers.stderr) assert.match(providers.stdout, /codex-cli/) assert.match(providers.stdout, /ollama/) + assert.match(providers.stdout, /grok-cli.*support=experimental/) + assert.match(providers.stdout, /opencode-cli.*support=experimental/) + assert.match(providers.stdout, /grok\tkind=api/) +}) + +test('doctor reports a healthy local provider as stable JSON without secrets', () => { + const fixtureBin = join(root, 'test/fixtures/bin') + const run = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'codex-cli', '--json', + ], { + cwd: root, + encoding: 'utf8', + env: { ...process.env, PATH: `${fixtureBin}:${process.env.PATH ?? ''}` }, + }) + + assert.equal(run.status, 0, run.stderr) + const report = JSON.parse(run.stdout) + assert.equal(report.schemaVersion, 1) + assert.equal(report.provider, 'codex-cli') + assert.equal(report.support, 'stable') + assert.equal(report.ok, true) + assert.equal(report.checks.find(check => check.name === 'version').status, 'pass') +}) + +test('doctor catches missing binaries and unsupported versions offline', () => { + const missing = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'opencode-cli', '--json', + ], { + cwd: root, + encoding: 'utf8', + env: { ...process.env, PATH: '/usr/bin:/bin' }, + }) + assert.equal(missing.status, 1) + const missingReport = JSON.parse(missing.stdout) + assert.equal(missingReport.checks.find(check => check.name === 'executable').detail, 'not found') + + const fixtureBin = join(root, 'test/fixtures/bin') + const oldVersion = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'codex-cli', '--json', + ], { + cwd: root, + encoding: 'utf8', + env: { ...process.env, CODEX_FIXTURE_VERSION: 'codex-cli 0.0.1', PATH: `${fixtureBin}:${process.env.PATH ?? ''}` }, + }) + assert.equal(oldVersion.status, 1) + assert.match(oldVersion.stdout, /unsupported version 0\.0\.1/) +}) + +test('unknown local CLI versions warn locally and fail in CI', () => { + const fixtureBin = join(root, 'test/fixtures/bin') + const args = ['dist/src/cli.js', 'doctor', '--provider', 'codex-cli', '--json'] + const local = spawnSync(process.execPath, args, { + cwd: root, encoding: 'utf8', + env: { ...process.env, CI: '', CODEX_FIXTURE_VERSION: 'codex development build', PATH: `${fixtureBin}:${process.env.PATH ?? ''}` }, + }) + assert.equal(local.status, 0) + assert.equal(JSON.parse(local.stdout).checks.find(check => check.name === 'version').status, 'warn') + + const ci = spawnSync(process.execPath, args, { + cwd: root, encoding: 'utf8', + env: { ...process.env, CI: 'true', CODEX_FIXTURE_VERSION: 'codex development build', PATH: `${fixtureBin}:${process.env.PATH ?? ''}` }, + }) + assert.equal(ci.status, 1) + assert.match(ci.stdout, /unknown version/) +}) + +test('CI rejects an unknown local CLI version before model execution', () => { + const fixtureBin = join(root, 'test/fixtures/bin') + const run = spawnSync(process.execPath, [ + 'dist/src/cli.js', '--provider', 'codex-cli', '--stdin', '--no-fail', + ], { + cwd: root, + input: 'export const answer = 42\n', + encoding: 'utf8', + env: { ...process.env, CI: 'true', CODEX_FIXTURE_VERSION: 'codex development build', PATH: `${fixtureBin}:${process.env.PATH ?? ''}` }, + }) + assert.equal(run.status, 2) + assert.match(run.stderr, /unknown version/) + assert.doesNotMatch(run.stdout, /Code review —/) +}) + +test('doctor reports missing API credentials without echoing values', () => { + const env = { ...process.env } + for (const key of Object.keys(env)) if (key.endsWith('_API_KEY') || key === 'LLM_API_KEY') delete env[key] + const secret = 'never-echo-this-key' + const run = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'openai', '--model', 'fixture-model', '--json', + ], { cwd: root, encoding: 'utf8', env: { ...env, OPENAI_API_KEY: secret } }) + assert.equal(run.status, 0) + assert.doesNotMatch(`${run.stdout}\n${run.stderr}`, new RegExp(secret)) + assert.equal(JSON.parse(run.stdout).checks.find(check => check.name === 'credentials').detail, 'configured') + + const missing = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'openai', '--model', 'fixture-model', '--json', + ], { cwd: root, encoding: 'utf8', env }) + assert.equal(missing.status, 1) + assert.equal(JSON.parse(missing.stdout).checks.find(check => check.name === 'credentials').detail, 'missing') +}) + +test('doctor treats an unknown provider as invalid CLI usage', () => { + const run = spawnSync(process.execPath, [ + 'dist/src/cli.js', 'doctor', '--provider', 'not-a-provider', '--json', + ], { cwd: root, encoding: 'utf8' }) + + assert.equal(run.status, 2) + const report = JSON.parse(run.stdout) + assert.equal(report.ok, false) + assert.equal(report.checks[0].detail, 'unsupported provider') }) diff --git a/test/fixtures/bin/codex b/test/fixtures/bin/codex index a0b9225..a0189df 100755 --- a/test/fixtures/bin/codex +++ b/test/fixtures/bin/codex @@ -2,6 +2,10 @@ import { writeFileSync } from 'node:fs' const prompt = process.argv.at(-1) ?? '' +if (process.argv.includes('--version')) { + process.stdout.write(process.env.CODEX_FIXTURE_VERSION ?? 'codex-cli 1.0.0\n') + process.exit(0) +} if (process.env.CODEX_FIXTURE_HANG === '1') setInterval(() => {}, 1_000) if ( diff --git a/test/provider-registry.test.mjs b/test/provider-registry.test.mjs new file mode 100644 index 0000000..aad7307 --- /dev/null +++ b/test/provider-registry.test.mjs @@ -0,0 +1,17 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { providerRegistry, resolveProviderId } from '../dist/src/provider-registry.js' + +test('registry keeps API and local provider identities separate', () => { + const entries = providerRegistry() + const byId = Object.fromEntries(entries.map(entry => [entry.id, entry])) + assert.equal(byId.grok.kind, 'api') + assert.equal(byId['grok-cli'].kind, 'cli') + assert.equal(byId['opencode-cli'].kind, 'cli') + assert.equal(byId.grok.support, 'stable') + assert.equal(byId['grok-cli'].support, 'experimental') + assert.equal(byId['opencode-cli'].support, 'experimental') + assert.equal(resolveProviderId('api'), 'anthropic') + assert.ok(entries.some(entry => entry.id === 'azureOpenAI')) + assert.equal(entries.some(entry => entry.id === 'createRouter'), false) +})