Skip to content

Openrouter diag branch#615

Closed
seonghobae wants to merge 13 commits into
mainfrom
openrouter-diag-branch
Closed

Openrouter diag branch#615
seonghobae wants to merge 13 commits into
mainfrom
openrouter-diag-branch

Conversation

@seonghobae

Copy link
Copy Markdown
Contributor

This pull request adds support for using the OpenRouter free model endpoint as an additional provider for LLM-based workflows, including Strix and OpenCode review jobs. It introduces configuration, secret management, and diagnostic workflows for OpenRouter, and ensures that all relevant scripts and error messages are updated to recognize OpenRouter as a supported provider. Additionally, it updates Python dependencies for compatibility and security.

OpenRouter Provider Integration

  • Added openrouter as an enabled provider in .github/workflows/opencode-review.yml and configured the OpenRouter free model endpoint with API key support. [1] [2]
  • Updated the model candidate list and fallback logic to include openrouter/openrouter/free in OpenCode and Strix workflows, ensuring OpenRouter is used when available. [1] [2] [3]
  • Added OpenRouter API key handling and validation throughout Strix and OpenCode workflows, including environment variables, secret masking, and input file preparation. [1] [2] [3] [4] [5] [6] [7] [8]

Diagnostics and Testing

  • Introduced .github/workflows/openrouter-diag.yml, a diagnostic workflow that verifies OpenRouter API key validity, endpoint health, and CLI integration.

Error Handling and Messaging

  • Updated error messages and test scripts to mention OpenRouter as a supported provider, ensuring clear communication when unsupported models are selected. [1] [2] [3]

Dependency Updates

  • Updated pyasn1 to version 0.6.4 in both requirements-strix-ci.txt and requirements-strix-ci-hashes.txt for compatibility and security improvements. [1] [2]

Cross-Provider Fallbacks

  • Improved logic for resolving API base URLs and fallback credentials when switching between providers (OpenAI, OpenRouter, GitHub Models) in Strix scripts.

These changes ensure that OpenRouter is a first-class, configurable LLM provider in CI workflows, with robust support for secrets, error handling, and diagnostics.

seonghobae and others added 10 commits July 23, 2026 10:46
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Write trimmed provider API keys to llm_api_key_file in Strix workflow and lock this behavior in the workflow contract self-test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 23, 2026 05:03

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

Adds a GitHub Actions workflow to diagnose OpenRouter connectivity/auth and exercise OpenCode CLI with the OpenRouter free model.

Changes:

  • Introduces an openrouter-diag.yml workflow triggered by manual dispatch or pushes to openrouter-diag-branch
  • Adds curl-based probes for chat completions, tool-calls, and key-info endpoints
  • Downloads and runs OpenCode CLI against an OpenRouter-configured model, capturing logs for debugging

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@@ -0,0 +1,111 @@
name: OpenRouter Diagnostic
Comment on lines +13 to +15
diag:
runs-on: ubuntu-latest
steps:
Comment on lines +30 to +34
status="$(curl -sS -o "$response_file" -w '%{http_code}' \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d "$body" \
https://openrouter.ai/api/v1/chat/completions)"
echo "Key length: ${key_len} characters."

body='{"model":"openrouter/free","messages":[{"role":"user","content":"Say OK"}],"max_tokens":10}'
response_file="$(mktemp)"
Copilot AI review requested due to automatic review settings July 23, 2026 05:04

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

.github/workflows/openrouter-diag.yml:1

  • The workflow file starts with a UTF-8 BOM character (visible before name:). This can cause subtle YAML parsing/lint issues; please remove the BOM so the file starts with plain ASCII name:.
name: OpenRouter Diagnostic

.github/workflows/openrouter-diag.yml:48

  • Like the first probe, this tool-call curl request will pass the step on non-2xx HTTP responses (e.g., auth failure or rate-limit), which reduces the usefulness of the diagnostic signal.
          tool_status="$(curl -sS -o "$response_file" -w '%{http_code}' \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer ${OPENROUTER_API_KEY}" \
            -d "$tool_body" \
            https://openrouter.ai/api/v1/chat/completions)"
          echo "HTTP status: $tool_status"
          head -c 1500 "$response_file"

.github/workflows/openrouter-diag.yml:56

  • The /auth/key response body is always printed. This is more information than needed for a health check and can unnecessarily expose key/account metadata in workflow logs. Consider printing the body only on failure and failing the job on non-2xx status.
          key_status="$(curl -sS -o "$response_file" -w '%{http_code}' \
            -H "Authorization: Bearer $OPENROUTER_API_KEY" \
            https://openrouter.ai/api/v1/auth/key)"
          echo "HTTP status: $key_status"
          head -c 1500 "$response_file"

.github/workflows/openrouter-diag.yml:111

  • The OpenCode CLI run captures and prints logs but never fails the job when opencode run exits non-zero. As written, the workflow can report success even when the OpenCode/OpenRouter integration is broken.
          echo "opencode exit code: $rc"
          echo "--- stdout (first 4000 bytes) ---"
          head -c 4000 out.json; echo
          echo "--- stderr (first 6000 bytes) ---"
          head -c 6000 err.log; echo

Comment on lines +30 to +38
status="$(curl -sS -o "$response_file" -w '%{http_code}' \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d "$body" \
https://openrouter.ai/api/v1/chat/completions)"
echo "HTTP status: $status"
echo "Response (first 1500 bytes):"
head -c 1500 "$response_file"
echo
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 23, 2026 07:17
@seonghobae
seonghobae force-pushed the openrouter-diag-branch branch from ab1118c to 5265f10 Compare July 23, 2026 07:17

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (4)

.github/workflows/openrouter-diag.yml:1

  • The workflow file appears to start with a UTF-8 BOM (the key is rendered as "\ufeffname"), which can cause YAML parsing/key recognition issues. Remove the BOM so the first key is exactly name:.
name: OpenRouter Diagnostic

.github/workflows/openrouter-diag.yml:37

  • This probe prints the HTTP status but never fails the step when the endpoint returns a non-2xx response, so the diagnostic can report success even when the OpenRouter chat completion request is failing.
          echo "HTTP status: $status"
          echo "Response (first 1500 bytes):"
          head -c 1500 "$response_file"

.github/workflows/openrouter-diag.yml:49

  • The tool-call probe also doesn't gate on the HTTP status code, which means tool support/auth regressions won't fail the diagnostic workflow run.
          echo "HTTP status: $tool_status"
          head -c 1500 "$response_file"
          echo

.github/workflows/openrouter-diag.yml:57

  • The key-info endpoint probe prints the status/response but doesn't fail on non-2xx results, so invalid/insufficient credentials could still produce a green diagnostic run.
          echo "HTTP status: $key_status"
          head -c 1500 "$response_file"
          echo

Comment thread .github/workflows/openrouter-diag.yml Outdated
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 23, 2026 07:31

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (2)

.github/workflows/openrouter-diag.yml:1

  • The workflow file starts with a UTF-8 BOM character before name:. This can break YAML parsing and causes the key to be \ufeffname instead of name, so GitHub may ignore the workflow name (or fail to load the workflow). Remove the BOM so the first line begins with plain name:.
name: OpenRouter Diagnostic

.github/workflows/openrouter-diag.yml:46

  • After sanitizing the API key into trimmed_key, the later curl calls still use the raw ${OPENROUTER_API_KEY} value in the Authorization header. Reuse the sanitized variable to keep header handling consistent and avoid newline/header-injection edge cases.
          tool_status="$(curl -sS -o "$response_file" -w '%{http_code}' \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer ${OPENROUTER_API_KEY}" \
            -d "$tool_body" \
            https://openrouter.ai/api/v1/chat/completions)"

Comment on lines +20 to +34
set -euo pipefail
if [ -z "${OPENROUTER_API_KEY:-}" ]; then
echo '::error::OPENROUTER_API_KEY secret is empty or not visible to this workflow.'
exit 1
fi
key_len="$(printf '%s' "$OPENROUTER_API_KEY" | wc -c | tr -d ' ')"
echo "Key length: ${key_len} characters."

body='{"model":"openrouter/free","messages":[{"role":"user","content":"Say OK"}],"max_tokens":10}'
response_file="$(mktemp)"
status="$(curl -sS -o "$response_file" -w '%{http_code}' \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $OPENROUTER_API_KEY" \
-d "$body" \
https://openrouter.ai/api/v1/chat/completions)"
Comment on lines +3 to +7
on:
workflow_dispatch:
push:
branches:
- openrouter-diag-branch
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@seonghobae seonghobae closed this Jul 23, 2026
@seonghobae
seonghobae deleted the openrouter-diag-branch branch July 23, 2026 07:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants