Skip to content

copilot -p fails with 401 on GHEC data residency since 1.0.81-1 — prompt mode model-catalog fetch hits api.githubcopilot.com instead of the tenant endpoint #4527

Description

@AvitalLivshits

Describe the bug

Summary

On a GitHub Enterprise Cloud with data residency tenant (<tenant>.ghe.com), non-interactive prompt mode (copilot -p) fails at startup with Authentication failed, while interactive mode (copilot) works perfectly with the exact same credentials, on the same machine, in the same shell.

The cause is visible in --log-level debug: after the CLI correctly authenticates and exchanges tokens against the tenant endpoints (api.<tenant>.ghe.com and copilot-api.<tenant>.ghe.com), the model-catalog fetch falls back to the public api.githubcopilot.com. A data-residency Copilot token is not valid there, so the request returns 401 unauthorized: AuthenticateToken authentication failed, and prompt mode aborts with exit code 1.

This is a regression. Bisecting across locally cached builds pins the break to 1.0.81-1: 1.0.81-0 and everything before it works, 1.0.81-1 and 1.0.81-3 fail.

This is the same class of defect as #76 ("Copilot CLI ends up sending traffic to the non subscription-based api.githubcopilot.com"), which was fixed in v0.0.332 by "Switched to using per-subscription Copilot API endpoints". That fix has regressed, but only on the prompt-mode code path.

What the user sees

$ copilot -p "hello"
Error: Authentication failed (Request ID: CF07:274F1E:1D73F43:23D810D:6A85932F)

Your GitHub token may be invalid, expired, or lacking the required permissions.

To resolve this, try the following:
  • Start 'copilot' and run the '/login' command to re-authenticate
  • If using a Fine-Grained PAT, ensure it has the 'Copilot Requests' permission enabled
  • If using COPILOT_GITHUB_TOKEN, GH_TOKEN or GITHUB_TOKEN environment variable, verify the token is valid and not expired
  • Run 'gh auth status' to check your current authentication status

Every suggestion in that message is a dead end — the token is valid and interactive mode proves it. The message is actively misleading for this failure mode.

Version bisect (the key evidence)

Same machine, same credentials, same command, only --prefer-version differs:

Version copilot -p "Reply with exactly: OK"
1.0.78-2 OK
1.0.79 OK
1.0.79-5 OK
1.0.81-0 OK
1.0.81-1 Error: Authentication failed
1.0.81-3 Error: Authentication failed

Regression introduced in 1.0.81-1. Nothing in the bundled changelog.json for 1.0.81-1 mentions auth, hosts, endpoints, or model routing, so this looks unintentional.

Distinct hosts contacted (root cause, side by side)

Extracted from --log-level debug logs of the identical command on the same machine:

1.0.81-0 — WORKING

api.<tenant>.ghe.com
copilot-api.<tenant>.ghe.com

1.0.81-3 — FAILING

api.github.com                            <-- new
api.githubcopilot.com                     <-- new, returns 401
api.<tenant>.ghe.com
copilot-api.<tenant>.ghe.com
copilot-telemetry-service.<tenant>.ghe.com

The working build never contacts a public github.com endpoint. The failing build contacts two.

Interactive mode on the failing build (1.0.81-3) also never contacts api.githubcopilot.com — its Copilot traffic goes exclusively to copilot-api.<tenant>.ghe.com. So the mis-routing is specific to prompt mode, not to the build's auth layer as a whole.

Affected version

GitHub Copilot CLI 1.0.81-3.

Last known good: 1.0.81-0. First broken: 1.0.81-1.

Steps to reproduce the behavior

  1. Authenticate against a GHEC data-residency tenant:
    copilot login --host https://<tenant>.ghe.com
    
  2. Confirm interactive mode works — start copilot, send any prompt, get a normal response. ✅
  3. Run the same prompt non-interactively in the same shell:
    copilot -p "hello"
    
  4. Observe Error: Authentication failed and exit code 1. ❌
  5. Confirm it is a regression:
    copilot --prefer-version 1.0.81-0 -p "hello"   # succeeds
    copilot --prefer-version 1.0.81-1 -p "hello"   # fails
    

Expected behavior

copilot -p should resolve the Copilot model-catalog endpoint from the authenticated host's per-subscription configuration — copilot-api.<tenant>.ghe.com — exactly as interactive mode does and as 1.0.81-0 did. It should never fall back to api.githubcopilot.com for an account whose token is scoped to a data-residency tenant.

Secondarily, when the model-catalog fetch 401s, the error should name the endpoint and host that were actually used. The current generic "your token may be invalid" text sends users down four unrelated remediation paths, none of which can fix this.

Additional context

Debug log excerpt (1.0.81-3, failing)

Trimmed to the relevant lines; {"log.target":…} suffixes removed for readability. Note the successful tenant token exchange at :44.671, immediately followed by the public-endpoint call at :45.377 and the 401.

11:27:32.862 [DEBUG] [rust:log] creating entry with service copilot-cli, user https://<tenant>.ghe.com:<user>
11:27:32.864 [DEBUG] [rust:log] starting new connection 'Some("api.<tenant>.ghe.com")'
11:27:34.020 [DEBUG] [rust:log] starting new connection 'Some("api.github.com")'
11:27:34.741 [INFO]  [managedSettings] server policy fetch skipped: no authenticated GitHub host available
11:27:35.831 [DEBUG] [rust:copilot_runtime::auth::gh] `gh auth token` exited non-zero
             {"exit_code":"Some(1)","stderr":"no oauth token found for github.com"}
11:27:36.328 [DEBUG] [rust:log] starting new connection 'Some("api.<tenant>.ghe.com")'
11:27:44.671 [DEBUG] [rust:log] starting new connection 'Some("copilot-api.<tenant>.ghe.com")'   <-- correct endpoint
11:27:45.377 [DEBUG] [rust:log] starting new connection 'Some("api.githubcopilot.com")'          <-- WRONG endpoint
11:27:45.891 [ERROR] Error loading models: Error: 401 "unauthorized: AuthenticateToken authentication failed\n"
11:27:45.892 [DEBUG] runPromptMode: executePromptDirectly returned succeeded=false
11:27:45.892 [DEBUG] runPromptMode: exiting with code 1

Two secondary observations from the same log, which may or may not be related to the same root cause:

  • [managedSettings] server policy fetch skipped: no authenticated GitHub host available fires early in prompt mode, suggesting the active host is not yet resolved at that point in startup. This line does not appear in the working build's log.
  • gh auth token is invoked against github.com rather than the authenticated tenant host, even with GH_HOST / COPILOT_GH_HOST explicitly set to the tenant. This looks like the same fallback-to-default-host pattern that Login to Github Enterprise Cloud non-interactivly doesn't work #296 fixed for the login path.

Ruled out

All of the following were tested individually and none change the outcome on 1.0.81-3:

Attempt Result
COPILOT_GH_HOST=<tenant>.ghe.com still 401
GH_HOST=<tenant>.ghe.com still 401
Both host vars together still 401
COPILOT_GITHUB_TOKEN=$(gh auth token -h <tenant>.ghe.com) (valid gho_ token, copilot scope) still 401
Host vars + explicit token together still 401
--model claude-sonnet-4.5 (to try to skip catalog resolution) still 401
Removing a stale second https://github.com entry from loggedInUsers in ~/.copilot/config.json still 401

Also verified healthy:

  • gh auth status → logged in to <tenant>.ghe.com, token scopes 'copilot', 'gist', 'read:org', 'repo', 'workflow'.
  • Windows Credential Manager contains https://<tenant>.ghe.com:<user>.copilot-cli, and the debug log shows it being read successfully as the first credential.
  • The tenant endpoints themselves are reachable and return 200s throughout startup (managed settings, user info, telemetry all succeed against api.<tenant>.ghe.com).
  • The account has no working github.com credential, which is correct and expected — there is no reason for prompt mode to need one.

Suggested fix

Make the prompt-mode startup path resolve the Copilot API base URL from the authenticated account's per-subscription endpoint, the same way the interactive path does, rather than defaulting to api.githubcopilot.com. This is the same fix that landed for #76 in v0.0.332; it appears the prompt-mode model-catalog call was either missed or regressed in 1.0.81-1.

Additionally, please consider surfacing the failing URL in the Authentication failed message. A single line such as model catalog request to https://api.githubcopilot.com/models returned 401 would have made this self-diagnosing instead of requiring a debug-log dive and a six-version bisect.

Related issues

Environment

  • Copilot CLI: 1.0.81-3 (installed via winget; launcher at %LOCALAPPDATA%\Microsoft\WinGet\Links\copilot.exe)
  • Operating system: Windows 11 Pro Insider Preview, 10.0.26310
  • CPU architecture: x86_64 (AMD64)
  • Node.js: v24.18.1 (bundled runtime)
  • Terminal emulator: Windows Terminal
  • Shell: PowerShell
  • Host: GitHub Enterprise Cloud with data residency (<tenant>.ghe.com)
  • Auth method: copilot login --host OAuth token in Windows Credential Manager (no token env vars set)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions