Skip to content

Grok preflight rejects deployment-key logins and accepts an unlisted default model #131

Closed

Description

@andrewfree

preflightPassed for Grok (run.ts, after #130) has two gaps. Both reproduce on main c3a177d with a stand-in grok whose grok models prints the text below and whose model call succeeds.

  1. Deployment-key logins fail. grok 1.0.46 prints You are authenticated via deployment key. for that login. The pass check accepts only logged in or You are using XAI_API_KEY., so the runner retries once and records unauthenticated (exit 77) without running the model.

    You are authenticated via deployment key.
    Available models:
      * grok-4.7 (default)
    
  2. An unlisted default model passes. grok prints Default model: <slug> (not in the list below) when the configured default is missing from the account's list. grokModelAvailable matches that line as a whole token, so a lane for grok-4.7 passes preflight even though only grok-4.6 is listed.

    You are logged in with grok.com.
    Default model: grok-4.7 (not in the list below)
    Available models:
      * grok-4.6
    

Expected: case 1 passes preflight; case 2 fails with unavailable-model (exit 69) before the model call. A fix could accept the deployment-key line and match the model only against the lines under Available models:.

Activity

  1. ericlitman commented on Oct 11, 2026

    @ericlitman
    Owner

    Design pass, 2026-10-11 (bundled with #141, #142, #132; see the plan on #141)

    Plan:

    • Accept You are authenticated via deployment key. as a passing Grok login line.
    • grokModelAvailable matches the model only in the lines after Available models:. A missing header means the model is unavailable.
    • Whole-output matching is deleted.

    Tests: both reported repros, a default-only mention, a neighbouring slug, and the existing logged-in and XAI_API_KEY cases. Fixtures that put the header and the model on one line are adjusted.

    Unverified assumption: supported grok versions keep this text format. If a version drops the header, the lane fails loudly with unavailable-model.

  2. added a commit that references this issue on Oct 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions