feat(models): move Sol to gpt-6-sol and Grok to grok-4.7 - #83
rasmushjulskov wants to merge 1 commit into
Conversation
GPT-6 Sol (Sep 22) supersedes GPT-5.6 Sol in Codex, which now labels 5.6 Sol as an older model. Grok 4.7 (Sep 21) is the Grok CLI default and matches Grok 4.6 on price and speed. Update the provider-dispatch matrix, first-run sheet, skill and playbook defaults, and docs. Fable and Opus already use rolling aliases and are unchanged.
|
This pull request does not currently match the merge queue conditions, so it cannot be queued from here. The box comes back if it matches again. |
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| | sol | gpt-5.6-sol-max | codex | gpt-6-sol | max | low medium high xhigh max | - | | ||
| | grok | grok-4.7-xhigh-fast | grok | grok-4.7 | xhigh | low medium high xhigh max | - | |
There was a problem hiding this comment.
Changing these matrix keys strands existing installations on the old model descriptors. Runtime normalization only handles versioned Claude aliases, so persisted codex:gpt-5.6-sol and grok:grok-4.6 entries continue dispatching the old versions. When users rerun /setup-pstack, its exact provider/model matching treats those entries as inconsistent state and stops before it can preserve their role assignments on the new releases. Add an explicit migration for the prior Sol and Grok descriptors before matrix matching.
| | sol | gpt-5.6-sol-max | codex | gpt-6-sol | max | low medium high xhigh max | - | | ||
| | grok | grok-4.7-xhigh-fast | grok | grok-4.7 | xhigh | low medium high xhigh max | - | |
There was a problem hiding this comment.
The repository requires installed Claude Code and Codex behavioral checks for changes to this surface, but these new model routes have not received those live probes. Static consistency tests cannot show that gpt-6-sol@max works through the native and external Codex routes or that Grok CLI lists and executes grok-4.7@xhigh. Run and record the required probes before retaining the documentation’s claim that the frontier panel was verified in fresh sessions.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
![openswebot[bot]](https://avatars.githubusercontent.com/in/4340220?s=60&v=4){kind=small}
| | sol | gpt-5.6-sol-max | codex | gpt-6-sol | max | low medium high xhigh max | - | | ||
| | grok | grok-4.7-xhigh-fast | grok | grok-4.7 | xhigh | low medium high xhigh max | - | |
There was a problem hiding this comment.
🟠 Existing model sheets fail the version switch
Changing these (provider, model) keys leaves existing sheets seeded from the previous defaults with codex:gpt-5.6-sol and grok:grok-4.6. The only read-time migration handles versioned Fable/Opus, so installed users continue dispatching to the old Sol/Grok models; when they rerun /setup-pstack, step 3 treats those previously generated descriptors as unmatched, stops before probes, and cannot save the new choices without manually replacing each affected lane. Migrate the two previously generated descriptors during read-time routing and setup loading, preserving role, order, and effort, before matching the new matrix.
(Refers to lines 14-15)
Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.
| feature, refactoring: grok:grok-4.7@xhigh | ||
| bug-fix: codex:gpt-6-sol@max | ||
| perf-issue: codex:gpt-6-sol@max |
There was a problem hiding this comment.
🟡 Model change lacks required installed-surface verification
This changes the real Sol and Grok probe/dispatch defaults, but the PR explicitly says neither new descriptor has been live-probed through /setup-pstack. The repository's AGENTS.md requires the exact candidate installed and the changed behavior exercised on the real user surface in every affected harness, with version, surface, action, and observed result recorded; a PR without that evidence must remain a draft. This PR is currently non-draft. Keep it in draft until the installed Claude Code and Codex lanes for these descriptors have been observed and the evidence recorded.
(Refers to lines 91-93)
Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.
|
Thanks, @rasmushjulskov. The Grok 4.7 half of this PR landed in Open Pstack 1.5.0 (#98, released as v1.5.0): the matrix and first-run defaults now use |
rasmushjulskov
left a comment
rasmushjulskov
left a comment
There was a problem hiding this comment.
QA review (bot-assisted under @rasmushjulskov)
Reviewed PR body, the 18-file +32/−32 matrix/docs rename, mergeability, and prior thread (maintainer note + Greptile/OpenSWE).
Current state (important)
- GitHub reports
mergeable_state: dirty— this branch does not cleanly apply toericlitman/open-pstackmain. - Maintainer already noted the Grok 4.7 half landed in Open Pstack 1.5.0 via #98 (
grok:grok-4.7@xhighin matrix/first-run). Re-landing those same string swaps here will conflict or no-op; the remaining unique value of this PR is the Sol →gpt-6-solhalf (plus any docs still on 5.6).
What looks solid
- Scope is intentionally narrow: Model column + first-run sheet + role defaults + README/reference/UPSTREAM/CHANGES wording. Historical changelog narrative and sample-string fixtures were left alone as claimed.
- Asymmetry called out in the PR is visible in the diff: Sol Upstream cell stays
gpt-5.6-sol-maxwhile Model becomesgpt-6-sol; Grok upstream cell moves togrok-4.7-xhigh-fast. That matches the “Cursor upstream hasn’t moved Sol yet” rationale. - Excluding
ultraeffort for Sol (auto-delegation vs no-recursive-agents) is the right product call if still accurate forgpt-6-sol. - Static suite claim (158 tests / typecheck / skill-collision) is appropriate for a string-matrix change; it does not substitute for live route probes.
Findings / risks
- Rebase / reshape before anything else — Do not merge as-is. Rebase onto current
main, drop the already-shipped Grok edits, and leave a Sol-only (or Sol + any still-stale docs) diff. Until then review of the full patch is partly reviewing superseded work. - Existing sheet migration (agree with prior P1) — Read-time normalization is described as Claude-version-only. Persisted
codex:gpt-5.6-sol(and any leftovergrok:grok-4.6) will keep dispatching old models;/setup-pstackexact-match can treat them as inconsistent and stop. A Sol bump needs an explicit prior-descriptor → new-descriptor migration (role/order/effort preserved) or a documented one-shot operator step. Docs-only rename is incomplete product behavior. - Live probes still missing — PR itself says the
/setup-pstackbehavioral lanes forcodex:gpt-6-sol@max(and formerly Grok) were not run. For a dispatch-matrix change that is the real QA gate: Claude Code + Codex installed surfaces, record version/surface/action/result. Static green ≠ route works. - Dual source of truth risk while dirty — Arena/architect/interrogate/how/swarm/playbooks all duplicate the descriptors. After rebase, grep the tree for stray
gpt-5.6-sol/grok-4.6outside intentional historical prose and the Sol upstream-max cell.
Suggested follow-ups
- Close or convert to a Sol-only PR after rebase; reference #98 so reviewers do not re-litigate Grok.
- Add migration + probe evidence (or keep draft until both exist).
Event: COMMENT — not APPROVE (dirty + migration + unverified routes); not REQUEST_CHANGES solely because rebase may supersede half the patch, but I would not merge without addressing items 1–3.
|
Thanks for this, @rasmushjulskov. I appreciate the time you put into it. Open Pstack now accepts contributions as detailed issues rather than pull requests, so I'm closing this one. If it's still relevant, please open an issue describing the problem, how to reproduce it, and the change you'd suggest. I'll take it from there. |
2 participants
Summary
Moves two model families in the provider-dispatch matrix to their current releases:
codex:gpt-5.6-solcodex:gpt-6-solgrok:grok-4.6grok:grok-4.7gpt-5.6-solas an "Older coding model". OpenAI reports that 6 Sol makes about half as many mistakes as 5.6 Sol.grok modelson Grok CLI 1.0.41, at the same price and speed as 4.6. Cursor upstream also moved its Grok defaults to 4.7 in70b2dc8.What changed
provider-dispatch.md, plus the note about which Grok CLI model is pinned.setup-pstack.codex-tools.md.README.md,docs/reference.md,UPSTREAM.mdandCHANGES.md.Left as they were:
gpt-5.6-sol-max, because Cursor upstream hasn't moved yet.gpt-6-solalso offersultra, but that effort delegates work automatically, which conflicts with the runner's no-recursive-agents rule, so it stays excluded.Verification
bun run test: 158 pass, 0 fail.bun run typecheck: clean.PSTACK_STATIC_ONLY=1 bash tests/skill-collision-repro.sh: ok.UPSTREAM.mdasks for (a live/setup-pstackprobe ofcodex:gpt-6-sol@maxandgrok:grok-4.7@xhigh).