Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 20 additions & 34 deletions docs/architecture/ai-hints.md
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,7 @@ Adding a provider is one spec; settings, IPC and the settings UI read the regist

- **The model is not a setting, except for Custom.** Each named provider always uses its spec's
default model, and any model saved earlier is ignored. Students can't judge which model tutors
well, and a free-text model field let a saved `openrouter/free` bypass the routing below. Custom
has no sensible default, so it keeps a required Model field.
well. Custom has no sensible default, so it keeps a required Model field.
- **Per-provider settings.** `config.ai.providers[id]` stores the key (plus base URL and model for
Custom) for each provider separately, so switching providers doesn't discard a key already
entered.
Expand All @@ -87,60 +86,47 @@ Adding a provider is one spec; settings, IPC and the settings UI read the regist
The earlier OpenRouter-only key field (`openRouterApiKeyEnc`) was dropped without migration. The
feature had not shipped.

### OpenRouter model routing
### OpenRouter model

The default provider has to work with a free key, and free models are the least reliable part of
the whole feature. Each request sends a primary `model` plus OpenRouter's `models` fallback array,
and OpenRouter tries them in order, moving on when one is down, rate-limited, or has left the free
tier:
The default provider has to work with a free key. Every request uses `openrouter/free`, OpenRouter's
router over all zero-cost models, and sends no fallback list.

1. `google/gemma-4-31b-it:free` (primary)
2. `qwen/qwen3.8-27b:free`
3. `nvidia/nemotron-3-super-120b-a12b:free`
4. `openrouter/free` (last resort)
**Why the router alone.** Free-tier membership changes without notice (see `../llm-integration.md`,
Option 9), so any named free model eventually disappears or is rate-limited, and a hard-coded list
needs upkeep with every change. The router always resolves to something that is currently free, so
the default keeps working with nothing to maintain.

The three fallbacks are the most OpenRouter is documented to accept (its equivalent `fallbacks`
parameter caps at three).
**The cost is quality.** The router picks from every zero-cost model, and some can't tutor. In
testing it produced:

**Why named models come first.** On its own, `openrouter/free` picks from every zero-cost model, and
in testing it produced unusable replies:

- a ~2B agent-tuned model (`liquid/lfm-2.5-2.6b`) answered in its raw tool-call syntax
- a ~2B agent-tuned model (`liquid/lfm-2.5-2.6b`) answering in its raw tool-call syntax
(`<|tool_call_start|>[read(filePath=…)]<|tool_call_end|>`), because it wanted to read files and no
tools were offered;
- a content-safety classifier (`nvidia/nemotron-3.5-content-safety`) answered "User Safety: safe".

The named models are mid-sized, instruction-tuned, and follow the hint policy reasonably well.
- a content-safety classifier (`nvidia/nemotron-3.5-content-safety`) answering "User Safety: safe".

**Why `openrouter/free` is still the last resort.** The tradeoff is availability against quality.
Free models are rate-limited per model, and at busy times all three named models can be throttled
together. Without the router, the student gets a rate-limit error and has to wait. With it, they
usually still get an answer, but occasionally from a model that can't tutor. Availability wins: the
feature is only useful if it answers, and a bad reply is visible and easy to retry.

What limits the damage when the router picks badly:
What limits the damage:

- The system prompt says the model has no tools and must reply only in text. Weaker agent-tuned
models then answer in prose more often, but not reliably.
- **Regenerate** retries the whole chain. That lands on a named model once its rate limit has reset,
or on a different random model if it hasn't.
- **Regenerate** sends the request again, which usually lands on a different model.
- The classifier and code-only models can't be steered by the prompt. When they answer, the reply is
obviously wrong rather than subtly misleading, which is the lesser failure for a tutor.
- Students who want consistent answers can switch to a paid provider, or to Google's free tier, in
Settings.

**Alternatives rejected:**

- **Named free models first, with `openrouter/free` as the last fallback.** This was the previous
setup (a primary model plus three fallbacks, the most OpenRouter accepts). It gave better replies
while the named models stayed free, but they had to be checked and replaced by hand whenever they
left the free tier or were throttled.
- **Stripping tool-call tokens from the output.** This treats one model family's symptom, and the
answer underneath still comes from a model too small to tutor.
- **Choosing a free model at runtime from `/models`.** The listing has price and context length but
no quality signal. It can't tell a 2B model or a classifier from a 30B instruct model.
- **A paid default model.** It would be reliable, but every student would need a funded account,
which is what the free default exists to avoid.

**Upkeep.** Free-tier membership changes without notice (see `../llm-integration.md`, Option 9).
When replies start coming from the router regularly, check the named models against
`https://openrouter.ai/api/v1/models` and replace any that have left the free tier. The response's
`model` field shows which model actually answered.

## 5. Tutoring policy

`ai/prompt.ts` builds the system prompt. The policy depends on the session kind:
Expand Down
8 changes: 4 additions & 4 deletions docs/architecture/exercises-root.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ needed. Running setup from an onboarding screen asked for that work before the
lesson did, and it required the CLI to be installed first.

The first time a learner clicks Start, the app walks through three steps. The
introduction is shown once. The other two are checked on every Start, from the
main process, so the Start button in the app and the one in the lesson page
behave the same.
introduction repeats on every Start until they check "Don't show this again".
The other two are checked on every Start, from the main process, so the Start
button in the app and the one in the lesson page behave the same.

1. **Introduction.** A short description of an exercise: a folder of starting
files, work in the terminal, then Verify. Hands-on practicals are mentioned
because they have no Verify step.
because they have no Verify step. A checkbox opts out of seeing it again.
2. **Tools.** `gitmastery` must be on PATH. On Windows, Git Bash must be
installed too, because the in-app terminal uses it. Neither check runs the
CLI. On Windows the step says to restart the app, because PATH is read at
Expand Down
42 changes: 42 additions & 0 deletions docs/architecture/walkthrough.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
# App walkthrough

A two-step tour runs on first launch and whenever the learner clicks Help (the
question mark in the header). Each step dulls everything except one area and
explains it; the learner clicks Next to move on. There is no skip: the tour is
two clicks long. After the last step the app behaves as normal.

1. **Lessons.** The terminal column and header are dimmed.
2. **Terminal.** The lesson page, tours panel and header are dimmed, and the
terminal gets a brand outline.

"Seen" lives in renderer `localStorage` (`gm-walkthrough-seen`). It is a
display preference with no meaning to the main process, like the other
renderer-only flags.

## The lesson page is dimmed from inside the page

The lesson site is a native `WebContentsView` that paints above all DOM, so a
React overlay cannot dull it. For step 2 the renderer sends `wcv-set-dimmed`,
and main injects a fixed, full-page layer into the lesson page itself. The
colour mirrors the `--gm-dim` token for the current theme, so it matches the
dimmed DOM panes. Main keeps the flag and re-applies it on `dom-ready`, so the
dim survives a navigation or a page that finishes loading mid-tour. The layer
also swallows clicks, so the learner cannot wander off during the step.

## Cards always sit in the terminal column

For the same reason, a card placed over the lesson pane would be hidden. Both
step cards render inside the terminal column: centred over the dimmed terminal
in step 1, and along the bottom of the terminal in step 2. They are compact
callouts, so the title is Inter semibold like a toast rather than the serif
modal title.

## Rejected

- **Hide the native view for step 2.** Simple, but the lesson pane goes blank
instead of dull, and the learner loses the sense of where the lessons are.
- **A tour library (spotlight / popover).** Such libraries position DOM
popovers and cutouts, which cannot reach the native view.
- **Render the card inside the lesson page.** It could sit anywhere, but its
buttons would need a round trip through the page bridge and main, and it
would have to re-implement the house styles in injected CSS.
18 changes: 4 additions & 14 deletions src/electron/ai/llmProviders.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,19 +31,10 @@ export type ProviderSpec = AiProviderInfo & {
const OPENROUTER_BASE_URL = "https://openrouter.ai/api/v1";

/**
* Named free models first; OpenRouter moves down the list when one is down,
* rate-limited, or no longer free. `openrouter/free` is the last resort and
* picks any zero-cost model, some of which cannot tutor — see
* docs/architecture/ai-hints.md. Free-tier membership changes without notice;
* revisit the named models when they stop answering.
* OpenRouter's router over every zero-cost model. Some of those cannot tutor —
* see docs/architecture/ai-hints.md.
*/
const OPENROUTER_PRIMARY_MODEL = "google/gemma-4-31b-it:free";
/** At most three: OpenRouter caps the equivalent `fallbacks` list at three. */
const OPENROUTER_FALLBACK_MODELS = [
"qwen/qwen3.8-27b:free",
"nvidia/nemotron-3-super-120b-a12b:free",
"openrouter/free",
];
const OPENROUTER_FREE_MODEL = "openrouter/free";

/** Optional OpenRouter attribution headers. */
const OPENROUTER_HEADERS = {
Expand Down Expand Up @@ -109,7 +100,7 @@ const SPECS: ProviderSpec[] = [
label: "OpenRouter",
description:
"Free models with a free account. Recommended if you don't already pay for an AI API.",
defaultModel: OPENROUTER_PRIMARY_MODEL,
defaultModel: OPENROUTER_FREE_MODEL,
keyUrl: "https://openrouter.ai/keys",
keyPlaceholder: "sk-or-v1-…",
keyRequired: true,
Expand All @@ -127,7 +118,6 @@ const SPECS: ProviderSpec[] = [
probe("OpenRouter", `${OPENROUTER_BASE_URL}/key`, {
Authorization: `Bearer ${apiKey}`,
}),
providerOptions: { openrouter: { models: OPENROUTER_FALLBACK_MODELS } },
},
{
id: "openai",
Expand Down
Loading
Loading