Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
65ab07a
Record the figures a session summary needs in each history row
charlesnutter Sep 23, 2026
87130a2
Add the session summary: generation rate, ttft, cache, time split, en…
charlesnutter Sep 23, 2026
59e8aa2
Render the Session section below the per-turn figures, collapsed by d…
charlesnutter Sep 23, 2026
435a5b4
Log session trees and sub-agent sessions under debug
charlesnutter Sep 23, 2026
566bff3
Roll up a turn's sub-agents onto its line and into the session totals
charlesnutter Sep 23, 2026
d9bdbfe
Read latest-request engines when a step finishes streaming, not after…
charlesnutter Sep 23, 2026
066c0c9
Give sub-agent time its own share of the session time split
charlesnutter Sep 23, 2026
eaeef90
Expect same-engine sub-agents in counter-difference windows, and labe…
charlesnutter Sep 23, 2026
ff525d3
Lay the sidebar out as two independent boxes of labelled rows
charlesnutter Sep 24, 2026
c30909b
Resolve theme colours defensively; the runtime theme differs from its…
charlesnutter Sep 24, 2026
0612d88
Shade boxes with the theme's raised.high, one step above the sidebar
charlesnutter Sep 24, 2026
a5ca89f
Pad boxes 2 columns at the sides so padding reads even
charlesnutter Sep 24, 2026
7c21eca
Correct two comments to the 32-cell box interior
charlesnutter Sep 24, 2026
d36f124
Keep a padding row above and below a collapsed box
charlesnutter Sep 24, 2026
ab3f7d6
Cut history rows at the panel edge; pool the headline rate per model
charlesnutter Sep 24, 2026
03f7f2d
Revert history rows to one wrapped text; truncation garbled them
charlesnutter Sep 24, 2026
02451c6
Bring the changelog's examples up to the box layout
charlesnutter Sep 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,49 @@
## [Unreleased]
### Added
- A **Session** section below the per-turn figures, collapsed by default:
click its heading to open it. Collapsed, it still shows the session's
generation speed (`▸ Session · 14 turns 48.2 tok/s`). Open, it
shows generation tok/s with a trend of recent turns, TTFT median and
worst, cache hit rate, how the time split between generating, waiting
for the first token and everything else, the engine's own averages
(MTP or draft acceptance, prefill rate) where the engine provides them,
and retries. Only the current model's turns count; the heading says so
when the model changed partway through. The session's tokens, context
and cost are left to OpenCode's own sidebar.
- Sub-agent roll-ups. A turn that started sub-agents adds a line to the
per-turn figures (`sub-agents 2 · 4,210 tok`, then `38.10s` and
`$0.012` on the rows below): their tokens and cost summed, and the time from the first starting to the last
finishing. Rates are never combined across them. The Session section
totals them in a `sub-agents` row, and its time split gives the real time
sub-agents were running its own share instead of folding it into `other`.
- On `vllm`, `sglang`, `vllmmlx`, `aphrodite`, `lmdeploy`, `llamacpp`,
`llamafile`, `splash` and `omlx`, a turn whose sub-agent used the same
engine is no longer declined: the engine's window is expected to hold the
turn's requests and tokens plus the sub-agents'. Those figures then cover
both, and the line says `incl. sub-agents`. Rate and TTFT stay the turn's
own. Built and tested against captures; not yet run live.
- `background` option (default on): shades each box one step above the
sidebar's own background.

### Changed
- The sidebar is two boxes, the last turn and the session, each opened and
closed independently by clicking its heading. Figures are laid out one
per line as label and value; a figure with parts continues on the next
line. Collapsed, a heading keeps one figure. The heading names the
engine, not the model, which OpenCode already shows under the prompt.
- mlx-serve no longer shows a non-streamed request's whole-request rate:
it includes prefill, so it is not generation speed.
- `mtplx`, `koboldcpp` and `mlxserve` are read when each step finishes
streaming, not when OpenCode marks the step ended. For a step that calls
tools, "ended" comes only after the tools have run, and a sub-agent on the
same engine had replaced the step's figures by then, so the turn was
declined with `engine data skipped: overlapping requests`. Measured on
MTPLX: 194 tokens (the step's own) at the end of streaming, 163 (the
sub-agent's) at "ended".
- The history panel's headline rate is generation tok/s on the newest
turn's model (tokens over streaming time, as in the Session box), instead
of a mean of per-turn rates across every model.

## [0.2.4] – 2026-09-23
### Changed
- `mtplx` is read at the end of every step, not once per turn, so a turn
Expand Down
31 changes: 24 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,24 @@ the provider.


```
MTPLX arsis-dev-ukisai-swift-…
38.1 tok/s ttft 9.06s
prefill 452 tok/s
37 tok 10.03s
MTP 3.70x 99/96/80%
▾ MTPLX · last turn

speed 34.4 tok/s
ttft 17.19s
prefill 460 tok/s
tokens 1,233
time 207.37s
MTP 3.42x
accepted 91/79/64%
sub-agent 191 tok
23.91s

▸ Session · 14 turns 48.2 tok/s
```

Two boxes, each opened and closed by clicking its heading: the last turn,
and the session so far.

Requires [**OpenCode 2**](https://opencode.ai/v2/docs). For the v1 line
(OpenCode 1.18.x), see
[opencode-engine-hud](https://github.com/charlesnutter/opencode-engine-hud).
Expand Down Expand Up @@ -70,7 +81,7 @@ Collapsed, the line keeps one figure rather than becoming a bare label:

## Configuration

One key, because one figure is genuinely a preference. Everything else
Two keys, because two things are genuinely preferences. Everything else
appears exactly when its underlying data exists and stays silent when it
does not — there is nothing to choose.

Expand All @@ -79,7 +90,7 @@ does not — there is nothing to choose.
"plugins": [
{
"package": "@banburist/opencode-headsup",
"options": { "showContext": false }
"options": { "showContext": false, "background": true }
}
]
}
Expand All @@ -92,6 +103,12 @@ Adds a `13% prompt/limit` line, computed as
config in `opencode.json`. Labeled as `prompt/limit` rather than
`context used`.

**background** (default `true`)

Puts your theme's offset panel shade behind each box. Turn it off for a
theme or terminal with a transparent background, where the shade can
disappear.

### Endpoints

Engine endpoints use the defaults below, overridable per key or by env var.
Expand Down
52 changes: 30 additions & 22 deletions adapters/koboldcpp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
// without the TUI runtime.

import { httpJson, type HttpOptions } from "../http"
import { nn, ni, short } from "../format"
import { nn, ni } from "../format"
import { rowsOf, timeRows, viewText, nt, type Row, type TurnView } from "../rows"
/**
* The fields of /api/extra/perf this plugin reads. The endpoint returns more
* (image/TTS/transcription counters, horde bookkeeping, seeds) that describe
Expand Down Expand Up @@ -228,33 +229,40 @@ export function combineKoboldSteps(
}

/**
* `host.total` is the turn's total from OpenCode -- what the user waited,
* retries included -- and wins over the engine's prefill + decode time.
* The turn as labelled rows. `host.total` is the turn's total from OpenCode
* -- what the user waited, retries included -- and wins over the engine's
* prefill + decode time. TTFT is the host's, labelled as such: KoboldCpp
* reports none, and nothing on these rows takes its numerator from one
* source and its denominator from the other.
*/
export function koboldView(
t: KoboldTurn,
hostTtft?: number,
host: { total?: number; retries?: number } = {}
): TurnView {
const rows: Row[] = [
...rowsOf("speed", [t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : ""]),
...rowsOf("ttft", [hostTtft !== undefined ? `${nn(hostTtft, 2)}s (host)` : ""]),
...rowsOf("prefill", [t.prefillTokS !== undefined ? `${ni(t.prefillTokS)} tok/s` : ""]),
["tokens", nt(t.completionTokens)],
...timeRows(host.total ?? t.prefillS + t.decodeS, host.retries),
...rowsOf("draft", [t.draftAcceptRate !== undefined ? `${ni(t.draftAcceptRate * 100)}% accepted` : ""]),
]
const notes =
t.generationsInWindow !== undefined && t.generationsInWindow > 1
? [`${ni(t.generationsInWindow)} generations this turn`, "(last shown only)"]
: []
return { engine: "KoboldCpp", rows, notes, key: t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : undefined }
}

/** The view as text; kept for tests that look for a figure. */
export function formatKoboldLine(
t: KoboldTurn,
model: string,
_model: string,
hostTtft?: number,
host: { total?: number; retries?: number } = {}
): string {
// Host-derived, and labelled as such. No derived figure on this line takes
// its numerator from one source and its denominator from the other -- ttft
// is measured directly, so nothing crosses the seam.
const ttftLabel = hostTtft !== undefined ? ` ttft ${nn(hostTtft, 2)}s (host)` : ""
return [
`KoboldCpp ${short(model)}`,
t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s${ttftLabel}` : ttftLabel.trim(),
t.prefillTokS !== undefined ? `prefill ${ni(t.prefillTokS)} tok/s` : "",
`${ni(t.completionTokens)} tok ${nn(host.total ?? t.prefillS + t.decodeS, 2)}s${
(host.retries ?? 0) > 0 ? ` (${host.retries} ${host.retries === 1 ? "retry" : "retries"})` : ""
}`,
t.draftAcceptRate !== undefined ? `draft ${ni(t.draftAcceptRate * 100)}% accepted` : "",
t.generationsInWindow !== undefined && t.generationsInWindow > 1
? `${ni(t.generationsInWindow)} generations this turn (last shown only)`
: "",
]
.filter(Boolean)
.join("\n")
return viewText(koboldView(t, hostTtft, host))
}

export async function fetchKoboldPerf(
Expand Down
45 changes: 27 additions & 18 deletions adapters/llamacpp.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@

import { sumLabeledMetric } from "../prometheus-text"
import { httpText, type HttpOptions } from "../http"
import { nn, ni, short } from "../format"
import { nn, ni } from "../format"
import { rowsOf, timeRows, viewText, nt, type Row, type TurnView } from "../rows"

export interface LlamaCppCounters {
promptTokens: number
Expand Down Expand Up @@ -121,30 +122,38 @@ export function llamaCppIsThisTurn(t: LlamaCppTurn, hostTokens: number | undefin
}

/**
* The turn as labelled rows under `label` (llama.cpp or llamafile).
* `host.total` is the turn's total from OpenCode -- what the user waited,
* retries included -- and wins over the engine's own timings.
* retries included -- and wins over the engine's own timings. TTFT is the
* host's, labelled: llama.cpp reports none of its own.
* `host.includesSubagents`: the window also held same-engine sub-agents'
* requests, so the token count covers theirs too, and says so.
*/
export function llamaCppView(
t: LlamaCppTurn,
label: string,
hostTtft?: number,
host: { total?: number; retries?: number; includesSubagents?: boolean } = {}
): TurnView {
const rows: Row[] = [
...rowsOf("speed", [t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : ""]),
...rowsOf("ttft", [hostTtft !== undefined ? `${nn(hostTtft, 2)}s (host)` : ""]),
...rowsOf("prefill", [t.prefillTokS !== undefined ? `${ni(t.prefillTokS)} tok/s` : ""]),
...rowsOf("tokens", [nt(t.completionTokens), host.includesSubagents ? "incl. sub-agents" : ""]),
...timeRows(host.total ?? t.decodeS + t.prefillS, host.retries),
]
return { engine: label, rows, notes: [], key: t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : undefined }
}

/** The view as text; kept for tests that look for a figure. */
export function formatLlamaCppLine(
t: LlamaCppTurn,
label: string,
model: string,
_model: string,
hostTtft?: number,
host: { total?: number; retries?: number } = {}
host: { total?: number; retries?: number; includesSubagents?: boolean } = {}
): string {
// Host-derived, and labelled as such. No derived figure on this line takes
// its numerator from one source and its denominator from the other -- ttft
// is measured directly, so nothing crosses the seam.
const ttftLabel = hostTtft !== undefined ? ` ttft ${nn(hostTtft, 2)}s (host)` : ""
return [
`${label} ${short(model)}`,
t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s${ttftLabel}` : ttftLabel.trim(),
t.prefillTokS !== undefined ? `prefill ${ni(t.prefillTokS)} tok/s` : "",
`${ni(t.completionTokens)} tok ${nn(host.total ?? t.decodeS + t.prefillS, 2)}s${
(host.retries ?? 0) > 0 ? ` (${host.retries} ${host.retries === 1 ? "retry" : "retries"})` : ""
}`,
]
.filter(Boolean)
.join("\n")
return viewText(llamaCppView(t, label, hostTtft, host))
}

export async function fetchLlamaCppCounters(
Expand Down
54 changes: 29 additions & 25 deletions adapters/mlxserve.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
// No JSX/solid-js imports, so it stays unit-testable (test/mlxserve.test.mjs).

import { httpJson, type HttpOptions } from "../http"
import { nn, ni, short } from "../format"
import { nn, ni } from "../format"
import { rowsOf, timeRows, viewText, nt, type Row, type TurnView } from "../rows"

/** One record from /v1/metrics/requests, as the server names its fields. */
export interface MlxServeRequest {
Expand Down Expand Up @@ -272,35 +273,38 @@ export function combineMlxServeSteps(
}

/**
* `host.total` is the turn's total from OpenCode -- what the user waited,
* retries included -- and wins over the record's request duration.
* The turn as labelled rows. `host.total` is the turn's total from OpenCode
* -- what the user waited, retries included -- and wins over the record's
* request duration. Only a streamed record's rate is shown: a non-streamed
* record's `tokens_per_second` is completion / whole duration, prefill
* included, which is not generation speed (OpenCode always streams).
*/
export function mlxServeView(
t: MlxServeTurn,
host: { total?: number; retries?: number } = {}
): TurnView {
const rows: Row[] = [
...rowsOf("speed", [t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : ""]),
...rowsOf("ttft", [t.decodeTokS !== undefined && t.ttft !== undefined ? `${nn(t.ttft, 2)}s` : ""]),
["tokens", nt(t.completionTokens)],
...rowsOf("prompt", [t.promptTokens !== undefined ? nt(t.promptTokens) : ""]),
...timeRows(host.total ?? t.totalS, host.retries),
]
const notes: string[] = []
// A cold start loaded the model mid-request; without this the turn reads
// as a tenfold slowdown rather than a one-off load.
if (t.coldStart) notes.push("cold start (model loaded)")
if (t.requests > 1 && t.steps === undefined) notes.push(`${ni(t.requests)} requests this turn`)
return { engine: "mlx-serve", rows, notes, key: t.decodeTokS !== undefined ? `${nn(t.decodeTokS)} tok/s` : undefined }
}

/** The view as text; kept for tests that look for a figure. */
export function formatMlxServeLine(
t: MlxServeTurn,
model: string,
_model: string,
host: { total?: number; retries?: number } = {}
): string {
// decodeTokS and overallTokS are never both set; they are not comparable,
// so the whole-request one is labelled rather than shown as a decode rate.
const rate =
t.decodeTokS !== undefined
? `${nn(t.decodeTokS)} tok/s${t.ttft !== undefined ? ` ttft ${nn(t.ttft, 2)}s` : ""}`
: t.overallTokS !== undefined
? `${nn(t.overallTokS)} tok/s (whole request)`
: ""
return [
`mlx-serve ${short(model)}`,
rate,
`${ni(t.completionTokens)} tok${t.promptTokens !== undefined ? ` ${ni(t.promptTokens)} prompt` : ""} ${nn(host.total ?? t.totalS, 2)}s${
(host.retries ?? 0) > 0 ? ` (${host.retries} ${host.retries === 1 ? "retry" : "retries"})` : ""
}`,
// A cold start loaded the model mid-request; without this the turn reads
// as a tenfold slowdown rather than a one-off load.
t.coldStart ? "cold start (model loaded)" : "",
t.requests > 1 && t.steps === undefined ? `${ni(t.requests)} requests this turn` : "",
]
.filter(Boolean)
.join("\n")
return viewText(mlxServeView(t, host))
}

export async function fetchMlxServeRequests(
Expand Down
Loading
Loading