fix(openai-api): report real token usage in chat completions - #8
Open
fancyboi999 wants to merge 36 commits into
Open
fix(openai-api): report real token usage in chat completions#8fancyboi999 wants to merge 36 commits into
fancyboi999 wants to merge 36 commits into
Conversation
Contributor
Author
|
@syzsunshine219 maintainer follow-up: this PR remains clean and mergeable, and current No conflict rebuild is needed here. Could you review it when convenient? |
merge main to v1.0.5
fix: unify cross-agent action backgrounds
fix: simplify memory tool call headings
feat(memory): commit episode routing on turn complete
fix(memory): separate retrieval content and validate L3 evidence
fix(memory): finalize episode rewards and processing
chore: docs
feat(memory): link related memory details
…emTensor#139) Co-authored-by: antalike <> Co-authored-by: Cursor <cursoragent@cursor.com>
* feat: merge pkg shell * feat: merge pkg shell * feat: merge pkg shell * feat: merge pkg shell
fix(backend): share scan permission per installation
fix(backend): refine onboarding permissions and actions
fix: surface structured quota exhaustion errors
feat(memory): add time-filtered trace recall
feat: trust Codex hooks and reveal account identifiers
…faults (MemTensor#7) loadConfig() caught YAML parse and schema validation errors, logged a console.warn, and returned a fresh default Config. Any onboarded settings (including BYOK provider credentials) were silently discarded and the CLI kept running against defaults with no non-zero exit, contradicting the project's fail-loud philosophy. Throw a new ConfigError (ConfigLoadError for bad YAML/schema, EnvValueError for missing env var references, both now exported) when the config file exists but is invalid. A missing config file still returns the defaults unchanged, since first-run `memmy onboard` depends on that path. The CLI entrypoint (main.ts) catches ConfigError specifically and prints a concise `memmy: <reason>` message with a non-zero exit code instead of an unhandled stack trace; other entrypoints (serve/gateway route through the same CLI main(), and the frontend-bridge/websocket HTTP layer already has generic catch blocks that turn thrown errors into clean error responses).
…allback (MemTensor#9) MemmyMemoryHook.beforeRun/afterRun/sessionStart/sessionEnd caught every connection error from the memory service and only recorded it on an unread `lastError` field, so an unreachable service (e.g. default http://127.0.0.1:18960 down) degraded completely silently: no CLI/log warning, and the LLM turn context received neither a recall block nor any notice, indistinguishable from "memory checked, found nothing". This contradicted the README's fail-loud promise and drove agents to silently fall back to the local MEMORY.md/history.jsonl store. - Replace the blanket try/catch with per-phase handling that emits a deduped console.warn (once per session, reset on recovery) and injects an honest <memmy_memory_status status="unavailable"> notice into the user message so the model is told memory wasn't checked, instead of implying nothing relevant was found. - Drop the stale in-flight turn entry on a failed beforeRun so afterRun doesn't try to complete a turn that was never opened server-side. - Document the new status tag in the memory context protocol prompt. Failures still never crash the turn (fail-loud, not fail-crash).
The OpenAI-compatible /v1/chat/completions endpoint always returned
usage: {prompt_tokens: 0, completion_tokens: 0, total_tokens: 0},
because chatCompletionResponse() hardcoded zeros instead of reading
the usage the agent runtime already tracks (AgentRunner accumulates
per-iteration usage in runner.ts, and AgentLoop.runAgentLoop already
normalizes it into `lastUsage`, but that value never left the loop).
Thread the turn's accumulated usage through the existing pipeline
instead of introducing a new channel: runAgentLoop now returns it
alongside its other turn outputs, TurnContext carries it, and
assembleOutbound attaches it to OutboundMessage.metadata.usage. The
API entrypoint reads it from there, sums it across the empty-response
retry path, and reports it in the JSON response. Streaming responses
now also honor stream_options.include_usage by emitting a trailing
usage-only chunk (choices: []), matching the OpenAI SSE convention.
fancyboi999
force-pushed
the
fix/openai-api-usage
branch
from
August 4, 2026 09:08
0ec980c to
6c8cc1b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
memmy serve的 OpenAI 兼容 API/v1/chat/completions返回的usage恒为{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}。把 memmy 当 OpenAI 端点接入其他工具时计量完全失真。根因
src/entrypoints/openai-like-api/server.ts的chatCompletionResponse()硬编码了三个 0。真实 usage 一直存在：AgentRunner.run()跨工具调用迭代累计 usage，AgentLoop.runAgentLoop()也做了归一化（lastUsage），但从未随 turn 结果传到 API 入口。修复
沿既有的 per-turn 对象传递 usage，不依赖共享的
lastUsage字段，避免并发 turn 竞争：runAgentLoop()在v1.0.5已有的errorCategory之后追加本 turn 的归一化 usageTurnContext.usage由stateRun()填充，assembleOutbound()再挂到OutboundMessage.metadata.usagestream_options.include_usage，设置时在[DONE]前追加choices: []的 usage 尾块；未设置时行为不变冲突处理
本分支已 rebase 到当前
v1.0.5。唯一冲突来自runAgentLoop()返回元组：base 新增了errorCategory，本 PR 新增了usage。最终同时保留两者，顺序为errorCategory第 7 项、usage第 8 项，避免改变 base 现有调用方语义。验证
npm run typecheck：通过npm run build：通过schema-validation.test.ts的旧 fallback 预期；在未包含本 PR 改动的纯v1.0.5@80b07d7上可独立复现。v1.0.5已通过 fix(config): fail loud on invalid config instead of silently using defaults #7 改为 invalid config fail-loud，但该旧测试仍期待 fallback；本 PR 不顺带修改这项无关基线问题。usage={"prompt_tokens":6304,"completion_tokens":5,"total_tokens":6309}，总数校验通过include_usage: true：finish chunk 之后、[DONE]之前收到choices: []的 usage chunk；6341+5=6346include_usage：0 个 usage chunk，保留兼容行为未覆盖说明
processSystemMessage()（cron/系统通道，不经 OpenAI API）未接 usage，属于有意边界