fix: caller-controlled scope IDs grow Runtime caches without bound - #1325
fix: caller-controlled scope IDs grow Runtime caches without bound#1325thunguo wants to merge 3 commits into
Conversation
Signed-off-by: ThunGuo <tew@apache.org>
There was a problem hiding this comment.
Pull request overview
This PR introduces a bounded, lease-aware LRU cache for scope-local built-in Runtime resources to prevent unbounded memory growth when callers use many one-off scope_id values (especially via long-running HTTP/MCP server processes). It also adds configuration, metrics, documentation, and regression tests around the new behavior.
Changes:
- Add a lease-aware
ScopeCacheto bound inactive scope-local Runtime resources (default 128) and evict corresponding relational/provider scope resources together. - Add
RuntimeConfig.scope_cache_size(and env-based configuration via existing nested settings) plus/metricsreporting for active/cached scope counts without usingscope_idlabels. - Add regression tests covering one-off scope IDs, convergence to capacity, lock waiter/holder safety, and environment-driven configuration.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/test_server.py | Verifies server settings load runtime.scope_cache_size from environment. |
| tests/test_server_metrics.py | Adds regression test ensuring one-off scope IDs don’t produce unbounded Runtime scope retention and that metrics avoid scope_id labels. |
| tests/builtin/runtime/test_scope_cache.py | Adds targeted concurrency/eviction regression test for the scope lease + lock interaction under cache pressure. |
| src/powercontext/server/metrics.py | Adds powercontext_server_runtime_scopes{state=...} gauge and setter. |
| src/powercontext/server/factory.py | Wires Runtime scope cache observer into server metrics. |
| src/powercontext/builtin/runtime/relational.py | Adds evict() to drop per-scope cached contexts and locks together. |
| src/powercontext/builtin/runtime/config.py | Adds RuntimeConfig.scope_cache_size with default and validation. |
| src/powercontext/builtin/runtime/composition.py | Plumbs scope_cache_size, evictor, and observer into BuiltinRuntime composition. |
| src/powercontext/builtin/runtime/application.py | Integrates ScopeCache, ensures Runtime close clears scope cache, and refines operation lifecycle tracking for nested operations. |
| src/powercontext/builtin/runtime/_scope_cache.py | New lease-aware, bounded LRU scope cache implementation with eviction hooks and observer support. |
| docs/en/docs/reference/configuration.md | Documents POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE. |
| docs/zh/docs/reference/configuration.md | Documents POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def observe(cached: int, active: int) -> None: | ||
| nonlocal same_scope_leases | ||
| if cached == 1 and active == 1: | ||
| same_scope_leases += 1 | ||
| if same_scope_leases == 2: | ||
| both_same_scope_operations_started.set() |
There was a problem hiding this comment.
[P1] 当前 Head 仍未修复这个等待条件。ScopeCacheCounts.cached 现在只统计 inactive entries,因此两个 same-scope operation 都持有 lease 时观察值是 cached=0, active=1;这里等待的 cached=1, active=1 永远不会出现,both_same_scope_operations_started.wait() 没有超时并会挂住测试。请在修复 _locked() 的 P0 后同步更新 predicate(或改用直接的 lease/lock 同步信号),并给等待增加有界超时,避免回归再次把测试任务永久挂起。
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
| yield context | ||
|
|
||
| def _lock(self, scope_id: str) -> asyncio.Lock: | ||
| return self._scope_cache.lock(validate_scope_id(scope_id)) |
There was a problem hiding this comment.
[P0] 删除 _locks 后 _locked() 仍在访问它
这里把 scope lock 存储切到了 _scope_cache,但紧接着的 _locked() 仍执行 self._locks.setdefault(...)。exact Head 的公开 HTTP 用例在第一个 POST /v1/context/prepare 就稳定抛出 AttributeError: BuiltinRuntime has no attribute _locks;prepare、Memory 写入、Review、Skill 等使用 _locked() 的路径都会被阻断。请让 _locked() 在当前 scope lease 内取得 _scope_cache.lock()(并保留 tracing/acquire/release 语义),再用真实 HTTP 入口补回归测试。
| @@ -0,0 +1,135 @@ | |||
| """Bounded lifecycle for scope-local Runtime resources.""" | |||
There was a problem hiding this comment.
[P1] 两个新增 Python 文件缺少许可证头
exact Head 的 License Check 已明确报告 invalid: 2,缺失文件是本文件和 tests/builtin/runtime/test_scope_cache.py,因此当前强制门禁退出 1。请按仓库现有 Python 文件格式在两个文件顶部补齐 Apache 2.0 许可证头,并重新运行 License Check。
Which issue or RFC does this PR close?
Closes #1322.
Rationale for this change
Caller-controlled
scope_idvalues could permanently populate several process-level Runtime and relational-provider caches. A long-running HTTP or MCP Server therefore exhibited linear memory growth when receiving valid requests with many one-off scope IDs, even when no Source or Memory data was persisted.The Runtime needs a bounded lifecycle for inactive scope-local objects while preserving per-scope serialization for operations that currently hold or are waiting for a lock.
What changes are included in this PR?
RuntimeConfig.scope_cache_sizeand thePOWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZEenvironment variable.powercontext_server_runtime_scopes{state="active"}powercontext_server_runtime_scopes{state="cached"}scope_idas a label.Are there any user-facing changes?
Yes, all changes are additive:
RuntimeConfig.scope_cache_sizedefaults to128.POWERCONTEXT_SERVER_RUNTIME_SCOPE_CACHE_SIZE./metricsnow exposes active and cached scope counts using a boundedstatelabel.There are no breaking HTTP, MCP, OpenAPI, or persisted-data changes. If an inactive scope is evicted, its ephemeral services are recomposed on the next request; persisted scope data is unaffected.
How was this change tested?
Targeted Runtime, provider, Server, and metrics tests:
Result:
67 passed.Repository validation:
Result:
464 passed, 7 skipped, 1 deselected.The deselected inference tracing test was also run independently and fails because the current Pydantic AI instrumentation includes schema-retry content in span attributes. No inference or tracing implementation is changed by this PR.
Manual validation sent 250 successful
POST /v1/context/preparerequests using distinct scope IDs:250.128, with0active scopes after the requests completed.AI usage statement