Handle missing tiktoken encoding data in api token counting

[bhavana-giri]

Summary

• make working-memory token counting resilient when tiktoken cannot load cl100k_base
• cache the encoding after the first successful load and fall back to a character-based estimate when unavailable
• add regression coverage for both direct token counting and the GET /v1/working-memory/{session_id}?model_name=... API path

Testing

• git commit pre-commit hooks (ruff, ruff format, typos, trailing whitespace, EOF checks)
• python3 -m py_compile agent_memory_server/api.py tests/test_issue_237.py

Closes #237.

---

Note

Low Risk
Low risk: changes are limited to token-counting/truncation paths and add a conservative fallback plus regression tests to prevent 500s when tiktoken can’t load.

Overview
Prevents GET /v1/working-memory/{session_id} (and summarization/truncation logic) from failing when tiktoken.get_encoding("cl100k_base") cannot load by introducing a cached encoder with a 5-minute backoff and a character-based token estimate fallback.

Refactors token counting to use _count_text_tokens throughout and adds regression tests covering both direct token counting and the API path, including retry/backoff behavior in _get_tiktoken_encoding.

^{Reviewed by Cursor Bugbot for commit d1bddbc. Bugbot is set up for automated code reviews on this repo. Configure here.}

[jit-ci]

Hi, I’m Jit, a friendly security platform designed to help developers build secure applications from day zero with an MVS (Minimal viable security) mindset.

In case there are security findings, they will be communicated to you as a comment inside the PR.

Hope you’ll enjoy using Jit.

Questions? Comments? Want to learn more? Get in touch with us.

[nkanu17]

lgtm but it's worth checking the copilot review comments

[Copilot]

Pull request overview

This PR makes working-memory token counting resilient when tiktoken can’t load the cl100k_base encoding (e.g., in air-gapped / restricted-egress environments), preventing API requests and summarization/truncation logic from failing due to missing tokenizer data.

Changes:

• Add a cached tiktoken encoding loader with a safe character-based fallback token estimator.
• Route working-memory token counting and summarization token calculations through a shared _count_text_tokens() helper.
• Add regression tests covering both direct token counting and the GET /v1/working-memory/{session_id}?model_name=... path when tiktoken.get_encoding() raises.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
agent_memory_server/api.py	Adds cached encoding load + fallback estimation and updates working-memory token counting/summarization to use it.
tests/test_issue_237.py	Adds regression coverage ensuring token counting + working-memory GET do not 500 when tiktoken encoding load fails.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

[Copilot]

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[bsbodden]

LGTM! Thanks!