fix(state): persist json checkpoints as utf-8

[Ghraven]

Problem

JsonProvider persists checkpoint JSON with the platform default text encoding in both the sync and async paths. The matching checkpoint readers also rely on the default encoding.

That makes checkpoint round-trips environment-dependent when serialized state contains non-ASCII text on systems whose default encoding is not UTF-8.

Before / after

Before this change, checkpoint writes and reads used the process/platform default encoding.

After this change, sync and async checkpoint writes/readers use encoding="utf-8", matching JSON's expected interoperable encoding and keeping checkpoint behavior stable across platforms.

Closes #7256

Verification

• python -m py_compile lib/crewai/src/crewai/state/provider/json_provider.py lib/crewai/tests/test_checkpoint.py
• git diff --check
• direct smoke test of JsonProvider sync + async non-ASCII checkpoint round trip via file import: passed

I also attempted:

• python -m pytest lib/crewai/tests/test_checkpoint.py -k "JsonProviderFork or checkpoint_uses_utf8" -q

That local run was blocked by environment setup in this bare checkout: first the configured --timeout=60 option required a missing pytest-timeout plugin, then collection required repo dependencies such as jsonref. The added tests are narrow and exercise the same provider paths as the passing direct smoke test.

Note: this replaces #7255, which the first-time-contributor issue gate closed before the tracking issue existed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 3d8f8401-4d77-496c-9ed5-b44551ff93f0

📥 Commits

Reviewing files that changed from the base of the PR and between 9b4e950 and c4d9372.

📒 Files selected for processing (1)

• lib/crewai/tests/test_checkpoint.py

🚧 Files skipped from review as they are similar to previous changes (1)

• lib/crewai/tests/test_checkpoint.py

Included review availability: Your plan provides up to 10 included reviews per hour; 9 remain after this review.

---

📝 Walkthrough

Walkthrough

JsonProvider now uses explicit UTF-8 encoding for checkpoint reads and writes. Tests verify that non-ASCII JSON is stored as UTF-8 bytes and survives synchronous and asynchronous round trips.

Changes

JSON checkpoint encoding

Layer / File(s)	Summary
UTF-8 checkpoint I/O and validation lib/crewai/src/crewai/state/provider/json_provider.py, lib/crewai/tests/test_checkpoint.py	Checkpoint operations now specify UTF-8 encoding. Tests verify non-ASCII JSON writes, reads, UTF-8 file bytes, and round-trip data equality.

Merge Risk: ⚪ Minimal · up to c4d93

Checkpoint JSON reads and writes now consistently use UTF-8 for synchronous and asynchronous paths, with focused coverage for non-ASCII round trips. No current merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 8 functions across 2 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the primary change: persisting JSON checkpoints as UTF-8.
Description check	✅ Passed	The description explains the problem, solution, linked issue, verification steps, and test limitations. It omits some template headings and checkbox confirmations, but it is mostly complete and releva…
Linked Issues check	✅ Passed	The changes satisfy issue #7256 by adding explicit UTF-8 encoding to synchronous and asynchronous checkpoint reads and writes, with tests for non-ASCII data.
Out of Scope Changes check	✅ Passed	The changes are limited to the requested encoding fixes and focused checkpoint tests. No unrelated changes are present.

• Fix all pre-merge checks with AI

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[Vidit-Ostwal]

The provider change is the right fix and it hits all four I/O paths (open / aiofiles.open write + Path.read_text / aiofiles.open read). In scope, no rebase needed.

The new tests will not collect: they call Path(...) but never import pathlib.Path. That is a NameError on first run. Please add from pathlib import Path (or drop Path and read bytes with open(path, "rb")).

Nit, not blocking: read_bytes() == data.encode("utf-8") is the right assertion, but on UTF-8 CI it also passes without this patch. Fine as a regression guard once the import is fixed.

CodeRabbit's docstring-coverage warning is noise here — no new functions were added.

[Vidit-Ostwal]

Path is used here and again in test_acheckpoint_uses_utf8_for_non_ascii_json, but test_checkpoint.py does not import it (and this PR does not add from pathlib import Path). These tests will raise NameError as soon as they run. The author noted they could not run pytest locally, and Actions on this first-time fork PR have not run the test suite yet, so this was not caught.

Add the import, or use open(path, "rb").read() to stay consistent with the os.path style in this class.

[coderabbitai]

Note

GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer.

[Ghraven]

Thanks for catching that, and sorry I missed the import on the first pass.

I added from pathlib import Path in lib/crewai/tests/test_checkpoint.py and pushed the update. I also re-ran the focused checkpoint coverage locally:

uv run pytest lib/crewai/tests/test_checkpoint.py -k "utf8_for_non_ascii_json" -n0 --allowed-hosts=127.0.0.1,localhost

Result: 2 passed, 64 deselected.