Skip to content

Keep truncated UTF-8 prefixes and skip ignored binaries before decode - #14

Merged
mikkel merged 2 commits into
mainfrom
cursor/main-green-utf8-0192
Oct 13, 2026
Merged

mikkel merged 2 commits into
mainfrom
cursor/main-green-utf8-0192

Conversation

@mikkel

@mikkel mikkel commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Summary

  • A truncated UTF-8 tail keeps its valid prefix when the tail is a legal partial character (b"a\xc3" and b"a\xe0\xa0" decode to "a", including a tail past the 8192-byte probe). Illegal partials stay binary, including 61e080, 61eda0, 61f080, and 61f490. on_binary="include" still returns the original bytes.
  • Ignored paths are excluded before their bytes are decoded. Regressions cover both cases.
  • Vendored tests/fixtures/parity/relevance.json byte-identical from 255BITS/patchling cursor/main-green-utf8-915b at f26c770a026f2554da3972c10b7f7918d075fd56. sha256 dbe5c7033b279be2a0b923d82dffb1349039b051e18a3650b224570374b8914b.

The hardcoded tests/test_packaging.py::test_version_constant_is_0_8_1 failure on v0.9.1 (51841f2, run 38342864453) is already fixed on main by v0.9.2 (43127320): packaging tests read patchling.__version__ instead of "0.8.1". This branch does not reintroduce a hardcoded version.

PyPI to commit

Compared sdist and wheel package files from pypi.org with the tag trees (generated PKG-INFO, *.dist-info, and patchling.egg-info ignored). Each release matches its tag and not the parent or the neighboring releases. The only sdist-only file is a 38-byte setuptools setup.cfg (tag_build / tag_date) that is not in git.

  • 0.9.0 → 6077de5e5aaf9b1130b2e9960b6cefeb354893d3 (tag v0.9.0)
  • 0.9.1 → 51841f2996ae6910bf6df160728b49533cba7dc4 (tag v0.9.1)
  • 0.9.2 → 43127320f5f8e681910760a71d91e12ab8d67c30 (annotated tag v0.9.2, tag object 7671175c4bb7ab0b50d7dac5b9ac5f48ad2b8c21)

Test plan

  • pytest -q tests offline (171 passed, 2 skipped live tests). LIVE_LLM was unset.
  • CI Core API Tests on 2069eaf
Open in Web Open in Cursor 

cursoragent and others added 2 commits October 13, 2026 14:54
A short tail such as b"a\xc3" stays "a" at any length. Ignored paths are
excluded before those bytes are decoded. Vendor the JS parity fixture from
cursor/main-green-utf8-915b at 034aa2f.

Co-authored-by: ntc-ai <ntc-ai@users.noreply.github.com>
61e080, 61eda0, 61f080, and 61f490 stay binary. The decoder already
rejects those second bytes, so the filter implementation is unchanged.

Co-authored-by: ntc-ai <ntc-ai@users.noreply.github.com>
@mikkel
mikkel marked this pull request as ready for review October 13, 2026 16:31
@mikkel
mikkel merged commit bf9c8f0 into main Oct 13, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants