Repository navigation
Count frontmatter field lengths in characters, not bytes - #95
Merged
Merged
Conversation
The name, description, and compatibility limits (64, 1024, 500) were enforced with len(), which counts UTF-8 bytes, while the spec and the error messages speak of characters. A 984-character CJK description is 2952 bytes and was rejected as "exceeds 1024 characters (2952)". Count Unicode code points instead, the same unit the skills-ref reference validator uses. The judge's sanitizeStringField cap moves to 1024 characters for the same reason: it was cutting spec-compliant multibyte descriptions to a third of their length before scoring. Fixes #94
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #94.
What changed
structure/frontmatter.go: thename(64),description(1024), andcompatibility(500) limits are now measured withutf8.RuneCountInStringinstead oflen(). The counts printed in the error and pass messages use the same unit, and the limits are named constants.judge/judge.go:sanitizeStringFieldcaps at 1024 characters instead of 1024 bytes. Before this, a spec-compliant 1024-character CJK description (3072 bytes) was cut to about a third of its length before being sent to the judge.Why code points
The spec says "characters" without defining the unit. The
skills-refreference validator uses Pythonlen(), which is code points, and Codex CLI (the one major harness that enforces the description limit, by truncating its catalog entry at 1024) counts bychar_indices. Counting code points matches both. Cline counts UTF-16 units and Crush counts bytes, so those two will still disagree on emoji and multibyte text respectively, but code points is the reference behavior.The
namepattern stays ASCII-only; the reference validator's Unicode-alphanumeric acceptance is a separate question.Verification
go test ./...andgolangci-lint runare clean. No CHANGELOG entry, following the pattern of adding those in the release commit.