fix: detect attachment mime from file contents#23291
Open
kitlangton wants to merge 5 commits intodevfrom
Open
Conversation
Some screenshot files keep a .png name even when their bytes are JPEG, which causes providers like Anthropic to reject the attachment. Sniff common binary signatures before building the data URL so the declared MIME matches the payload.
Drive read-tool attachment classification from a small file header sample so mislabeled files are treated consistently. Reuse the same sampling helper for binary detection and strengthen the regression to cover the branch decision, not just the data URL MIME.
Use the scoped FileSystem.open and File.readAlloc APIs from effect-smol instead of raw fs/promises handles. This keeps read-tool sampling and binary detection inside Effect, removes promise wrappers from the hot path, and matches the repo's existing filesystem service boundary.
Extract shared media MIME helpers so read, webfetch, and message conversion use the same image/PDF rules. Add an Anthropic regression that preserves JPEG tool-result media through model-message conversion and provider transforms.
Use one sampled buffer for both MIME sniffing and binary detection so text-file reads do not reopen the file just to classify it. Simplify the shared media helper to explicit predicates so callers do not mix overlapping kind and predicate APIs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
readtool attachments so the declared MIME matches the payload bytes.pngname even when the actual file contents are JPEG, which Anthropic rejects as an invalid image payloadimage/jpegTesting