Repository navigation
Add strip endpoint backed by an ExifTool process pool - #1
Conversation
🟢 Tier S · Ready to merge
Adds a Latest changes: The new commits move upload and response handling to disk-backed streaming, add the pool size to build-info metrics, use ExifTool status output for rewrite results, and expand integration coverage and documentation to TIFF, GIF, WebP, AVIF, and HEIC.
Note @jaysomani does not have write access to this repository, so Hansi does not approve automatically. A maintainer can review and approve. 📂 Walkthrough · 11
Reviewed the commits since |
What does this PR do?
Adds
POST /strip, which removes metadata such as location, camera, and capture time from an image and returns the stripped file. The ICC color profile and orientation are kept, and pixel data is never re-encoded. This is the service Appwrite will call for buckets withstripMetadataenabled (appwrite/appwrite#14351), replacing the per-worker ExifTool processes that used ~2 GB per container.internal/exiftool: a fixed pool of ExifTool processes in-stay_openmode, warmed up at startup. Each command uses a numbered-execute{N}/{readyN}marker, so a stuck command's output can never leak into the next request. A process that times out, is cancelled, or exits is killed and started again on its next use./stripaccepts a raw body or a multipartfilefield. The format is detected from the content, so a renamed file is still stripped. Responses:200stripped file,422file ExifTool can't rewrite,413/400bad input,504timeout,503shutting down. Callers should keep the original on anything but200.MAX_CONCURRENT_STRIPS(default: CPUs),STRIP_TIMEOUT(30s),MAX_REQUEST_SIZE(25MiB, Appwrite strips files up to 20 MB). Buffered uploads are bounded separately from running strips, following autogravity.The scaffold is unchanged apart from passing the pool into
newApplicationand addinginternalto thegofmtcheck.Test Plan
make test-integrationagainst the pinned ExifTool 13.55: GPS, camera, date, and XMP creator removed from JPEG and PNG; orientation kept; ICC profile kept byte for byte; text file left unchanged (ErrNotStripped); recovery after a timeout and after a killed process; 12 concurrent strips on 2 processes;Closewakes a waiting strip; a failed startup leaves no processes behind.MAX_CONCURRENT_STRIPS=4:Related PRs and Issues