You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fixes SAM2 segmentation when using the current Transformers SAM2 processor.
Sam2Processor no longer returns reshaped_input_sizes in its processed inputs, but the InvokeAI SAM2 pipeline attempts to access that field during mask post-processing
Removing the obsolete argument allows SAM2 mask post-processing to complete successfully.
No correctness defect found in the change itself. The one-line removal is verified correct against the pinned dependency. Two coverage/accuracy issues remain.
Medium (test gap): invokeai/backend/image_util/segment_anything/segment_anything_2_pipeline.py:101-104 — this PR fixes a third-party contract drift (Sam2Processor dropped reshaped_input_sizes from its output) and adds no test. There is no test anywhere covering either SAM pipeline: tests/backend/image_util/ contains only test_color_conversion.py and test_vendor_mutable_defaults.py, and a repo-wide search for *segment* / *sam* test files returns nothing. The identical construct still lives in the SAM1 path at invokeai/backend/image_util/segment_anything/segment_anything_pipeline.py:92, so the next upstream transformers bump that drops reshaped_input_sizes from SamProcessor reproduces issue [bug]: Segment Anything V2 fails every time #9551 verbatim with zero CI signal. The failure mode is a runtime AttributeError inside an invocation, i.e. a user-visible generation failure, not an import-time or type error, so neither lint nor typecheck catches it.
To expose this issue, add a test that runs SegmentAnything2Pipeline.segment() against a randomly initialized Sam2Model(Sam2Config()) plus a real Sam2Processor(image_processor=Sam2ImageProcessor()) and asserts the returned tensor is torch.bool with shape [num_masks, 3, original_height, original_width]. This needs no downloaded weights and runs in seconds. The same test shape should be added for SegmentAnythingPipeline so the still-present reshaped_input_sizes access in the SAM1 path is pinned by a test rather than by luck.
Low (stale, now actively misleading comment): invokeai/backend/image_util/segment_anything/segment_anything_2_pipeline.py:6 — the comment reads # Import SAM2 components - these should be available in transformers 4.56.0+. After this change the code depends on the 5.x processor output contract (no reshaped_input_sizes), and pyproject.toml:67 pins transformers>=5.5,<5.6. The comment advertises support for a version range the patched code no longer targets. Relatedly, the constructor docstring at invokeai/backend/image_util/segment_anything/segment_anything_2_pipeline.py:22 claims the processor "can be Sam2Processor or Sam2VideoProcessor", but the type annotation is Sam2Processor and invokeai/app/invocations/segment_anything.py:127 only ever constructs Sam2Processor. Both should be corrected while the file is open; this is a documentation-accuracy issue, not a runtime defect.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
backendPRs that change backend filespythonPRs that change python filespython-testsPRs that change python tests
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes SAM2 segmentation when using the current Transformers SAM2 processor.
Sam2Processorno longer returnsreshaped_input_sizesin its processed inputs, but the InvokeAI SAM2 pipeline attempts to access that field during mask post-processingRemoving the obsolete argument allows SAM2 mask post-processing to complete successfully.
Processing an image with the SAM2 processor:
returns:
There is no
reshaped_input_sizesRelated Issues / Discussions
Closes #9551
QA Instructions
Tested locally with:
facebook/sam2.1-hiera-largeBefore the change, SAM2 segmentation consistently failed with
KeyError: 'reshaped_input_sizes'After the change, SAM2 segmentation completes successfully
Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)