fix(bedrock): fold streaming tool input deltas back into tool args (#6149) - #6152
fix(bedrock): fold streaming tool input deltas back into tool args (#6149)#6152tsushanth wants to merge 1 commit into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthroughThis PR fixes a streaming bug where Bedrock tool calls receive empty arguments. The fix introduces a helper function to parse accumulated streaming deltas, applies it to both sync and async streaming handlers at content-block termination, and validates the behavior with regression tests. ChangesBedrock streaming tool-call argument accumulation
🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Apologies @kimnamu — your #6150 was open ~9 hours before this and addresses the same root cause in the same file ( |
Closes #6149. Streaming twin of #4972 (non-streaming case already fixed by #5415).
_handle_streaming_converseand_ahandle_streaming_converseaccumulate Bedrock Converse'scontentBlockDeltaJSON chunks into a localaccumulated_tool_inputstring but never fold that string back intocurrent_tool_use["input"]. AtcontentBlockStopthe code readscurrent_tool_use.get("input", {}), which the start event left as the empty dict, so every streaming tool call reaches_handle_tool_executionwith{}and pydantic raisesField required [type=missing].The fix is a focused parse-on-stop, shared between the sync and async paths:
_parse_streaming_tool_input(accumulated, fallback)that returnsjson.loads(accumulated)when deltas arrived, falls back to the start-block'sinputdict otherwise (some providers send the full input up front), and degrades to{}onJSONDecodeErrorso a malformed partial doesn't crash the streamcontentBlockStopbranches now call the helper instead of readingcurrent_tool_use["input"]directlycurrent_tool_use["input"]is intentionally left untouched in the dict that's later re-appended tomessages({"toolUse": current_tool_use}) — the assistant turn echoed back to the model uses Bedrock's own start-block payload, not the parsed args, and changing that shape is out of scope here.Regression test
lib/crewai/tests/llms/bedrock/test_bedrock.py::test_streaming_tool_call_accumulates_input_deltas:contentBlockStart(toolUse) → contentBlockDelta('{"city":') → contentBlockDelta(' "Paris"}') → contentBlockStop → messageStop— exactly the shape the issue describes.available_functions["get_weather"]that records itscityarg.{"city": "Paris"}.Verified locally — fails against
mainwithAssertionError: {} == {'city': 'Paris'}and theWARNING ... Bedrock streaming returned empty contentlog; passes with the fix.No async regression test added: the existing
test_bedrock_async.pycases are all skipped in CI (VCR doesn't play back aiobotocore), and the parsing logic is shared with the sync path through_parse_streaming_tool_input, so the sync test exercises the relevant code in both call sites.Summary by CodeRabbit
Release Notes