Skip to content

fix: move the remaining sonnet 4 calls to claude sonnet 5.5 and the merge decisions to sonnet 5 (lfx-cdp-insights-ops#820) - #4997

Merged
ulemons merged 1 commit into
mainfrom
fix/issue-820-sonnet-5-5-everywhere
Oct 16, 2026
Merged

ulemons merged 1 commit into
mainfrom
fix/issue-820-sonnet-5-5-everywhere

Conversation

@ulemons

@ulemons ulemons commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Summary

Step 2 of #820, after #4996 proved Claude Sonnet 5.5 in production on the project evaluation. Bedrock retired Claude Sonnet 4 on 2026-10-16; this moves every remaining use of it, as asked by the CDP team.

  • Seven LLM_SETTINGS query types → Claude Sonnet 5.5 (us.anthropic.claude-sonnet-5-5): member enrichment, squash of multi-value attributes, squash of work experiences, repo categories, repo collections, select most relevant domain, fake organization analysis. As in fix: move project evaluation to claude sonnet 5.5 after bedrock retired sonnet 4 (lfx-cdp-insights-ops#820) #4996: temperature removed (Claude 5 models return 400 on non-default sampling parameters) and thinking: { type: 'between_tools' } (the lowest thinking setting on Sonnet 5.5; no tools, so the response is text only). LlmService already reads the first text block.
  • Merge-suggestion workflows (members and organizations) → Claude Sonnet 5 (us.anthropic.claude-sonnet-5), not 5.5: they force the decision tool with tool_choice: { type: 'tool' }, which Sonnet 5.5 rejects with a 400 (tool_choice: type "tool" and "any" are not supported for this model). The documented replacement is tool_choice: auto plus strict: true, but on Amazon Bedrock only auto is available, and with auto the model may answer in text and send the decision into retries. Sonnet 5 still accepts forced tool use and thinking: { type: 'disabled' }, costs the same as 5.5 ($2 / $10 per MTok), and is active with EOL no sooner than 2027-06-30. temperature removed for the same reason as above.
  • No Sonnet 4 reference is left in the code; the enum value stays for prompt history. Haiku 4.5 is untouched.

Behaviour note: without temperature: 0, answers are no longer fully deterministic across repeated calls. This is unavoidable on the Claude 5 family.

Changes

  • enums/llm.ts: CLAUDE_SONNET_5.
  • llm.ts: region and pricing for Sonnet 5; the seven settings above.
  • mergeMembersWithLLM.ts, mergeOrganizationsWithLLM.ts: model id and arguments.

Before deploying

Confirm the AWS account has Sonnet 5.5 and Sonnet 5 enabled in Bedrock model access (5.5 is already used by #4996; Sonnet 5 is listed as open access).

Test plan

  • vitest types and merge_suggestions_worker, pnpm lint --deny-warnings, pnpm tsc-check
  • Deploy: backend, members-enrichment worker, profiles worker (fake organization analysis), merge-suggestions worker, projects-evaluation worker
  • processMemberSources runs without the "Activity Retry Limit Reached" alert
  • llmPromptHistory shows the new model ids for the moved types; merge decisions keep producing tool_use results

Closes linuxfoundation/lfx-cdp-insights-ops#820

🤖 Generated with Claude Code

…erge decisions to sonnet 5

Refs linuxfoundation/lfx-cdp-insights-ops#820

Signed-off-by: Umberto Sgueglia <usgueglia@contractor.linuxfoundation.org>
Copilot AI balanced review requested due to automatic review settings October 16, 2026 11:23
@ulemons ulemons self-assigned this Oct 16, 2026
@cursor

cursor Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core enrichment and merge-decision LLM paths; model and sampling changes can alter merge outcomes and enrichment quality until validated in production.

Overview
Migrates off retired Claude Sonnet 4 on Bedrock before EOL: seven shared LLM_SETTINGS query types (member enrichment, attribute/work-experience squash, repo categories/collections, domain selection, fake-org analysis) now use Claude Sonnet 5.5 with thinking: { type: 'between_tools' } instead of temperature: 0.

Member and organization merge-suggestion workflows switch to Claude Sonnet 5 with thinking: { type: 'disabled' } so forced tool_choice for merge decisions keeps working (5.5 rejects that pattern on Bedrock). Types add CLAUDE_SONNET_5 with us-east-1 region and pricing aligned to Sonnet 5.5.

Responses may be slightly less deterministic without temperature: 0, which is required on the Claude 5 family.

Reviewed by Cursor Bugbot for commit 9e4be7d. Bugbot is set up for automated code reviews on this repo. Configure here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Live Bedrock validation remains pending for Sonnet 5 workflows that automatically merge profiles.

0 open findings

What changed in this PR

Completes the model migration started in #4996, moving remaining active Sonnet 4 calls to newer models.

Changes:

  • Moves seven shared query types to Sonnet 5.5 with updated thinking settings.
  • Moves member and organization merge decisions to Sonnet 5, preserving forced tool use.
  • Adds Sonnet 5 model, region, and pricing definitions.
File Description
services/​libs/​types/​src/​llm.ts Updates shared model settings and adds Sonnet 5 region and pricing.
services/​libs/​types/​src/​enums/​llm.ts Adds the Sonnet 5 model identifier.
services/​apps/​merge_suggestions_worker/​src/​workflows/​mergeOrganizationsWithLLM.ts Migrates organization merge decisions to Sonnet 5.
services/​apps/​merge_suggestions_worker/​src/​workflows/​mergeMembersWithLLM.ts Migrates member merge decisions to Sonnet 5.

🧠 Review effort: Balanced


💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ulemons
ulemons merged commit 7ac420d into main Oct 16, 2026
15 checks passed
@ulemons
ulemons deleted the fix/issue-820-sonnet-5-5-everywhere branch October 16, 2026 11:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants