feat: resolve member contributors for unknown ecosystems via llm repo inference by mbani01 · Pull Request #5000 · linuxfoundation/crowd.dev

mbani01 · 2026-10-16T12:34:47Z

Summary

Purls from ecosystems CDP does not track are now accepted by POST /akrites-external/packages/member-contributors/jobs and routed to a new collection path instead of 400. Malformed purls (pkg:, pkg:hackage) still get 400.
A new workflow collectMemberContributorsFromInferredRepo asks Claude Sonnet 5.5 on Bedrock for up to three ranked candidate GitHub repositories, keeps those with confidence ≥ 0.7, takes the first that exists on GitHub, fetches its contributors without storing them, resolves affiliations for new identities, and returns the sync body with repoUrlSource: "llm_inferred" as the workflow result. Below-threshold, unknown, non-GitHub or missing repositories end the job as unresolvable with a reason. Truncated or refused LLM answers are retried or surfaced, never recorded as a confident unresolvable.
The poll endpoint serves unresolvable, reads inferred-repo done bodies from Temporal, and answers failed with a resubmit reason once the result has left Temporal retention. A resubmission while a result is still readable gets 409 with error.context.jobId.
Job rows record the repository used, its source, LLM confidence, per-call token usage and estimated cost, and a submission counter for repeat requests. Each LLM call is also logged with tokens, cost and stop reason. At most two LLM calls per job.
No repos, packages, package_repos or repo_contributors row is created or updated from an LLM answer. Supported-ecosystem jobs are unchanged apart from counting repeat submissions.
The response body builder moved from the backend into @crowd/data-access-layer so the worker and the API share it.

Migration

V1792150054__member_contributor_jobs_llm_repo_resolution.sql widens the two CHECK constraints on member_contributor_jobs and adds typed columns. Apply it before deploying the backend: the poll handler now selects the new columns on every path. Constraint names were verified against prod.

Closes linuxfoundation/lfx-cdp-insights-ops#502

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…ess layer Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…ults Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…openapi Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…affiliations Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…m unresolvable Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…b row Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

cursor · 2026-10-16T12:34:57Z

PR Summary

Medium Risk
Changes external job submit/poll semantics, adds LLM and Temporal-dependent result retention, and requires a DB migration before deploy; supported-ecosystem GitHub collection path is largely unchanged.

Overview
Member-contributors jobs now accept purls from ecosystems CDP does not track, routing them to a new llm_repo_resolution collection path instead of 400. Malformed purls are validated with isWellFormedPurl; only those still return 400.

For unknown ecosystems, submit creates jobs without a package/repo lookup, starts collectMemberContributorsFromInferredRepo, and tracks repeat submissions (incrementMemberContributorJobSubmissions). 409 responses use a ConflictError with error.context.jobId when a prior inferred-repo result is still readable in Temporal. Poll adds unresolvable, serves inferred done bodies from the workflow result (not DB tables), and maps expired/missing workflow results to failed with a resubmit reason.

The worker adds Bedrock-based repo inference (confidence threshold + GitHub existence checks), contributor fetch/affiliation for inferred repos, and job metadata writes (LLM usage, inferred repo_url, submission count). No package/repo/contributor rows are persisted for the LLM path beyond job state.

OpenAPI documents the new statuses, repoUrlSource: llm_inferred, and conflict shape. A migration widens member_contributor_jobs constraints and adds LLM/repo columns. Response shaping moves into @crowd/data-access-layer (memberContributorsBody) for API/worker sharing.

^{Reviewed by Cursor Bugbot for commit dbf88c8. Bugbot is set up for automated code reviews on this repo. Configure here.}

CLAassistant · 2026-10-16T12:36:33Z

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

Copilot

🟡 Changes recommended

Unresolved correctness issues affect inference outcomes, usage accounting, input validation, and duplicate-job prevention.

4 open findings

What changed in this PR

Extends member-contributor collection to unsupported ecosystems through LLM-based GitHub repository inference, without persisting inferred package/repository mappings or contributor rows.

Changes:

Adds inference, repository verification, contributor fetching, and affiliation resolution.
Extends job submission, polling, usage tracking, and result-retention handling.
Shares response builders between the API and worker, with tests and API documentation.

File	Description
services/libs/data-access-layer/src/osspckgs/memberContributorsBody.ts	Shares response builders and adds inferred-repo bodies.
services/libs/data-access-layer/src/osspckgs/memberContributorsBody.test.ts	Tests inferred response formatting.
services/libs/data-access-layer/src/osspckgs/memberContributorJobs.ts	Adds inference metadata and job operations.
services/libs/data-access-layer/src/osspckgs/memberContributorJobs.test.ts	Tests new job reads and writes.
services/libs/data-access-layer/src/osspckgs/index.ts	Exports shared response builders.
services/libs/data-access-layer/src/osspckgs/identityAffiliations.ts	Looks up affiliations by identity keys.
services/libs/data-access-layer/src/osspckgs/identityAffiliations.test.ts	Tests identity-key affiliation queries.
services/libs/data-access-layer/src/index.ts	Exports shared response builders.
services/apps/packages_worker/src/workflows/index.ts	Registers the inferred-repo workflow.
services/apps/packages_worker/src/member-contributors/workflows.ts	Orchestrates inferred-repo collection.
services/apps/packages_worker/src/member-contributors/job/startInferredRepo.ts	Starts inferred-repo jobs.
services/apps/packages_worker/src/member-contributors/job/resolveRepoViaLlm.ts	Selects and verifies repository candidates.
services/apps/packages_worker/src/member-contributors/job/llmRepoInference.ts	Requests and parses LLM candidates.
services/apps/packages_worker/src/member-contributors/job/finish.ts	Records unresolvable outcomes.
services/apps/packages_worker/src/member-contributors/job/fetchGithubRepoContributors.ts	Fetches contributors without storing them.
services/apps/packages_worker/src/member-contributors/job/canonicalGithubUrl.ts	Normalizes candidate GitHub URLs.
services/apps/packages_worker/src/member-contributors/job/buildInferredRepoBody.ts	Builds affiliated contributor responses.
services/apps/packages_worker/src/member-contributors/job/__tests__/startInferredRepo.test.ts	Tests inferred-job startup.
services/apps/packages_worker/src/member-contributors/job/__tests__/resolveRepoViaLlm.test.ts	Tests repository selection and failures.
services/apps/packages_worker/src/member-contributors/job/__tests__/llmRepoInference.test.ts	Tests model responses and usage logging.
services/apps/packages_worker/src/member-contributors/job/__tests__/fetchGithubRepoContributors.test.ts	Tests fetching and heartbeats.
services/apps/packages_worker/src/member-contributors/job/__tests__/canonicalGithubUrl.test.ts	Tests URL normalization.
services/apps/packages_worker/src/member-contributors/job/__tests__/buildInferredRepoBody.test.ts	Tests inferred response construction.
services/apps/packages_worker/src/member-contributors/github/mapContributors.ts	Extracts reusable contributor-login mapping.
services/apps/packages_worker/src/member-contributors/activities.ts	Exports new collection activities.
services/apps/packages_worker/src/member-contributors/__tests__/workflows.test.ts	Tests orchestration and retry settings.
services/apps/packages_worker/src/llm/bedrockClient.ts	Adds timed Bedrock invocation.
services/apps/packages_worker/src/config.ts	Reads Bedrock credentials.
services/apps/packages_worker/src/activities.ts	Registers new worker activities.
services/apps/packages_worker/package.json	Adds Zod dependency.
pnpm-lock.yaml	Locks the worker’s Zod dependency.
backend/src/osspckgs/migrations/V1792150054__member_contributor_jobs_llm_repo_resolution.sql	Extends job constraints and metadata columns.
backend/src/api/public/v1/packages/submitMemberContributorsJob.ts	Routes inferred jobs and handles resubmissions.
backend/src/api/public/v1/packages/submitMemberContributorsJob.test.ts	Tests inferred submissions and conflicts.
backend/src/api/public/v1/packages/memberContributorsJob.ts	Extends job response types.
backend/src/api/public/v1/packages/memberContributorCollectionPaths.ts	Routes unsupported ecosystems and validates purls.
backend/src/api/public/v1/packages/memberContributorCollectionPaths.test.ts	Tests routing and validation.
backend/src/api/public/v1/packages/getMemberContributorsJob.ts	Reads workflow results and handles expiration.
backend/src/api/public/v1/packages/getMemberContributorsJob.test.ts	Tests inferred-job polling.
backend/src/api/public/v1/packages/getAkritesExternalMemberContributors.ts	Uses shared response builders.
backend/src/api/public/v1/packages/getAkritesExternalMemberContributors.test.ts	Adjusts shared-library mocks.
backend/src/api/public/v1/akrites-external/openapi.yaml	Documents inferred jobs and response states.

Files not reviewed (1)

pnpm-lock.yaml: Generated file

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+  if (inference.status === 'unparseable') {
+    return unresolvable(inference.reason)


cursor

Stale Bugbot comment from a previous run.

…olution Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

…esult Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot

🔵 Needs a closer look

Terminal Temporal failures after database completion can leave job polling returning HTTP 500 instead of a recoverable failed status.

1 open finding

Handle Bedrock refusals as retryable failures

3 resolved since last review

Files not reviewed (1)

pnpm-lock.yaml: Generated file

Previously missed (1)

In code that hasn't changed since last review

Handle terminal workflow failures after committing done

backend/src/api/public/v1/packages/getMemberContributorsJob.ts:71

The workflow commits done before returning its Temporal result. If it is terminated or cancelled in that interval, the row stays done because failMemberContributorJob excludes done rows. result() then rejects with WorkflowFailedError, so subsequent polls return HTTP 500 instead of a job status. Catch terminal workflow failures and return failed with a resubmit reason, consistent with submission allowing replacement of these executions. Continue propagating transient Temporal errors, and add a polling test for this case.

🧠 Review effort: Balanced

…normally Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot

🔵 Needs a closer look

Bedrock refusals still become unresolvable results rather than following the promised retry or failure path.

1 open finding

Handle Bedrock refusals as retryable failures

Files not reviewed (1)

pnpm-lock.yaml: Generated file

🧠 Review effort: Balanced

Copilot

🟡 Changes recommended

GitHub rate-limit waits can exhaust inference retries, repeat paid calls, and fail otherwise resolvable jobs.

2 open findings

Files not reviewed (1)

pnpm-lock.yaml: Generated file

🧠 Review effort: Balanced

+  startToCloseTimeout: '3 minutes',
+  retry: { ...RETRY, maximumAttempts: 2 },


cursor

Cursor Bugbot has reviewed your changes and found 2 potential issues.

^{Reviewed by Cursor Bugbot for commit dbf88c8. Configure here.}

cursor · 2026-10-16T13:59:12Z

+    collectionPath: LLM_REPO_RESOLUTION_PATH,
+    packageId: null,
+    repoId: null,
+  })


Purl type case breaks job dedup

Medium Severity

Active-job reuse and the still-readable 409 both key inferred-repo jobs on the raw purl string, while purlType already treats the type as case-insensitive. pkg:hackage/aeson and pkg:Hackage/aeson therefore miss each other, so a second inference can start while the first job is pending or its Temporal result is still readable.

Additional Locations (2)

services/libs/data-access-layer/src/osspckgs/memberContributorJobs.ts#L74-L86

services/libs/data-access-layer/src/osspckgs/memberContributorJobs.ts#L189-L203

^{Reviewed by Cursor Bugbot for commit dbf88c8. Configure here.}

Fixed in #5010 (commit 744e699 there): on the inferred-repo path the purl type is lowercased before the active-job lookup, the readable-result check, the row insert and the workflow id. The fix was pushed to this branch after it had been merged, so it ships in that follow-up PR rather than in #5000.

cursor · 2026-10-16T13:59:12Z

+    throw new Error(`GitHub contributors fetch failed for ${input.repoUrl}`)
+  }
+  const logins = toGithubContributorLogins(fetched.contributors)
+  return { status: logins.length === 0 ? 'empty' : 'fetched', logins }


Large repos fail inferred jobs

Medium Severity

fetchGithubRepoContributors turns every error from fetchRepoContributors into a retryable failure, including the helper’s 10-page cap. A correctly inferred repo with more than 1000 contributors never completes: the job fails after retries, and a resubmit spends another LLM call and fails the same way.

^{Reviewed by Cursor Bugbot for commit dbf88c8. Configure here.}

Not reachable in practice: the GitHub contributors endpoint links only the first 500 author emails to GitHub users and omits anonymous contributors unless anon=1 is passed, which this code never does (docs: https://docs.github.com/en/rest/repos/repos#list-repository-contributors). With per_page=100 that is at most 5 pages, so the 10-page cap cannot trigger for this call. The same helper and cap already serve the supported-ecosystem path.

mbani01 added 19 commits October 16, 2026 11:28

feat: extend member contributor jobs for llm repo resolution

5a8ce21

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: look up affiliations for a list of identity keys

d08a68e

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

refactor: move the member contributors body builder into the data acc…

248f5f6

…ess layer Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: route unknown ecosystems to llm repo resolution

36c12ff

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: accept member contributors jobs for unknown ecosystems

a27ed2c

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: serve unresolvable and llm inferred member contributors job res…

9a9bd7d

…ults Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

docs: describe inferred-repo member contributors jobs in the akrites …

f348f87

…openapi Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: infer package repositories with claude sonnet 5.5 on bedrock

41fdfa0

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: resolve an inferred repository against github before collecting

c475ced

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: fetch github contributors for a repository without storing them

43e7a7b

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: build the inferred-repo member contributors body from identity …

804a4f6

…affiliations Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

feat: collect member contributors from an llm inferred repository

863d798

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

style: format member contributor jobs data access layer

a07a596

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: reject malformed purls before llm repo resolution

44467b4

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: surface truncated and refused llm answers instead of marking the…

d64a6b2

…m unresolvable Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: record llm usage per call and the attempted repository on the jo…

f86c44e

…b row Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

docs: document the conflict jobId and merge duplicate activity exports

2d7c4b7

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: accept uppercase purl types on llm repo resolution

92b231a

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

test: consolidate member contributors llm repo resolution tests

e2e6f78

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot AI balanced review requested due to automatic review settings October 16, 2026 12:34

Copilot started reviewing on behalf of mbani01 October 16, 2026 12:35

View session

mbani01 self-assigned this Oct 16, 2026

Copilot AI reviewed Oct 16, 2026

View reviewed changes

cursor Bot reviewed Oct 16, 2026

View reviewed changes

Comment thread backend/src/api/public/v1/packages/submitMemberContributorsJob.ts

mbani01 added 3 commits October 16, 2026 12:46

fix: reject purls with malformed percent encoding before llm repo res…

1c0d1a4

…olution Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: treat a done job whose workflow is still closing as a readable r…

b015757

…esult Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

fix: persist the usage of truncated llm attempts before retrying

c8c660f

Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot AI balanced review requested due to automatic review settings October 16, 2026 13:07

Copilot started reviewing on behalf of mbani01 October 16, 2026 13:07

View session

Copilot AI reviewed Oct 16, 2026

View reviewed changes

fix: answer failed instead of 500 when a done job's workflow ended ab…

5264d1f

…normally Refs linuxfoundation/lfx-cdp-insights-ops#502 Signed-off-by: Mouad BANI <mouad-mb@outlook.com>

Copilot AI balanced review requested due to automatic review settings October 16, 2026 13:18

Copilot started reviewing on behalf of mbani01 October 16, 2026 13:19

View session

Copilot AI reviewed Oct 16, 2026

View reviewed changes

Merge branch 'main' into feat/issue-502-llm-repo-resolution

dbf88c8

Copilot AI balanced review requested due to automatic review settings October 16, 2026 13:51

Copilot started reviewing on behalf of mbani01 October 16, 2026 13:52

View session

mbani01 merged commit e43d35b into main Oct 16, 2026
18 checks passed

mbani01 deleted the feat/issue-502-llm-repo-resolution branch October 16, 2026 13:57

Copilot AI reviewed Oct 16, 2026

View reviewed changes

Comment thread services/apps/packages_worker/src/member-contributors/workflows.ts

Comment on lines +104 to +105

startToCloseTimeout: '3 minutes',

retry: { ...RETRY, maximumAttempts: 2 },

cursor Bot reviewed Oct 16, 2026

View reviewed changes

mbani01 mentioned this pull request Oct 16, 2026

fix: dedupe llm repo resolution jobs on the lowercased purl type #5010

Merged

		if (inference.status === 'unparseable') {
		return unresolvable(inference.reason)

		startToCloseTimeout: '3 minutes',
		retry: { ...RETRY, maximumAttempts: 2 },

Conversation

mbani01 commented Oct 16, 2026

Summary

Migration

Uh oh!

cursor Bot commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

Uh oh!

CLAassistant commented Oct 16, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🔵 Needs a closer look

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot Oct 16, 2026

Choose a reason for hiding this comment

Purl type case breaks job dedup

Uh oh!

mbani01 Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cursor Bot Oct 16, 2026

Choose a reason for hiding this comment

Large repos fail inferred jobs

Uh oh!

mbani01 Oct 16, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cursor Bot commented Oct 16, 2026 •

edited

Loading

cursor Bot left a comment •

edited

Loading

mbani01 Oct 16, 2026 •

edited

Loading