Project Requirements gate: accept DEVIN_ORG_ID from a variable or secret; report setup errors on the PR

[devin-ai-integration]

Summary

The first manual run of the Project Requirements gate (run 37666022613) failed with Missing configuration: DEVIN_ORG_ID. The error appeared only in the Actions log; nothing was posted on the PR.

1. Org ID from a variable or a secret. The workflow only read vars.DEVIN_ORG_ID, so an org ID saved under Secrets came through as an empty string.

- DEVIN_ORG_ID: ${{ vars.DEVIN_ORG_ID }}
+ DEVIN_ORG_ID: ${{ vars.DEVIN_ORG_ID || secrets.DEVIN_ORG_ID }}

2. Setup errors show up on the PR. Config.from_env used to fail before the GitHub client existed, so main() could only print the error. It now also calls report_setup_error:

github_env = env minus DEVIN_MAX_ACU / DEVIN_TIMEOUT_MINUTES / DEVIN_POLL_SECONDS
config = Config.from_env(github_env, require_devin=False)   # GitHub settings only
sha = github.pull_request()["head"]["sha"]
github.set_status(sha, "error", f"Gate error: {message}")
github.upsert_comment(render_error(f"{message}. Set it under Settings > Secrets and variables > Actions (see docs/...)"))

The Devin numeric settings are removed first so that a malformed one, such as DEVIN_MAX_ACU=oops, is also reported on the PR. If the GitHub settings are missing too, or a GitHub call fails, the function logs a warning instead and the original error is preserved. main(http=http_json) now takes the HTTP function as a parameter so tests can call it.

3. Tests. This PR was also the gate's first live run, and it returned fail on test coverage. To address that:

• New tests read the DEVIN_ORG_ID expression straight from project-requirements.yml and resolve its || chain against vars/secrets. They cover variable only, secret only, variable taking precedence over secret, and both missing.
• main() tests cover a missing DEVIN_ORG_ID, a missing DEVIN_API_KEY, and a malformed DEVIN_MAX_ACU. Each asserts an error status, an actionable comment, exit code 1, and no Devin calls. Removing the reporter call, or the numeric-setting filter, makes these tests fail.
• project-requirements-tests.yml now also runs when project-requirements.yml changes.

The setup table and the comment behavior in the docs are updated.

Link to Devin session: https://nasa-jpl-demo.devinenterprise.com/sessions/4be0d92fba4f4ec3b8e00342c356f8aa
Open in Devin Desktop: https://nasa-jpl-demo.devinenterprise.com/desktop/session/4be0d92fba4f4ec3b8e00342c356f8aa?variant=devin
Requested by: @jl-0

---

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

Note

Newer findings are available below. Devin Review posted a newer report on this PR, in addition to the findings presented here.

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)

[github-actions]

Project Requirements: PASS

PR #45 changes only the Project Requirements gate's CI tooling. The gate now reads DEVIN_ORG_ID from a repository variable first and falls back to a secret. When configuration is missing or invalid, it posts an error commit status and a PR comment that links the setup doc, as long as it still has GitHub access. All three requirements pass: the gate reference doc is updated to match, the change adds no site-specific coupling, and the new behavior has nominal and off-nominal tests that run in CI. All 35 gate-script tests pass locally.

Requirement	Verdict	Notes
1. Documentation parity	Pass	Under the criteria this is not a user-facing TIG change. It touches only the CI gate (.github/scripts, .github/workflows) and does not change the tig CLI, images, demos, examples, or install requirements. The gate's own maintainer doc was still updated accurately. In docs/reference/project-requirements-gate.md, the Setup table now says DEVIN_ORG_ID can be a variable or a secret and is read from the variable first, which matches vars.DEVIN_ORG_ID || secrets.DEVIN_ORG_ID in project-requirements.yml. 'How it works' now says configuration errors such as a missing DEVIN_ORG_ID or DEVIN_API_KEY appear in the PR comment with a fix, which matches report_setup_error, whose comment names Settings > Secrets and variables > Actions and SETUP_DOC. The page is already linked from docs/README.md.
2. Broad deployability	Pass	No organization-specific hosts, credentials, or toolchains are added. The change makes configuration more flexible: DEVIN_ORG_ID can be a variable or a secret, and DEVIN_API_URL keeps its public default. Error reporting uses only the configurable GitHub API URL and the Python standard library. The gate stays an optional CI integration and is not part of the core CLI or images.
3. Test coverage	Pass	The new behavior is the variable/secret fallback for DEVIN_ORG_ID and reporting setup errors on the PR. Nominal tests cover: variable only, secret only, and both set (variable wins); and report_setup_error setting an error status and posting a comment that links the setup doc. Off-nominal tests cover: DEVIN_ORG_ID missing from both, which raises 'Missing configuration'; no GitHub access, which only prints a warning; the GitHub API failing with 403 without crashing; and main() returning exit code 1 and reporting on the PR for a missing DEVIN_ORG_ID, a missing DEVIN_API_KEY, or a non-integer DEVIN_MAX_ACU, without calling the Devin API. project-requirements-tests.yml now also runs when project-requirements.yml changes, so the workflow-expression test runs in CI. All 35 tests pass locally. Two small gaps, not blocking: the workflow test reads the ${{ a || b }} expression with a regex instead of evaluating it the way GitHub Actions does; and test_report_setup_error_survives_github_failure does not assert that the warning is printed.

---

Evaluated at e045977 · Devin session · workflow run

Criteria: .github/project-requirements/criteria.md. A fail blocks the merge; a maintainer can waive it by adding the requirements-waived label.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud