Skip to content

M7-Release: Prepare the v1.0.0 release PR with go/no-go evidence - #182

Merged
akurkar07 merged 3 commits into
mainfrom
devin/issue-72-release-v1.0.0
Oct 6, 2026
Merged

akurkar07 merged 3 commits into
mainfrom
devin/issue-72-release-v1.0.0

Conversation

@akurkar07

@akurkar07 akurkar07 commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Integrator refresh (wave 7): branch updated with main after #178 (heat1d mutants) and #183 (mutant catalogue) landed, plus one CHANGELOG fix citing #183. All evidence below is re-run on the new head 07cf387: V1-AC2 now covers 14 tests (13 package mutants), local check_v1_acceptance.py 94 passed / gate PASSED, pytest -q 623 passed / 3 skipped, validate_mutants.py 13/13 mutants, 4/4 references.

Summary

Release metadata for v1.0.0. Only release-metadata files change; there is no feature or bug work.

Closes #72

Go/no-go: V1 acceptance on head 07cf387917bfaefa15fc03bc8c2dab58ea0e1f93

CI run: V1 Acceptance #37430960755 (job v1-acceptance, artifact v1-acceptance). The release dry run reruns the same gate in its "V1 acceptance gate (final tags only)" step. A local run on the same SHA (Windows, real Docker) also returned 94 passed, V1 acceptance gate: PASSED.

ID Test node id(s) Tests CI run Result
V1-AC1 tests/acceptance/test_v1_reference_gates.py (module) 9 37430960755 pass
V1-AC2 tests/acceptance/test_v1_mutants.py::test_mutation_registry_is_non_empty, ::test_mutant_passes_weak_profile_and_fails_expected_gate[*] 14 37430960755 pass
V1-AC3 tests/acceptance/test_v1_oracle_separation.py, tests/acceptance/test_task_boundaries.py (modules) 51 37430960755 pass
V1-AC4 tests/integration/test_v1_replay_determinism.py::test_repeated_replay_yields_identical_evaluator_outcomes[oscillator-update-order], ::test_perturbed_replay_response_changes_evaluator_outcomes 2 37430960755 pass
V1-AC5 tests/acceptance/test_v1_report_reconstruction.py::test_report_totals_equal_enumerated_sample_records 1 37430960755 pass
V1-AC6 tests/acceptance/test_v1_report_reconstruction.py::test_report_tables_regenerate_offline_byte_for_byte 1 37430960755 pass
V1-AC7 tests/acceptance/test_v1_ci_coverage.py (module) 7 37430960755 pass
V1-AC8 tests/acceptance/test_v1_dataset_export.py (module) 9 37430960755 pass

The full v1-acceptance.json ("ok": true, all eight passed) is posted as a PR comment and is also the CI artifact.

Workflow runs on head SHA (all success)

Workflow Event Run Conclusion
Tests (lint, typecheck, test 3.10/3.11/3.12, task-tests x4, acceptance) pull_request 37430960679 success
Task Validation (validate) pull_request 37430960650 success
Reproduce Smoke Run (reproduce-smoke) pull_request 37430960705 success
V1 Acceptance (v1-acceptance, no continue-on-error) pull_request 37430960755 success
Release, workflow_dispatch tag=v1.0.0 dry_run=true workflow_dispatch 37430973005 success (build green: metadata check, V1 gate, wheel version matches tag, fresh-venv smoke test; release skipped as intended for dry runs)

Acceptance criteria

  • Release PR against main: CHANGELOG [1.0.0] - 2026-10-06, CITATION version: 1.0.0 / date-released: 2026-10-06, classifier 5 - Production/Stable, continue-on-error removed. Verified by the diff and by check_release_metadata.py below.
  • uv run python scripts/check_release_metadata.py --tag v1.0.0 exits 0 on the head. Run locally (OK: release metadata consistent for v1.0.0) and in the release dry run's "Check release metadata" step.
  • uv run python scripts/check_v1_acceptance.py --report v1-acceptance.json exits 0, with all eight IDs passed. Run locally (exit 0) and in CI V1 Acceptance. JSON posted as a comment.
  • Every workflow run on the head SHA concluded success. See the table above (GET /actions/runs?head_sha=07cf387…).
  • release.yml dry run with tag=v1.0.0 is green on the head. Run 37430973005.
  • Go/no-go table with one row per V1-AC ID. Above.
  • Post-merge maintainer steps. Below.

Local test output (Windows, Python 3.12, Docker Desktop running)

uv run python scripts/check_release_metadata.py --tag v1.0.0   -> OK: release metadata consistent for v1.0.0 (exit 0)
uvx cffconvert --validate -i CITATION.cff                      -> Citation metadata are valid according to schema version 1.2.0.
uv run python scripts/check_v1_acceptance.py --report v1-acceptance.json -> 94 passed, 532 deselected; V1-AC1..AC8 passed; V1 acceptance gate: PASSED (exit 0)
uv run pytest -q                                               -> 623 passed, 3 skipped (exit 0)
uv run ruff check . / ruff format --check src tests tasks scripts / mypy src -> clean
tasks/{oscillator,kepler,heat1d,wave1d}: uv run pytest tests -q -> 19 / 20 / 20 / 23 passed

Maintainer post-merge steps (agents do not create tags or releases)

  1. git tag -a v1.0.0 <merge-sha> -m "InvariantLab v1.0.0", then git push origin v1.0.0.
  2. Confirm the Release workflow run for the tag published dist/* (sdist and wheel) to the GitHub Release.
  3. Confirm that invariantlab version from the release wheel prints 1.0.0, e.g. uv venv /tmp/il && uv pip install --python /tmp/il dist/invariantlab-1.0.0-py3-none-any.whl && /tmp/il/bin/invariantlab version.

Recommended, not done here because it needs maintainer settings: main still has no branch protection (GET /branches/main/protection returns 404). Make V1 Acceptance and Tests required checks, and consider cutting v1.0.0-rc.1 through the release workflow first (#121). The date-released/CHANGELOG date is 2026-10-06. If the tag is pushed on a later day, update both dates in a follow-up PR (check_release_metadata.py only checks that they are valid ISO dates).

Written by Devin

Co-Authored-By: Devin AI <devin-ai-integration[bot]@users.noreply.github.com>
@akurkar07

Copy link
Copy Markdown
Owner Author

v1-acceptance.json from uv run python scripts/check_v1_acceptance.py --report v1-acceptance.json on head e2db51f9e42b8d2de13302074f5c78566a05e29f (local Windows run with Docker; CI artifact v1-acceptance in run 37427809405 is from the same gate).

v1-acceptance.json (ok: true, V1-AC1..AC8 passed)
{
 "ok": true,
 "pytest_exit_code": 0,
 "criteria": {
  "V1-AC1": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_every_v1_task_is_covered",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[oscillator]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[kepler]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[heat1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[wave1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_gated_cases_include_domain_length_other_than_one[heat1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_gated_cases_include_domain_length_other_than_one[wave1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_missing_gate_fails_the_check",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_broken_reference_fails_a_scientific_gate",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC2": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutation_registry_is_non_empty",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[kepler/non-conservative-velocity-damping]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[kepler/unit-error-au-rounding]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/early-termination]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/float32-position]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/hard-coded-fixture-shortcut]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/non-conservative-damping]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/courant-not-squared]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/dirichlet-wrong-node]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/sign-error-startup]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/unstable-time-recurrence]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/update-order-overwrite]",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC3": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab.verification import analytical-invariantlab.verification]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[import invariantlab.verification.solvers-invariantlab.verification.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab.mutations.registry import REGISTRY-invariantlab.mutations.registry]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from tests.scientific.helpers import oracle-tests.scientific.helpers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab import verification-invariantlab.verification]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab import mutations-invariantlab.mutations]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from tests import scientific-tests.scientific]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[import invariantlab.verification_helpers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[from invariantlab import verification_helpers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[import invariantlab.mutations_extra.registry]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[from tests import scientific_helpers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[oscillator]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[kepler]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[heat1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[wave1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[oscillator]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[kepler]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[heat1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[wave1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_agent_facing_task_sources_do_not_import_trusted_evaluator",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_task_namespace_has_no_stale_m2_placeholders",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[oscillator-import solver-expected0]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[kepler-from solver import solve-expected1]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[heat1d-from tasks.heat1d.src import solver-expected2]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[wave1d-import tasks.wave1d.src.solver-expected3]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_tests_do_not_import_candidate_solver_modules",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_task_boundaries.py::test_agent_workspace_contains_no_scientific_tests",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_ignores_names_constants_annotations_and_docstrings",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_preserves_operators_and_attribute_names[return x + v * dt-return x - v * dt]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_preserves_operators_and_attribute_names[return np.sin(angle)-return np.cos(angle)]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_functions_are_not_copied_into_agent_facing_code",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import invariantlab.verification.solvers-invariantlab.verification.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from invariantlab.verification.solvers import solve-invariantlab.verification.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from invariantlab.verification import solvers-invariantlab.verification.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from .solvers import solve-.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from . import solvers-.solvers]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import solver-solver]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from solver.integrator import solve-solver.integrator]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import tasks.oscillator.src.solver-tasks.oscillator.src.solver]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from tasks import oscillator-tasks]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from candidate_runner import run-candidate_runner]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[import numpy as np]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[from scipy.integrate import solve_ivp]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[from invariantlab.verification import analytical]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_modules_do_not_import_agent_facing_code",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[oscillator]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[kepler]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[heat1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[wave1d]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_candidate_stage_mount_exposes_no_hidden_files",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_hidden_file_checker_rejects_pre_fix_single_mount_layout",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC4": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/integration/test_v1_replay_determinism.py::test_repeated_replay_yields_identical_evaluator_outcomes[oscillator-update-order]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/integration/test_v1_replay_determinism.py::test_perturbed_replay_response_changes_evaluator_outcomes",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC5": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_report_reconstruction.py::test_report_totals_equal_enumerated_sample_records",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC6": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_report_reconstruction.py::test_report_tables_regenerate_offline_byte_for_byte",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC7": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[task validation]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[replay smoke run]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[report reconstruction]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[V1 acceptance gate]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_ci_scripts_are_not_stubs",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_trigger_parser_rejects_non_pr_or_other_branch",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_stub_marker_detection",
     "outcome": "passed"
    }
   ]
  },
  "V1-AC8": {
   "status": "passed",
   "tests": [
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_exported_records_have_full_provenance",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_exported_dataset_passes_credential_scan",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[env]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[authz]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[api_key]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[sk]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[hf]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[ghp]",
     "outcome": "passed"
    },
    {
     "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[github]",
     "outcome": "passed"
    }
   ]
  }
 },
 "unknown_ids": []
}

Written by Devin

@akurkar07
akurkar07 marked this pull request as ready for review October 6, 2026 07:15
akurkar07 and others added 2 commits October 6, 2026 07:38
Co-Authored-By: Devin AI <devin-ai-integration[bot]@users.noreply.github.com>
@akurkar07

Copy link
Copy Markdown
Owner Author

v1-acceptance.json from uv run python scripts/check_v1_acceptance.py --report v1-acceptance.json on head 07cf387 (Windows, Docker): all eight IDs passed.

v1-acceptance.json
{
  "ok": true,
  "pytest_exit_code": 0,
  "criteria": {
    "V1-AC1": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_every_v1_task_is_covered",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[oscillator]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[kepler]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[heat1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_reference_passes_every_scientific_gate[wave1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_gated_cases_include_domain_length_other_than_one[heat1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_gated_cases_include_domain_length_other_than_one[wave1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_missing_gate_fails_the_check",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_reference_gates.py::test_broken_reference_fails_a_scientific_gate",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC2": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutation_registry_is_non_empty",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[heat1d/discretisation-grid-spacing]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[heat1d/sixth-order-stencil-ftcs-limit]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[kepler/non-conservative-velocity-damping]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[kepler/unit-error-au-rounding]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/early-termination]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/float32-position]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/hard-coded-fixture-shortcut]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[oscillator/non-conservative-damping]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/courant-not-squared]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/dirichlet-wrong-node]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/sign-error-startup]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/unstable-time-recurrence]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_mutants.py::test_mutant_passes_weak_profile_and_fails_expected_gate[wave1d/update-order-overwrite]",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC3": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab.verification import analytical-invariantlab.verification]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[import invariantlab.verification.solvers-invariantlab.verification.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab.mutations.registry import REGISTRY-invariantlab.mutations.registry]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from tests.scientific.helpers import oracle-tests.scientific.helpers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab import verification-invariantlab.verification]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from invariantlab import mutations-invariantlab.mutations]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_detects_trusted_imports[from tests import scientific-tests.scientific]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[import invariantlab.verification_helpers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[from invariantlab import verification_helpers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[import invariantlab.mutations_extra.registry]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_boundary_checker_allows_benign_near_prefixes[from tests import scientific_helpers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[oscillator]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[kepler]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[heat1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_rejects_missing_expected_task_root[wave1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[oscillator]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[kepler]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[heat1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_repository_boundary_scans_each_expected_task_source_root[wave1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_agent_facing_task_sources_do_not_import_trusted_evaluator",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_task_namespace_has_no_stale_m2_placeholders",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[oscillator-import solver-expected0]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[kepler-from solver import solve-expected1]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[heat1d-from tasks.heat1d.src import solver-expected2]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_boundary_checker_detects_candidate_solver_imports[wave1d-import tasks.wave1d.src.solver-expected3]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_scientific_tests_do_not_import_candidate_solver_modules",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_task_boundaries.py::test_agent_workspace_contains_no_scientific_tests",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_ignores_names_constants_annotations_and_docstrings",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_preserves_operators_and_attribute_names[return x + v * dt-return x - v * dt]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_normalised_body_preserves_operators_and_attribute_names[return np.sin(angle)-return np.cos(angle)]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_functions_are_not_copied_into_agent_facing_code",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import invariantlab.verification.solvers-invariantlab.verification.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from invariantlab.verification.solvers import solve-invariantlab.verification.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from invariantlab.verification import solvers-invariantlab.verification.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from .solvers import solve-.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from . import solvers-.solvers]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import solver-solver]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from solver.integrator import solve-solver.integrator]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[import tasks.oscillator.src.solver-tasks.oscillator.src.solver]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from tasks import oscillator-tasks]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_detects_forbidden_forms[from candidate_runner import run-candidate_runner]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[import numpy as np]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[from scipy.integrate import solve_ivp]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_import_checker_allows_independent_dependencies[from invariantlab.verification import analytical]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_oracle_modules_do_not_import_agent_facing_code",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[oscillator]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[kepler]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[heat1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_agent_workspace_exposes_no_hidden_files[wave1d]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_candidate_stage_mount_exposes_no_hidden_files",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_oracle_separation.py::test_hidden_file_checker_rejects_pre_fix_single_mount_layout",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC4": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/integration/test_v1_replay_determinism.py::test_repeated_replay_yields_identical_evaluator_outcomes[oscillator-update-order]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/integration/test_v1_replay_determinism.py::test_perturbed_replay_response_changes_evaluator_outcomes",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC5": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_report_reconstruction.py::test_report_totals_equal_enumerated_sample_records",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC6": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_report_reconstruction.py::test_report_tables_regenerate_offline_byte_for_byte",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC7": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[task validation]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[replay smoke run]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[report reconstruction]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_pull_request_ci_runs_required_job[V1 acceptance gate]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_ci_scripts_are_not_stubs",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_trigger_parser_rejects_non_pr_or_other_branch",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_ci_coverage.py::test_stub_marker_detection",
          "outcome": "passed"
        }
      ]
    },
    "V1-AC8": {
      "status": "passed",
      "tests": [
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_exported_records_have_full_provenance",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_exported_dataset_passes_credential_scan",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[env]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[authz]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[api_key]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[sk]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[hf]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[ghp]",
          "outcome": "passed"
        },
        {
          "nodeid": "tests/acceptance/test_v1_dataset_export.py::test_credential_scan_fails_on_injected_canary[github]",
          "outcome": "passed"
        }
      ]
    }
  },
  "unknown_ids": []
}

Written by Devin

@akurkar07
akurkar07 merged commit b517aea into main Oct 6, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

M7-Release: Prepare the v1.0.0 release PR with go/no-go evidence

1 participant