feat(064): iris_coverage — ObjectScript line coverage#84
Merged
Conversation
- spec.md: single iris_coverage tool, mode-based (start/stop/report/run/check) - package param: auto-discover classes via %Dictionary.ClassDefinition - test_path: compiled class pattern + /noload always - Lift Evidence Requirement section with benchmark task coverage-001 - Clarifications from session 2026-07-11 - plan.md: architecture, ObjectScript code template, test strategy - tasks.md: 15 tasks across 4 phases; Phase 3 = lift gate before Polish - skills/iris-coverage-run: fix test_path format (slash→dot) and /noload
- spec: fix summary (single tool not three), mode union adds 'check', Implementation Notes: execute_via_generator not docker exec - tasks: T001/T002 add package expansion + mode=check unit test coverage, T007 fix class name (IrisDevTest.SqlPower not USER.IrisDevTest.SqlPower), T016 add cargo llvm-cov coverage gate (constitution §VIII)
…stem.LineByLine New iris_coverage MCP tool (mode=check/run/start/stop/report) that wraps %Monitor.System.LineByLine to measure which executable lines of a target class list were executed during a test run. Key implementation notes: - All ObjectScript uses single-line form (no curly-brace blocks) — valid in both compiled ClassMethod context (execute_via_generator/HTTP) and docker exec terminal - execute_coverage_code() tries HTTP first, falls back to docker exec automatically - strip_iris_banner() now strips terminal continuation prompts (USER 1S1>, etc.) - Code is unrolled per-routine at generation time — no nested For/While blocks - gmheap ≥ 256 required; mode=check validates monitor availability via dry Start() - iris_coverage added to merged_added toolset only (not all_tools — valid A/B test) Lift evidence: +0.50 (baseline 0.50 → merged 1.00) on COV-01 benchmark task. 22 unit tests + 2 live integration tests, all pass.
…91% → 73.20%) Adds targeted unit tests for uncovered pure-Rust logic paths in: - coverage.rs: parse_check_output, parse_coverage_output, parse_package_expand_output, build_coverage_run_code, build_package_expand_code, build_coverage_check_code (+43 tests) - mod.rs: ConnectionState, Toolset, SQL macro translation, host-var extraction (+21 tests) - interop.rs: status parsing, message body output, error formatting (+15 tests) - global.rs, observability.rs, symbols_local.rs: parsing and formatting paths (+60 tests) Coverage gap in tools/mod.rs (48.58%) remains — the ~8265 uncovered lines are async tool handler bodies that require live IRIS. Closing that gap requires integration test error-path coverage or extracting logic into pure functions (tracked separately).
… sentinel fix, coverage=true on iris_test - Add cobertura_path param to IrisCoverageParams; when TestCoverage unavailable, response includes cobertura_skipped field explaining why - Add testcoverage_available() async helper + testcoverage_available field in every check/run response; check response also includes testcoverage_hint when not installed - Fix build_package_expand_code: Package.% → Package. (literal percent broke %STARTSWITH, causing NO_CLASSES for all package queries) - Add COVERAGE_DATA_START sentinel in build_coverage_run_code after Stop(); update parse_coverage_output to skip RunTest stdout before sentinel (fixes class name corruption: 'All PASSEDIrisDevTest.SqlPower' → 'IrisDevTest.SqlPower') - Add coverage/coverage_classes/coverage_target_pct fields to TestParams; iris_test handler runs iris_coverage mode=run after test pass when coverage=true, embeds result in response["coverage"] - Bump version 0.9.1 → 0.9.2 - Add objectscript-coverage skill + docs/tools.md Coverage section - Add COV-02/COV-03 benchmark tasks + COV system prompts in runner - Benchmark lift: +0.67 (baseline 2.17 → merged 2.83; spec gate ≥+0.20 ✓) - 37 unit tests + 10 live integration tests, all pass
- docs/tools.md: add iris_coverage row to Code table, add ## Coverage section with mode table, quick reference, TestCoverage note, iris_test shorthand - iris-agentic-dev.toml: add objectscript-coverage to [provides] skills list
…provements - test_iris_global_unit.rs: add parse_get_output/parse_list_output/parse_subtree_output to top-level import; replace global:: qualified calls with unqualified; remove inner use statements that were unused (functions already in scope from top-level import) - test_tools_fixes.rs: len() > 0 → !is_empty() (clippy::len_zero) - test_iris_admin_observability_unit.rs: assert_eq!(x, true) → assert!(x) - test_interop_depth_unit.rs: assert!(true) → comment (always-true assertion) - benchmark/021/runner/claude_code.py: PYPR merged prompt now inlines skill content via _load_skill() rather than requiring tool call; COV category routing preserved - benchmark/021/runner/judge.py: PYPR category scoring note; rubric cleanup - skills/pyprod/SKILL.md, light-skills/pyprod/SKILL.md: import clarity improvements
Package expand prefix is 'Package.' not 'Package.%' — percent is a literal character in %STARTSWITH, not a wildcard. Tests were written before this fix.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
iris_coverageMCP tool (mode=check/run/start/stop/report) wrapping%Monitor.System.LineByLinecoverage=trueflag oniris_testfor inline test+coverage in one callpackage=param via%Dictionary.ClassDefinitiontestcoverage_availablefield in every check/run response;cobertura_pathsupport when TestCoverage IPM installedobjectscript-coverageskill added to manifestNotable fixes
COVERAGE_DATA_STARTsentinel separates RunTest stdout from coverage data; parser skips before itPackage.%→Package.(literal%broke%STARTSWITH)iris_coverageplaced correctly betweeniris_containersandiris_credential_listTests
Release notes
See
specs/064-objectscript-coverage/release-notes-0.9.2.md