Skip to content

Fix false Codecov project coverage drops by uploading R and JS reports together (#254) - #344

Open
Nishita-shah1 wants to merge 11 commits into
masterfrom
fix/254-codecov-failure
Open

Fix false Codecov project coverage drops by uploading R and JS reports together (#254)#344
Nishita-shah1 wants to merge 11 commits into
masterfrom
fix/254-codecov-failure

Conversation

@Nishita-shah1

Copy link
Copy Markdown

Fixes #254

Summary

  • Codecov project was failing on documentation-only commits because R and JS coverage were uploaded separately. If the JS job failed, an R-only report became the baseline, so the next full R+JS upload looked like a coverage drop.
  • Coverage jobs now save artifacts instead of uploading immediately. A follow-up job uploads both reports together, and only if both R and JS jobs succeed.
  • CRAN checks stay in their own job. Texlive is installed only there.

Test plan

  • Confirm CI runs R_coverage and JS_coverage in parallel, then Upload combined coverage to Codecov.
  • Confirm the combined upload job is skipped if either coverage job fails.
  • Confirm CRAN still runs independently.
  • On the PR, Codecov patch and project should not fail just because one coverage job failed on a previous commit.

@codecov

codecov Bot commented Aug 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 73.07%. Comparing base (8f009ed) to head (a2bb257).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #344   +/-   ##
=======================================
  Coverage   73.07%   73.07%           
=======================================
  Files         165      165           
  Lines        8933     8933           
=======================================
  Hits         6528     6528           
  Misses       2405     2405           
Flag Coverage Δ
javascript ?
r ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@tdhock

tdhock commented Aug 19, 2026

Copy link
Copy Markdown
Collaborator

@ANAMASGARD please review?

@ANAMASGARD ANAMASGARD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nishita-shah1 the combined coverage upload approach looks correct for #254. I left a few inline comments
after these changes, could you please verify one failure case where either coverage job fails and the Codecov upload is skipped, followed by a docs-only/no-code-change run showing 0.00% project coverage change? that would confirm #254 end-to-end.

Comment thread .github/scripts/save-r-coverage.R Outdated
Comment thread .github/workflows/tests.yaml
Comment thread .github/workflows/tests.yaml Outdated
@@ -1,4 +1,5 @@
acontext("render animint in pdf")
skip_if_not(nzchar(Sys.which("pdflatex")), "pdflatex not available")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Nishita-shah1 after removing Texlive from the coverage jobs, this test skips there. The CRAN job has Texlive, but build.sh uses filter="compiler", so this renderer test is not selected there either.

could we please keep this test running in at least one CI job, so the coverage test scope remains consistent with master?

Comment thread .github/workflows/tests.yaml
Nishita-shah1 and others added 5 commits August 24, 2026 21:30
Comment on lines +148 to +152
- name: run PDF renderer test
run: Rscript -e 'testthat::test_file("tests/testthat/test-renderer3-knit-print-pdf.R")'

- name: run CRAN build
run: bash build.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry, my earlier suggestion missed the normal test setup. test_file() sources the helpers directly, but animint2 is not loaded, so toRGB() is unavailable. could we load the same packages as tests/testthat.R before running this test?

Suggested change
- name: run PDF renderer test
run: Rscript -e 'testthat::test_file("tests/testthat/test-renderer3-knit-print-pdf.R")'
- name: run CRAN build
run: bash build.sh
- name: run PDF renderer test
run: Rscript -e 'library(testthat); library(animint2); library(XML); test_file("tests/testthat/test-renderer3-knit-print-pdf.R")'
- name: run CRAN build
run: bash build.sh

@ANAMASGARD

Copy link
Copy Markdown
Contributor

Sir @tdhock the PR now looks good can you please give it a last review before merge .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

incorrect codecov failures

3 participants