Repository navigation
Fix graph_calculate_power() edge cases for marginal power and alpha (#47) - #112
Merged
Merged
Conversation
When `power_marginal` contains 0 or 1, the simulated p-values are deterministically 1 or 0. A p-value of exactly 0 combined with a closure weight of 0 makes `p <= weight * alpha` evaluate to TRUE for a hypothesis that is not in the intersection, which broke two fast power paths: * `graph_test_shortcut_fast()` (Bonferroni) looped forever: an already-rejected hypothesis kept re-appearing in `rejected_step`, so `any(rejected_step)` stayed TRUE while `rejected` never changed. * `graph_test_closure_fast()` (Hochberg/Simes/parametric) over-rejected: `rowMaxs()` reduced over hypotheses before masking by intersection membership, so a hypothesis with p = 0 marked intersections it does not belong to as rejected. For a two-hypothesis Holm graph with `power_marginal = c(1, 0)` this returned local power (1, 1) instead of (1, 0). Fix both by restricting rejections to hypotheses that are actually in the current intersection: add a `!rejected` guard in the shortcut (which also terminates the loop) and mask `rej_hyps` by `matrix_intersections` before the row reduction in the closure path. The power paths now agree with the exact `graph_test_shortcut()` / `graph_test_closure()` results on the boundary. Add regression tests covering marginal power 0/1 for Bonferroni and Hochberg. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Code Coverage SummaryDiff against mainResults for commit: ec74a47 Minimum allowed coverage is ♻️ This comment has been updated with latest results |
Marginal power is defined relative to the significance level, so the mapping to a noncentrality parameter, qnorm(1 - alpha) - qnorm(1 - power_marginal), is only meaningful for alpha strictly inside (0, 1). At alpha 0 or 1 it returns NaN when power_marginal is also 0 or 1 (erroring with "missing value where TRUE/FALSE needed"), or silently wrong power otherwise (e.g. alpha = 0 rejected every hypothesis). Validate alpha in (0, 1) in power_input_val() so these inputs give a clear error. Interior alpha with boundary marginal power (0 or 1) stays valid and is handled by the preceding commit. Add regression tests for the four alpha 0/1 cases. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #47. Covers every thread in the issue (body + both comments).
1. Boundary marginal power 0/1 (#47 (comment))
When
power_marginalcontains 0 or 1, the simulated p-values are deterministically 1 or 0. A p-value of exactly 0 combined with a closure weight of 0 makesp <= weight * alphaevaluate toTRUEfor a hypothesis that is not in the intersection, breaking two fast power paths for the two-hypothesis Holm graph withpower_marginal = c(1, 0):graph_test_shortcut_fast()(Bonferroni) looped forever — an already-rejected hypothesis kept re-appearing inrejected_step, soany(rejected_step)stayedTRUEwhilerejectednever changed.graph_test_closure_fast()(Hochberg/Simes/parametric) over-rejected —matrixStats::rowMaxs()reduced over hypotheses before masking by intersection membership, returning local power(1, 1)instead of(1, 0).Fix: restrict rejections to hypotheses actually in the current intersection — a
!rejectedguard in the shortcut (which also terminates the loop) and maskingrej_hypsbymatrix_intersectionsbefore the row reduction in the closure. Only the degeneratep == 0 & weight == 0boundary changes; with p > 0 the comparisons are identical. The fast power paths now agree with the exactgraph_test_shortcut()/graph_test_closure().c(1, 0)(1, 0)c(1, 0)(1, 1)(1, 0)c(0, 1),c(1, 1),c(0, 0)2. alpha of exactly 0 or 1 (@EeethB's comment)
power_marginalis the marginal power at level alpha, so the noncentrality mappingqnorm(1 - alpha) - qnorm(1 - power_marginal)is only defined foralphastrictly inside (0, 1). At 0 or 1 it produced:NaN→"missing value where TRUE/FALSE needed"(e.g.alpha = 0, power_marginal = c(0, 0)), oralpha = 0rejected every hypothesis —(1, 1)— andalpha = 1rejected none).Fix:
power_input_val()now requiresalphastrictly between 0 and 1, with a clear message. Interior alpha with boundary marginal power (0 or 1) remains valid and is handled by item 1. This is the "null check" @EeethB suggested.3.
force_closurediscrepancy (issue body)The original report compared
force_closure = TRUEvs the default formarginal_power = c(0, 0, 0). Those arguments (force_closure,sim_seed) no longer exist in the current API —graph_calculate_power()now picks the shortcut path for all-Bonferroni and the closure path otherwise, internally. With item 1 fixed, both internal paths agree with the exact closed test on the boundary, so this discrepancy is resolved. No separate change needed.On "should validation exclude 0/1 for marginal power?"
No.
power_marginalof 0 or 1 is a legitimate boundary (never / always reject) and now computes correctly. Onlyalphaof 0/1 is excluded, since it is genuinely undefined for a power calculation.Tests
Added regression tests in
test-graph_calculate_power.R: marginal power 0/1 for Bonferroni and Hochberg (incl. agreement with the exact closed test), and the four alpha 0/1 cases. Full suite passes (560 passed, 0 failed, 13 skipped for optionaligraph/gMCP).🤖 Generated with Claude Code