Skip to content

Count a timeout retry's flips instead of forbidding them - #29

Merged
jserv merged 1 commit into
sysprog21:mainfrom
alanhc:timeout-retry-flip
Sep 18, 2026
Merged

jserv merged 1 commit into
sysprog21:mainfrom
alanhc:timeout-retry-flip

Conversation

@alanhc

@alanhc alanhc commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

a_timed_out_goal_is_retried_at_double_the_timeout asserted that no goal
flips, but the fixture has provable goals, and whether one of them times out at
1 s and then proves at 2 s depends on the host. Fixes #28.

Change

prover-timeout.c has one assertion, typed_nocast_slow_assert, that no
prover discharges. RTE adds signed-overflow obligations for its cubes, and
those are provable. The test now requires that:

  • at least one goal timed out at 1 s,
  • at least one of those is still unproved after the retry, and
  • typed_nocast_slow_assert is never among the flips. The match is exact,
    because the overflow obligations share its prefix
    (typed_nocast_slow_assert_rte_signed_overflow).

It does not assert that still_unproved and flipped add up to
timed_out_first_pass: timeout_retry_report derives still_unproved by
subtracting the flip count, so that equality holds by construction and says
nothing about WP.

The checks for the retry, the doubled timeout and the cache are unchanged. The
doc comment no longer says the flip set stays empty.

Testing

The goal name comes from running Frama-C 33.0 WP on the fixture directly
(-wp-rte -wp-model Typed+nocast -wp-timeout 1 -wp-fct slow), which reports
[Timeout] typed_nocast_slow_assert.

  • Ubuntu 24.04 riscv64 in qemu-system-riscv64 (TCG), Frama-C 33.0, Alt-Ergo
    2.6.3. Before this change the test failed alone and in every full stdio run,
    with the same test body as on main. At 5b2c13a it passed 3 of 3 alone.
  • macOS 26.6 (Apple Silicon), same Frama-C and Alt-Ergo: a4e4b95 passes alone,
    and cargo clippy --all-targets is clean.

The full gate was not rerun, because the change is limited to this one test.


Summary by cubic

Fixes the timeout retry test so it counts flips instead of forbidding them, resolving a host-dependent failure on riscv64. The test previously asserted no goal could flip; now it requires at least one timed-out goal to remain unproved and never lets typed_nocast_slow_assert itself flip. Retry, doubled timeout, and cache checks are unchanged. The timeout_retry_report doc comment now notes the live test can't force a flip on demand. Fixes #28.

Written for commit 2a14875. Summary will update on new commits.

Review in cubic

cubic-dev-ai[bot]

This comment was marked as resolved.

Comment thread tests/test-mcp-stdio.rs Outdated
Comment thread tests/test-mcp-stdio.rs
Comment on lines +7689 to +7692
/// The assertion here is one no prover discharges, so it times out again and
/// never flips. The overflow obligations RTE adds are provable, and whether one
/// exceeds T and proves within 2T is a fact about the machine: on a riscv64 VM
/// one does. So flips are counted rather than forbidden, and the unit test

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment now says a live flip can happen, but the doc comment on timeout_retry_report in src/mcp/server.rs still says "the live test only ever reaches this with an empty flip set". Update that sentence in this PR, or the two comments contradict each other.

a_timed_out_goal_is_retried_at_double_the_timeout asserted that every
goal timing out at 1 s stayed unproved at 2 s and that nothing flipped.
prover-timeout.c has one assertion no prover discharges, but RTE adds
signed-overflow obligations for its cubes, and those are provable. Its
own comment says how many of them exhaust one second is a fact about
the machine, and the same holds for whether one then proves within two.

On Ubuntu 24.04 riscv64 under qemu-system-riscv64, with Frama-C 33.0 and
Alt-Ergo 2.6.3, it failed alone and in every full stdio run: two goals
timed out, one stayed unproved, and
typed_nocast_slow_assert_rte_signed_overflow flipped.

The test now requires that at least one goal times out, that the goal
named typed_nocast_slow_assert is still a timeout once the retry is
done, and that it is never among the flips, matched exactly since the
overflow obligations share its prefix. Pinning that goal by name is what
keeps the flip check from passing vacuously if the id ever changes, and
it subsumes the report's still_unproved, which is only the flip count
subtracted from the timeout count. The retry, the doubled timeout and
the cache check are unchanged.

timeout_retry_report said the live test only ever reaches it with an
empty flip set. It does not any more, so its doc comment says instead
that the test cannot produce a flip on demand.

Fixes sysprog21#28.
@jserv
jserv merged commit 86dff30 into sysprog21:main Sep 18, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Timeout retry test assumes no goal flips on a slow host

2 participants