Skip to content

[FLINK-40655][runtime] Fix flaky partition release test - #29183

Open
qiuyanjun888 wants to merge 1 commit into
apache:masterfrom
qiuyanjun888:codex/flink-40655-fix-flaky-partition-release-test
Open

qiuyanjun888 wants to merge 1 commit into
apache:masterfrom
qiuyanjun888:codex/flink-40655-fix-flaky-partition-release-test

Conversation

@qiuyanjun888

Copy link
Copy Markdown

What is the purpose of the change

ExecutionVertexTest.testResetForNewExecutionReleasesPartitions can fail intermittently because it asserts that the partition-release future is incomplete immediately after execution.markFinished(). Partition release may already be triggered asynchronously at that point.

Brief change log

  • Remove the race-prone intermediate completion assertion.
  • Retain the final assertion that verifies the expected ResultPartitionID is released after resetForNewExecution().

Verifying this change

This change is covered by the existing test:

  • ExecutionVertexTest#testResetForNewExecutionReleasesPartitions
  • Passed in 10 consecutive local runs on the latest master.

Does this pull request potentially affect one of the following parts:

  • Dependencies: no
  • Public API: no
  • Serializers: no
  • Runtime per-record code paths: no
  • Deployment or recovery: no
  • S3 file system connector: no

Documentation

  • Does this pull request introduce a new feature? no

Was generative AI tooling used to co-author this PR?
  • Yes

Generated-by: Codex (GPT-6)

Remove the race-prone intermediate completion assertion while retaining the final partition release verification.

Generated-by: Codex (GPT-6)
@flinkbot

flinkbot commented Sep 15, 2026

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands The @flinkbot bot supports the following commands:
  • @flinkbot run azure re-run the last Azure build

@qiuyanjun888

Copy link
Copy Markdown
Author

@MartijnVisser could you please review?

@MartijnVisser MartijnVisser left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The release happens inside startScheduling, not after markFinished. Execution#deploy hops to the scheduler's general executor and back, and when the continuation lands on that thread the main thread assertion in forMainThread trips, so the deployment is marked failed and the partitions get released. Passing a DirectScheduledExecutorService there, as IntermediateResultPartitionTest does, keeps it on one thread. That was green over 20000 runs, against 5 failures in 12000 on master.

@@ -96,8 +96,6 @@ void testResetForNewExecutionReleasesPartitions() throws Exception {

execution.markFinished();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both runs linked on the ticket fail at line 95, the assertion before this call. Removing the one after it leaves the flake in place.


execution.markFinished();

assertThat(releasePartitionsFuture).isNotDone();

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also drops the check that finishing the producer does not release its blocking partitions. The final assertion cannot catch that, since the future keeps whichever release completed first.

@davidradl davidradl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

withdrawing my approval - thanks for the insights @MartijnVisser

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants