Repository navigation
Native ignores activity wait_cancellation_completed on whole-run cooperative cancellation #958
Description
Problem
Native Server ignores an activity's wait_cancellation_completed cancellation policy when a whole run is cancelled cooperatively. It always delivers the cancellation as try_cancel. A PHP Server workflow that waits for the worker to acknowledge an activity stop before continuing doesn't wait on native.
Found while triaging durable-workflow/workflow#923, by reading the source at native-3.0.1 (923d45b3). Nothing was run.
Evidence (native-3.0.1)
rust/src/runtime/cooperative.rs:469-481: the activity branch of run-level cancellation delivery callsdeliver_activity_cancellationunconditionally. Its comment says native "records no explicit activity cancellation policy, so it has no waiting or abandoning delivery".- The policy is stored with the activity (
rust/src/runtime/store.rsaround 4614-4619) and honoured only inside cancellation scopes (rust/src/runtime/cancellation_scopes.rsaround 1741-1767). - Server PR Rust #325: activity cancellation acknowledgement like Server 2.11.1 #745 (merged 2026-10-25) says this difference was logged in the port log, but
docs/rust-server-port.mdat the tag has no such entry. - Native has no test of this policy for activities on whole-run cancellation. Existing tests cover it for child workflows and scopes.
Not affected: task selection after a retry. Native targets the live retry task and attempt, and acknowledgement fencing and duplicate acknowledgement are correct. See a_retried_activity_acknowledges_on_its_live_task in rust/tests/execution/activity_cancellation_ack.rs. That was the workflow#923 defect, and native doesn't have it.
Reproduction (expected vs native)
- Schedule an activity with
cancellation_policy: "wait_cancellation_completed"andmax_attempts: 2. Fail attempt 1 retryably, so attempt 2 holds a new live task, then lease it. - Request workflow cancellation and deliver it (
deliver-cancellationfor that activity sequence). - Expected (PHP Server 2.11.5+ / Core 2.15.1+): the live attempt/task is cancelled, and
ActivityCancelledsnapshots the live task. Cooperative delivery then waits until the worker acknowledges the stop (ActivityCancellationAcknowledged), and only then is the workflow resumed with the cancellation. This was verified on PHP for both policies; see workflow#923 comments 6443785394 and 6444058199. - Native: the delivery is treated as
try_cancel, so the workflow continues without waiting for the acknowledgement.
Acceptance
- Whole-run cooperative cancellation honours the stored activity policy (
try_cancel,wait_cancellation_completed, andabandonif supported), matching PHP Server behaviour or a documented decision. - A native test covers cancel-after-retry with
wait_cancellation_completed: delivery holds until the live-task acknowledgement, and duplicate acknowledgement returns the same event. - The port log records the decision.
Tracked under #325. Filed by the workflow repo agent; Server/native owners decide scope and priority.
Activity
rmcdaniel commentedon Nov 2, 2026
Confirmed and taking it. Review tier B: new write-path behavior for cooperative delivery; no tier-A area. Design, following PHP Core at
637ea0b5(DefaultWorkflowTaskBridge::deliverCancellation,ActivityCancellationWait,ActivityCancellationAcknowledgement,V2PortableCancellationDeliveryTest):Policy. Read from
ActivityScheduled.payload.activity.cancellation_policy, defaulting totry_cancel. The same applies to each activity member of a parallel delivery.try_cancel: unchanged. The live attempt and task are cancelled and the delivery happens immediately.wait_cancellation_completed: the live attempt is cancelled as now (ActivityCancelledsnapshots the live task). If any targeted execution isn't yet resolved, the delivery is held: the workflow task's payload getscancellation_activity_wait = {request_id, root_request_id, cleanup_deadline_at, execution_ids}, the task completes, and the run goes towaiting. The answer is{"delivered":false,…,"reason":"cancellation_waiting_for_activity","claim_released":true}, the same shape as the child wait. An execution counts as resolved when it completed or failed before the request (noActivityCancelled), was cancelled with no started attempt, or its live attempt'sActivityCancellationAcknowledgedis recorded.abandon: the activity isn't cancelled, and delivery happens immediately. As in PHP, this needs the activity's finiteschedule_to_closedeadline. Without one, native refuses409 cancellation_activity_abandon_not_supported(PHP throws).Resume. When the acknowledgement records
ActivityCancellationAcknowledged, the same transaction creates one ready workflow task with{"resume_source_kind":"activity_cancellation_acknowledged","resume_source_id":<ack event>,"workflow_command_id":<request>}. It does this only if all of these hold:waitingand the cleanup deadline hasn't passed;execution_idsinclude this execution, and every listed execution is resolved.A duplicate acknowledgement returns the original event and resumes nothing.
Redelivery and deadline. The resumed task re-sends the delivery. The call boundary is re-validated, as for child waits, and now every execution is resolved, so it delivers. If no acknowledgement arrives, the existing cleanup-deadline repair finishes the run cancelled, never delivering late. A late acknowledgement is recorded with
received_after_deadline:trueand doesn't resume.Tests (native execution suite, every engine):
try_cancelis unchanged;abandondelivers without cancelling, and is refused withoutschedule_to_close.The decision goes in
docs/rust-server-port.md. It ships in the next native patch.Native: accept an explicit activity cancellation policy, as Core does (…