Lossless cancelable and joinOrCancel

[reardonj]

Context

There are a number of open bugs relating to data loss during races:

• Add Fiber#joinOrCancel #4620
• Queue.take.timeout(...) loses elements on cancelation #4571
• Possible Issue with Timeout in Cats-Effect 3.6.x which resulted in Http4s Connection leaks #4489

Fundamentally, these all must occur because there is no way to do all of the following together:

• start a fiber
• observe cancelation
• cancel the started fiber
• return the result of the started fiber if it completes before it gets canceled

In particular, the fiber that started the other fiber doesn't know it's getting canceled until it observes cancelation (ie. onCancel runs). At this point it is no longer possible for the fiber to complete, it must cancel. So all it can do is terminate its child fibers and carry on¹.

I had previously tried to solve this problem with a new onCancelRequested combinator (#4633 ), but that would break other Fs. This implementation instead follows @djspiewak's suggestion to base the solution on an @armanbilge's cancelable fix in #3491 which would give cancelable different behavior in IO, but also leave a working (but not ideal) implementation for other Fs already.

New cancelable Behaviour

In IO, cancelable can be implemented without hoisting the operation to a separate thread, by invoking the callback when cancelation is requested. Partly based on Arman's previous attempt in #3491

This changes the behavior of cancelable in IO vs other Fs. Specifically if this code is canceled:

ioa
  .onCancel(IO.println("onCancel"))
  .cancelable(IO.println("cancelable"))

In IO, the message "cancelable" will print first, while an F using the default GenSpawn will print "onCancel" first, as it's cancelable is implemented in terms of onCancel. cancelable doesn't describe any details of its semantics, so we're taking the liberty of choosing a different way to implement it 😸

Changing the behavior this way is extremely useful! By invoking cancelable finalizers before the fiber observes cancelation, we are able to give the operation a chance to return a result before the fiber is canceled and becomes unable to return a result, but also try canceling the operation to make sure cancelation does actually happen if we can't complete.

Changes to Use This Behaviour

Unsafe usages of join.onCancel(cancel) are replaced with usages of cancelable so that they can become lossless in IO. This does make these usages less efficient in other Fs, as we now start an extra fiber and join it. But in IO, joinOrCancel will result in exactly one of getting outcome of the fiber or the fiber being canceled². cancelable remains lossy by default, but this is unfixable without new semantics which would break compatibility.

This change also required a new cancelable that accepts a poll, to avoid introducing a cancelation boundary while setting up the cancelable machinery.

The implementation of fromCompletableFuture is also updated to use cancelable instead of it's bespoke cont implementation. This implementation is no better by default, but will no longer lose data in IO.

Implementation

IOFiber

cancelable in IO is now handled as a new primitive IO.Cancelable class. This operation introduces a second finalizer (referred to as acks to differentiation from the actual finalizers) stack to IOFiber. When the fiber receives a cancelation request, all acks are immediately run in parallel and any acks that get added after this point are also immediately started. This behavior is intended to drive the fiber towards cancelation as quickly as possible, with the expectation that acks are safe to run at any time, unlike finalizers, which clean up resources. The fiber waits for all acks to complete before running finalizers, since the acks could depend on a resource that will be disposed by a finalizer. If the IO.Cancelable would come off the stack, the fiber will also wait on the ack to complete³

IO.racePair is also updated to use cancelable instead of async cancelation so it does not lose data during a race following @armanbilge's earlier implementation.

polling cancelable

To fully plug the race, I needed a new cancelable that accepted a poll parameter. The existing problematic code (join.onCancel(cancel)) was typically wrapped in a poll so that the join could be canceled. Just changing this to poll(joinOrCancel) is insufficient, as there is now a cancelation boundary before the finalizer is installed, leading to possible leaks of the fiber. For the default implementation, we needed to be able to make cancelable unmask only after everything is set up, so a poll needs to be passed in. The IO implementation ignores the poll, since its handling ignores masking entirely.

Footnotes

1. You can do a little better if you know the fiber is returning a resource, and clean up that resource, but you still can't get data back out. ↩

2. or non-termination, of course. ↩

3. The ack should already be completed, since the ack cancelable action should ↩