Skip to content

DeallocTests 4.0 (4/5): configuration, warning severity and grace period - #28

Open
DanielCech wants to merge 18 commits into
dc/release-4.0/3-tracking-hints-difrom
dc/release-4.0/4-configuration
Open

DanielCech wants to merge 18 commits into
dc/release-4.0/3-tracking-hints-difrom
dc/release-4.0/4-configuration

Conversation

@DanielCech

@DanielCech DanielCech commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

Part 4 of 5. DeallocTests 4.0 series (merge in order): #25 groundwork → #26 expectDeallocation → #27 tracking, hints, DI → #28 configuration → #29 remove legacy, docs. Why 4.0, the breaking changes and the alternatives we looked at: see #25.

Configuration for a test, a suite or an XCTest class: timeout, leaks as warnings, and a grace period that tells "released late" from "leaked".

Why

  • Some screens legitimately take longer to go away; passing timeout: to every call is noise.
  • A project adopting dealloc tests finds leaks on day one, and needs to see them without blocking CI.
  • On a loaded CI, or with a store finishing a short Task after its screen closes, an object can go away just after the timeout. That's bounded retention, not a leak, and flaky failures erode trust in the tests.

What changes

@Suite(.deallocationTimeout(.seconds(5)))
struct ScreenDeallocTests { … }

@Suite(.deallocationIssues(.warning))          // adopting: report, don't fail
struct LegacyDeallocTests { … }

// XCTest has no traits: around a test's code, or around invokeTest() for a whole class
override func invokeTest() {
    withDeallocationConfiguration({ $0.severity = .warning }) {
        super.invokeTest()
    }
}
  • One rule: timeout describes the object, so expectDeallocation still takes it; severity and grace period are policies, so they're configuration only.
  • An object is checked with the configuration in effect where it was tracked, so an XCTest teardown check or a nested trait doesn't fall back to the defaults.
  • Warnings: a non-strict expected failure in XCTest, Issue.record(severity: .warning) on Swift 6.3+, an intermittent known issue before.
  • Grace period (3 s): an object still alive at the timeout is watched a little longer. If it goes away then, it's a warning ("released after 3.2 sec … bounded retention, not a leak"). Passing checks never wait for it, and it's skipped when leaks are warnings anyway. .deallocationGracePeriod(.zero) turns it off.

How to review

Order File What
1 Expectation/DeallocationConfiguration.swift struct, task-local, traits, sync/async withDeallocationConfiguration
2 Expectation/DeallocationTracker.swift configuration captured at track time, the grace-period wait
3 Expectation/IssueReporting.swift severity
4 Diagnostics/LeakReport.swift late-release message

Source +221, tests +245. The grace period is the most optional piece of the series; it can be split out or dropped without affecting the rest.

Testing

ConfigurationTests (call vs test vs suite precedence, nested trait order), GracePeriodTests, XCTest configuration cases. No test asserts on wall-clock time, so they hold on a loaded runner. Green on macOS (both trait settings) and the iOS Simulator; a CI job on the latest Xcode compiles and tests the Swift 6.3 warning path (Issue.record(severity:)).

🤖 Generated with Claude Code

DanielCech and others added 3 commits October 5, 2026 17:55
`expectDeallocation { makeObject() }` creates an object, runs its
lifecycle, releases it and checks that it deallocates. It holds a weak
reference, so any class can be checked without conformances, and it
polls with a deadline instead of sleeping. A leak is reported at the
line of the test, to exactly one framework (Swift Testing or XCTest).

Lifecycles run the code paths where leaks appear: `.custom`,
`.loadView` (UIKit, AppKit), `.present` and `.push` with an optional
interaction, and `.hosting` for SwiftUI. UIKit steps wait for UIKit's
state instead of completion handlers, so a test can't hang.

The package has tests again; CI runs them on macOS with and without
the trait and on the iOS Simulator. DeallocTester keeps working.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- `trackForDeallocation` checks objects owned by the tested one inside
  `expectDeallocation`, and the system under test of any unit test:
  an XCTestCase method, or the `.checksDeallocation` Swift Testing trait.
- Failure messages list likely causes found in the leaked object's
  stored properties: closures, Tasks, Combine subscriptions, timers and
  reference cycles. Only for confirmed leaks, phrased as suggestions,
  because Mirror can't tell weak properties from strong ones.
- `expectDeallocation(of:resolvedFrom:)` resolves a dependency from an
  AsyncContainer, releases the shared instances and checks it; a value
  type is reported with its concrete type.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`DeallocationConfiguration` (timeout, severity, grace period) is set
per Swift Testing test or suite with traits, and in XCTest with
`withDeallocationConfiguration` around a test's code or `invokeTest()`.
`timeout` stays a parameter of `expectDeallocation` because it
describes the object; the rest is configuration only. An object is
checked with the configuration in effect where it was tracked.

`.deallocationIssues(.warning)` reports leaks without failing the test,
for adopting dealloc tests in an existing project. An object still
alive at the timeout is watched for a grace period (3 s); if it goes
away then, it's reported as bounded retention, a warning, instead of a
leak. Passing checks never wait for it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DanielCech and others added 5 commits October 5, 2026 18:13
Doc comments are kept for public API only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Doc comments are kept for public API only.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two tests asserted on wall-clock time and failed on a loaded CI
runner (6.2 s and 1.9 s against a 1 s limit). The suite timeout test
already proves the 100 ms timeout through the failure message, so its
clock check goes. The rule that warnings skip the grace period is now
`DeallocationConfiguration.effectiveGracePeriod`, which the tracker
uses and the test checks directly.

CI also runs the tests with the latest Xcode, so the Swift 6.3
warning path (Issue.record(severity:)) is compiled and tested.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Trackers incorrectly apply the first object’s configuration to every tracked object, and warning emission lacks effective assertions.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity

Open (2)
What changed in this PR

Adds configurable deallocation timeouts, warning severity, and grace-period handling across Swift Testing and XCTest.

Changes:

  • Adds task-local configuration and Swift Testing traits.
  • Reports leaks as warnings and distinguishes late releases from leaks.
  • Expands configuration, reporting, and CI coverage.
File Description
.github/​workflows/​ci.yml Tests the Swift 6.3 warning path.
Sources/​DeallocTests/​Diagnostics/​LeakReport.swift Adds grace-period and late-release messages.
Sources/​DeallocTests/​Expectation/​DeallocationConfiguration.swift Defines configuration APIs and traits.
Sources/​DeallocTests/​Expectation/​DeallocationTracker.swift Applies timeout, grace-period, and severity policies.
Sources/​DeallocTests/​Expectation/​ExpectDeallocation.swift Uses configured default timeouts.
Sources/​DeallocTests/​Expectation/​ExpectDeallocation+DependencyInjection.swift Applies configuration to DI checks.
Sources/​DeallocTests/​Expectation/​IssueReporting.swift Adds warning-level reporting.
Sources/​DeallocTests/​Expectation/​TrackForDeallocation.swift Documents configuration capture.
Tests/​DeallocTestsTests/​ConfigurationTests.swift Tests configuration precedence.
Tests/​DeallocTestsTests/​ExpectDeallocationTests.swift Shortens configured tracking timeout.
Tests/​DeallocTestsTests/​GracePeriodTests.swift Tests grace-period behavior.
Tests/​DeallocTestsTests/​LeakReportTests.swift Tests new diagnostic messages.
Tests/​DeallocTestsTests/​TrackForDeallocationXCTests.swift Covers XCTest configuration behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread Sources/DeallocTests/Expectation/DeallocationTracker.swift Outdated
Comment thread Sources/DeallocTests/Expectation/IssueReporting.swift
DanielCech and others added 5 commits October 6, 2026 10:16
- `.present(style: .popover)` anchors the popover to the host view;
  UIKit throws for a popover without a source view on iPad.
- `afterRelease` also runs when the lifecycle can't run, so a cache
  filled by the factory is still cleared. Only the check is skipped.
- `.hosting` waits until the view has appeared and started its tasks
  (with the usual 10 s deadline) instead of sleeping a fixed 50 ms, so
  a loaded runner can't remove the view before onAppear and .task run.
- A cancelled test reports nothing: neither a CancellationError thrown
  by an interaction nor a UIKit wait cut short by cancellation.
- The expected-leak SwiftUI test cancels the leaked model's task, so it
  doesn't keep running for the rest of the test process.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rows

The trait skipped verification when the test body threw, so a failing
test silently checked nothing. It now verifies before rethrowing, like
XCTest's teardown blocks, which also run after a thrown error.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
# Conflicts:
#	Sources/DeallocTests/Expectation/IssueReporting.swift
- Each tracked object keeps the configuration in effect where it was
  tracked (timeout, grace period, severity). The tracker used the
  first object's configuration for all of them, so objects tracked in
  a different configuration scope were checked with the wrong values.
  One polling loop now checks every object against its own deadlines.
- The warning tests assert that a warning is emitted with the expected
  message: in Swift Testing through withKnownIssue matching severity
  .warning (Swift 6.3+), in XCTest through an XCTestObservation that
  records expected failures. Before, they only checked that the test
  didn't fail.
- The throwing-test tracking check uses a 100 ms timeout like its
  neighbours.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@DanielCech
DanielCech added this pull request to stack #30 October 9, 2026 08:03
DanielCech and others added 2 commits October 15, 2026 11:14
The test window falls back to UIScreen.main.bounds when there is no window scene.
visionOS has no UIScreen, so linking the package from a visionOS test target failed
to compile. A window without a scene is never shown there anyway, and
waitUntilVisible already reports that a host app is needed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
DanielCech and others added 2 commits October 15, 2026 11:14
With a 2 s grace period, a loaded CI runner released the object after the grace
period and the warning became a failure. A passing check ends as soon as the
object goes away, so the grace period can be long without slowing the test down.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@ipek ipek left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed the incremental changes in PR #28. One reproduced issue is noted inline.

let elapsed = clock.now - start

pending.removeAll { check in
if check.trackedObject.object == nil {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking for nil first can classify a release after timeout + gracePeriod as a warning instead of a leak. With a 100 ms timeout, 200 ms grace period, and the main actor delayed, I reproduced a release at about 554 ms that yielded a warning and passing test. Could the tracker record the actual deinit time and compare that with the deadline?

`perform` swallowed every CancellationError, so an interaction whose child
task threw one while the test task was alive ended silently and the rest
of the interaction never ran. The guard in `reportIssue` already handles a
cancelled test, so every error now goes through it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
(cherry picked from commit 2eb60fc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants