Skip to content

redo: cap spool memory hot set to avoid periodic sink stalls #6033

Description

@asddongmen

Bug report

With the new-architecture redo pipeline from #5956, a sustained write workload eventually changes from continuous sink progress into long zero-throughput windows followed by large bursts.

A heap profile from the affected capture retained about 1.78 GiB under pkg/sink/codec/common.NewMsg in dmlWriter.writeEncodedEventsToSpool. The redo spool currently derives its memory quota as 20% of its disk quota. With the default 10 GiB disk quota, this creates an independent 2 GiB in-memory hot set, even when the changefeed memory quota is only 1 GiB. CPU profiles then show heavy GC plus EventStore/Pebble contention.

Expected behavior

The redo spool should keep a bounded hot set in memory and spill the remaining encoded events to its local disk quota, so it does not dominate the capture heap or cause periodic sink stalls.

Proposed fix

Keep the existing 20% behavior for small spool quotas, but cap redo spool memory at 256 MiB. For the default 10 GiB disk quota this uses a 2.5% memory ratio. Add focused unit coverage for default, small, and invalid quota inputs.

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions