Skip to content

History cloud sync re-parses the full listens file per song #362

Description

@LargeModGames

Per-song cloud sync (#361) re-reads and JSON-parses all of listens.jsonl on every finished track to find records newer than last_synced_at (sync_history_to_cloud_with_client in src/infra/history.rs).

This is fine at current sizes: it runs on a background task, takes tens of milliseconds for a few MB of history, and is what makes the sync self-healing after failures. But it is O(total history size) once per song, forever growing, and the file read is blocking I/O on an async worker rather than spawn_blocking.

When to act: users report hitching during heavy playback, or listens files reach tens of MB.

Known fix:

  • Persist the byte offset of the last-synced record and read only the tail of the file.
  • Wrap the read in tokio::task::spawn_blocking.

Not urgent; filed so the scaling cliff and its fix are on record.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions