Skip to content

Cache: invalidate a cache=True source when its data changes #39

Description

@jvdd

Problem

A source registered with cache=True asserts that its data is static for the process lifetime. Nothing checks that. The response cache, the cube cache, the resolved min/max bounds and the sorted flag on LFQueryBuilder all live until the process ends or the source is re-registered with raw data or a new builder. Re-registering the same builder object invalidates nothing and only warns (flexviz/server.py, register_source).

Architecture.md and the register_source docstring cite issue #27 for this. #27 is a pull request. This issue replaces that reference.

Proposal

Pick one of:

  1. A fingerprint per source (file size and mtime for a local Parquet scan; the frame id for a resident frame), checked on each request and used to drop every cache entry keyed on that source when it changes.
  2. An explicit invalidate(name) endpoint and CLI command, with no automatic check.

Option 2 is smaller and keeps the static-data contract explicit. Option 1 is what a user who edits a file expects.

Done when

  • A changed file behind a cache=True scan is served fresh after invalidation, including its bounds and sorted flag.
  • Re-registering the same builder object with cache=True drops the stale entries instead of warning.
  • The docs name the chosen behaviour and the limits.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions