Design: invalidate fixture cache when source definition changes

[navidemad]

Context

PR #61 fixed a NoMethodError when child fixtures were saved against a parent whose cache existed on disk but was not mounted in the current process. During review, a reviewer surfaced a deeper, pre-existing concern:

The lazy-load path can silently use stale parent cache data when the parent cache file exists but does not match the current parent fixture definition. Fixture#generate skips regeneration when @cache.exists? && !force, and Cache#exists? treats a disk file as sufficient. The new child-save path reads parent data directly from that file, and ActiveRecordCoder#generate merges parent_data.keys into the captured model set. No definition digest, mtime check, schema/version key, or parent-definition validation was found in Cache, FileCache, Fixture, or Runner.

This was deliberately scoped out of #61 because it is not new behavior — but it is a real correctness gap and warrants a separate design discussion.

Today's behavior

A fixture cache on disk is treated as valid forever, regardless of:

• Changes to the fixture's Definition block in source code.
• Changes to the underlying schema (new columns, dropped tables, type changes).
• Changes to Coder subclasses (new coders registered, custom encode/decode updated).
• Changes to the Rails / Ruby version that wrote the cache.

The only safety nets today are:

1. FK verification at mount time (ActiveRecord.verify_foreign_keys_for_fixtures), which only catches a subset of schema drift and only at load.
2. Manual FIXTURE_KIT_PRESERVE_CACHE opt-out and db:test:prepare.

What needs deciding

1. What identifies a "valid" cache?

Candidates:

• Source digest of the Definition block (and any procs it closes over — likely needs RubyVM::InstructionSequence or source-location-based hashing).
• mtime of the fixture file that declared the fixture. Cheaper, more brittle (touching the file invalidates).
• Schema fingerprint (digest of db/schema.rb, or ActiveRecord::Migration.current_version).
• Coder fingerprint (registered coder classes + their source digest).
• FixtureKit version string (cheap, catches gem upgrades).

A useful cache fingerprint is probably a tuple of several of these, not one. The cheapest worthwhile combo is (fixture_kit_version, schema_version).

2. Where does the metadata live?

• New top-level key in the cache JSON ("meta": { "version": ..., "schema": ..., "definition_digest": ... }). Simple, requires migrating existing caches.
• Sidecar .meta.json file. Avoids touching the existing format. More disk I/O.
• Filename suffix (fixture.json.v2.abc123.json). Visible but ugly.

3. What happens on mismatch?

• Raise with a clear message ("cache version mismatch — regenerate"). Conservative.
• Silently regenerate. Convenient but hides bugs in the fingerprint logic.
• Warn + regenerate. Probably the right middle ground; logging is cheap.

4. Backward compatibility

Existing users have a populated cache directory. Options:

• Treat a cache without a meta block as version-0 and force regeneration. One-time cost per fixture, no manual migration needed.
• Add a CLI / rake task to migrate caches in place.
• Bump major version of the gem and document the cache reset as a breaking change.

Out of scope

• Auto-regeneration logic itself. Fixture#generate(force:) already exists; this issue is about when to set force: true automatically.
• Cross-process locking. Worth tracking but separate.

Why this is filed as a discussion, not a PR

Each of the four questions above has reasonable answers in multiple directions, and the choices interact (a heavier fingerprint is more correct but more expensive; mismatch behavior is shaped by how cheap regeneration is in practice). Picking a strategy unilaterally and shipping a PR risks landing the wrong tradeoff. Looking for maintainer steer on at least Q1 and Q3 before coding.

Related

• Lazy-load parent cache content when saving a child fixture #61 — fix for the lazy-load case that surfaced this concern
• Wrap corrupt cache reads in CacheCorruptError #62 — follow-up: wrap corrupt cache errors (also from Lazy-load parent cache content when saving a child fixture #61 review)

[ngan]

Cache busting is too difficult to get right. This is why we default to building caches for every test run (overwrite). If you know what you're doing, you can opt in view environment variable. This means you need to clear out the cache yourself if you know you're doing something that would affect it.