[Vedaant-J]

Stack: #53 → #112 → #50 → #60 → #64; #114 on #112; #115 on #64. Merge bottom to top.

Summary

Every graph snapshot, training table and model bundle now carries a self-contained _provenance.json that leads back to the source table versions (and locations, wherever the source backend can name them), query, plan and engine build that produced it. Today a model has no id, an artifact records no lineage, an index works only on the machine that built it, and scoring needs the training job's Graph.

This PR adds kumo.MLOps / with kumo.mlops(...) to store each artifact once under one artifact root (s3://, gs://, local, or a (connector, path) file store from #112); GraphSnapshot.export / load_from_connector to move a checksummed index between machines; snapshot.source_changes(graph) to see which tables changed since a snapshot; and Trainer.predict(snapshot) to score a stored snapshot with the trained query. MLOps(on_error='raise') turns a failed store into a RuntimeError after the call finishes. Off unless asked for.

Usage

cfg = kumo.MLOps("s3://<bucket>/kumo")             # store=True: snapshots, tables, models
with kumo.mlops(cfg):                               # or mlops=cfg on one call
    snapshot = graph.snapshot()
    result = trainer.fit(pq.generate_training_table(snapshot), snapshot=snapshot)

# Later, on another machine with no Graph code:
snapshot = GraphSnapshot.load_from_connector(cfg, id=result.snapshot_id)
trainer = Trainer.load_from_connector(cfg, path=f"models/{result.id}")
scores = trainer.predict(snapshot)

Back-compat

Without a block or mlops=, existing calls behave as before.

Situation	Behaviour
Index built before this PR	Reused, not rebuilt; its manifest is backfilled with "reconstructed": true.
Bundle published before this PR	Loads and predicts identically; id computed at load, manifest is None.
Artifact from this PR read by an older engine	Extra files are ignored.
A single S3 or GCS object source	Versioned by VersionId or entity tag, so its snapshot gets a new id once.
verify_model_bundle return value	64 hex instead of a 16-hex digest no caller used.
DatabricksConnector(identity=...)	Unchanged; identity is now optional (ambient auth).
Manifest, sidecar or root write fails	A warning; the call completes as before. A failed store raises a RuntimeError instead, once the call has finished, with MLOps(on_error='raise').

Testing

• kumo-engine: test_snapshot_export.py (round trip, reuse, interrupted and concurrent exports, corrupt markers, source_changes, on_error), test_snapshot_manifest.py, test_training_table_export.py, test_holdout_df.py (published model's holdout, loaded and fetched holdout, empty test split), test_bundle_provenance.py, test_query_from_snapshot.py, test_filestore_sink.py (identical bundle reused), test_mlops.py, test_graph_mismatch.py, test_artifacts.py.
• kumo-core typed source versions, bundle ids and export sidecars; kumo-s3 test_sink_delete.py; Snowflake, Databricks and BigQuery identity tests assert version_kind.
• DiskGraph: Rust range_checksums_cover_every_listed_file_and_the_manifest; pybind test_exception_exports.py.
• Env-gated live S3 and GCS round trips in test_snapshot_export_live.py.

Change map: see the kumo-pr-change-map comment.

🤖 Generated with Claude Code

[Vedaant-J]

Change map

File	Function / scope	What changed	Why
packages/kumo-engine/kumo/engine/_mlops.py	new: MLOps, mlops, resolve, on_root, raise_on_error, warn_failure	Artifact root, store kinds and on_error; root checked when built; block or mlops= argument	One switch stores every artifact under one root
packages/kumo-engine/kumo/engine/__init__.py	exports	Exports MLOps, mlops, the three manifests, SnapshotOutputConfig	Public API of this PR
packages/kumo-engine/kumo/engine/_manifest.py	new: Manifest, SnapshotManifest, TrainingTableManifest, ModelManifest, Artifact, as_artifact, artifacts	Frozen manifest records; stubs, embedded parents, unknown keys kept; each lists its stored artifacts	Self-contained lineage readable from any one artifact
packages/kumo-engine/kumo/engine/_provenance.py	new: environment, source_table, source_location	Environment stamp and per-table location, version, version_kind	Manifests name builds and source versions
packages/kumo-engine/kumo/engine/_snapshot_io.py	new: export, read, load_into, store	Generation export committed by kumo_snapshot.json; checksummed parallel load	Use an index on a machine that did not build it
packages/kumo-engine/kumo/engine/graph.py	Graph.snapshot, GraphSnapshot.export, load_from_connector, source_changes, manifest, schema_changes	Writes or backfills the snapshot manifest; export, load, source checks; warnings name schema changes	Snapshot lineage and portability
packages/kumo-engine/kumo/engine/pquery/training_table.py	TrainingTable.id, manifest, load_from_connector, finish_generated	Content-derived id, manifest, export sidecar, store under the root	Training-table lineage and reload
packages/kumo-engine/kumo/engine/pquery/predictive_query.py	PredictiveQuery._from_snapshot, generate_training_table	Binds a query to a snapshot; mlops= on table generation	Score from a snapshot without a Graph
packages/kumo-engine/kumo/engine/pquery/_custom_source.py	stage_source_table	Returns the staged parts and their source version	Custom tables record their version
packages/kumo-engine/kumo/engine/pquery/prediction_table.py	prediction table staging	Unpacks the new stage_source_table return	Follows the signature change
packages/kumo-engine/kumo/engine/trainer/trainer.py	TrainingResult.id, manifest, snapshot_id, predict; Trainer.fit, _fit, _store_model, result, predict(snapshot)	fit wraps _fit: bundle manifest and id, holdout in bundle, store under the root, store failures per on_error; empty test split refused	Model lineage; score a snapshot with the trained query
packages/kumo-engine/kumo/engine/trainer/_provenance.py	new: FitRecord, bundle_manifest, embedded_model, read_provenance	Builds, writes and reads the model manifest in a bundle	Model manifest kept apart from the trainer
packages/kumo-engine/kumo/engine/trainer/_spec.py	TrainingSpec.graph_config, listed_file, read_bundle_artifact_paths	Spec records graph config; bundle id returned; shared atomic writes	Graph-mismatch warnings and model ids
packages/kumo-engine/kumo/engine/trainer/output_config.py	new: SnapshotOutputConfig	Connector and path for a snapshot export	Argument of GraphSnapshot.export
packages/kumo-engine/kumo/engine/trainer/__init__.py	exports	Exports SnapshotOutputConfig	Public API
packages/kumo-engine/kumo/engine/connector/__init__.py	new: connector_for_root	Maps an s3://, gs://, file:// or local root to a connector	URI artifact roots
packages/kumo-engine/kumo/engine/connector/databricks.py	DatabricksConnector.__init__	identity optional; None uses ambient SDK auth	Jobs and notebooks need no explicit identity
packages/kumo-engine/kumo/engine/connector/file.py	table_location	Absolute path of a file table	Manifests record source locations
packages/kumo-core/src/kumo_core/sources/identity.py	new: SourceVersion, version_kind	Version string carrying its kind and dependencies	Tell exact versions from hour buckets
packages/kumo-core/src/kumo_core/sources/object_store.py	_source_version, copies	Single object versioned by VersionId or entity tag; copies public	Same-second rewrites change the version
packages/kumo-core/src/kumo_core/sources/object_store_backend.py	data_version	Returns the typed version unchanged	Keep version_kind
packages/kumo-core/src/kumo_core/sources/object_store_sessions.py	table_location	Full scheme://bucket/key without credentials	Manifests record source locations
packages/kumo-core/src/kumo_core/artifacts.py	verify_model_bundle, bundle_id_from_manifest, add_bundle_file	64-hex bundle id excluding _provenance.json; list one bundle file	Stable model id across manifest edits
packages/kumo-core/src/kumo_core/artifact_export.py	export_selected_files	Best-effort sidecars= written before _SUCCESS	Training-table export carries its manifest
packages/kumo-core/src/kumo_core/object_store_sink.py	holds_export, holds_bundle, write_files, publish_model	Skips uploading a complete identical copy; passes sidecars	Re-export and re-publish are cheap
packages/kumo-core/src/kumo_core/io/filesystem.py	new: atomic_write_bytes, fsync_directory	Shared atomic local write	Replaces three copies
packages/kumo-core/src/kumo_core/train/_artifact_storage.py	JSON writer	Uses the shared atomic write	Remove duplicate code
packages/kumo-snowflake/src/kumo_snowflake/identity.py	version stamp	Returns SourceVersion with its kind	Typed versions in manifests
packages/kumo-snowflake/src/kumo_snowflake/session.py	table_location	account/"DB"."SCHEMA"."TABLE"	Manifests record source locations
packages/kumo-databricks/src/kumo_databricks/identity.py	version stamp	Returns SourceVersion with its kind	Typed versions in manifests
packages/kumo-databricks/src/kumo_databricks/session.py	table_location	Fully qualified table name	Manifests record source locations
packages/kumo-bigquery/src/kumo_bigquery/identity.py	version stamp	Returns SourceVersion with its kind	Typed versions in manifests
packages/kumo-bigquery/src/kumo_bigquery/session.py	table_location	Fully qualified table name	Manifests record source locations
packages/kumo-s3/src/kumo_s3/sink.py	S3SinkBackend.delete_file	Deletes without leaving an empty prefix object	Root write probe leaves nothing behind
packages/diskgraph/diskgraph-ingest/src/snapshot_mirror.rs	new: snapshot_range_checksums; read_manifest	Parallel CRC-32 per range of every index file	Content check of an exported snapshot
packages/diskgraph/diskgraph-ingest/Cargo.toml	dependencies	Adds crc32fast	CRC-32 implementation
packages/diskgraph/Cargo.lock	lockfile	Locks crc32fast	New dependency
packages/diskgraph/diskgraph-pybind/src/lib.rs	module init	Exposes snapshot_range_checksums, __version__, BUILD_COMMIT	Python access; build stamp in manifests
packages/diskgraph/diskgraph-pybind/build.rs	build_commit	Stamps the git commit at build time	Dev builds share one version
packages/diskgraph/diskgraph-pybind/python/diskgraph/__init__.py	exports	Re-exports the new names	Python access
packages/diskgraph/third_party/artifact-manifest.json	crc32fast entry	Adds the crate and lockfile hash	License inventory
packages/kumo-engine/setup.py	_StampBuildCommit	Writes BUILD_COMMIT into the wheel	Manifests name the engine build
THIRD-PARTY.txt	crc32fast notice	Adds the notice for the ingest binary	License compliance
3rdparty/notices/inventory.json	crc32fast entry	Adds the crate	License inventory
packages/kumo-engine/test/conftest.py	graph fixture	Shared two-table synthetic graph	Reused by new tests
packages/kumo-engine/test/test_mlops.py	new	Root forms, checks, store kinds, on_error, precedence	Covers MLOps
packages/kumo-engine/test/test_snapshot_manifest.py	new	Fresh, backfilled, failed and round-tripped snapshot manifests	Covers snapshot lineage
packages/kumo-engine/test/test_snapshot_export.py	new	Export, load, reuse, concurrency, corruption, source_changes, on_error	Covers snapshot portability
packages/kumo-engine/test/test_snapshot_export_live.py	new	Env-gated S3 and GCS round trip	Real object stores
packages/kumo-engine/test/test_training_table_export.py	manifest tests	Manifest, sidecar, failures, store once	Covers training-table lineage
packages/kumo-engine/test/test_bundle_provenance.py	new	Model manifest, id, old bundles, warm-start chains	Covers model lineage
packages/kumo-engine/test/test_query_from_snapshot.py	new	predict(snapshot), argument errors, fresh scratch	Covers scoring from a snapshot
packages/kumo-engine/test/test_graph_mismatch.py	new	Schema-change and same-schema warnings	Covers graph checks
packages/kumo-engine/test/test_artifacts.py	new	artifacts() handles a new manifest kind	Covers the extension seam
packages/kumo-engine/test/test_holdout_df.py	holdout tests	Loaded or fetched model keeps its holdout; empty test split	Covers bundled holdout
packages/kumo-engine/test/test_filestore_sink.py	new test	Identical bundle publish reuses the copy	Covers holds_bundle
packages/kumo-engine/test/test_trainer_registry.py	manifest replace test	Patches the shared atomic write; checks bundle id	Follows the refactor
packages/kumo-engine/test/test_warm_start.py	warm-start test	Asserts the flat warm-start parent	Covers model parents
packages/kumo-engine/test/test_databricks_connector.py	identity tests	Connector without identity is ambient	Covers optional identity
packages/kumo-engine/test/test_semantic_carry_forward.py	fit fakes	Fakes get a non-empty test split	Fit now rejects an empty test split
packages/kumo-engine/test/test_training_explanations.py	fit fakes	Fakes get a non-empty test split	Fit now rejects an empty test split
packages/kumo-engine/test/connector/test_custom_source.py	staging tests	Unpack the new return value	Follows the signature change
packages/kumo-engine/test/test_custom_sql.py	staging test	Unpacks the new return value	Follows the signature change
packages/kumo-engine/test/test_predictive_query.py	staging stubs	Stubs return parts and version	Follows the signature change
packages/kumo-core/tests/sources/parquet/test_object_store.py	version tests	Kinds; single object by provider identity	Covers typed versions
packages/kumo-core/tests/sources/unit/test_sql_identity.py	new test	Custom SQL version keeps its dependencies	Covers typed versions
packages/kumo-core/tests/test_artifact_export.py	new test	Sidecars best effort, before _SUCCESS	Covers sidecars
packages/kumo-core/tests/test_artifacts.py	new test	Bundle id ignores provenance only	Covers bundle ids
packages/kumo-s3/tests/test_sink_delete.py	new	Delete leaves no prefix marker	Covers delete_file
packages/kumo-snowflake/tests/test_identity.py	stamp tests	Assert version_kind	Covers typed versions
packages/kumo-databricks/tests/test_identity.py	stamp tests	Assert version_kind	Covers typed versions
packages/kumo-bigquery/tests/test_identity.py	stamp tests	Assert version_kind	Covers typed versions
packages/diskgraph/diskgraph-pybind/python/tests/test_exception_exports.py	new tests	Build identity exported; checksums match zlib	Covers pybind exports
packages/kumo-engine/docs/mlops.md	new guide	Quickstart, ids, manifests, export and load, reference	User docs
packages/kumo-engine/README.md	docs list	Links the MLOps guide	Discoverability
test/connector/test_file_connector.py	test_get_data_version_s3_single_file	Expects provider version	Single-object versions changed
packages/kumo-adls/src/kumo_adls/reader.py	data_version	Returns a SourceVersion marked exact for one object, listing otherwise	Manifests say whether a version names one object or a listing
packages/kumo-adls/tests/test_identity_listing.py	new test	Checks ADLS versions are marked exact or listing	Keeps the version kind from regressing

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

• Last reviewed commit: 75f9e80
• Latest commit, not reviewed: c0d7000

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

No actionable comments were generated in the recent review.

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/kumo-engine/.coderabbit.yaml
• Review profile: QUIET
• Plan: Enterprise
• Run ID: d6ee79fd-a832-4873-b67d-fdca81f9f3f8

📥 Commits

Reviewing files that changed from the base of the PR and between 045fff7 and 75f9e80.

📒 Files selected for processing: 4

• packages/kumo-engine/kumo/engine/_manifest.py
• packages/kumo-engine/kumo/engine/_provenance.py
• packages/kumo-engine/kumo/engine/graph.py
• packages/kumo-engine/test/test_snapshot_manifest.py

💤 Files with no reviewable changes: 1

• packages/kumo-engine/kumo/engine/_manifest.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

Walkthrough

The diskgraph Python API now exposes supported on-disk format versions. Kumo Engine adds shared manifest and provenance utilities, records source and snapshot metadata for graph snapshots, and uses shared atomic writes for training outputs. Object-store sessions add a canonical table URI method.

Sequence Diagram(s)

sequenceDiagram
  participant GraphSnapshot
  participant source_table
  participant source_location
  participant ObjectStoreSessions
  participant new_manifest
  participant environment
  participant scheduler_context
  participant write_manifest
  participant atomic_write
  GraphSnapshot->>source_table: Build source-table metadata
  source_table->>source_location: Resolve source coordinate
  source_location->>ObjectStoreSessions: Resolve object-store table URI
  GraphSnapshot->>new_manifest: Build snapshot manifest
  new_manifest->>environment: Collect environment metadata
  new_manifest->>scheduler_context: Collect scheduler metadata
  GraphSnapshot->>write_manifest: Persist manifest
  write_manifest->>atomic_write: Write manifest atomically

Loading

Merge Risk: Moderate · previous review 75f9e

Some snapshots can identify the wrong source file or object, making their lineage unreliable. Correct those provenance paths before merging; also address the suppressed backfill warning.

Pre-merge checks: 2 · 1

Failed checks: 1 inconclusive

Check name	Status	Explanation	Resolution
Change Map Design Review		The required conversation comment beginning with <!-- kumo-pr-change-map --> is not available in the supplied context or repository checkout. The available summary states the change map's broad impl…	Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including its Change map and linked rationale. Then assess each proposed change against the repository diff and existing behavior.

Passed checks: 2 passed

Check name	Status	Explanation
Pr Size		The pull request changes 11 files. This does not exceed the failure threshold of 40 files.
Public Resource Access		The pull request adds no concrete external URL, repository location, or package source location that is required to build, test, use, or contribute. The only URI-like additions are the schematic `sche…

Full details: Public Resource Access

Explanation

The pull request adds no concrete external URL, repository location, or package source location that is required to build, test, use, or contribute. The only URI-like additions are the schematic scheme://bucket/key docstring and the s3://bucket-a/... local test fixture, both excluded by the check. /opt/ml/config/processingjobconfig.json is a local runtime path, not an external resource reference.

Full details: Change Map Design Review

Explanation

The required conversation comment beginning with <!-- kumo-pr-change-map --> is not available in the supplied context or repository checkout. The available summary states the change map's broad implementation, but it does not provide the required claims about the observed failure, requested release requirement, inspected existing behavior, smallest sufficient change, verification, and stopping condition. Those claims are necessary for this design review.

Resolution

Provide the complete PR conversation comment beginning with <!-- kumo-pr-change-map -->, including its Change map and linked rationale. Then assess each proposed change against the repository diff and existing behavior.

Finishing Touches

Fix failing CI checks

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

[coderabbitai]

Actionable comments posted: 2

---

ℹ️ Review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/kumo-engine/.coderabbit.yaml
• Review profile: QUIET
• Plan: Enterprise
• Run ID: 27abe888-d440-4212-bd1a-b59a2746c5b0

📥 Commits

Reviewing files that changed from the base of the PR and between 09bc7e5 and eed1867.

📒 Files selected for processing: 4

• packages/kumo-core/src/kumo_core/sources/object_store_sessions.py
• packages/kumo-engine/kumo/engine/_provenance.py
• packages/kumo-engine/kumo/engine/graph.py
• packages/kumo-engine/test/test_snapshot_manifest.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 11 remain after this review.

[coderabbitai]

Note

Quiet mode is enabled, so only the most important comments were posted inline. Other review comments are grouped below.

🟡 Other comments: 1

packages/kumo-engine/kumo/engine/graph.py: 1

269-269: 🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win

Track backfill warnings only after a backfill failure.

If a fresh manifest write fails, Line 269 adds its path to _warned_backfill. If the index is later reused and its backfill also fails, the backfill failure produces no warning. Add the path to this set only when fresh is false.

Proposed change

         if fresh or path not in _warned_backfill:
-            _warned_backfill.add(path)
+            if not fresh:
+                _warned_backfill.add(path)
             warnings.warn(

---

ℹ️ Review info

⚙️ Run configuration

• Configuration used: Repository: NVIDIA/kumo-engine/.coderabbit.yaml
• Review profile: QUIET
• Plan: Enterprise
• Run ID: fec48a97-da42-43cd-9e74-6708ef97c388

📥 Commits

Reviewing files that changed from the base of the PR and between eed1867 and 045fff7.

📒 Files selected for processing: 4

• packages/kumo-engine/kumo/engine/_manifest.py
• packages/kumo-engine/kumo/engine/_provenance.py
• packages/kumo-engine/kumo/engine/graph.py
• packages/kumo-engine/test/test_snapshot_manifest.py

Included review availability: This review used your included allowance. Your plan provides up to 12 included reviews per hour; 9 remain after this review.

[aaron-kumo]

The separation between snapshots, model bundles, and explicit scoring inputs looks sound. My main concern is preserving enough information now for future restoration, freshness, and compatibility checks. We can defer those policies, but information omitted from today’s artifacts will be difficult to recover later.

P2 — Record the DiskGraph producer version/build.

environment() records engine, core, Torch, and Python versions, but omits the DiskGraph package that actually writes the index. Format constants identify the declared storage formats, but cannot distinguish implementations with different behavior or fixes under the same format version. Please record the installed DiskGraph version and a build identifier when available. This should be diagnostic provenance; package-version differences should not automatically imply incompatibility.

Discussion — Should the artifact retain the Custom SQL definition?

Could we retain literal SQL alongside its fingerprint, with an explicit option to omit/redact it for distribution? The product already persists literal SQL in table definitions and exposes it through SDK objects and generated scripts. Hash-only storage introduces a more restrictive policy here and loses the ability to inspect transformations, explain changes, or reconstruct the source definition.

SQL can contain sensitive literals or business logic, so an explicit retention policy makes sense. Is there a specific sharing or privacy requirement driving unconditional omission? If not, I’d favor retaining the SQL, dialect, resolution namespace, and referenced tables. The exact-text fingerprint and normalized-query fingerprint should also have distinct purposes; the existing Custom SQL identity already uses normalization.

Discussion — Can source identity and version evidence be structured before publication?

Could we preserve a credential-free source descriptor and typed version evidence rather than only {connector, location, version}? For example, a Snowflake table coordinate needs account context, and an actual source version, a metadata timestamp, and an hour-bucket fallback have different guarantees.

For Custom SQL, the version calculation already gathers dependency versions and folds in a staleness bucket. Could those components be retained alongside the resulting hash? Otherwise future freshness checks cannot explain whether the query, a dependency, or only the bucket changed.

I’m not suggesting implementing every comparison now. A small versioned representation that retains the evidence—including observation time, fallback status, and observed versus reconstructed provenance—would leave those decisions open.

Discussion — Should publication offer a metadata-completeness requirement?

Best-effort manifest writing seems appropriate for preserving existing local snapshot() behavior. Should export/publication optionally require complete provenance, though? Otherwise a newly created snapshot whose manifest write failed can become a durable artifact without the evidence needed for future compatibility checks. Legacy artifacts could remain supported with an explicit incomplete-provenance state.

[Vedaant-J]

P2 — Record the DiskGraph producer version/build.
Done in 2552bc1: environment now records diskgraph (diskgraph.__version__) and diskgraph_build_commit (diskgraph.BUILD_COMMIT, stamped from git at build time by diskgraph-pybind/build.rs, "unknown" when built without git). It is diagnostic only; a different DiskGraph build never makes an index unusable.

Discussion — Should the artifact retain the Custom SQL definition?
We'll retain the Custom SQL text by default (with its dialect, next to the hash and dependency versions), the same as exact source locations (0d73b76). graph.snapshot(record_sources=False) omits both, recording only the connector and table name and a hash of the SQL, for deployments that can't carry source details in artifacts that travel (exports, bundles). The training-table manifest (#55) and bundle provenance (#56) follow the snapshot's choice; the choice now travels on GraphSnapshot.record_sources (72e2932), and a custom training table whose snapshot's choice is unknown records the table name only (#55, c5baa35).

Your two remaining points, both in 60e1ab6:

• Resolution namespace: the Custom SQL source record now has namespace, the context unqualified table names in the query resolve in: [database, schema] (Snowflake), [catalog, schema] (Databricks) or [project, dataset] (BigQuery), read from the connector with the same attributes the Custom SQL identity uses (CUSTOM_SQL_NAMESPACE_ATTRS). It is source detail, so it is omitted with record_sources=False, like sql and dialect.
• Normalized-SQL fingerprint: normalized_sql_fingerprint sits next to the exact-text hash (location = sql:<sha256>). It reuses the existing Custom SQL identity normalisation (custom_sql_synthetic_name, the name hosted and the SDK give the query's table), with no new normaliser, so whitespace, comment and keyword-case edits leave it unchanged while the exact-text hash changes; comparing the two tells a reformatting from a real query change. It is a hash, so it is kept with record_sources=False.

Discussion — Can source identity and version evidence be structured before publication?
Yes, in 8b16cd1. Each source-table entry now has version_kind (exact, timestamp, hour_bucket, unknown), set by the connector that produced the version, and source_changes (#54) uses it instead of guessing from hour-aligned values. Custom SQL keeps the dependency table versions it folds into its hash (dependencies), so a later check can tell a dependency change from a bucket rollover. Snowflake locations now include the account (<account>/"DB"."SCHEMA"."TABLE", no credentials).

Discussion — Should publication offer a metadata-completeness requirement?
Not as a requirement for now, so export never fails where snapshots work today. Instead, export (#54, 3b1d2bd) writes a manifest marked "reconstructed": true (empty source_tables, null build fields) when a snapshot has none, so missing provenance is stated explicitly in the artifact.

CodeRabbit: object-store locations keep their percent-encoding, a FileConnector table records its resolved file (orders.parquet), and a failed fresh manifest write no longer suppresses a later backfill warning (8b16cd1).