Expose computed volume_path during initialize#5550
Draft
radakam wants to merge 3 commits into
Draft
Conversation
Collaborator
Integration test reportCommit: 7eba8b8
24 interesting tests: 15 SKIP, 7 KNOWN, 2 flaky
Top 33 slowest tests (at least 2 minutes):
|
radakam
force-pushed
the
volume-path-support
branch
from
6211576 to
0399701
Compare
radakam
changed the title
radakam
force-pushed
the
volume-path-support
branch
from
3aa9504 to
b30a845
Compare
radakam
force-pushed
the
volume-path-support
branch
from
2827a4e to
a240d86
Compare
Add a computed, read-only volume_path field to volume resources, set to
the Unity Catalog path /Volumes/{catalog}/{schema}/{name} during the
initialize phase. This enables other resources to reference
${resources.volumes.<key>.volume_path}.
- InitializeVolumePaths resolves catalog_name/schema_name/name references
locally to compute the path without mutating the original references, so
validate and plan keep showing the references.
- volume_path is only set when catalog, schema, and name resolve to
concrete values; unresolved or malformed references leave it empty.
- Computation runs after PythonMutator: volume_path is a computed field the
PyDABs Volume model does not declare, so it must not be exposed to Python.
Cover behaviors the initial volume_path change did not exercise: - unresolved_volume_path: a volume whose name is only known at deploy cannot have volume_path computed at plan time, so a reference to it resolves to an empty string (no error). Documented as known badness. - volume_path_job_ref: the motivating use case from #4233, a job parameter referencing ${resources.volumes.<key>.volume_path}. - Unit test for resolving a reference to an unset volume_path on an existing volume (resolves to empty rather than erroring).
radakam
force-pushed
the
volume-path-support
branch
from
a240d86 to
3334a23
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Expose a computed, read-only
volume_pathonresources.volumes.*so configs can reference a volume's UC path (/Volumes/{catalog}/{schema}/{name}) via${resources.volumes.<key>.volume_path}instead of hardcoding it.Volume.VolumePath(bundle:"readonly") +ComputeVolumePath(), which only builds the path whencatalog_name,schema_name, andnameare fully resolved.InitializeVolumePathsmutator: computesvolume_path, resolving${resources.catalogs/schemas...name}locally for the computation only — the original references are preserved in config sovalidate/planare unchanged.ResolveVolumePathReferencesOnlyResourcesmutator: resolves onlyresources.volumes.*.volume_pathreferences and skips everything else.InitializeafterPythonMutator(computed/read-only field not exposed to PyDABs, like thedeploymentmetadata).volume_pathbefore Terraform conversion (API-computed) and mark italwaysSkipin config sync.Why
Addresses DECO-26236 and fixes #4233.
Users had to hardcode
/Volumes/<catalog>/<schema>/<name>wherever they referenced a bundle-managed volume, duplicating config and breaking when the catalog/schema/name changes. A symbolicvolume_pathkeeps references consistent with the definition.Tests
ComputeVolumePath(resolved vs. unresolved/malformed${...}),InitializeVolumePaths, volume-path-only resolution, and the Terraform drop.resource_deps/remote_field_volume_path—comment: ${resources.volumes.bar.volume_path}resolves to the computed path while the originalschema_namereference is preserved.resource_deps/volume_path_job_ref— a job referencingvolume_path.resource_deps/unresolved_volume_path—volume_pathis not set when components stay unresolved.volumes/change-comment,change-name,change-schema-name) andbad_syntax/non_existent_fieldreference outputs.