feat: add new granularities to gooddata-dbt and gooddata-pipelines - #1714
Open
okrcmaGD wants to merge 1 commit into
Open
feat: add new granularities to gooddata-dbt and gooddata-pipelines#1714okrcmaGD wants to merge 1 commit into
okrcmaGD wants to merge 1 commit into
Conversation
📝 WalkthroughWalkthroughThis change adds optional second-based date granularities to LDM extension and dbt workflows. The option is disabled by default and requires the organization feature flag. Documentation and tests cover both modes. ChangesSecond-based date granularities
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant LdmExtensionManager
participant LdmExtensionDataProcessor
participant DateDatasetBuilder
LdmExtensionManager->>LdmExtensionDataProcessor: Pass enable_second_granularities
LdmExtensionDataProcessor->>DateDatasetBuilder: Use configured date granularities
DateDatasetBuilder-->>LdmExtensionDataProcessor: Build date datasets
sequenceDiagram
participant deploy_ldm
participant DbtModelTables
participant TimestampDateDataset
deploy_ldm->>DbtModelTables: Pass CLI option
DbtModelTables->>TimestampDateDataset: Add second timestamp granularities
TimestampDateDataset-->>DbtModelTables: Return generated dataset
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py`:
- Line 64: Restore the public LdmExtensionDataProcessor.DATE_GRANULARITIES class
attribute as a deprecated alias referencing _DATE_GRANULARITIES, preserving
existing caller access without duplicating the granularity values.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: e731f3c6-6548-4baf-bbfc-5d9c6522845b
📒 Files selected for processing (10)
docs/content/en/latest/pipelines/ldm_extension/_index.mdpackages/gooddata-dbt/README.mdpackages/gooddata-dbt/src/gooddata_dbt/args.pypackages/gooddata-dbt/src/gooddata_dbt/dbt/base.pypackages/gooddata-dbt/src/gooddata_dbt/dbt/tables.pypackages/gooddata-dbt/src/gooddata_dbt/dbt_plugin.pypackages/gooddata-dbt/tests/test_tables.pypackages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.pypackages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/ldm_extension_manager.pypackages/gooddata-pipelines/tests/test_ldm_extension/test_input_processor.py
| """Create GoodData LDM from validated custom datasets and fields.""" | ||
|
|
||
| DATE_GRANULARITIES: list[str] = [ | ||
| _DATE_GRANULARITIES: list[str] = [ |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n -C 2 '\bDATE_GRANULARITIES\b|\b_DATE_GRANULARITIES\b' packages docsRepository: gooddata/gooddata-python-sdk
Length of output: 6156
🏁 Script executed:
#!/bin/bash
set -euo pipefail
printf '%s\n' '--- input_processor.py outline ---'
ast-grep outline packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py
printf '%s\n' '--- relevant source ---'
sed -n '1,180p' packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py
printf '%s\n' '--- package exports and public references ---'
rg -n -C 2 'InputProcessor|input_processor|DATE_GRANULARITIES|_DATE_GRANULARITIES' \
packages/gooddata-pipelines README.md docs 2>/dev/null || true
printf '%s\n' '--- package metadata ---'
fd -H -i 'pyproject.toml|setup.py|__init__.py|README*|CHANGELOG*' packages/gooddata-pipelines \
--exec sh -c 'echo "--- $1"; sed -n "1,220p" "$1"' sh {}Repository: gooddata/gooddata-python-sdk
Length of output: 47817
🏁 Script executed:
#!/bin/bash
set -euo pipefail
path='packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py'
printf '%s\n' '--- current and parent declarations ---'
printf '%s\n' 'Current:'
git show "HEAD:${path}" | sed -n '55,105p'
printf '%s\n' 'Parent:'
git show "HEAD^:${path}" | sed -n '55,105p'
printf '%s\n' '--- history summary for the target file ---'
git log --oneline -8 -- "$path"
printf '%s\n' '--- imports of the processor module or constant ---'
rg -n 'from gooddata_pipelines\.ldm_extension\.input_processor|import gooddata_pipelines\.ldm_extension\.input_processor|DATE_GRANULARITIES' \
--glob '*.py' --glob '*.md' --glob '*.rst' .Repository: gooddata/gooddata-python-sdk
Length of output: 6583
Keep LdmExtensionDataProcessor.DATE_GRANULARITIES as a deprecated alias for _DATE_GRANULARITIES. Existing callers can access this public class attribute, and removing it causes AttributeError.
🧰 Tools
🪛 Ruff (0.16.1)
[warning] 64-83: Mutable default value for class attribute
(RUF012)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py`
at line 64, Restore the public LdmExtensionDataProcessor.DATE_GRANULARITIES
class attribute as a deprecated alias referencing _DATE_GRANULARITIES,
preserving existing caller access without duplicating the granularity values.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #1714 +/- ##
==========================================
+ Coverage 78.30% 78.39% +0.09%
==========================================
Files 271 271
Lines 18689 18748 +59
==========================================
+ Hits 14634 14698 +64
+ Misses 4055 4050 -5 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
JIRA: CQ-2783 risk: low
|
Note GitHub couldn't provide a complete incremental comparison for this pull request, so CodeRabbit is performing a full review instead. This review may take a little longer. |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py (1)
85-99: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAvoid sharing mutable granularity lists.
CatalogDeclarativeDateDatasetretains the supplied list, and_date_from_fieldpasses the same list to every dataset. Copy_date_granularitiesin__init__and when constructing each dataset. AddClassVarannotations if Ruff RUF012 is enabled.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py` around lines 85 - 99, Prevent shared mutable granularity lists by copying _date_granularities in CatalogDeclarativeDateDataset.__init__ and copying it again when _date_from_field constructs each dataset. Annotate class-level mutable granularity constants with ClassVar if required by Ruff RUF012.Source: Linters/SAST tools
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In
`@packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.py`:
- Around line 85-99: Prevent shared mutable granularity lists by copying
_date_granularities in CatalogDeclarativeDateDataset.__init__ and copying it
again when _date_from_field constructs each dataset. Annotate class-level
mutable granularity constants with ClassVar if required by Ruff RUF012.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 54f40155-b3eb-4148-bdd9-7d4e0953a987
📒 Files selected for processing (10)
docs/content/en/latest/pipelines/ldm_extension/_index.mdpackages/gooddata-dbt/README.mdpackages/gooddata-dbt/src/gooddata_dbt/args.pypackages/gooddata-dbt/src/gooddata_dbt/dbt/base.pypackages/gooddata-dbt/src/gooddata_dbt/dbt/tables.pypackages/gooddata-dbt/src/gooddata_dbt/dbt_plugin.pypackages/gooddata-dbt/tests/test_tables.pypackages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/input_processor.pypackages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/ldm_extension_manager.pypackages/gooddata-pipelines/tests/test_ldm_extension/test_input_processor.py
🚧 Files skipped from review as they are similar to previous changes (8)
- docs/content/en/latest/pipelines/ldm_extension/_index.md
- packages/gooddata-dbt/README.md
- packages/gooddata-dbt/src/gooddata_dbt/args.py
- packages/gooddata-pipelines/tests/test_ldm_extension/test_input_processor.py
- packages/gooddata-pipelines/src/gooddata_pipelines/ldm_extension/ldm_extension_manager.py
- packages/gooddata-dbt/src/gooddata_dbt/dbt_plugin.py
- packages/gooddata-dbt/tests/test_tables.py
- packages/gooddata-dbt/src/gooddata_dbt/dbt/base.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
JIRA: CQ-2783
risk: low
Summary by CodeRabbit
New Features
Documentation
Tests