Skip to content

Commit 16d73f5

Browse files
georgeRobertsondependabot[bot]stevenhsd
authored
Release v084 (#137)
* build(deps-dev): bump pymdown-extensions from 10.21.3 to 11.0.1 (#134) Bumps [pymdown-extensions](https://github.com/facelessuser/pymdown-extensions) from 10.21.3 to 11.0.1. - [Release notes](https://github.com/facelessuser/pymdown-extensions/releases) - [Commits](facelessuser/pymdown-extensions@10.21.3...11.0.1) --- updated-dependencies: - dependency-name: pymdown-extensions dependency-version: 11.0.1 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps-dev): bump cryptography from 48.0.1 to 50.0.0 (#133) Bumps [cryptography](https://github.com/pyca/cryptography) from 48.0.1 to 50.0.0. - [Changelog](https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst) - [Commits](pyca/cryptography@48.0.1...50.0.0) --- updated-dependencies: - dependency-name: cryptography dependency-version: 50.0.0 dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: georgeRobertson <50412379+georgeRobertson@users.noreply.github.com> * fix: ensure that format for times can be accessed and supplied in duckdb casting (#135) --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: stevenhsd <56357022+stevenhsd@users.noreply.github.com>
2 parents 0c4d3ed + fdd0315 commit 16d73f5

6 files changed

Lines changed: 147 additions & 99 deletions

File tree

poetry.lock

Lines changed: 52 additions & 55 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

pyproject.toml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -69,7 +69,7 @@ optional = true
6969
behave = "1.3.3"
7070
coverage = "7.11.0"
7171
moto = {extras = ["s3"], version = "4.2.14"}
72-
cryptography = "48.0.1" # dependency of `moto`
72+
cryptography = "50.0.0" # dependency of `moto`
7373
requests = "2.33.0" # dependency of `moto`
7474
Werkzeug = "3.1.6"
7575
pytest = "9.0.3"
@@ -106,7 +106,7 @@ click = "8.2.1"
106106
mkdocs = "1.6.1"
107107
mkdocstrings = { version = "1.0.3", extras = ["python"] }
108108
griffelib = "2.0.1"
109-
pymdown-extensions = "10.21.3"
109+
pymdown-extensions = "11.0.1"
110110
zensical = "0.0.46"
111111

112112
[tool.ruff]

src/dve/core_engine/backends/implementations/duckdb/duckdb_helpers.py

Lines changed: 24 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
from dve.core_engine.backends.utilities import DEFAULT_ISO_FORMATS, datetime_format_to_regex
2525
from dve.core_engine.constants import RECORD_INDEX_COLUMN_NAME
2626
from dve.core_engine.type_hints import URI, EntityName
27+
from dve.metadata_parser.utilities import resilient_get
2728
from dve.parser.file_handling.service import LocalFilesystemImplementation, _get_implementation
2829

2930

@@ -451,23 +452,27 @@ def get_duckdb_cast_statement_from_annotation(
451452
raise ValueError(f"dict must be `typing.TypedDict` subclass, got {type_annotation!r}")
452453

453454
for type_ in type_annotation.mro():
454-
_date_format: str = getattr( # type: ignore
455-
type_, "DATE_FORMAT", DEFAULT_ISO_FORMATS.get(type_, DEFAULT_ISO_FORMATS.get(datetime))
456-
)
457-
dt_cast_statement = rf"CASE WHEN REGEXP_FULL_MATCH(TRIM({quoted_name}), '{datetime_format_to_regex(_date_format)}') THEN TRY_STRPTIME(TRIM({quoted_name}), '{_date_format}') ELSE NULL END" # pylint: disable=C0301
458-
459-
# datetime is subclass of date, so needs to be handled first
460-
if issubclass(type_, datetime):
461-
stmt = rf"TRY_CAST({dt_cast_statement} as TIMESTAMP)"
462-
return stmt
463-
if issubclass(type_, date):
464-
stmt = rf"TRY_CAST({dt_cast_statement} as DATE)"
465-
return stmt
466-
if issubclass(type_, time):
467-
stmt = rf"TRY_CAST({dt_cast_statement} as TIME)"
468-
return stmt
469-
duck_type = get_duckdb_type_from_annotation(type_)
470-
if duck_type:
471-
stmt = f"TRIM({quoted_name})"
472-
return _cast_as_ddb_type(stmt, type_) if parent_element else stmt
455+
if issubclass(type_, (date, time)):
456+
_date_format: str = resilient_get(
457+
type_, "DATE_FORMAT", "TIME_FORMAT"
458+
) or DEFAULT_ISO_FORMATS.get(
459+
type_, DEFAULT_ISO_FORMATS.get(datetime)
460+
) # type: ignore
461+
dt_cast_statement = rf"CASE WHEN REGEXP_FULL_MATCH(TRIM({quoted_name}), '{datetime_format_to_regex(_date_format)}') THEN TRY_STRPTIME(TRIM({quoted_name}), '{_date_format}') ELSE NULL END" # pylint: disable=C0301
462+
463+
# datetime is subclass of date, so needs to be handled first
464+
if issubclass(type_, datetime):
465+
stmt = rf"TRY_CAST({dt_cast_statement} as TIMESTAMP)"
466+
return stmt
467+
if issubclass(type_, date):
468+
stmt = rf"TRY_CAST({dt_cast_statement} as DATE)"
469+
return stmt
470+
if issubclass(type_, time):
471+
stmt = rf"TRY_CAST({dt_cast_statement} as TIME)"
472+
return stmt
473+
else:
474+
duck_type = get_duckdb_type_from_annotation(type_)
475+
if duck_type:
476+
stmt = f"TRIM({quoted_name})"
477+
return _cast_as_ddb_type(stmt, type_) if parent_element else stmt
473478
raise ValueError(f"No equivalent DuckDB type for {type_annotation!r}")

0 commit comments

Comments
 (0)