fix(ingest): normalize Markdown front-matter dates before storage - #253
Merged
codeforstartups merged 4 commits intoSep 25, 2026
Conversation
Add tests for front matter date normalization in MarkdownSource.
codeforstartups
approved these changes
Sep 25, 2026
codeforstartups
left a comment
Owner
There was a problem hiding this comment.
Correct bug fix for #246 — yaml.safe_load() turns unquoted front-matter dates into datetime.date/datetime objects that DynamoDB's TypeSerializer rejects. _normalize_front_matter recursively converts them to ISO-8601 strings (handling nested dicts/lists too). Verified locally: ruff clean, 19 markdown-source tests pass. Merging — thanks @Shraddhameduri! 🙌
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #246
What
yaml.safe_load()converts unquoted YAML front-matter dates intodatetime.date/datetime.datetimeobjects, which DynamoDB'sTypeSerializerrejects at write time (TypeError: Unsupported type "datetime.date"). This PR normalizes parsed front-matter in the ingestion layer before it reaches storage.Changes
src/dynavec/ingest.py: new recursive_normalize_front_matter()helper (date/datetime →.isoformat(), recurses through dicts and lists, scalars untouched). Applied inMarkdownSource._front_matter(), which also covers theS3SourceMarkdown path since it reuses the same parser.tests/test_markdown_source.py: 5 new tests — unquoted dates → ISO strings, timestamps → ISO strings, nested dates in dicts/lists, quoted strings and scalars untouched, and an end-to-end check that ingested metadata passes boto3'sTypeSerializerafter_to_dynamo.Test results
pytest tests/test_markdown_source.py— 19 passedpytest tests/test_ingest.py tests/test_s3_source.py— 25 passed