Repository navigation
fix(dump): put _updated_at last in whole-table downloads - #27
Merged
Merged
Conversation
BigQuery stores system columns first (_id, _updated_at, user fields), and the dump built its SELECT list and CSV header from that physical order. Reorder so _updated_at trails the user columns, matching datastore_search's projection.
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
QA report: after appending records via the API and downloading the resource as CSV, the first two columns were
_idand_updated_at._updated_atshould be the last column.Cause
BigQuery stores the system columns first (
_id,_updated_at, then user fields).GET /dump/{resource_id}built both itsEXPORT DATASELECT list and the CSV header from that physical order.datastore_searchalready moves_updated_atafter the user columns; dump did not.Fix
_dump_column_order(schema)inbigquery/export.pymoves_updated_atlast.dump()uses it for both the SELECT list and the CSV header, so the two stay aligned. This applies to csv, gzip, ndjson and parquet./dump/queryis unchanged: it keeps the user's SQL projection order.Notes
table.modified, so an unchanged table can still serve a previously cached file with the old order until the table is written to again or the cache entry ages out.Tests
test_dump_column_order_moves_updated_at_lastandtest_dump_column_order_without_updated_at_is_unchanged.pytest tests/test_datastore_dump.py tests/engines: 124 passed.