Skip to content

fix(dump): put _updated_at last in whole-table downloads - #27

Merged
luccasmmg merged 1 commit into
mainfrom
fix/dump-updated-at-last
Oct 12, 2026
Merged

luccasmmg merged 1 commit into
mainfrom
fix/dump-updated-at-last

Conversation

@luccasmmg

Copy link
Copy Markdown
Member

Problem

QA report: after appending records via the API and downloading the resource as CSV, the first two columns were _id and _updated_at. _updated_at should be the last column.

Cause

BigQuery stores the system columns first (_id, _updated_at, then user fields). GET /dump/{resource_id} built both its EXPORT DATA SELECT list and the CSV header from that physical order. datastore_search already moves _updated_at after the user columns; dump did not.

Fix

  • New _dump_column_order(schema) in bigquery/export.py moves _updated_at last.
  • dump() uses it for both the SELECT list and the CSV header, so the two stay aligned. This applies to csv, gzip, ndjson and parquet.
  • /dump/query is unchanged: it keeps the user's SQL projection order.

Notes

  • Cached exports are keyed on table.modified, so an unchanged table can still serve a previously cached file with the old order until the table is written to again or the cache entry ages out.
  • Not tested against real BigQuery; covered by unit tests only.

Tests

  • Added test_dump_column_order_moves_updated_at_last and test_dump_column_order_without_updated_at_is_unchanged.
  • pytest tests/test_datastore_dump.py tests/engines: 124 passed.

BigQuery stores system columns first (_id, _updated_at, user fields),
and the dump built its SELECT list and CSV header from that physical
order. Reorder so _updated_at trails the user columns, matching
datastore_search's projection.
@coderabbitai

coderabbitai Bot commented Oct 12, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: e22eafe5-daba-41cb-b66b-112f4a09009d

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@luccasmmg
luccasmmg merged commit d63e695 into main Oct 12, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant