Refuse a date past the year 9999 before the arithmetic - #3
Merged
Conversation
Versions 2 and 3 carry the date as an unsigned 32 bit count of minutes since 2020-01-01, which runs to 4,294,967,295 and lands on 15 February 10186. datetime stops at the end of 9999, so the addition raised OverflowError for any count from 4,197,074,400 upwards, and a read that promises never to raise raised on caller data. The same bytes read fine in Java, PHP and JavaScript, so this is the runtime's limit rather than a fault in the data. The reader now compares the count with the largest one datetime can hold, derived from datetime.max, and reports IMPLEMENTATION_CAPACITY_EXCEEDED before the arithmetic on both reading contracts. Two bytes of hours in version 1 reach June 2027, so that field needs no guard, and the reader and tests say so. test_date_range pins the boundary on both contracts, covering the maximum count, the first count past the runtime, the last count inside it, and the version 1 maximum. The coverage test that named IMPLEMENTATION_CAPACITY_EXCEEDED unreachable now produces it, and MALFORMED_ENVELOPE keeps its own test as the one still unreachable. With the guard removed, five tests error with the OverflowError the guard prevents.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
Versions 2 and 3 carry the creation date as an unsigned 32 bit count of minutes since 2020-01-01. The wire therefore allows up to 4,294,967,295 minutes, which lands on 15 February 10186.
datetime.maxis the last moment of the year 9999, soBASE_DATE + timedelta(minutes=minutes)inowid/parse.pyraisedOverflowErrorfor any count from 4,197,074,400 upwards. A caller supplied envelope could makeparse_bytes,parse_prefixandparse, which promise never to raise, raise.The rule applied
The same bytes read fine in Java, PHP and JavaScript, whose date types reach that far, so the envelope is structurally consistent and the finding is this runtime's limit. That is what
IMPLEMENTATION_CAPACITY_EXCEEDEDmeans. The count is compared with the largest onedatetimecan hold before the arithmetic, never caught after it.Before and after for a caller
Before, on an envelope whose four date bytes were
FF FF FF FF:After:
The framed
parse_prefixbehaves the same way and consumes nothing. Every count up to 4,197,074,399 (9999-12-31 23:59) parses exactly as before, and nothing changes for creation.Changes
owid.io.MAXIMUM_MINUTES, derived fromdatetime.maxso it cannot drift from the runtime.owid/parse.pycompares the count with it before the addition and reportsIMPLEMENTATION_CAPACITY_EXCEEDED. Version 1's two byte hour count reaches 65,535 hours, which is June 2027, so it has no guard and the comment says why.ParseStatus.IMPLEMENTATION_CAPACITY_EXCEEDEDdocumentation and the README status row now say what produces the status.tests/test_date_range.pycovers the maximum count, the first count past the runtime, the last count inside it, the boundary constant against the runtime, and the version 1 maximum, on both reading contracts. Intests/test_parse_contract.pythe test that namedIMPLEMENTATION_CAPACITY_EXCEEDEDunreachable now produces it, andMALFORMED_ENVELOPEkeeps its own test as the one member still unreachable.Verification
python -m unittest discover -s tests, 111 tests before and 117 after, all passing.OverflowError: date value out of range(test_maximum_count_is_capacity_exceededandtest_first_count_beyond_the_runtime_is_capacity_exceededon both contracts, andtest_implementation_capacity_is_a_date_the_runtime_cannot_hold). Restored, all 117 pass.Not changed
owid.io.Reader.read_dateis the private raising reader kept only so the tests can pin its messages. It still raisesOverflowErroron such a count, which is consistent with a path that raises by design and is not reachable from the public surface.Produced with AI assistance under James Rosewell's direction and needs human review.