Skip to content

Refuse a date past the year 9999 before the arithmetic - #3

Merged
jwrosewell merged 1 commit into
mainfrom
harden/date-beyond-runtime-range
Aug 31, 2026
Merged

Refuse a date past the year 9999 before the arithmetic#3
jwrosewell merged 1 commit into
mainfrom
harden/date-beyond-runtime-range

Conversation

@jwrosewell

Copy link
Copy Markdown
Contributor

What was wrong

Versions 2 and 3 carry the creation date as an unsigned 32 bit count of minutes since 2020-01-01. The wire therefore allows up to 4,294,967,295 minutes, which lands on 15 February 10186. datetime.max is the last moment of the year 9999, so BASE_DATE + timedelta(minutes=minutes) in owid/parse.py raised OverflowError for any count from 4,197,074,400 upwards. A caller supplied envelope could make parse_bytes, parse_prefix and parse, which promise never to raise, raise.

The rule applied

The same bytes read fine in Java, PHP and JavaScript, whose date types reach that far, so the envelope is structurally consistent and the finding is this runtime's limit. That is what IMPLEMENTATION_CAPACITY_EXCEEDED means. The count is compared with the largest one datetime can hold before the arithmetic, never caught after it.

Before and after for a caller

Before, on an envelope whose four date bytes were FF FF FF FF:

Owid.parse_bytes(raw)
# raised OverflowError: date value out of range

After:

result = Owid.parse_bytes(raw)
# result.ok is False, result.owid is None,
# result.status is ParseStatus.IMPLEMENTATION_CAPACITY_EXCEEDED

The framed parse_prefix behaves the same way and consumes nothing. Every count up to 4,197,074,399 (9999-12-31 23:59) parses exactly as before, and nothing changes for creation.

Changes

  • owid.io.MAXIMUM_MINUTES, derived from datetime.max so it cannot drift from the runtime.
  • owid/parse.py compares the count with it before the addition and reports IMPLEMENTATION_CAPACITY_EXCEEDED. Version 1's two byte hour count reaches 65,535 hours, which is June 2027, so it has no guard and the comment says why.
  • The ParseStatus.IMPLEMENTATION_CAPACITY_EXCEEDED documentation and the README status row now say what produces the status.
  • tests/test_date_range.py covers the maximum count, the first count past the runtime, the last count inside it, the boundary constant against the runtime, and the version 1 maximum, on both reading contracts. In tests/test_parse_contract.py the test that named IMPLEMENTATION_CAPACITY_EXCEEDED unreachable now produces it, and MALFORMED_ENVELOPE keeps its own test as the one member still unreachable.

Verification

  • python -m unittest discover -s tests, 111 tests before and 117 after, all passing.
  • Neutralisation, with the guard disabled: 5 tests error with OverflowError: date value out of range (test_maximum_count_is_capacity_exceeded and test_first_count_beyond_the_runtime_is_capacity_exceeded on both contracts, and test_implementation_capacity_is_a_date_the_runtime_cannot_hold). Restored, all 117 pass.

Not changed

owid.io.Reader.read_date is the private raising reader kept only so the tests can pin its messages. It still raises OverflowError on such a count, which is consistent with a path that raises by design and is not reachable from the public surface.

Produced with AI assistance under James Rosewell's direction and needs human review.

Versions 2 and 3 carry the date as an unsigned 32 bit count of minutes
since 2020-01-01, which runs to 4,294,967,295 and lands on 15 February
10186. datetime stops at the end of 9999, so the addition raised
OverflowError for any count from 4,197,074,400 upwards, and a read that
promises never to raise raised on caller data.

The same bytes read fine in Java, PHP and JavaScript, so this is the
runtime's limit rather than a fault in the data. The reader now
compares the count with the largest one datetime can hold, derived from
datetime.max, and reports IMPLEMENTATION_CAPACITY_EXCEEDED before the
arithmetic on both reading contracts.

Two bytes of hours in version 1 reach June 2027, so that field needs no
guard, and the reader and tests say so.

test_date_range pins the boundary on both contracts, covering the
maximum count, the first count past the runtime, the last count inside
it, and the version 1 maximum. The coverage test that named
IMPLEMENTATION_CAPACITY_EXCEEDED unreachable now produces it, and
MALFORMED_ENVELOPE keeps its own test as the one still unreachable.
With the guard removed, five tests error with the OverflowError the
guard prevents.
@jwrosewell
jwrosewell merged commit 0a77e39 into main Aug 31, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant