Skip to content

Bound the declared payload length, and the domain on read and write - #1

Merged
jwrosewell merged 6 commits into
mainfrom
fix/payload-length-before-allocation
Aug 30, 2026
Merged

Bound the declared payload length, and the domain on read and write#1
jwrosewell merged 6 commits into
mainfrom
fix/payload-length-before-allocation

Conversation

@jwrosewell

@jwrosewell jwrosewell commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Defect and format rule

The parser previously accepted a payload length that left bytes after the 64-byte signature. This PR requires complete byte-array and Base64 input to contain exactly one envelope. The public reader consumes one OWID and leaves following framed bytes. A declaration that does not agree with the bytes present is rejected before the payload is copied.

The unsigned 32-bit field itself permits payloads from 0 through 4,294,967,295 bytes. A large value is not malformed when the matching bytes are present. Runtime capacity and application policy are separate from format validity.

The domain had the same shape of fault and it is now bounded too. The domain is stored as text followed by a zero terminator, and the parser found the end of it by walking forward to that terminator with nothing to stop the walk, so a buffer whose terminator was missing or corrupted was read all the way to its end. That was the last unbounded read left in the envelope parse. The walk now stops at the greatest number of characters a domain name can hold, so a hostile buffer costs that maximum rather than its own length, and a domain longer than a domain name may be is refused as well.

The maximum is 253 characters. RFC 1035 section 2.3.4, "Size limits", restricts the total length of a domain name, counting label octets and label length octets, to 255 octets or less. That figure is the wire format, which spends one length octet on every label and one zero octet on the root, whereas OWID stores the presentation form, the text example.com, where the dots stand in for the label length octets and the root has no text at all. Exactly two of those 255 octets therefore have no character here. The number is held in one named constant beside the signature length and the comment on it carries this reasoning, so a reader can check where 253 came from without leaving the file. The bound owes nothing to any application policy.

Bounding only the read left the two halves of the library disagreeing with each other, because a creator configured with a domain longer than 253 characters still produced an OWID that this same library refused to parse, so the fault landed on whoever read it rather than on the creator that caused it. A library that can emit what it cannot read is worse than one that does neither. The specification is being changed to match in owid#6, so that the limit binds a creator as well as a consumer, and this PR now enforces the half PHP owns on both sides.

Where the write is refused

Two points, and both are cheap.

  1. The Creator constructor, which is the earliest point at which the caller can be told, so a wrongly configured domain is refused when the domain is supplied rather than when an OWID is later serialized. The check sits before the crypto instance is looked at, so a creator that would sign an unreadable OWID is never built and nothing is signed with such a domain.
  2. Io::writeString, which is the only place a domain is serialized, so a value that reached the public Owid domain field by some other route is still refused. That refusal happens while the data to sign is being built, which is before the signing key is used, so no signature is calculated over a value that is about to be thrown away.

Both raise OwidException::domainTooLong and both read OwidException::MAXIMUM_DOMAIN_LENGTH, the same named constructor and the same constant the read bound already uses, so the two halves report the one condition the one way. The message of that named constructor is reworded so it is true from either side, and it still names the maximum. It still does not name the domain. On the write side the domain does come from the caller's own configuration rather than from an attacker, so echoing it back would be safe there and might help an operator, but writeString cannot tell which of the two routes a value arrived by, and a caller that has just handed in a domain already has it, along with a stack trace naming the call site. One message for one condition is worth more here than a longer message on one of the two paths.

Current changes

  • Rejects payload-length mismatches, trailing bytes, and short signatures consistently with the .NET reference.
  • Documents the protocol range and tells callers to limit the complete transport body before buffering or Base64 decoding. After parsing, strlen($owid->payload) is the allocation-free structured size for downstream policy.
  • Renames the large-declaration test to describe the actual fault, which is that the declared bytes are absent.
  • Adds a matching 1 MiB payload test to both PHPUnit and the dependency-free runner.
  • Caches the input string length and unpacks 32-bit values directly at the current offset, avoiding repeated length calls and temporary four-byte strings.
  • Preserves public reader composition for framed or concatenated input while keeping complete byte-array and Base64 parsing strict about trailing bytes.
  • Bounds the domain read at the published maximum, refusing a domain that has no terminator within it and one that is longer than a domain name may be. The refusal uses the library exception type through a new named constructor, which is needed because the message has to name the maximum and must not repeat the offending bytes back, of which there may be no end.
  • Searches for the terminator with a call that takes the window as an argument, so no more bytes are examined than the bound allows, rather than one that runs to the end of the buffer.
  • Bounds the write at the same maximum, in the Creator constructor and in Io::writeString, so the library cannot produce an OWID it would then refuse to read.
  • Adds domain tests to both PHPUnit and the dependency-free runner, covering a domain of the greatest length, one character more, a buffer with no terminator at all, characters filling the bound exactly, and the library's own signed output. The cost of the unterminated case is timed over two buffers sixteen times apart, so the check rests on the cost not growing with the buffer rather than on how fast the machine is.
  • Adds write-side tests covering both ways of making a creator, serialization and the building of the data to sign, a domain of exactly the maximum still being written and parsing back unchanged, and the ordering that puts the refusal before any signature is calculated.
  • Changes the two test helpers that deliberately build an over-long domain for the read side so they append the zero terminator themselves rather than calling Io::writeString, which now refuses the very bytes those read tests are built from.
  • Corrects the README, which said the domain had no separate maximum, and describes the read bound, the write bound and the payload length check together.

An empty domain, and any domain at or under the maximum, behave exactly as before. This is a refusal at the top of the range and nothing else.

A malformed envelope declaring 4 GiB but carrying only a few bytes still fails with work and memory proportional to the bytes actually supplied. A matching payload is processed in proportion to its real size and remains subject to PHP string, platform, address-space, and available-memory capacity.

Validation

The full PHPUnit suite reports 77 tests and 5,195 assertions passing, and the dependency-free repository runner reports 113 checks passed. Before the write bound those figures were 74 tests with 5,187 assertions and 105 checks, and before the read bound they were 69 tests with 3,172 assertions and 98 checks, so every test that already passed still passes untouched. The changed PHP files also pass php -l.

The new tests were checked against the defect they describe. With only the two write-side checks reverted and the new tests left in place, PHPUnit fails all three new tests, being testCreatorRefusesDomainOverMaximum because the creator is built, testWriteRefusesDomainOverMaximum because asByteArray returns bytes, and testRefusalHappensBeforeAnySignature because the constructor reaches the crypto check and reports instance of Crypto cannot be used to generate a signature instead of naming the maximum, which is the ordering that test exists to pin down. The dependency-free runner fails seven of its eight new checks. The eighth, which serializes a domain of exactly the maximum and parses it back, passes either way on purpose, because it is the control that shows the refusal is at the top of the range and not below it.

The read bound was checked the same way when it was added. With only the read bound reverted, PHPUnit failed testDomainOneOverMaximumIsRefused, because the over-long domain was accepted, and testUnterminatedDomainIsRefusedWithoutReadingTheBuffer, because 1,000 refusals over a 16 MiB buffer took 1.87s against 0.11s over a 1 MiB one, which is the cost growing with the buffer. The dependency-free runner failed the three matching checks. PHP cannot count the bytes a single call examines, so the bound is shown by that timing rather than measured directly.

References: the format-validation fix is owid-dotnet#5, the .NET size-policy follow-up is owid-dotnet#6, and the specification change that makes the limit bind a creator as well as a consumer is owid#6.

This change was produced with AI assistance under James Rosewell's direction and needs human review before merge.

…locating

Owid::fromReader read the payload through Io::readByteArray, which takes
the sender's declared count and slices that many bytes from the buffer.
Io::readBytes already refused a count beyond the end of the buffer
before slicing, so this port never allocated by the declared number,
but nothing checked that the count left exactly the 64 byte signature
after the payload. A count short of the payload let payload bytes be
read as the signature, and bytes after the signature were ignored, so
malformed OWIDs that the reference fix in owid-dotnet now refuses still
parsed here.

The count is now checked before anything is sized by it. A valid OWID
is the declared payload followed by the 64 byte signature and nothing
else, so the new Io::readPayload requires the count to equal the bytes
remaining less the signature length, and any other count, short or
long, is refused with the existing exception type through the new
OwidException::payloadLengthMismatch, which names the declared length
and the bytes present. Owid::fromReader reads the payload through
readPayload, so an envelope with a byte after the signature, previously
ignored, or a signature shorter than 64 bytes is refused as malformed.
Io::readByteArray is unchanged because it is already bounded by
readBytes and is not tied to the signature, and its comment now says
so. The domain terminator scan uses strpos, which stops at the end of
the buffer, and the date reads go through the bounded readBytes, so no
other count driven read needed a change.

tests/PayloadLengthTest.php covers a matching envelope, the library's
own signed output, off by one counts, a trailing byte, a short
signature, declared lengths of 64 MiB, 2 GiB and 0xFFFFFFFF each
refused 1,000 times inside a second and with under 64 KiB of peak
memory where PHP can measure it, and an empty payload. tests/run.php,
the dependency free fallback runner, carries the same checks.
The domain in an OWID envelope is stored as text followed by a zero
terminator, and the parse found the end of it by walking forward to
that terminator. Nothing stopped the walk, so a buffer whose
terminator was missing or corrupted was read all the way to its end.
That is work an attacker controls, in the same class as the declared
payload length this branch already checks, and it was the last
unbounded read left in the envelope parse.

The search now stops after 253 characters and the envelope is refused
when no terminator has been found by then, which also refuses a domain
longer than a domain name may be. Refusing costs the bound rather than
the length of the input, because the search takes the window as an
argument instead of running to the end of the buffer.

RFC 1035 section 2.3.4, "Size limits", restricts the total length of a
domain name, counting label octets and label length octets, to 255
octets or less. That figure is the wire format, which spends one
length octet on every label and one zero octet on the root, whereas
OWID stores the presentation form, the text example.com, where the
dots stand in for the label length octets and the root has no text at
all. Exactly two of those 255 octets therefore have no character here,
so the maximum is 253. The number is held in one named constant beside
the signature length, and the comment on the constant carries this
reasoning so it can be checked without leaving the file.

Refusals use the library exception type. A new named constructor sits
alongside the existing ones because the message has to name the
maximum and must not repeat the offending bytes back, of which there
may be no end.

Tests cover a domain of the greatest length, one character more, a
buffer with no terminator at all, a run of characters filling the
bound exactly, and what the library itself signs. The cost of the
unterminated case is timed over two buffers sixteen times apart, so
the check rests on the cost not growing with the buffer rather than on
how fast the machine is. The dependency free runner in tests/run.php
gains the same checks. The README no longer says the domain has no
maximum.
@jwrosewell jwrosewell changed the title Check the declared payload length before allocating Bound the declared payload length and the domain read Aug 30, 2026
The read was bounded at the greatest number of characters a domain
name can hold but the write was not, so a creator configured with a
longer domain still produced an OWID that this same library refused to
parse, and the fault landed on whoever read it rather than on the
creator that caused it.

The domain is now refused at two points. The Creator constructor
checks it first, which is the earliest point the caller can be told
and is before the crypto instance is looked at, so nothing is ever
signed with a domain that could not be read back. Io::writeString
checks it again, so a domain that reached the public Owid domain field
by some other route is still refused, and that refusal happens while
the data to sign is being built rather than after a signature has been
calculated.

Both points raise OwidException::domainTooLong and reuse
OwidException::MAXIMUM_DOMAIN_LENGTH, so the two halves report the one
condition the one way. The message of that named constructor is
reworded to be true from either side and it still names the maximum.
It does not name the domain, because writeString cannot tell whether
the value came from the caller's own configuration or from bytes that
some other route filled in.

An empty domain, and any domain at or under the maximum, behave
exactly as before. The two test files that deliberately build an over
long domain for the read side now append the terminator themselves
rather than going through writeString, because the write side would
otherwise refuse the bytes those read tests are built from.

The PHPUnit suite goes from 74 tests to 77 and the dependency free
runner from 105 checks to 113. With the two checks removed and the new
tests kept, all three new PHPUnit tests fail and seven of the eight
new runner checks fail, the eighth being the control at exactly the
maximum which is meant to pass either way.
@jwrosewell jwrosewell changed the title Bound the declared payload length and the domain read Bound the declared payload length, and the domain on read and write Aug 30, 2026
@jwrosewell
jwrosewell merged commit 5697866 into main Aug 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant