Bound the declared payload length, and the domain on read and write - #1
Merged
Merged
Conversation
…locating Owid::fromReader read the payload through Io::readByteArray, which takes the sender's declared count and slices that many bytes from the buffer. Io::readBytes already refused a count beyond the end of the buffer before slicing, so this port never allocated by the declared number, but nothing checked that the count left exactly the 64 byte signature after the payload. A count short of the payload let payload bytes be read as the signature, and bytes after the signature were ignored, so malformed OWIDs that the reference fix in owid-dotnet now refuses still parsed here. The count is now checked before anything is sized by it. A valid OWID is the declared payload followed by the 64 byte signature and nothing else, so the new Io::readPayload requires the count to equal the bytes remaining less the signature length, and any other count, short or long, is refused with the existing exception type through the new OwidException::payloadLengthMismatch, which names the declared length and the bytes present. Owid::fromReader reads the payload through readPayload, so an envelope with a byte after the signature, previously ignored, or a signature shorter than 64 bytes is refused as malformed. Io::readByteArray is unchanged because it is already bounded by readBytes and is not tied to the signature, and its comment now says so. The domain terminator scan uses strpos, which stops at the end of the buffer, and the date reads go through the bounded readBytes, so no other count driven read needed a change. tests/PayloadLengthTest.php covers a matching envelope, the library's own signed output, off by one counts, a trailing byte, a short signature, declared lengths of 64 MiB, 2 GiB and 0xFFFFFFFF each refused 1,000 times inside a second and with under 64 KiB of peak memory where PHP can measure it, and an empty payload. tests/run.php, the dependency free fallback runner, carries the same checks.
The domain in an OWID envelope is stored as text followed by a zero terminator, and the parse found the end of it by walking forward to that terminator. Nothing stopped the walk, so a buffer whose terminator was missing or corrupted was read all the way to its end. That is work an attacker controls, in the same class as the declared payload length this branch already checks, and it was the last unbounded read left in the envelope parse. The search now stops after 253 characters and the envelope is refused when no terminator has been found by then, which also refuses a domain longer than a domain name may be. Refusing costs the bound rather than the length of the input, because the search takes the window as an argument instead of running to the end of the buffer. RFC 1035 section 2.3.4, "Size limits", restricts the total length of a domain name, counting label octets and label length octets, to 255 octets or less. That figure is the wire format, which spends one length octet on every label and one zero octet on the root, whereas OWID stores the presentation form, the text example.com, where the dots stand in for the label length octets and the root has no text at all. Exactly two of those 255 octets therefore have no character here, so the maximum is 253. The number is held in one named constant beside the signature length, and the comment on the constant carries this reasoning so it can be checked without leaving the file. Refusals use the library exception type. A new named constructor sits alongside the existing ones because the message has to name the maximum and must not repeat the offending bytes back, of which there may be no end. Tests cover a domain of the greatest length, one character more, a buffer with no terminator at all, a run of characters filling the bound exactly, and what the library itself signs. The cost of the unterminated case is timed over two buffers sixteen times apart, so the check rests on the cost not growing with the buffer rather than on how fast the machine is. The dependency free runner in tests/run.php gains the same checks. The README no longer says the domain has no maximum.
The read was bounded at the greatest number of characters a domain name can hold but the write was not, so a creator configured with a longer domain still produced an OWID that this same library refused to parse, and the fault landed on whoever read it rather than on the creator that caused it. The domain is now refused at two points. The Creator constructor checks it first, which is the earliest point the caller can be told and is before the crypto instance is looked at, so nothing is ever signed with a domain that could not be read back. Io::writeString checks it again, so a domain that reached the public Owid domain field by some other route is still refused, and that refusal happens while the data to sign is being built rather than after a signature has been calculated. Both points raise OwidException::domainTooLong and reuse OwidException::MAXIMUM_DOMAIN_LENGTH, so the two halves report the one condition the one way. The message of that named constructor is reworded to be true from either side and it still names the maximum. It does not name the domain, because writeString cannot tell whether the value came from the caller's own configuration or from bytes that some other route filled in. An empty domain, and any domain at or under the maximum, behave exactly as before. The two test files that deliberately build an over long domain for the read side now append the terminator themselves rather than going through writeString, because the write side would otherwise refuse the bytes those read tests are built from. The PHPUnit suite goes from 74 tests to 77 and the dependency free runner from 105 checks to 113. With the two checks removed and the new tests kept, all three new PHPUnit tests fail and seven of the eight new runner checks fail, the eighth being the control at exactly the maximum which is meant to pass either way.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Defect and format rule
The parser previously accepted a payload length that left bytes after the 64-byte signature. This PR requires complete byte-array and Base64 input to contain exactly one envelope. The public reader consumes one OWID and leaves following framed bytes. A declaration that does not agree with the bytes present is rejected before the payload is copied.
The unsigned 32-bit field itself permits payloads from 0 through 4,294,967,295 bytes. A large value is not malformed when the matching bytes are present. Runtime capacity and application policy are separate from format validity.
The domain had the same shape of fault and it is now bounded too. The domain is stored as text followed by a zero terminator, and the parser found the end of it by walking forward to that terminator with nothing to stop the walk, so a buffer whose terminator was missing or corrupted was read all the way to its end. That was the last unbounded read left in the envelope parse. The walk now stops at the greatest number of characters a domain name can hold, so a hostile buffer costs that maximum rather than its own length, and a domain longer than a domain name may be is refused as well.
The maximum is 253 characters. RFC 1035 section 2.3.4, "Size limits", restricts the total length of a domain name, counting label octets and label length octets, to 255 octets or less. That figure is the wire format, which spends one length octet on every label and one zero octet on the root, whereas OWID stores the presentation form, the text
example.com, where the dots stand in for the label length octets and the root has no text at all. Exactly two of those 255 octets therefore have no character here. The number is held in one named constant beside the signature length and the comment on it carries this reasoning, so a reader can check where 253 came from without leaving the file. The bound owes nothing to any application policy.Bounding only the read left the two halves of the library disagreeing with each other, because a creator configured with a domain longer than 253 characters still produced an OWID that this same library refused to parse, so the fault landed on whoever read it rather than on the creator that caused it. A library that can emit what it cannot read is worse than one that does neither. The specification is being changed to match in owid#6, so that the limit binds a creator as well as a consumer, and this PR now enforces the half PHP owns on both sides.
Where the write is refused
Two points, and both are cheap.
Creatorconstructor, which is the earliest point at which the caller can be told, so a wrongly configured domain is refused when the domain is supplied rather than when an OWID is later serialized. The check sits before the crypto instance is looked at, so a creator that would sign an unreadable OWID is never built and nothing is signed with such a domain.Io::writeString, which is the only place a domain is serialized, so a value that reached the publicOwiddomain field by some other route is still refused. That refusal happens while the data to sign is being built, which is before the signing key is used, so no signature is calculated over a value that is about to be thrown away.Both raise
OwidException::domainTooLongand both readOwidException::MAXIMUM_DOMAIN_LENGTH, the same named constructor and the same constant the read bound already uses, so the two halves report the one condition the one way. The message of that named constructor is reworded so it is true from either side, and it still names the maximum. It still does not name the domain. On the write side the domain does come from the caller's own configuration rather than from an attacker, so echoing it back would be safe there and might help an operator, butwriteStringcannot tell which of the two routes a value arrived by, and a caller that has just handed in a domain already has it, along with a stack trace naming the call site. One message for one condition is worth more here than a longer message on one of the two paths.Current changes
strlen($owid->payload)is the allocation-free structured size for downstream policy.Creatorconstructor and inIo::writeString, so the library cannot produce an OWID it would then refuse to read.Io::writeString, which now refuses the very bytes those read tests are built from.An empty domain, and any domain at or under the maximum, behave exactly as before. This is a refusal at the top of the range and nothing else.
A malformed envelope declaring 4 GiB but carrying only a few bytes still fails with work and memory proportional to the bytes actually supplied. A matching payload is processed in proportion to its real size and remains subject to PHP string, platform, address-space, and available-memory capacity.
Validation
The full PHPUnit suite reports 77 tests and 5,195 assertions passing, and the dependency-free repository runner reports 113 checks passed. Before the write bound those figures were 74 tests with 5,187 assertions and 105 checks, and before the read bound they were 69 tests with 3,172 assertions and 98 checks, so every test that already passed still passes untouched. The changed PHP files also pass
php -l.The new tests were checked against the defect they describe. With only the two write-side checks reverted and the new tests left in place, PHPUnit fails all three new tests, being
testCreatorRefusesDomainOverMaximumbecause the creator is built,testWriteRefusesDomainOverMaximumbecauseasByteArrayreturns bytes, andtestRefusalHappensBeforeAnySignaturebecause the constructor reaches the crypto check and reportsinstance of Crypto cannot be used to generate a signatureinstead of naming the maximum, which is the ordering that test exists to pin down. The dependency-free runner fails seven of its eight new checks. The eighth, which serializes a domain of exactly the maximum and parses it back, passes either way on purpose, because it is the control that shows the refusal is at the top of the range and not below it.The read bound was checked the same way when it was added. With only the read bound reverted, PHPUnit failed
testDomainOneOverMaximumIsRefused, because the over-long domain was accepted, andtestUnterminatedDomainIsRefusedWithoutReadingTheBuffer, because 1,000 refusals over a 16 MiB buffer took 1.87s against 0.11s over a 1 MiB one, which is the cost growing with the buffer. The dependency-free runner failed the three matching checks. PHP cannot count the bytes a single call examines, so the bound is shown by that timing rather than measured directly.References: the format-validation fix is owid-dotnet#5, the .NET size-policy follow-up is owid-dotnet#6, and the specification change that makes the limit bind a creator as well as a consumer is owid#6.
This change was produced with AI assistance under James Rosewell's direction and needs human review before merge.