From 765fff490f2577f4bb109a8fe48c1590a9ad4082 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Sun, 30 Aug 2026 23:59:06 +0100 Subject: [PATCH 1/5] Harden parsing and close unsigned OWID construction Reading an OWID means reading whatever a caller was handed, which on a public end point is anything at all, so malformed data is an ordinary outcome rather than an exceptional one. Reporting it by throwing costs the construction and unwinding of an exception for every bad input, and whoever is sending the data chooses how often that happens. Owid.tryParse and Owid.tryParseBytes now report three facts every time, being whether it worked, the OWID only when it did, and a named OwidParseStatus either way. The new OwidReader walks the buffer by index and checks every read against what is left, so a bad envelope is a comparison that fails. It deliberately does not call the old parser and catch, because the exception would still be built and unwound and only the surface would look different. Base 64 decoding is written out here for the same reason, as the strict JDK decoder throws and the lenient one drops every character outside the alphabet, which would report text that is not base 64 at all as a missing OWID. A declared payload that cannot leave exactly the signature the version requires is BYTE_COUNT_MISMATCH whichever way the bytes fall short, including where the buffer also ended early. UNEXPECTED_END is for data that stops inside a field before the length is read. An OWID is only worth anything because it is signed, so a caller can no longer build one. The constructor is package private and the fields are final with no setters, so an instance reaches calling code only from a successful read or from Creator.createString and Creator.createBytes, which own the version, domain, date and signature. The payload and signature are handed out as copies, because a Java byte array is mutable. Creator.sign, signWithOthers, signString and signBytes are gone, as are Owid.fromBase64, Owid.fromByteArray and the Io reader they used. OwidSignatureStatus and verifyDetailed keep "could not check" apart from "does not match". A key that cannot be obtained or cannot be decoded leaves the signature unjudged, and reporting that as invalid would read as an attack rather than as the outage it is. The end point helper no longer repeats the format query parameter back in its refusal, since that value comes from whoever called the end point and a refusal is often logged. This is a deliberate breaking change to the source a consumer compiles in. The README carries a before and after table for every removed call. --- README.md | 167 ++++++++-- .../swancommunity/owid/CapacityException.java | 39 +++ .../java/com/swancommunity/owid/Creator.java | 108 ++++--- .../com/swancommunity/owid/Endpoints.java | 7 +- src/main/java/com/swancommunity/owid/Io.java | 144 +-------- .../java/com/swancommunity/owid/Owid.java | 306 ++++++++++-------- .../com/swancommunity/owid/OwidException.java | 9 +- .../swancommunity/owid/OwidParseResult.java | 100 ++++++ .../swancommunity/owid/OwidParseStatus.java | 96 ++++++ .../com/swancommunity/owid/OwidReader.java | 294 +++++++++++++++++ .../owid/OwidSignatureStatus.java | 82 +++++ .../owid/OwidVerificationResult.java | 62 ++++ .../java/com/swancommunity/owid/Version.java | 14 +- .../ConstructionBoundaryTest.java | 176 ++++++++++ .../owidconsumer/ReadmeExampleTest.java | 97 ++++++ .../com/swancommunity/owid/CreatorTest.java | 62 +++- .../swancommunity/owid/DomainLengthTest.java | 63 ++-- .../java/com/swancommunity/owid/Envelope.java | 105 ++++++ .../com/swancommunity/owid/FixturesTest.java | 12 +- .../java/com/swancommunity/owid/IoTest.java | Bin 4859 -> 6446 bytes .../com/swancommunity/owid/ParseAssert.java | 65 ++++ .../swancommunity/owid/ParseContractTest.java | 305 +++++++++++++++++ .../swancommunity/owid/PayloadLengthTest.java | 46 +-- .../owid/SignatureStatusTest.java | 187 +++++++++++ .../swancommunity/owid/WireVectorsTest.java | 21 +- 25 files changed, 2151 insertions(+), 416 deletions(-) create mode 100644 src/main/java/com/swancommunity/owid/CapacityException.java create mode 100644 src/main/java/com/swancommunity/owid/OwidParseResult.java create mode 100644 src/main/java/com/swancommunity/owid/OwidParseStatus.java create mode 100644 src/main/java/com/swancommunity/owid/OwidReader.java create mode 100644 src/main/java/com/swancommunity/owid/OwidSignatureStatus.java create mode 100644 src/main/java/com/swancommunity/owid/OwidVerificationResult.java create mode 100644 src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java create mode 100644 src/test/java/com/example/owidconsumer/ReadmeExampleTest.java create mode 100644 src/test/java/com/swancommunity/owid/Envelope.java create mode 100644 src/test/java/com/swancommunity/owid/ParseAssert.java create mode 100644 src/test/java/com/swancommunity/owid/ParseContractTest.java create mode 100644 src/test/java/com/swancommunity/owid/SignatureStatusTest.java diff --git a/README.md b/README.md index 8efaba1..6efe78a 100644 --- a/README.md +++ b/README.md @@ -42,9 +42,10 @@ limit for the complete envelope. This library validates that the declared payload length agrees with the bytes present before it sizes or copies the payload. A large declaration without -the corresponding bytes is malformed and is rejected without allocating the -declared size. A matching large payload is not malformed merely because it is -large, and parsing work and memory use scale with the bytes actually present. +the corresponding bytes is malformed, and is reported as +`BYTE_COUNT_MISMATCH` without allocating the declared size. A matching large +payload is not malformed merely because it is large, and parsing work and +memory use scale with the bytes actually present. The in-memory APIs remain subject to Java's signed `int` array indexing, address-space and available-memory limits, so a single Java byte array cannot @@ -62,7 +63,8 @@ on behalf of the application. ## Installation -Build and install with Maven. The project targets Java 21. +Build and install with Maven. The library is compiled for Java 8, and the +test suite runs on Java 8, 11, 17 and 21. ``` mvn install @@ -80,10 +82,15 @@ Then depend on it from another Maven project. ## Usage +The example below is compiled and run by the test suite as +`ReadmeExampleTest`, so a change to the library that would break it fails the +build rather than leaving a documented example that no longer works. + ```java import com.swancommunity.owid.Creator; import com.swancommunity.owid.Crypto; import com.swancommunity.owid.Owid; +import com.swancommunity.owid.OwidParseResult; import java.util.Collections; @@ -91,44 +98,145 @@ import java.util.Collections; Crypto crypto = Crypto.generate(); Creator creator = Creator.create("example.com", crypto); -// Create and sign an OWID with a payload. -Owid owid = creator.signString("Hello World"); +// Create a signed OWID with a payload. An OWID is signed from the moment it +// exists, so there is never an unsigned one to hold. +Owid owid = creator.createString("Hello World"); // Serialize to base 64 for storage or transmission. String encoded = owid.asBase64(); -// Later, or elsewhere, decode and verify with the public key. -Owid copy = Owid.fromBase64(encoded); -String publicPem = crypto.publicKeyPem(); -boolean valid = copy.verifyWithPublicKey(publicPem, Collections.emptyList()); +// Later, or elsewhere, read it back. Reading answers rather than throwing, +// because whatever arrives from outside may not be an OWID at all. +OwidParseResult result = Owid.tryParse(encoded); +if (result.isSuccess()) { + Owid copy = result.getValue(); + String publicPem = crypto.publicKeyPem(); + boolean valid = copy.verifyWithPublicKey( + publicPem, Collections.emptyList()); +} else { + // result.getStatus() names which of the expected problems it was, and + // result.getValue() is null. +} ``` Chaining covers other OWIDs with the same signature. The same others, in the same order, must be supplied when verifying as were supplied when signing. ```java -Owid root = creator.signString("root"); - -Owid party = new Owid(); -party.setPayload("party".getBytes()); -creator.signWithOthers(party, java.util.List.of(root)); +Owid root = creator.createString("root"); +Owid party = creator.createString("party", Collections.singletonList(root)); // Verifies with the root as the single other, fails without it. -party.verifyWithCrypto(crypto, java.util.List.of(root)); // true -party.verifyWithCrypto(crypto, Collections.emptyList()); // false +party.verifyWithCrypto(crypto, Collections.singletonList(root)); // true +party.verifyWithCrypto(crypto, Collections.emptyList()); // false ``` +## Reading, and why it does not throw + +An OWID is read from whatever a caller was handed, which on a public end +point means anything at all, so malformed data is an ordinary outcome rather +than an exceptional one. `Owid.tryParse` and `Owid.tryParseBytes` therefore +report three facts every time. + +| Fact | Where | +|------|-------| +| Whether it worked | `isSuccess()` | +| The OWID, only when it worked | `getValue()`, null otherwise | +| A named reason, either way | `getStatus()` | + +The statuses are the cross language vocabulary, so a failure means the same +thing whichever language read the bytes. + +| Status | Meaning | +|--------|---------| +| `PARSED` | The bytes are a structurally valid OWID. | +| `MISSING_INPUT` | Nothing was supplied to read. | +| `INVALID_INPUT_TYPE` | Not reachable in Java, where the compiler already refuses anything that is not a string or a byte array. | +| `INVALID_BASE64` | The string is not base 64, so there are no bytes to read. | +| `UNSUPPORTED_VERSION` | The first byte names a version this library does not know. | +| `UNEXPECTED_END` | The data stopped in the middle of a field. | +| `INVALID_DOMAIN_ENCODING` | The domain is unterminated, or longer than the published maximum. | +| `BYTE_COUNT_MISMATCH` | The declared payload count disagrees with the bytes present. | +| `IMPLEMENTATION_CAPACITY_EXCEEDED` | Larger than this runtime can hold. Not reachable from the byte array surface, because a Java array cannot exceed `Integer.MAX_VALUE` bytes and so can never agree with a larger declaration. | +| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. | + +Reading and verifying are separate questions, and reading fetches no key and +performs no cryptography. Bytes that are a well formed OWID read successfully +even when the signature does not match, and only `verifyDetailed` then +reports that it does not. + +`verifyDetailed` keeps "does not match" apart from "could not check", because +a key that cannot be obtained or cannot be decoded leaves the signature +unjudged, and reporting that as invalid would read as an attack rather than +as the outage it is. + +| Status | Meaning | +|--------|---------| +| `SIGNATURE_VALID` | Genuine for this data and this key. | +| `SIGNATURE_INVALID` | Well formed and does not match. The only status that means the identifier should be distrusted. | +| `INVALID_SIGNATURE_LENGTH` | A signature field of the wrong length reached the check. | +| `KEY_UNAVAILABLE` | No key was supplied, or the one supplied cannot verify. | +| `INVALID_KEY` | Key material arrived and cannot be decoded or used. | +| `IMPLEMENTATION_CAPACITY_EXCEEDED` | More work than this runtime can hold. | +| `VERIFICATION_ERROR` | The check could not be completed for a reason that is not the identifier's fault. | + +## How an OWID comes into being + +An OWID is only worth anything because it is signed, so a caller cannot build +one. There is no public constructor and no setter, and an instance reaches +calling code by exactly two routes. + +1. A successful read of a complete serialized OWID, through `Owid.tryParse` + or `Owid.tryParseBytes`. +2. A creator signing one into existence, through `createString` or + `createBytes`. + +An unsigned OWID is indistinguishable from a signed one to the code +downstream of it, and the difference only surfaces later when a verification +fails somewhere nobody is watching, which is why there is no way to obtain +one. There is no public way to sign an OWID either, because with nothing +unsigned to hold there is nothing outside to sign, and signing a parsed OWID +again would replace the signature its fields were read with. + +The fields are read only for the same reason. The signature covers them as +they arrived, so a caller changing one afterwards would hold something the +signature no longer describes. `getPayload` and `getSignature` hand back +copies, because a Java byte array is mutable. + +## Migrating from the earlier surface + +| Before | After | +|--------|-------| +| `Owid.fromBase64(value)` | `Owid.tryParse(value)` | +| `Owid.fromByteArray(buffer)` | `Owid.tryParseBytes(buffer)` | +| `new Owid()`, then `setPayload`, then `creator.sign(owid)` | `creator.createBytes(payload)` | +| `creator.signString(value)` | `creator.createString(value)` | +| `creator.signBytes(value)` | `creator.createBytes(value)` | +| `new Owid()`, then `creator.signWithOthers(owid, others)` | `creator.createBytes(payload, others)` | +| `owid.setVersion`, `setDomain`, `setDate`, `setPayload` | no replacement, the state is read only | +| `Version.fromByte(b)` | no replacement, an unknown version byte is `UNSUPPORTED_VERSION` from a read | + +The parse surfaces do not throw for malformed data, so a caller that wrapped +the old ones in `try`/`catch` reads the status instead. `OwidException` is +still raised for the caller's own mistakes, such as an invalid creator +domain, a null payload, or a field that cannot be serialized. + ## Interface - `Owid` holds the version, domain, date to the minute in UTC, payload bytes, - and signature bytes. - - `Owid.fromBase64` and `Owid.fromByteArray` parse a signed OWID. + and signature bytes, all read only. + - `Owid.tryParse` and `Owid.tryParseBytes` read a signed OWID and report + why rather than throwing. - `asBase64` and `asByteArray` serialize a signed OWID. - `payloadAsString` decodes the payload as UTF-8. `payloadAsPrintable` returns zero padded lower case hexadecimal with no separator. - `payloadAsBase64` returns the payload as base 64. + `payloadAsBase64` returns the payload as base 64. `getPayloadLength` + reports the payload size without copying it. - `verifyWithCrypto` and `verifyWithPublicKey` return whether the signature, covering this OWID and any others provided, is valid. + - `verifyDetailed` and `verifyDetailedWithPublicKey` answer the same + question with a status, keeping a key that could not be used apart from a + signature that does not match. - `ageMinutes` returns the minutes elapsed since creation. - `Crypto` holds the keys. - `Crypto.generate` creates a new P-256 key pair. @@ -141,10 +249,10 @@ party.verifyWithCrypto(crypto, Collections.emptyList()); // false - An empty, whitespace, or null PEM is rejected with a clear message rather than an opaque cryptography error. - `Creator` binds a domain to a signing `Crypto`. - - `sign` and `signWithOthers` set the OWID domain to the creator domain, the - date to the current time, and the version to the current version, then - sign. - - `signString` and `signBytes` create and sign a new OWID. + - `createString` and `createBytes` create a complete signed OWID, setting + the domain to the creator domain, the date to the current time and the + version to the current version. Both take an optional list of other OWIDs + to cover with the same signature. - `Endpoints` provides framework agnostic helpers for the well known end points. - `creatorResponse` returns JSON with the fields `domain`, `name`, @@ -181,7 +289,8 @@ An empty OWID is written as the single byte `0x00`. It marks an absent optional OWID inside a larger byte array. Base 64 decoding accepts the standard alphabet with or without the trailing -padding. Encoding always emits padding. +padding, and skips line breaks and spaces. Anything else in the string is +reported as `INVALID_BASE64`. Encoding always emits padding. ## Testing @@ -193,8 +302,12 @@ mvn test The tests round trip the canonical wire format vectors byte for byte, verify cross language signed fixtures including the chained case, confirm that a -flipped signature byte fails verification, and cover the binary read and write -helpers, the crypto, the creator, and the end point helpers. +flipped signature byte fails verification, and cover the binary write +helpers, the crypto, the creator, and the end point helpers. They also cover +the parse contract, being every status the reading surfaces report together +with a run of malformed buffers that must never throw, and the construction +boundary, which is checked from a package outside the library because a check +made from inside it would measure nothing. ## License diff --git a/src/main/java/com/swancommunity/owid/CapacityException.java b/src/main/java/com/swancommunity/owid/CapacityException.java new file mode 100644 index 0000000..1adf211 --- /dev/null +++ b/src/main/java/com/swancommunity/owid/CapacityException.java @@ -0,0 +1,39 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +/** + * Raised when an OWID is well formed but larger than this runtime can hold, + * being a Java array limited to {@link Integer#MAX_VALUE} bytes. + * + *

Kept apart from the other failures so that a verification result can + * report {@link OwidSignatureStatus#IMPLEMENTATION_CAPACITY_EXCEEDED} + * without reading the text of a message. Running out of room is not the same + * as the data being wrong, because the same OWID may be readable on a + * runtime with more of it.

+ * + *

Not public, because callers catch {@link OwidException} and the + * distinction is only needed inside the library.

+ */ +final class CapacityException extends OwidException { + + private static final long serialVersionUID = 1L; + + CapacityException(String message) { + super(message); + } +} diff --git a/src/main/java/com/swancommunity/owid/Creator.java b/src/main/java/com/swancommunity/owid/Creator.java index 2d42cc4..6295796 100644 --- a/src/main/java/com/swancommunity/owid/Creator.java +++ b/src/main/java/com/swancommunity/owid/Creator.java @@ -26,9 +26,14 @@ * Needed to create new OWIDs. * *

A creator binds the domain that hosts the well known end points to the - * crypto instance holding the signing key. Signing an OWID sets its domain to - * the creator domain, its date to the current time, and its version to the - * current version, then produces the signature.

+ * crypto instance holding the signing key. Creating an OWID sets its domain + * to the creator domain, its date to the current time and its version to the + * current version, signs it, and hands back the finished thing.

+ * + *

There is no way to sign an OWID that already exists, because there is no + * way to obtain an unsigned one, so nothing outside the library is available + * to be signed. Signing a parsed OWID again would replace the signature its + * fields were read with, which is why the library does not offer it.

*/ public final class Creator { @@ -109,63 +114,82 @@ public Crypto crypto() { } /** - * Signs the OWID provided, setting the domain to the creator domain, the - * date to the current time, and the version to the current version. + * Creates a new signed OWID for this creator carrying the string as the + * UTF-8 payload. * - * @param owid the OWID to sign - * @throws OwidException if a field cannot be encoded or the signing - * operation fails + * @param value the payload string + * @return the signed OWID + * @throws OwidException if the payload is null, a field cannot be + * encoded, or the signing operation fails */ - public void sign(Owid owid) throws OwidException { - signWithOthers(owid, Collections.emptyList()); + public Owid createString(String value) throws OwidException { + return createString(value, Collections.emptyList()); } /** - * Signs the OWID provided together with the other OWIDs provided. The same - * others, in the same order, must be passed when verifying. + * Creates a new signed OWID for this creator carrying the bytes as the + * payload. * - * @param owid the OWID to sign - * @param others the other OWIDs to cover with the signature - * @throws OwidException if a field cannot be encoded or the signing - * operation fails + * @param value the payload bytes + * @return the signed OWID + * @throws OwidException if the payload is null, a field cannot be + * encoded, or the signing operation fails */ - public void signWithOthers(Owid owid, List others) - throws OwidException { - owid.setVersion(Version.current()); - owid.setDomain(domain); - owid.setDate(Instant.now().truncatedTo(ChronoUnit.MINUTES)); - byte[] data = owid.dataForCrypto(others); - byte[] signature = crypto.signByteArray(data); - if (signature.length != Owid.SIGNATURE_LENGTH) { - throw Io.invalidSignatureLength(signature.length); - } - owid.setSignature(signature); + public Owid createBytes(byte[] value) throws OwidException { + return createBytes(value, Collections.emptyList()); } /** - * Creates a new signed OWID for the creator containing the string as the - * UTF-8 payload. + * Creates a new signed OWID carrying the string as the UTF-8 payload, + * with the other OWIDs covered by the same signature so that a tree can + * be verified as a whole. The same others, in the same order, must be + * passed when verifying. * - * @param value the payload string + * @param value the payload string + * @param others the other OWIDs to cover with the signature * @return the signed OWID - * @throws OwidException see {@link #sign(Owid)} + * @throws OwidException see {@link #createString(String)} */ - public Owid signString(String value) throws OwidException { - return signBytes(value.getBytes(StandardCharsets.UTF_8)); + public Owid createString(String value, List others) + throws OwidException { + if (value == null) { + throw new OwidException("payload is null"); + } + return createBytes(value.getBytes(StandardCharsets.UTF_8), others); } /** - * Creates a new signed OWID for the creator containing the bytes as the - * payload. + * Creates a new signed OWID carrying the bytes as the payload, with the + * other OWIDs covered by the same signature. * - * @param value the payload bytes + *

This is one of only two ways an OWID reaches calling code, the other + * being a successful read of a complete serialized one. The creator owns + * the version, the domain, the date and the signature, and a caller + * supplies the payload and nothing else, so there is no moment at which a + * partly built OWID exists for anyone to hold or pass on.

+ * + * @param value the payload bytes + * @param others the other OWIDs to cover with the signature * @return the signed OWID - * @throws OwidException see {@link #sign(Owid)} + * @throws OwidException see {@link #createBytes(byte[])} */ - public Owid signBytes(byte[] value) throws OwidException { - Owid owid = new Owid(); - owid.setPayload(value); - sign(owid); - return owid; + public Owid createBytes(byte[] value, List others) + throws OwidException { + if (value == null) { + throw new OwidException("payload is null"); + } + if (others == null) { + throw new OwidException("others is null"); + } + Version version = Version.current(); + Instant date = Instant.now().truncatedTo(ChronoUnit.MINUTES); + byte[] payload = value.clone(); + byte[] data = Owid.dataForCrypto( + version, domain, date, payload, others); + byte[] signature = crypto.signByteArray(data); + if (signature.length != Owid.SIGNATURE_LENGTH) { + throw Io.invalidSignatureLength(signature.length); + } + return new Owid(version, domain, date, payload, signature); } } diff --git a/src/main/java/com/swancommunity/owid/Endpoints.java b/src/main/java/com/swancommunity/owid/Endpoints.java index 8d83b06..8ccc1ed 100644 --- a/src/main/java/com/swancommunity/owid/Endpoints.java +++ b/src/main/java/com/swancommunity/owid/Endpoints.java @@ -102,8 +102,11 @@ public static String publicKeyResponse(Creator creator, String format) if ("spki".equals(format) || "pkcs".equals(format)) { return creator.crypto().subjectPublicKeyInfo(); } - throw new OwidException("format parameter 'spki' or 'pkcs' must be " - + "provided, received '" + format + "'"); + // The value is not repeated back, because it arrives on a query + // string from whoever called the end point and a refusal is often + // logged. + throw new OwidException( + "format parameter 'spki' or 'pkcs' must be provided"); } private static void appendField(StringBuilder json, String name, diff --git a/src/main/java/com/swancommunity/owid/Io.java b/src/main/java/com/swancommunity/owid/Io.java index 74616ba..77584dc 100644 --- a/src/main/java/com/swancommunity/owid/Io.java +++ b/src/main/java/com/swancommunity/owid/Io.java @@ -22,10 +22,14 @@ import java.time.Instant; /** - * Low level read and write helpers for the OWID binary format. The format - * uses little endian unsigned 32 bit integers, null terminated strings, and a - * fixed 64 byte signature. Version 1 stores the date as a two byte big endian - * count of hours. + * Low level write helpers, and the constants both halves of the library + * share, for the OWID binary format. The format uses little endian unsigned + * 32 bit integers, null terminated strings, and a fixed 64 byte signature. + * Version 1 stores the date as a two byte big endian count of hours. + * + *

Reading lives in {@link OwidReader}, which walks a buffer by index and + * reports why rather than throwing, because the bytes it reads come from + * outside.

* *

The class is not part of the public API. The methods are package private * so that the unit tests can exercise them directly.

@@ -63,138 +67,6 @@ static Instant baseDate() { return Instant.ofEpochSecond(BASE_DATE_EPOCH_SECONDS); } - /** Sequential reader over a byte buffer. */ - static final class Reader { - - private final byte[] buffer; - private int position; - - Reader(byte[] buffer) { - this.buffer = buffer; - this.position = 0; - } - - int position() { - return position; - } - - int readByte() throws OwidException { - if (position >= buffer.length) { - throw endOfBuffer(); - } - return buffer[position++] & 0xFF; - } - - /** - * Copies the next count bytes. The end of the buffer is checked - * before the copy is sized, so a count beyond the bytes present is - * refused without allocating. - */ - private byte[] readBytes(int count) throws OwidException { - if (count < 0) { - throw new OwidException("payload length is negative"); - } - long end = (long) position + count; - if (end > buffer.length) { - throw endOfBuffer(); - } - byte[] value = new byte[count]; - System.arraycopy(buffer, position, value, 0, count); - position += count; - return value; - } - - /** - * Reads the domain, being the bytes up to the null terminator. The - * search stops at {@link Io#MAXIMUM_DOMAIN_LENGTH} rather than at - * the end of the buffer, so a buffer whose terminator is missing or - * corrupted costs no more than the published maximum however long - * that buffer is, and a domain longer than the maximum is refused - * without reading past it. - */ - String readString() throws OwidException { - long lastTerminator = (long) position + MAXIMUM_DOMAIN_LENGTH; - int limit = (int) Math.min(buffer.length - 1L, lastTerminator); - int terminator = -1; - for (int i = position; i <= limit; i++) { - if (buffer[i] == 0) { - terminator = i; - break; - } - } - if (terminator < 0) { - if (lastTerminator < buffer.length) { - throw domainTooLong(); - } - throw endOfBuffer(); - } - String value = new String(buffer, position, terminator - position, - StandardCharsets.UTF_8); - position = terminator + 1; - return value; - } - - /** Reads an unsigned 32 bit integer in little endian byte order. */ - long readUInt32() throws OwidException { - return ((long) readByte()) - | ((long) readByte() << 8) - | ((long) readByte() << 16) - | ((long) readByte() << 24); - } - - /** - * Reads the length prefixed payload. The count is whatever the sender - * declared, so it is checked against the bytes actually present - * before anything is sized by it. A valid OWID is the declared - * payload followed by the signature and nothing else, so the count - * must equal the bytes remaining less the signature length, and any - * other count, short or long, is refused here. The same check refuses - * an envelope with bytes after the signature, which was previously - * accepted and ignored, and one whose signature is short. - */ - byte[] readByteArray() throws OwidException { - long count = readUInt32(); - long remaining = (long) buffer.length - position; - long expected = count + Owid.SIGNATURE_LENGTH; - if (remaining != expected) { - throw new OwidException("OWID payload length '" + count - + "' does not match the '" + remaining - + "' bytes present, of which the final '" - + Owid.SIGNATURE_LENGTH + "' must be the signature"); - } - return readBytes((int) count); - } - - /** Reads the fixed length signature. */ - byte[] readSignature() throws OwidException { - return readBytes(Owid.SIGNATURE_LENGTH); - } - - /** Reads the date using the encoding associated with the version. */ - Instant readDate(Version version) throws OwidException { - switch (version) { - case VERSION1: { - int high = readByte(); - int low = readByte(); - long hours = ((long) high << 8) | low; - return baseDate().plus(Duration.ofHours(hours)); - } - case VERSION2: - case VERSION3: { - long minutes = readUInt32(); - return baseDate().plus(Duration.ofMinutes(minutes)); - } - default: - throw new OwidException("OWID version '" - + (version.asByte() & 0xFF) + "' not supported"); - } - } - - private static OwidException endOfBuffer() { - return new OwidException("buffer ended before the OWID was complete"); - } - } - static void writeByte(ByteArrayOutputStream buffer, byte value) { buffer.write(value); } diff --git a/src/main/java/com/swancommunity/owid/Owid.java b/src/main/java/com/swancommunity/owid/Owid.java index b9cd6ba..5ac1bfb 100644 --- a/src/main/java/com/swancommunity/owid/Owid.java +++ b/src/main/java/com/swancommunity/owid/Owid.java @@ -20,7 +20,6 @@ import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; -import java.time.temporal.ChronoUnit; import java.util.Arrays; import java.util.Base64; import java.util.List; @@ -30,8 +29,22 @@ * *

An OWID records that the processor operating the domain handled the * payload, and any other OWIDs covered by the signature, at the date and time - * given. Once signed it is immutable. Any change to the fields will cause - * verification to fail.

+ * given.

+ * + *

An OWID is only worth anything because it is signed, so a caller cannot + * build one. An instance reaches calling code by one of two routes, being + * {@link #tryParse(String)} or {@link #tryParseBytes(byte[])} reading bytes + * that were already a complete OWID, or {@link Creator#createBytes(byte[])} + * and its companions signing one into existence. There is deliberately no way + * to assemble a half made one, because an unsigned OWID is indistinguishable + * from a signed one to the code downstream of it and the difference only + * surfaces later, when a verification fails somewhere nobody is watching.

+ * + *

The state is read only for the same reason. The signature covers the + * fields as they arrived, so a caller changing one afterwards would hold + * something the signature no longer describes. The payload and signature are + * handed out as copies, because a Java byte array is mutable and a caller + * writing into one it was given would otherwise reach inside the OWID.

* *

The serialized form places the fields in this order. Multi byte integers * are little endian, except the version 1 date which is big endian.

@@ -55,86 +68,74 @@ public final class Owid { */ public static final int SIGNATURE_LENGTH = 64; - private Version version; - private String domain; - private Instant date; - private byte[] payload; - private byte[] signature; - - /** - * Creates an empty unsigned OWID with the current version, an empty - * domain, the current date truncated to the minute, an empty payload, and - * no signature. - */ - public Owid() { - this.version = Version.current(); - this.domain = ""; - this.date = Instant.now().truncatedTo(ChronoUnit.MINUTES); - this.payload = new byte[0]; - this.signature = new byte[0]; - } + private final Version version; + private final String domain; + private final Instant date; + private final byte[] payload; + private final byte[] signature; /** - * Creates a new unsigned OWID with the domain, date, and payload provided - * and the current version. + * Builds an instance from fields a reader or the creator has already + * settled. * - * @param domain the domain associated with the creator - * @param date the creation date, used to the minute - * @param payload the payload bytes + *

Package private, so only this library can call it. That is the whole + * construction boundary, because a consumer compiled against the library + * cannot name this constructor at all, so there is no way to obtain an + * OWID that has not either been read from a complete serialized one or + * been signed by a {@link Creator}. The arrays are taken as given because + * every caller inside the library hands over an array nothing else + * holds.

*/ - public Owid(String domain, Instant date, byte[] payload) { - this.version = Version.current(); + Owid(Version version, String domain, Instant date, byte[] payload, + byte[] signature) { + this.version = version; this.domain = domain; this.date = date; - this.payload = payload.clone(); - this.signature = new byte[0]; + this.payload = payload; + this.signature = signature; } /** - * Creates an OWID from a base 64 encoded string. Decoding accepts the - * standard alphabet with or without the trailing padding. + * Reads a complete OWID from its base 64 form. + * + *

The value may be anything at all, because this is external data and + * failing to be an OWID is an ordinary outcome rather than an error. The + * result reports whether it worked, the OWID only when it did, and a + * named reason either way. Decoding accepts the standard alphabet with or + * without the trailing padding, and ignores line breaks and spaces.

+ * + *

A successful read says the bytes are a structurally valid OWID. It + * says nothing about whether the signature is genuine, which is a + * separate question answered by {@link #verifyDetailed(Crypto, List)}.

* - * @param value the base 64 encoded OWID - * @return the parsed OWID - * @throws OwidException if the string is not valid base 64, or the bytes - * are not a valid OWID + * @param value the base 64 encoded OWID, which may be null + * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and + * the reason the string is not an OWID */ - public static Owid fromBase64(String value) throws OwidException { - return fromByteArray(decodeBase64(value)); + public static OwidParseResult tryParse(String value) { + if (value == null || value.isEmpty()) { + return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); + } + byte[] buffer = OwidReader.decodeBase64(value); + if (buffer == null) { + return OwidParseResult.failed(OwidParseStatus.INVALID_BASE64); + } + return OwidReader.read(buffer); } /** - * Creates an OWID from its binary form. + * Reads a complete OWID from a buffer holding exactly one. * - * @param buffer the serialized OWID bytes - * @return the parsed OWID - * @throws OwidException if the first byte is not a known version, the - * buffer is too short for the remaining fields, the - * domain is unterminated or longer than the maximum - * published for a domain name, or the declared - * payload length does not leave exactly the 64 byte - * signature at the end + *

The buffer must be one whole OWID and nothing else. Bytes after the + * envelope are refused, because this library has no framed reader and so + * there is nothing else they could belong to.

+ * + * @param buffer the serialized OWID bytes, which may be null + * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and + * the reason the bytes are not an OWID */ - public static Owid fromByteArray(byte[] buffer) throws OwidException { - return fromReader(new Io.Reader(buffer)); - } - - /** Creates an OWID by reading the next fields from the reader. */ - static Owid fromReader(Io.Reader reader) throws OwidException { - Version version = Version.fromByte(reader.readByte()); - Owid owid = new Owid(); - owid.version = version; - if (version == Version.EMPTY) { - owid.domain = ""; - owid.payload = new byte[0]; - owid.signature = new byte[0]; - return owid; - } - owid.domain = reader.readString(); - owid.date = reader.readDate(version); - owid.payload = reader.readByteArray(); - owid.signature = reader.readSignature(); - return owid; + public static OwidParseResult tryParseBytes(byte[] buffer) { + return OwidReader.read(buffer); } /** @@ -163,7 +164,7 @@ public String asBase64() throws OwidException { /** Appends the OWID, including the signature, to the buffer provided. */ void toBuffer(ByteArrayOutputStream buffer) throws OwidException { - toBufferNoSignature(buffer); + writeNoSignature(buffer, version, domain, date, payload); Io.writeSignature(buffer, signature); } @@ -181,7 +182,9 @@ public static byte[] emptyByteArray() { * Appends the fields other than the signature to the buffer. This is the * data over which the signature is calculated. */ - void toBufferNoSignature(ByteArrayOutputStream buffer) throws OwidException { + private static void writeNoSignature(ByteArrayOutputStream buffer, + Version version, String domain, Instant date, byte[] payload) + throws OwidException { Io.writeByte(buffer, version.asByte()); Io.writeString(buffer, domain); Io.writeDate(buffer, date, version); @@ -194,23 +197,43 @@ void toBufferNoSignature(ByteArrayOutputStream buffer) throws OwidException { * form of each of the others in the order provided. */ byte[] dataForCrypto(List others) throws OwidException { - int length = byteCount(false); + return dataForCrypto(version, domain, date, payload, others); + } + + /** + * The same bytes for fields that are not yet an OWID, which is what the + * creator holds at the moment it signs. Nothing partly built exists, + * because the creator keeps loose fields until it has a signature and + * then builds the finished OWID in one step. + */ + static byte[] dataForCrypto(Version version, String domain, Instant date, + byte[] payload, List others) throws OwidException { + int length = byteCount(version, domain, payload, null, false); for (Owid other : others) { length = addLength(length, other.byteCount(true)); } ExactByteArrayOutputStream buffer = new ExactByteArrayOutputStream(length); - toBufferNoSignature(buffer); + writeNoSignature(buffer, version, domain, date, payload); for (Owid other : others) { other.toBuffer(buffer); } return buffer.toExactByteArray(); } + /** The exact number of bytes serialization will write. */ + private int byteCount(boolean includeSignature) throws OwidException { + return byteCount(version, domain, payload, signature, + includeSignature); + } + /** - * The exact number of bytes serialization will write. + * The exact number of bytes serialization will write for the fields + * given. The signature may be null when it is not being counted. */ - private int byteCount(boolean includeSignature) throws OwidException { + private static int byteCount(Version version, String domain, + byte[] payload, byte[] signature, boolean includeSignature) + throws OwidException { int dateLength; switch (version) { case VERSION1: @@ -246,7 +269,7 @@ private int byteCount(boolean includeSignature) throws OwidException { */ private static int addLength(int left, int right) throws OwidException { if (right < 0 || left > Integer.MAX_VALUE - right) { - throw new OwidException( + throw new CapacityException( "OWID byte length exceeds Java array capacity"); } return left + right; @@ -351,39 +374,97 @@ public boolean verifyWithPublicKey(String publicPem, List others) } /** - * Returns the byte version of the OWID. + * Asks whether the signature is genuine and reports why, keeping "does + * not match" apart from "could not check". * - * @return the version + *

A key that cannot be used leaves the signature unjudged, and saying + * that it is invalid would report an outage as an attack, so the two + * arrive as different statuses.

+ * + * @param crypto the crypto instance holding the public key, which may be + * null when no key could be obtained + * @param others the other OWIDs that were signed together with this one, + * in the same order as when signed + * @return the outcome of the check */ - public Version getVersion() { - return version; + public OwidVerificationResult verifyDetailed(Crypto crypto, + List others) { + if (crypto == null || crypto.canVerify() == false) { + return OwidVerificationResult.of( + OwidSignatureStatus.KEY_UNAVAILABLE); + } + if (signature.length != SIGNATURE_LENGTH) { + return OwidVerificationResult.of( + OwidSignatureStatus.INVALID_SIGNATURE_LENGTH); + } + byte[] data; + try { + data = dataForCrypto(others); + } catch (CapacityException e) { + return OwidVerificationResult.of( + OwidSignatureStatus.IMPLEMENTATION_CAPACITY_EXCEEDED); + } catch (OwidException e) { + return OwidVerificationResult.of( + OwidSignatureStatus.VERIFICATION_ERROR); + } + boolean valid; + try { + valid = crypto.verifyByteArray(data, signature); + } catch (OwidException e) { + return OwidVerificationResult.of( + OwidSignatureStatus.VERIFICATION_ERROR); + } + return OwidVerificationResult.of(valid + ? OwidSignatureStatus.SIGNATURE_VALID + : OwidSignatureStatus.SIGNATURE_INVALID); } /** - * Sets the byte version of the OWID. + * The same question as {@link #verifyDetailed(Crypto, List)}, starting + * from the public key in SPKI PEM form. + * + *

Key material that cannot be decoded reports + * {@link OwidSignatureStatus#INVALID_KEY}, because the fault is in the + * key rather than in the identifier.

* - * @param version the version + * @param publicPem the public key in SPKI PEM form, which may be null + * when no key could be obtained + * @param others the other OWIDs that were signed together with this + * one, in the same order as when signed + * @return the outcome of the check */ - public void setVersion(Version version) { - this.version = version; + public OwidVerificationResult verifyDetailedWithPublicKey(String publicPem, + List others) { + if (publicPem == null || publicPem.trim().isEmpty()) { + return OwidVerificationResult.of( + OwidSignatureStatus.KEY_UNAVAILABLE); + } + Crypto crypto; + try { + crypto = Crypto.newVerifyOnly(publicPem); + } catch (OwidException e) { + return OwidVerificationResult.of( + OwidSignatureStatus.INVALID_KEY); + } + return verifyDetailed(crypto, others); } /** - * Returns the domain associated with the creator. + * Returns the byte version of the OWID. * - * @return the domain + * @return the version */ - public String getDomain() { - return domain; + public Version getVersion() { + return version; } /** - * Sets the domain associated with the creator. + * Returns the domain associated with the creator. * - * @param domain the domain + * @return the domain */ - public void setDomain(String domain) { - this.domain = domain; + public String getDomain() { + return domain; } /** @@ -396,17 +477,9 @@ public Instant getDate() { } /** - * Sets the creation date and time. The serialized form uses the minute - * truncated value. - * - * @param date the date - */ - public void setDate(Instant date) { - this.date = date; - } - - /** - * Returns a copy of the payload bytes. + * Returns a copy of the payload bytes, so that writing into the array + * returned cannot alter an OWID whose signature was calculated over the + * original bytes. * * @return the payload */ @@ -425,16 +498,8 @@ public int getPayloadLength() { } /** - * Sets the payload bytes. - * - * @param payload the payload - */ - public void setPayload(byte[] payload) { - this.payload = payload.clone(); - } - - /** - * Returns a copy of the signature bytes. + * Returns a copy of the signature bytes, for the same reason as + * {@link #getPayload()}. * * @return the signature */ @@ -442,15 +507,6 @@ public byte[] getSignature() { return signature.clone(); } - /** - * Sets the signature bytes. - * - * @param signature the signature - */ - void setSignature(byte[] signature) { - this.signature = signature.clone(); - } - /** * Formats the OWID as a base 64 string, or the text of the error if it * cannot be encoded. @@ -491,14 +547,4 @@ public int hashCode() { result = 31 * result + Arrays.hashCode(signature); return result; } - - /** Decodes base 64 accepting input with or without trailing padding. */ - private static byte[] decodeBase64(String value) throws OwidException { - try { - return Base64.getMimeDecoder().decode(value); - } catch (IllegalArgumentException e) { - throw new OwidException("base 64 decoding failed because " - + e.getMessage(), e); - } - } } diff --git a/src/main/java/com/swancommunity/owid/OwidException.java b/src/main/java/com/swancommunity/owid/OwidException.java index 4177407..107c456 100644 --- a/src/main/java/com/swancommunity/owid/OwidException.java +++ b/src/main/java/com/swancommunity/owid/OwidException.java @@ -17,9 +17,16 @@ package com.swancommunity.owid; /** - * Checked exception thrown when creating, reading, signing, or verifying + * Checked exception raised when creating, serializing, signing or verifying * OWIDs fails. The message describes the cause and, where relevant, the * underlying exception is set as the cause. + * + *

Reading a serialized OWID does not raise this. Data that arrived from + * outside is expected to be malformed sometimes, so + * {@link Owid#tryParse(String)} and {@link Owid#tryParseBytes(byte[])} report + * an {@link OwidParseStatus} instead. What remains here is the caller's own + * mistakes, such as an invalid creator domain or a field that cannot be + * serialized, and failures of the cryptography.

*/ public class OwidException extends Exception { diff --git a/src/main/java/com/swancommunity/owid/OwidParseResult.java b/src/main/java/com/swancommunity/owid/OwidParseResult.java new file mode 100644 index 0000000..dd9174a --- /dev/null +++ b/src/main/java/com/swancommunity/owid/OwidParseResult.java @@ -0,0 +1,100 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +/** + * What a read of a serialized OWID produced, and why. + * + *

Every read reports the same three facts, so a caller never has to infer + * one of them from another. Whether it worked is + * {@link #isSuccess()}, the OWID is {@link #getValue()} and is present only + * on success, and the reason is {@link #getStatus()} either way.

+ * + *

The three move together. When {@link #isSuccess()} is true the value is + * not null and the status is {@link OwidParseStatus#PARSED}, and when it is + * false the value is null and the status names one of the expected + * problems.

+ * + *

A result carries no text taken from the input. The bytes came from + * outside, so putting them in a message would mean logging whatever an + * untrusted sender chose to send.

+ */ +public final class OwidParseResult { + + private final Owid value; + + private final OwidParseStatus status; + + private OwidParseResult(Owid value, OwidParseStatus status) { + this.value = value; + this.status = status; + } + + /** The result of a read that produced the OWID given. */ + static OwidParseResult parsed(Owid value) { + return new OwidParseResult(value, OwidParseStatus.PARSED); + } + + /** The result of a read that failed for the reason given. */ + static OwidParseResult failed(OwidParseStatus status) { + return new OwidParseResult(null, status); + } + + /** + * Whether the bytes were a complete, structurally valid OWID. This says + * nothing about whether the signature is genuine, which is a separate + * question answered by + * {@link Owid#verifyDetailed(Crypto, java.util.List)}. + * + * @return true when the read produced an OWID + */ + public boolean isSuccess() { + return status == OwidParseStatus.PARSED; + } + + /** + * The OWID that was read, or null when the read failed. Callers should + * test {@link #isSuccess()} first rather than testing this for null, + * because the status also says which of the expected problems it was. + * + * @return the OWID on success, otherwise null + */ + public Owid getValue() { + return value; + } + + /** + * Why the read succeeded or failed. + * + * @return {@link OwidParseStatus#PARSED} on success, otherwise the + * specific reason + */ + public OwidParseStatus getStatus() { + return status; + } + + /** + * The status name on its own. The input is deliberately absent, because + * a parse failure is often logged and the bytes came from outside. + * + * @return the status name + */ + @Override + public String toString() { + return status.name(); + } +} diff --git a/src/main/java/com/swancommunity/owid/OwidParseStatus.java b/src/main/java/com/swancommunity/owid/OwidParseStatus.java new file mode 100644 index 0000000..833baee --- /dev/null +++ b/src/main/java/com/swancommunity/owid/OwidParseStatus.java @@ -0,0 +1,96 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +/** + * Why a read of external data succeeded or failed. + * + *

Malformed data arriving from outside is expected rather than + * exceptional. An OWID is read from whatever a caller was handed, which on a + * public end point means anything at all, so every one of these outcomes is + * an ordinary result and not a fault. Reporting them by throwing costs the + * construction and unwinding of an exception for every bad input, and + * whoever is sending the data chooses how often that happens.

+ * + *

These names are the cross language vocabulary. Each implementation + * spells the surface in its own idiom, so the Java members are the Java + * naming convention for the same set of facts, and a failure means the same + * thing whichever language read the bytes.

+ */ +public enum OwidParseStatus { + + /** + * The bytes form a structurally valid OWID. This says nothing about the + * signature, which is a separate question with its own answer. + */ + PARSED, + + /** Nothing was supplied to read, being a null or empty value. */ + MISSING_INPUT, + + /** + * The input was supplied in a form this surface cannot read. Kept for + * the cross language vocabulary and not reachable in Java, where the + * compiler already refuses anything that is not a string or a byte + * array. + */ + INVALID_INPUT_TYPE, + + /** The string is not valid base 64, so there are no bytes to read. */ + INVALID_BASE64, + + /** The first byte names a version this implementation does not know. */ + UNSUPPORTED_VERSION, + + /** + * The data stopped in the middle of a field. Different from + * {@link #BYTE_COUNT_MISMATCH}, which is a declaration disagreeing with + * data that is all present. + */ + UNEXPECTED_END, + + /** + * The creator domain is not terminated, or is longer than the maximum + * published for a domain name. + */ + INVALID_DOMAIN_ENCODING, + + /** + * The declared payload byte count disagrees with the bytes actually + * present. Checked before anything is sized by the declaration, so a + * sender cannot make a reader allocate by claiming a large payload it + * did not send. + */ + BYTE_COUNT_MISMATCH, + + /** + * The envelope is structurally consistent but larger than this runtime + * can hold. Deliberately apart from the data being wrong, because the + * same bytes may be readable elsewhere. A Java byte array cannot hold + * more than {@link Integer#MAX_VALUE} bytes, so a declaration larger + * than that can never agree with the bytes present and this status is + * not reachable from the byte array surface. + */ + IMPLEMENTATION_CAPACITY_EXCEEDED, + + /** + * The envelope is malformed in a way none of the others describes. A + * fallback for the genuinely unclassified, not a substitute for naming a + * failure that is already understood. + */ + MALFORMED_ENVELOPE +} diff --git a/src/main/java/com/swancommunity/owid/OwidReader.java b/src/main/java/com/swancommunity/owid/OwidReader.java new file mode 100644 index 0000000..c36588a --- /dev/null +++ b/src/main/java/com/swancommunity/owid/OwidReader.java @@ -0,0 +1,294 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import java.nio.charset.StandardCharsets; +import java.time.Duration; +import java.time.Instant; + +/** + * Reads a complete OWID from a buffer, reporting why rather than throwing + * when the bytes are not one. + * + *

The buffer is walked by index and every read is checked against what is + * left, so a malformed envelope is a comparison that fails rather than an + * exception that unwinds. That matters because the data comes from outside, + * where whoever is sending it chooses how often the read fails and how large + * each attempt is, and an exception for every attempt is a cost they + * choose.

+ * + *

Nothing here calls the throwing code and catches it. The exception + * would still be built and unwound, so the cost would remain and only the + * surface would look different.

+ * + *

This is the exact buffer contract, meaning the envelope has to end + * where the buffer does. The library has no framed reader, so there is + * nothing that later bytes could belong to.

+ */ +final class OwidReader { + + private OwidReader() { + } + + /** Reads one complete OWID occupying the whole of the buffer. */ + static OwidParseResult read(byte[] buffer) { + if (buffer == null || buffer.length == 0) { + return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); + } + + int total = buffer.length; + Version version = Version.forByte(buffer[0] & 0xFF); + if (version == null) { + return OwidParseResult.failed( + OwidParseStatus.UNSUPPORTED_VERSION); + } + int at = 1; + + if (version == Version.EMPTY) { + // The marker for an absent optional OWID is the version byte and + // nothing else, so anything after it belongs to no field. + if (at != total) { + return OwidParseResult.failed( + OwidParseStatus.MALFORMED_ENVELOPE); + } + return OwidParseResult.parsed(new Owid( + version, "", Io.baseDate(), new byte[0], new byte[0])); + } + + // The domain, terminated by a zero byte and no longer than the + // maximum published for a domain name. The walk stops at that + // maximum rather than at the end of the buffer, so a buffer whose + // terminator is missing costs no more than the maximum however long + // that buffer is. + int start = at; + int limit = Math.min(total, start + Io.MAXIMUM_DOMAIN_LENGTH + 1); + String domain = null; + while (at < limit) { + if (buffer[at] == 0) { + domain = new String(buffer, start, at - start, + StandardCharsets.UTF_8); + at++; + break; + } + at++; + } + if (domain == null) { + // Either the buffer ended inside the domain, or the domain ran + // past the maximum without terminating. The second is a domain + // that cannot be valid rather than data that merely stopped, so + // the two are reported differently. + if (at >= total && at - start <= Io.MAXIMUM_DOMAIN_LENGTH) { + return OwidParseResult.failed( + OwidParseStatus.UNEXPECTED_END); + } + return OwidParseResult.failed( + OwidParseStatus.INVALID_DOMAIN_ENCODING); + } + + // The date, whose width depends on the version. + Instant date; + if (version == Version.VERSION1) { + if (total - at < 2) { + return OwidParseResult.failed( + OwidParseStatus.UNEXPECTED_END); + } + long hours = ((long) (buffer[at] & 0xFF) << 8) + | (buffer[at + 1] & 0xFF); + at += 2; + date = Io.baseDate().plus(Duration.ofHours(hours)); + } else { + if (total - at < 4) { + return OwidParseResult.failed( + OwidParseStatus.UNEXPECTED_END); + } + long minutes = readUInt32(buffer, at); + at += 4; + date = Io.baseDate().plus(Duration.ofMinutes(minutes)); + } + + if (total - at < 4) { + return OwidParseResult.failed(OwidParseStatus.UNEXPECTED_END); + } + long declared = readUInt32(buffer, at); + at += 4; + + // The declaration is the sender's claim about a payload not yet + // read, so it is compared with what is actually present before + // anything is sized by it. The subtraction is done in a long, and + // the declaration is read as unsigned into a long, so a buffer with + // fewer bytes left than a signature needs gives a negative count + // rather than wrapping, and a negative count can never equal a + // declaration. + // + // The disagreement is the finding even when the buffer also stopped + // early. What a reader can say for certain is that the declared + // payload cannot leave exactly the signature the version requires, + // and that is true whichever way the bytes fall short. Reporting it + // as a truncation instead would name a different fault for the same + // evidence. + long present = (long) (total - at) - Owid.SIGNATURE_LENGTH; + if (present != declared) { + return OwidParseResult.failed( + OwidParseStatus.BYTE_COUNT_MISMATCH); + } + + // The bytes are all here. Whether this runtime can hold them in one + // array is a separate question with a different answer, because the + // same envelope may be readable elsewhere. A Java byte array cannot + // exceed Integer.MAX_VALUE, so the count above can never agree with + // a larger declaration and this cannot fire today. It is kept so a + // future change to that arithmetic cannot silently truncate the cast + // below. + if (declared > Integer.MAX_VALUE) { + return OwidParseResult.failed( + OwidParseStatus.IMPLEMENTATION_CAPACITY_EXCEEDED); + } + + int payloadLength = (int) declared; + byte[] payload = new byte[payloadLength]; + System.arraycopy(buffer, at, payload, 0, payloadLength); + at += payloadLength; + + byte[] signature = new byte[Owid.SIGNATURE_LENGTH]; + System.arraycopy(buffer, at, signature, 0, Owid.SIGNATURE_LENGTH); + at += Owid.SIGNATURE_LENGTH; + + if (at != total) { + // Unreachable while the count check above holds, and kept so + // that a future change to that arithmetic cannot silently start + // accepting bytes after the signature. + return OwidParseResult.failed( + OwidParseStatus.MALFORMED_ENVELOPE); + } + + return OwidParseResult.parsed( + new Owid(version, domain, date, payload, signature)); + } + + /** + * Decodes base 64 without throwing, returning null when the string is not + * base 64. + * + *

Written out here rather than handed to {@code java.util.Base64} + * because neither JDK decoder answers the question this surface has to + * ask. The strict decoder throws, which is the cost this change exists to + * remove, and the MIME decoder silently drops every character outside the + * alphabet, so a string of nothing but rubbish would come back as an + * empty array and be reported as a missing OWID rather than as text that + * is not base 64 at all.

+ * + *

The standard alphabet is accepted with or without the trailing + * padding, because both are ordinary ways to carry an encoded OWID. + * Spaces, tabs and line breaks are skipped, since wrapped encodings are + * common and were accepted before. Anything else is refused.

+ */ + static byte[] decodeBase64(String value) { + int length = value.length(); + int significant = 0; + int padding = 0; + for (int i = 0; i < length; i++) { + char c = value.charAt(i); + if (isSkipped(c)) { + continue; + } + if (c == '=') { + padding++; + continue; + } + if (padding > 0 || c > 127 || DECODE[c] < 0) { + // Either a character outside the alphabet, or data after the + // padding that closes the last block. + return null; + } + significant++; + } + + // Padding only ever brings the final block up to four characters, so + // any other amount of it means the string was not produced by an + // encoder. + int remainder = significant % 4; + if (remainder == 1) { + return null; + } + if (padding > 0 && padding != (4 - remainder) % 4) { + return null; + } + + byte[] decoded = new byte[significant * 3 / 4]; + int bits = 0; + int held = 0; + int at = 0; + for (int i = 0; i < length; i++) { + char c = value.charAt(i); + if (c == '=' || isSkipped(c)) { + continue; + } + bits = (bits << 6) | DECODE[c]; + held++; + if (held == 4) { + decoded[at++] = (byte) (bits >> 16); + decoded[at++] = (byte) (bits >> 8); + decoded[at++] = (byte) bits; + bits = 0; + held = 0; + } + } + if (held == 2) { + decoded[at] = (byte) (bits >> 4); + } else if (held == 3) { + decoded[at++] = (byte) (bits >> 10); + decoded[at] = (byte) (bits >> 2); + } + return decoded; + } + + /** Layout whitespace, which an encoder may have used to wrap lines. */ + private static boolean isSkipped(char c) { + return c == ' ' || c == '\t' || c == '\r' || c == '\n'; + } + + /** + * The value of each standard alphabet character, and -1 for every other + * character below 128. + */ + private static final int[] DECODE = buildDecodeTable(); + + private static int[] buildDecodeTable() { + int[] table = new int[128]; + for (int i = 0; i < table.length; i++) { + table[i] = -1; + } + String alphabet = "ABCDEFGHIJKLMNOPQRSTUVWXYZ" + + "abcdefghijklmnopqrstuvwxyz0123456789+/"; + for (int i = 0; i < alphabet.length(); i++) { + table[alphabet.charAt(i)] = i; + } + return table; + } + + /** + * The four bytes at the offset as a little endian unsigned 32 bit value + * widened into a long, so the full wire range is compared without a + * signed int wrapping into a negative number. + */ + private static long readUInt32(byte[] buffer, int offset) { + return ((long) (buffer[offset] & 0xFF)) + | ((long) (buffer[offset + 1] & 0xFF) << 8) + | ((long) (buffer[offset + 2] & 0xFF) << 16) + | ((long) (buffer[offset + 3] & 0xFF) << 24); + } +} diff --git a/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java b/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java new file mode 100644 index 0000000..f027025 --- /dev/null +++ b/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java @@ -0,0 +1,82 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +/** + * The outcome of asking whether an OWID signature is genuine. + * + *

Only two of these say anything about the signature itself. The rest say + * the question could not be answered, which is a different thing and must + * never be reported as a forgery. A key that cannot be obtained, a key that + * cannot be decoded, or a provider that fails leaves the signature unjudged, + * and a caller acting on "invalid" would reject good identifiers during an + * outage.

+ * + *

That is not hypothetical. On 30 August 2026 the key end points served + * PEM a strict parser rejects and every offline verification against them + * failed while the keys and the identifiers were both fine. Reported as + * {@link #INVALID_KEY} that reads as the operational fault it was, whereas + * reported as {@link #SIGNATURE_INVALID} it would have read as an + * attack.

+ */ +public enum OwidSignatureStatus { + + /** The signature is genuine for this data and this key. */ + SIGNATURE_VALID, + + /** + * The signature is well formed and does not match, so the data does not + * belong to the key it claims. This is the only status that means the + * identifier should be distrusted. + */ + SIGNATURE_INVALID, + + /** + * A signature field of the wrong length reached a verification surface + * directly. Truncation in raw external input is a parse + * {@link OwidParseStatus#UNEXPECTED_END} or + * {@link OwidParseStatus#BYTE_COUNT_MISMATCH} instead, because there the + * envelope never formed. + */ + INVALID_SIGNATURE_LENGTH, + + /** + * No key was supplied, or the one supplied cannot verify. The signature + * was never examined. + */ + KEY_UNAVAILABLE, + + /** + * Key material arrived but cannot be decoded, imported, or used as the + * required type. The fault is in the key and not in the identifier. + */ + INVALID_KEY, + + /** + * The work required is more than this runtime can hold. Reaching it + * needs an OWID and its chain to approach the two gigabyte limit of a + * Java array. + */ + IMPLEMENTATION_CAPACITY_EXCEEDED, + + /** + * The check could not be completed for a reason that is not the + * identifier's fault, such as a cryptographic provider failing on valid + * inputs or a field that cannot be encoded for signing. + */ + VERIFICATION_ERROR +} diff --git a/src/main/java/com/swancommunity/owid/OwidVerificationResult.java b/src/main/java/com/swancommunity/owid/OwidVerificationResult.java new file mode 100644 index 0000000..71c68ae --- /dev/null +++ b/src/main/java/com/swancommunity/owid/OwidVerificationResult.java @@ -0,0 +1,62 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +/** + * What asking whether an OWID signature is genuine produced. + * + *

{@link #isValid()} is true only when the signature was examined and + * matched. Everything else is false, but the reasons are not + * interchangeable, because "does not match" and "could not check" call for + * different handling and {@link #getStatus()} keeps them apart.

+ */ +public final class OwidVerificationResult { + + private final OwidSignatureStatus status; + + private OwidVerificationResult(OwidSignatureStatus status) { + this.status = status; + } + + /** The result carrying the status given. */ + static OwidVerificationResult of(OwidSignatureStatus status) { + return new OwidVerificationResult(status); + } + + /** + * Whether the signature was examined and found genuine. + * + * @return true only for {@link OwidSignatureStatus#SIGNATURE_VALID} + */ + public boolean isValid() { + return status == OwidSignatureStatus.SIGNATURE_VALID; + } + + /** + * The outcome of the check. + * + * @return the signature status + */ + public OwidSignatureStatus getStatus() { + return status; + } + + @Override + public String toString() { + return status.name(); + } +} diff --git a/src/main/java/com/swancommunity/owid/Version.java b/src/main/java/com/swancommunity/owid/Version.java index 35b368f..aa5e01c 100644 --- a/src/main/java/com/swancommunity/owid/Version.java +++ b/src/main/java/com/swancommunity/owid/Version.java @@ -70,19 +70,19 @@ public static Version current() { } /** - * Maps a byte to the matching version. - * - * @param value the version byte - * @return the matching version - * @throws OwidException if the byte is not a known version + * Maps a byte to the matching version, or null when the byte is not a + * known version. Reading a version the implementation does not know is + * an ordinary outcome for data that arrived from outside, so it is + * answered rather than thrown, and the caller reports it as + * {@link OwidParseStatus#UNSUPPORTED_VERSION}. */ - public static Version fromByte(int value) throws OwidException { + static Version forByte(int value) { int unsigned = value & 0xFF; for (Version version : values()) { if (version.value == unsigned) { return version; } } - throw new OwidException("OWID version '" + unsigned + "' not supported"); + return null; } } diff --git a/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java b/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java new file mode 100644 index 0000000..c17af3b --- /dev/null +++ b/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java @@ -0,0 +1,176 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.example.owidconsumer; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; +import static org.junit.jupiter.api.Assertions.assertTrue; +import static org.junit.jupiter.api.Assertions.fail; + +import com.swancommunity.owid.Creator; +import com.swancommunity.owid.Crypto; +import com.swancommunity.owid.Owid; +import com.swancommunity.owid.OwidException; +import com.swancommunity.owid.OwidParseResult; +import com.swancommunity.owid.OwidParseStatus; +import java.lang.reflect.Constructor; +import java.lang.reflect.Field; +import java.lang.reflect.Method; +import java.lang.reflect.Modifier; +import java.nio.charset.StandardCharsets; +import java.util.Collections; +import org.junit.jupiter.api.Test; + +/** + * What a library user outside the OWID package can and cannot do. + * + *

This test lives in another package on purpose. The tests that sit beside + * the library share its package, so the compiler lets them reach things a + * consumer cannot, and a construction boundary asserted from inside would + * measure nothing. Everything here goes through the same public surface a + * consumer compiles against.

+ */ +class ConstructionBoundaryTest { + + /** + * There is no public constructor, so no caller can name one. This is the + * compiler's own rule rather than a check made at run time, and the + * reflective attempt below shows the runtime refuses it as well. + */ + @Test + void owidHasNoPublicConstructor() { + assertEquals(0, Owid.class.getConstructors().length, + "an OWID should not be constructible by a caller"); + for (Constructor constructor + : Owid.class.getDeclaredConstructors()) { + int modifiers = constructor.getModifiers(); + assertTrue(Modifier.isPublic(modifiers) == false + && Modifier.isProtected(modifiers) == false, + "every OWID constructor should be package private"); + } + } + + /** + * The runtime refuses the constructor as well, so the boundary is not + * only a compile time one. Reflection with setAccessible could still + * reach it, which is true of every package private member in Java and is + * the honest limit of the mechanism. + */ + @Test + void reflectiveConstructionIsRefused() { + for (Constructor constructor + : Owid.class.getDeclaredConstructors()) { + Object[] arguments = new Object[constructor.getParameterCount()]; + assertThrows(IllegalAccessException.class, + () -> constructor.newInstance(arguments), + "the runtime should refuse a package private constructor"); + } + } + + /** No field can be set or rebound from outside. */ + @Test + void owidHasNoPublicMutation() { + for (Method method : Owid.class.getMethods()) { + assertTrue(method.getName().startsWith("set") == false, + "an OWID should have no setter, but has " + + method.getName()); + } + for (Field field : Owid.class.getDeclaredFields()) { + if (Modifier.isStatic(field.getModifiers())) { + continue; + } + assertTrue(Modifier.isPrivate(field.getModifiers()), + "every OWID field should be private, but " + + field.getName() + " is not"); + assertTrue(Modifier.isFinal(field.getModifiers()), + "every OWID field should be final, but " + + field.getName() + " is not"); + } + } + + /** + * There is no public way to sign an OWID either, because with no way to + * obtain an unsigned one there is nothing outside to sign, and signing a + * parsed one again would replace the signature its fields were read with. + */ + @Test + void creatorHasNoPublicSigningOfAnOwid() { + for (Method method : Creator.class.getMethods()) { + if (method.getName().startsWith("sign") == false) { + continue; + } + fail("a creator should not sign a caller's OWID, but exposes " + + method.getName()); + } + } + + /** + * Writing into a byte array a caller was handed does not alter the OWID, + * because a Java array is mutable and the OWID hands out copies. + */ + @Test + void writingIntoReturnedArraysDoesNotAlterTheOwid() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + Owid owid = creator.createBytes(new byte[] {1, 2, 3}); + byte[] encoded = owid.asByteArray(); + + byte[] payload = owid.getPayload(); + byte[] signature = owid.getSignature(); + payload[0] = 99; + signature[0] ^= 0xFF; + + assertArrayEquals(new byte[] {1, 2, 3}, owid.getPayload(), + "the payload should be unchanged"); + assertNotEquals(99, owid.getPayload()[0], + "writing into the copy should not reach the OWID"); + assertArrayEquals(encoded, owid.asByteArray(), + "the OWID should serialise to the same bytes"); + assertTrue(owid.verifyWithCrypto(crypto, Collections.emptyList()), + "the OWID should still verify"); + } + + /** + * A library user can still do everything the old surface allowed, by the + * new route. Creating, chaining, serialising, reading back and verifying + * all work without ever naming a constructor. + */ + @Test + void aLibraryUserCanStillDoEverything() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + + Owid root = creator.createString("root"); + Owid party = creator.createBytes( + "party".getBytes(StandardCharsets.UTF_8), + Collections.singletonList(root)); + + OwidParseResult result = Owid.tryParse(party.asBase64()); + assertEquals(OwidParseStatus.PARSED, result.getStatus(), + "the created OWID should read back"); + Owid copy = result.getValue(); + + assertEquals(party, copy, "should read back an equal OWID"); + assertTrue(copy.verifyWithPublicKey(crypto.publicKeyPem(), + Collections.singletonList(root)), + "should verify with the same others"); + assertEquals("party", copy.payloadAsString(), + "should carry the payload it was created with"); + } +} diff --git a/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java new file mode 100644 index 0000000..222fa98 --- /dev/null +++ b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java @@ -0,0 +1,97 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.example.owidconsumer; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.swancommunity.owid.Creator; +import com.swancommunity.owid.Crypto; +import com.swancommunity.owid.Owid; +import com.swancommunity.owid.OwidException; +import com.swancommunity.owid.OwidParseResult; +import java.util.Collections; +import org.junit.jupiter.api.Test; + +/** + * The examples printed in the README, compiled and run. + * + *

A documented example that nothing compiles goes stale without anyone + * noticing, and in the Go port exactly that had already happened. Keeping the + * example here means a change to the library that would break it breaks the + * build instead.

+ * + *

The bodies below are the README text, with the assertions of a test + * added around it. Change one and change the other.

+ */ +class ReadmeExampleTest { + + @Test + void createSerializeReadBackAndVerify() throws OwidException { + // The creator operates a domain and holds the signing keys. + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + + // Create a signed OWID with a payload. An OWID is signed from the + // moment it exists, so there is never an unsigned one to hold. + Owid owid = creator.createString("Hello World"); + + // Serialize to base 64 for storage or transmission. + String encoded = owid.asBase64(); + + // Later, or elsewhere, read it back. Reading answers rather than + // throwing, because whatever arrives from outside may not be an OWID + // at all. + OwidParseResult result = Owid.tryParse(encoded); + if (result.isSuccess()) { + Owid copy = result.getValue(); + String publicPem = crypto.publicKeyPem(); + boolean valid = copy.verifyWithPublicKey( + publicPem, Collections.emptyList()); + + assertTrue(valid, "the OWID read back should verify"); + assertEquals("Hello World", copy.payloadAsString(), + "should carry the payload it was created with"); + } else { + // result.getStatus() names which of the expected problems it was, + // and result.getValue() is null. + org.junit.jupiter.api.Assertions.fail( + "the example should read back, but reported " + + result.getStatus()); + } + } + + @Test + void chainingCoversTheOtherOwids() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + + Owid root = creator.createString("root"); + Owid party = creator.createString( + "party", Collections.singletonList(root)); + + // Verifies with the root as the single other, fails without it. + assertTrue( + party.verifyWithCrypto( + crypto, Collections.singletonList(root)), + "should verify with the same others"); + assertFalse( + party.verifyWithCrypto(crypto, Collections.emptyList()), + "should fail to verify without the others"); + } +} diff --git a/src/test/java/com/swancommunity/owid/CreatorTest.java b/src/test/java/com/swancommunity/owid/CreatorTest.java index bae69d6..f5c3dd5 100644 --- a/src/test/java/com/swancommunity/owid/CreatorTest.java +++ b/src/test/java/com/swancommunity/owid/CreatorTest.java @@ -16,6 +16,7 @@ package com.swancommunity.owid; +import static org.junit.jupiter.api.Assertions.assertArrayEquals; import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertThrows; @@ -48,7 +49,7 @@ void verifyOnlyCryptoRejected() throws OwidException { void signSetsDomainVersionAndVerifies() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create("example.com", crypto); - Owid owid = creator.signString("Hello World"); + Owid owid = creator.createString("Hello World"); assertEquals("example.com", owid.getDomain(), "should set the creator domain"); assertEquals(Version.VERSION3, owid.getVersion(), @@ -63,9 +64,9 @@ void signSetsDomainVersionAndVerifies() throws OwidException { void signAndSelfVerifyThroughPem() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create("example.com", crypto); - Owid owid = creator.signString("payload"); + Owid owid = creator.createString("payload"); String encoded = owid.asBase64(); - Owid copy = Owid.fromBase64(encoded); + Owid copy = ParseAssert.parsed(Owid.tryParse(encoded)); assertTrue(copy.verifyWithPublicKey(crypto.publicKeyPem(), Collections.emptyList()), "the decoded OWID should verify"); } @@ -74,34 +75,71 @@ void signAndSelfVerifyThroughPem() throws OwidException { void tamperedSignedOwidFails() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create("example.com", crypto); - Owid owid = creator.signBytes(new byte[] {1, 2, 3}); + Owid owid = creator.createBytes(new byte[] {1, 2, 3}); byte[] bytes = owid.asByteArray(); bytes[bytes.length - 1] ^= 0x01; - Owid tampered = Owid.fromByteArray(bytes); + Owid tampered = ParseAssert.parsed(Owid.tryParseBytes(bytes)); assertFalse(tampered.verifyWithCrypto(crypto, Collections.emptyList()), "a tampered signature should not verify"); } @Test - void signWithOthersRoundTrips() throws OwidException { + void createWithOthersRoundTrips() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create("example.com", crypto); - Owid root = creator.signString("root"); - Owid party = new Owid(); - party.setPayload("party".getBytes()); - creator.signWithOthers(party, Collections.singletonList(root)); - assertTrue(party.verifyWithCrypto(crypto, Collections.singletonList(root)), + Owid root = creator.createString("root"); + Owid party = creator.createString( + "party", Collections.singletonList(root)); + assertTrue( + party.verifyWithCrypto( + crypto, Collections.singletonList(root)), "should verify with the same others"); assertFalse(party.verifyWithCrypto(crypto, Collections.emptyList()), "should fail to verify without the others"); } + /** + * A creator refuses a null payload rather than producing an OWID with + * nothing in it. This is a caller mistake in code rather than data that + * arrived from outside, so it stays an exception. + */ + @Test + void nullPayloadRefused() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + assertThrows(OwidException.class, + () -> creator.createBytes((byte[]) null), + "should refuse a null payload"); + assertThrows(OwidException.class, + () -> creator.createString((String) null), + "should refuse a null payload string"); + } + + /** + * The payload the creator was handed is copied, so a caller writing into + * the array afterwards cannot change the OWID the signature covers. + */ + @Test + void payloadHandedToCreatorIsCopied() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + byte[] payload = {1, 2, 3}; + + Owid owid = creator.createBytes(payload); + payload[0] = 99; + + assertArrayEquals(new byte[] {1, 2, 3}, owid.getPayload(), + "the OWID should keep the bytes it was signed over"); + assertTrue(owid.verifyWithCrypto(crypto, Collections.emptyList()), + "the OWID should still verify"); + } + @Test void fromPrivatePemCreatesWorkingCreator() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.fromPrivatePem("example.com", crypto.privateKeyPem()); - Owid owid = creator.signString("data"); + Owid owid = creator.createString("data"); assertTrue(owid.verifyWithCrypto(crypto, Collections.emptyList()), "should sign with the imported key"); } diff --git a/src/test/java/com/swancommunity/owid/DomainLengthTest.java b/src/test/java/com/swancommunity/owid/DomainLengthTest.java index 5a8054a..6f2ef7d 100644 --- a/src/test/java/com/swancommunity/owid/DomainLengthTest.java +++ b/src/test/java/com/swancommunity/owid/DomainLengthTest.java @@ -140,7 +140,7 @@ void maximumLengthDomainParses() throws OwidException { String domain = domainOfLength(MAXIMUM); byte[] bytes = envelope(ascii(domain)); - Owid owid = Owid.fromByteArray(bytes); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(bytes)); assertEquals(MAXIMUM, owid.getDomain().length(), "should read a domain of the published maximum length"); @@ -149,7 +149,8 @@ void maximumLengthDomainParses() throws OwidException { "should read the payload that follows the domain"); assertArrayEquals(bytes, owid.asByteArray(), "should write the same bytes back out"); - assertEquals(owid, Owid.fromByteArray(owid.asByteArray()), + assertEquals(owid, + ParseAssert.parsed(Owid.tryParseBytes(owid.asByteArray())), "should parse its own output to an equal OWID"); } @@ -161,8 +162,8 @@ void maximumLengthDomainParses() throws OwidException { void overMaximumLengthDomainRefused() { byte[] bytes = envelope(ascii(domainOfLength(MAXIMUM + 1))); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a domain one character over the maximum"); + ParseAssert.failed(Owid.tryParseBytes(bytes), + OwidParseStatus.INVALID_DOMAIN_ENCODING); } /** @@ -175,8 +176,8 @@ void missingTerminatorRefused() { byte[] bytes = filled(64 * 1024, (byte) 'a'); bytes[0] = Version.VERSION3.asByte(); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a buffer with no terminator"); + ParseAssert.failed(Owid.tryParseBytes(bytes), + OwidParseStatus.INVALID_DOMAIN_ENCODING); } /** @@ -191,10 +192,9 @@ void hostileDomainRefusedWithoutAllocating() { byte[] bytes = envelope(filled(HOSTILE_DOMAIN_LENGTH, (byte) 'a')); long before = allocatedBytes(); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a domain field of " - + HOSTILE_DOMAIN_LENGTH + " bytes"); + OwidParseResult result = Owid.tryParseBytes(bytes); long allocated = allocatedBytes() - before; + ParseAssert.failed(result, OwidParseStatus.INVALID_DOMAIN_ENCODING); assertTrue(allocated < ALLOCATION_BOUND, "refusing a " + HOSTILE_DOMAIN_LENGTH + " byte domain field allocated " @@ -212,8 +212,9 @@ void maximumLengthDomainWritten() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create(domain, crypto); - Owid signed = creator.signBytes(PAYLOAD); - Owid parsed = Owid.fromByteArray(signed.asByteArray()); + Owid signed = creator.createBytes(PAYLOAD); + Owid parsed = ParseAssert.parsed( + Owid.tryParseBytes(signed.asByteArray())); assertEquals(MAXIMUM, parsed.getDomain().length(), "should write and read a domain of the published maximum"); @@ -243,9 +244,7 @@ void overMaximumLengthDomainRefusedByCreator() throws OwidException { /** * The refusal comes before anything is signed. A creator cannot be built * with an over long domain even when the crypto instance holds no private - * key at all, and a domain that reaches an OWID by another route is - * refused as the bytes to sign are assembled, which is the step before - * the private key is used, leaving the OWID with no signature. + * key at all, so the private key is never reached. */ @Test void overMaximumLengthDomainRefusedBeforeSigning() throws OwidException { @@ -259,16 +258,25 @@ void overMaximumLengthDomainRefusedBeforeSigning() throws OwidException { () -> Creator.create(domain, verifyOnly), "should refuse the domain without reaching the crypto"); assertNamesMaximum(fromCreator); + } + + /** + * Assembling the bytes that would be signed refuses a domain over the + * maximum however it arrived. Only the library itself can reach this, + * because a caller cannot build an OWID with a domain of its own + * choosing, so the test asks the library directly. + */ + @Test + void overMaximumLengthDomainRefusedWhenAssemblingDataToSign() { + String domain = domainOfLength(MAXIMUM + 1); - Owid owid = new Owid(); - owid.setDomain(domain); - owid.setPayload(PAYLOAD); - OwidException fromData = assertThrows(OwidException.class, - () -> owid.dataForCrypto(Collections.emptyList()), + OwidException thrown = assertThrows(OwidException.class, + () -> Owid.dataForCrypto(Version.current(), domain, + Io.baseDate(), PAYLOAD, + Collections.emptyList()), "should refuse to assemble the bytes that would be signed"); - assertNamesMaximum(fromData); - assertEquals(0, owid.getSignature().length, - "nothing should have been signed"); + + assertNamesMaximum(thrown); } /** @@ -278,10 +286,8 @@ void overMaximumLengthDomainRefusedBeforeSigning() throws OwidException { */ @Test void overMaximumLengthDomainRefusedWhenSerialising() { - Owid owid = new Owid(); - owid.setDomain(domainOfLength(MAXIMUM + 1)); - owid.setPayload(PAYLOAD); - owid.setSignature(SIGNATURE); + Owid owid = new Owid(Version.current(), domainOfLength(MAXIMUM + 1), + Io.baseDate(), PAYLOAD, SIGNATURE); OwidException thrown = assertThrows(OwidException.class, owid::asByteArray, @@ -306,9 +312,10 @@ private static void assertNamesMaximum(OwidException thrown) { void libraryOutputParses() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create("51d.es", crypto); - Owid original = creator.signBytes(PAYLOAD); + Owid original = creator.createBytes(PAYLOAD); - Owid parsed = Owid.fromByteArray(original.asByteArray()); + Owid parsed = ParseAssert.parsed( + Owid.tryParseBytes(original.asByteArray())); assertEquals("51d.es", parsed.getDomain(), "should read the domain the library wrote"); diff --git a/src/test/java/com/swancommunity/owid/Envelope.java b/src/test/java/com/swancommunity/owid/Envelope.java new file mode 100644 index 0000000..764a74a --- /dev/null +++ b/src/test/java/com/swancommunity/owid/Envelope.java @@ -0,0 +1,105 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import java.io.ByteArrayOutputStream; +import java.nio.charset.StandardCharsets; +import java.util.Arrays; + +/** + * Builds serialized OWIDs a byte at a time for the tests. + * + *

The bytes are written by hand rather than through the library, so a test + * of the reader does not depend on the writer it is checking the reader + * against, and so a test can make the declared payload length and the bytes + * present disagree in ways the writer would never produce.

+ */ +final class Envelope { + + /** The domain used wherever a test does not care what the domain is. */ + static final String DOMAIN = "51d.es"; + + private Envelope() { + } + + /** A byte array of the length given, every byte set to the value. */ + static byte[] filled(int length, byte value) { + byte[] bytes = new byte[length]; + Arrays.fill(bytes, value); + return bytes; + } + + /** + * A signature shaped run of bytes, being the right length and nothing + * more. + */ + static byte[] signature() { + return filled(Owid.SIGNATURE_LENGTH, (byte) 0x99); + } + + /** + * A version 3 envelope, being the version byte, the domain with its + * terminator, four minute bytes, the declared payload length, the payload + * bytes given and the signature bytes given. + */ + static byte[] version3(String domain, long minutes, long declaredLength, + byte[] payload, byte[] signature) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + stream.write(Version.VERSION3.asByte()); + writeDomain(stream, domain.getBytes(StandardCharsets.UTF_8)); + writeLittleEndian(stream, minutes); + writeLittleEndian(stream, declaredLength); + stream.write(payload, 0, payload.length); + stream.write(signature, 0, signature.length); + return stream.toByteArray(); + } + + /** The smallest well formed version 3 envelope carrying the payload. */ + static byte[] version3(byte[] payload) { + return version3(DOMAIN, 1000L, payload.length, payload, signature()); + } + + /** + * A version 1 envelope, whose date is two big endian bytes counting hours + * rather than four little endian bytes counting minutes. + */ + static byte[] version1(String domain, long hours, long declaredLength, + byte[] payload, byte[] signature) { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + stream.write(Version.VERSION1.asByte()); + writeDomain(stream, domain.getBytes(StandardCharsets.UTF_8)); + stream.write((int) ((hours >> 8) & 0xFF)); + stream.write((int) (hours & 0xFF)); + writeLittleEndian(stream, declaredLength); + stream.write(payload, 0, payload.length); + stream.write(signature, 0, signature.length); + return stream.toByteArray(); + } + + private static void writeDomain(ByteArrayOutputStream stream, + byte[] domain) { + stream.write(domain, 0, domain.length); + stream.write(0); + } + + static void writeLittleEndian(ByteArrayOutputStream stream, long value) { + stream.write((int) (value & 0xFF)); + stream.write((int) ((value >> 8) & 0xFF)); + stream.write((int) ((value >> 16) & 0xFF)); + stream.write((int) ((value >> 24) & 0xFF)); + } +} diff --git a/src/test/java/com/swancommunity/owid/FixturesTest.java b/src/test/java/com/swancommunity/owid/FixturesTest.java index 8134910..17a7a3b 100644 --- a/src/test/java/com/swancommunity/owid/FixturesTest.java +++ b/src/test/java/com/swancommunity/owid/FixturesTest.java @@ -133,22 +133,22 @@ private void runFixtures(Fixtures fixtures) throws OwidException { Crypto crypto = Crypto.newVerifyOnly(fixtures.spki()); List none = Collections.emptyList(); - Owid simple = Owid.fromBase64(fixtures.simple()); + Owid simple = ParseAssert.parsed(Owid.tryParse(fixtures.simple())); assertTrue(simple.verifyWithCrypto(crypto, none), "simple should verify"); assertTrue(simple.verifyWithPublicKey(fixtures.spki(), none), "simple should verify by public key PEM"); - Owid utf8 = Owid.fromBase64(fixtures.utf8()); + Owid utf8 = ParseAssert.parsed(Owid.tryParse(fixtures.utf8())); assertTrue(utf8.verifyWithCrypto(crypto, none), "utf8 should verify"); org.junit.jupiter.api.Assertions.assertEquals(UTF8_TEXT, utf8.payloadAsString(), "utf8 payload text should match"); - Owid root = Owid.fromBase64(fixtures.chainRoot()); + Owid root = ParseAssert.parsed(Owid.tryParse(fixtures.chainRoot())); assertTrue(root.verifyWithCrypto(crypto, none), "chain root should verify alone"); - Owid party = Owid.fromBase64(fixtures.chainParty()); + Owid party = ParseAssert.parsed(Owid.tryParse(fixtures.chainParty())); assertTrue(party.verifyWithCrypto(crypto, Collections.singletonList(root)), "chain party should verify with the root as the other"); assertFalse(party.verifyWithCrypto(crypto, none), @@ -158,13 +158,13 @@ private void runFixtures(Fixtures fixtures) throws OwidException { for (String encoded : new String[] {fixtures.simple(), fixtures.utf8(), fixtures.chainRoot()}) { byte[] tampered = flipLastByte(Base64.getMimeDecoder().decode(encoded)); - Owid owid = Owid.fromByteArray(tampered); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(tampered)); assertFalse(owid.verifyWithCrypto(crypto, none), "a flipped signature byte should break verification"); } byte[] tamperedParty = flipLastByte(Base64.getMimeDecoder().decode(fixtures.chainParty())); - Owid party2 = Owid.fromByteArray(tamperedParty); + Owid party2 = ParseAssert.parsed(Owid.tryParseBytes(tamperedParty)); assertFalse(party2.verifyWithCrypto(crypto, Collections.singletonList(root)), "a flipped party signature byte should break verification"); } diff --git a/src/test/java/com/swancommunity/owid/IoTest.java b/src/test/java/com/swancommunity/owid/IoTest.java index 18c4c2086d6c58623f6e8467bcd291e31284a9b7..236295c126908d4a9592badf442db899b2db5c09 100644 GIT binary patch literal 6446 zcmd5=ZFAbj7XHq!IO9(uA>zeJXIiJ3i(_uXl;DoR&15p3&H{_rMC_{4%9xw+f4|RJ zNedM0)~Vfl`2a%NJ$qiB=e(%-jGjH>r$*1{tx49YiWZhStP*c{7!XC8Y@`Cb1HFDru$c!ktq}33=bg zQjD2cf#m`lk~Nw6_CgYT1g~CLoBYshu2!p1IAueVM$Nb|C2J1)Z+pX0?*$YtH(csi zW*MdO_gtkgcCsdsK%uFaK+RaJ$fP9LeKWF#6<(#vDjhW_GjqEVspNBKDzmAYUyfPWhwjBicR23%Ms$8bZ_kHk{c-<% zh=2d4?(if1+#jAbNWxo)i@Zrvwg74>_N1JJG?MIVr&y)J9M3sISt6%uuBI?eN4bb3 zMdn(j8Ws^yu~ZpHG=qXOiWwim@_$xq&n_3AM+9%PRzu9buu?!pn5}>cvs~s{*>z}E zYIahq!A6r>%2=-{jE=(3{2bp2u$&_5D(q$%@Xcz6!d-ULRIGcy=OWHdzD>g4+B{yQ zW`&chqd$vl5h@d&uB~*co#!^m?Z~E5EWLtml~qd_p5>{F!huJ94Nqu$;E9#&DG`Tn zak@5_aFSnj*llB(VM(p|%rS{Kki{)9w#3dvX_2lS0VFMC47_B(A#wsG!m-CQ7?yx9 z3t?#?9C*MT$D)Gd2w2I#*U6g;tY!FtCE8GIbOcz7@&H`05;O&4rXr4!K4k1@>Mj~I zkyF9BMY$}a&5Bkc{sI8gYGF9j=xZ69L^?#LlF^gl(VE~><`0zA_(FtAQaghf@9tbI z0U8%_3X#m831U`UaUQ__BsH^q3fePcutFo3h?!fmqzT~cL_vNYEnM~jNu}NB6~1mJ zZ?L9Wo5nyZ_05>={aULLK1r&s1y+}MpCbaX)8W$<{nGt7IPad(kMy;5)9N+|->0{T zAaDE^OL14%M$KpjAr}S=MwUv-wA~5nh2a^UgIqnAk~y7!b^BOK2`{%D=ahd#csZzt zHqG@Ea%W=`Y(N&i?+-7>y;1$dOW}~xQfXwz3{omMhZAX6QtF^@!imV_8Ds_ZhC2+> z&HG|U07+idLCVl1pU)ABA4$s<9bCdf*dYGkI2Sv2Ua_?})EBhy*WSel8Mm{wtH>lp z9td6nckawEQ~z;FA7u-Z1DFiOxyjQKC`=us*P=?ID*l-z$mVj&MdUBq(kszjSTs=9 z# z;>W;qfu*9=b9u|x)6f<6{bU0xUc7lgH9D^E%h=?S&!`)&bY^IY*j+BGa9wqG?{_i} zFFNuK!k_1n$M9HHhtc=q@Sv*1NAgB+kHoWRr8Seaq$U4w7jdXC-I12TVz=v)`!16| zy`c70!y|Jd33!ar^kD`jwDuPat&iSZg0-(s*K(d6hJ-=xkgwD&XTMkIXAZ!37J5e2STl7Qvu{ocxzw#S{FKJKwS zRnhLU&qsOM@d(j***R^s2YdDQ?h4szbA{}jx^HyFEai3|Lx;$^!@avZ#AcJm{N94w zqpikzyLl>RD@GKGWR}7VS;6V*(MoUU!f%i?-h?r~c?$C4}cW8pC_gcic z^o)h66+0cGmOjHsjsMd;S0u?m)g5b3f=;cw-k0 u+>Gi^`Twn)-VYT5{Seh|UESxKs%UzzAUoiVp?eQ#>ARN&t~Rc^t^F56|Io|; delta 774 zcmcIiPfHs?6jv)r-B>YowK<5sZWT5qtT7c4i-(F90;Pnuxl}xC%!^sN&TMBUjk@?1 zdU<%29>s$ca`WIf5d8+V(7SjnUYt!*wFl9Qb9uw?=KcP?{oKX)&+YE_P-LD0g;GhV zZ~>}{;PX28_zit1dCV0XAkZy?%gRs}o5ZAKjtp3`%4>cZz90tFZS2x{e58$U(Tl|Ahu~l)njB&+4vjy( z2VKsUf`8g>lU&-+wo_oHz`}KRE%4D<;cHgrrqqv0cx;t^^$gQq|JkFCl0~PfF*;5b zsbGv=S-kmlCw*_+$m$4oD!ZP2lbg!TXD8qN2U^TC^(tqluRRsKrqkD40yy4wGAS_#`vXmOVQgO>a@4-#x>rfD65#dM!~? h;FadJRI60zEC~%5eA{`yU} diff --git a/src/test/java/com/swancommunity/owid/ParseAssert.java b/src/test/java/com/swancommunity/owid/ParseAssert.java new file mode 100644 index 0000000..858bfba --- /dev/null +++ b/src/test/java/com/swancommunity/owid/ParseAssert.java @@ -0,0 +1,65 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +/** + * Assertions over a parse result, used everywhere a test reads an OWID. + * + *

Each one checks all three facts a result reports rather than only the + * one the test is interested in, so no test can pass while the result is + * internally inconsistent.

+ */ +final class ParseAssert { + + private ParseAssert() { + } + + /** + * Asserts the read worked and returns the OWID, checking that success, + * the value and the status agree. + */ + static Owid parsed(OwidParseResult result) { + assertNotNull(result, "a read should always report a result"); + assertTrue(result.isSuccess(), + "should have read an OWID but reported " + result.getStatus()); + assertEquals(OwidParseStatus.PARSED, result.getStatus(), + "a successful read should report PARSED"); + assertNotNull(result.getValue(), + "a successful read should hand back the OWID"); + return result.getValue(); + } + + /** + * Asserts the read failed for the reason given, and that nothing was + * handed back with it. + */ + static void failed(OwidParseResult result, OwidParseStatus expected) { + assertNotNull(result, "a read should always report a result"); + assertFalse(result.isSuccess(), + "should have refused the input but reported success"); + assertEquals(expected, result.getStatus(), + "should report the reason the input is not an OWID"); + assertNull(result.getValue(), + "a failed read should hand back no OWID"); + } +} diff --git a/src/test/java/com/swancommunity/owid/ParseContractTest.java b/src/test/java/com/swancommunity/owid/ParseContractTest.java new file mode 100644 index 0000000..db1fd27 --- /dev/null +++ b/src/test/java/com/swancommunity/owid/ParseContractTest.java @@ -0,0 +1,305 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.lang.reflect.Method; +import java.util.Arrays; +import java.util.Base64; +import java.util.Collections; +import java.util.Random; +import org.junit.jupiter.api.Test; + +/** + * The contract the reading surfaces keep for data that arrived from outside. + * + *

Every read reports three facts, being whether it worked, the OWID only + * when it did, and a named reason either way, and no expected failure throws. + * {@link ParseAssert} checks all three on every case here, so a test cannot + * pass while the result contradicts itself.

+ */ +class ParseContractTest { + + /** The bytes of a well formed version 3 envelope with a short payload. */ + private static byte[] wellFormed() { + return Envelope.version3(new byte[] {1, 2, 3}); + } + + /** + * A successful read reports all three facts, being success, a value, and + * the reason PARSED. + */ + @Test + void successReportsAllThreeFacts() { + OwidParseResult result = Owid.tryParseBytes(wellFormed()); + + assertTrue(result.isSuccess(), "should report success"); + assertNotNull(result.getValue(), "should hand back the OWID"); + assertEquals(OwidParseStatus.PARSED, result.getStatus(), + "should report PARSED"); + assertEquals(Envelope.DOMAIN, result.getValue().getDomain(), + "should read the fields"); + } + + /** Having nothing to say is allowed, so an empty payload is an OWID. */ + @Test + void emptyPayloadParses() { + Owid owid = ParseAssert.parsed( + Owid.tryParseBytes(Envelope.version3(new byte[0]))); + + assertEquals(0, owid.getPayloadLength(), + "should read an empty payload"); + } + + /** + * A one mebibyte payload is an OWID. The limit the format sets is the + * wire format's, and how much an application accepts is that + * application's policy rather than the parser's. + */ + @Test + void oneMebibytePayloadParsesFromBase64() { + byte[] payload = Envelope.filled(1024 * 1024, (byte) 0x5A); + String encoded = Base64.getEncoder().encodeToString( + Envelope.version3(payload)); + + Owid owid = ParseAssert.parsed(Owid.tryParse(encoded)); + + assertEquals(payload.length, owid.getPayloadLength(), + "should read the whole payload"); + assertArrayEquals(payload, owid.getPayload(), + "should read the payload unchanged"); + } + + /** Nothing to read is its own answer, on both surfaces. */ + @Test + void absentInputIsMissingInput() { + ParseAssert.failed(Owid.tryParse(null), OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.tryParse(""), OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.tryParseBytes(null), + OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.tryParseBytes(new byte[0]), + OwidParseStatus.MISSING_INPUT); + } + + /** + * Text that is not base 64 is reported rather than thrown, whether the + * characters are outside the alphabet, data follows the padding, or the + * length cannot be a whole number of blocks. + */ + @Test + void invalidBase64IsReported() { + String[] values = { + "not base 64 at all!", + "AAAAA*AA", + "AAAAA", + "AAAA=AAA", + "AAA==", + }; + for (String value : values) { + OwidParseResult result = assertDoesNotThrow( + () -> Owid.tryParse(value), + "should not throw for a value that is not base 64"); + ParseAssert.failed(result, OwidParseStatus.INVALID_BASE64); + } + } + + /** + * Base 64 without the trailing padding is a normal way to carry an + * encoded OWID, so it is read rather than refused. + */ + @Test + void unpaddedBase64IsAccepted() { + String padded = Base64.getEncoder().encodeToString(wellFormed()); + String unpadded = padded.replace("=", ""); + + Owid fromPadded = ParseAssert.parsed(Owid.tryParse(padded)); + Owid fromUnpadded = ParseAssert.parsed(Owid.tryParse(unpadded)); + + assertEquals(fromPadded, fromUnpadded, + "padding should make no difference to what is read"); + } + + /** A version byte this implementation does not know is named as such. */ + @Test + void unknownVersionIsReported() { + byte[] bytes = wellFormed(); + bytes[0] = 0x04; + + ParseAssert.failed(Owid.tryParseBytes(bytes), + OwidParseStatus.UNSUPPORTED_VERSION); + } + + /** + * Data that stops inside a field, before the payload length has even been + * read, is a truncation rather than a byte count disagreement. + */ + @Test + void truncatedFieldsAreUnexpectedEnd() { + byte[] complete = wellFormed(); + int domainEnd = 1 + Envelope.DOMAIN.length() + 1; + + // Inside the domain, with no terminator reached. + ParseAssert.failed( + Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd - 2)), + OwidParseStatus.UNEXPECTED_END); + + // Inside the date, two of its four bytes present. + ParseAssert.failed( + Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd + 2)), + OwidParseStatus.UNEXPECTED_END); + + // Inside the payload length field, two of its four bytes present. + ParseAssert.failed( + Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd + 6)), + OwidParseStatus.UNEXPECTED_END); + } + + /** + * The marker for an absent optional OWID is the single version byte, so + * it reads, and anything after it belongs to no field. + */ + @Test + void emptyMarkerParsesAndTrailingBytesDoNot() { + Owid owid = ParseAssert.parsed( + Owid.tryParseBytes(Owid.emptyByteArray())); + assertEquals(Version.EMPTY, owid.getVersion(), + "should read the empty marker"); + + ParseAssert.failed(Owid.tryParseBytes(new byte[] {0, 1}), + OwidParseStatus.MALFORMED_ENVELOPE); + } + + /** + * Parsing and verifying are two questions with two answers. An identifier + * whose bytes are a well formed OWID reads, and only then does asking + * about the signature report that it does not match. + */ + @Test + void structurallyValidWithWrongSignatureParsesThenFailsVerification() + throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + byte[] bytes = creator.createBytes(new byte[] {1, 2, 3}).asByteArray(); + + // A payload byte, so the envelope stays exactly the shape it was and + // only the signature stops describing the contents. + bytes[bytes.length - Owid.SIGNATURE_LENGTH - 1] ^= 0x01; + + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(bytes)); + + OwidVerificationResult verification = + owid.verifyDetailed(crypto, Collections.emptyList()); + assertEquals(OwidSignatureStatus.SIGNATURE_INVALID, + verification.getStatus(), + "the signature should be reported as not matching"); + assertFalse(verification.isValid(), + "a signature that does not match is not valid"); + } + + /** + * Nothing about a key can happen while a read fails, because neither + * reading surface is given one. The signatures of both are checked here + * rather than a call being counted, since a parameter that does not exist + * cannot be passed a key, and this library retrieves no keys of its own + * at all. + */ + @Test + void readingTakesNoKeyAndNoCrypto() { + int checked = 0; + for (Method method : Owid.class.getDeclaredMethods()) { + if (method.getName().startsWith("tryParse") == false) { + continue; + } + checked++; + for (Class parameter : method.getParameterTypes()) { + assertTrue(parameter == String.class + || parameter == byte[].class, + "reading should take only the data to read, but " + + method.getName() + " takes " + + parameter.getName()); + } + } + assertEquals(2, checked, "should have checked both read surfaces"); + } + + /** + * A run of malformed buffers, none of which throws and none of which + * hands back a value. The bytes come from a fixed seed so a failure can + * be reproduced, and half of them are near misses cut or corrupted from a + * good envelope, so the later checks are reached as well as the first + * one. + */ + @Test + void malformedInputNeverThrows() { + Random random = new Random(20260830L); + byte[] complete = wellFormed(); + + for (int i = 0; i < 2000; i++) { + byte[] bytes; + if (i % 2 == 0) { + bytes = new byte[random.nextInt(200)]; + random.nextBytes(bytes); + } else { + bytes = Arrays.copyOf(complete, + random.nextInt(complete.length + 8)); + if (bytes.length > 0) { + bytes[random.nextInt(bytes.length)] = + (byte) random.nextInt(256); + } + } + byte[] input = bytes; + + OwidParseResult result = assertDoesNotThrow( + () -> Owid.tryParseBytes(input), + "reading should never throw for malformed bytes"); + assertEquals(result.isSuccess(), result.getValue() != null, + "the value should be present exactly when it worked"); + assertEquals(result.isSuccess(), + result.getStatus() == OwidParseStatus.PARSED, + "the status should agree with the success outcome"); + + String encoded = Base64.getEncoder().encodeToString(input); + OwidParseResult fromText = assertDoesNotThrow( + () -> Owid.tryParse(encoded), + "reading should never throw for malformed text"); + assertEquals(fromText.isSuccess(), fromText.getValue() != null, + "the value should be present exactly when it worked"); + } + } + + /** + * A failure carries the status and nothing taken from the input, because + * a parse failure is often logged and the bytes came from outside. + */ + @Test + void failureCarriesNoneOfTheInput() { + String secret = "cGFzc3dvcmRwYXNzd29yZHBhc3N3b3Jk"; + + OwidParseResult result = Owid.tryParse(secret); + + ParseAssert.failed(result, OwidParseStatus.UNSUPPORTED_VERSION); + assertEquals(OwidParseStatus.UNSUPPORTED_VERSION.name(), + result.toString(), + "the result should say only which status it is"); + } +} diff --git a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java index 7630d8d..f580b85 100644 --- a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java +++ b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java @@ -18,7 +18,6 @@ import static org.junit.jupiter.api.Assertions.assertArrayEquals; import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; import java.io.ByteArrayOutputStream; @@ -110,9 +109,9 @@ private static long allocatedBytes() { * last 64 bytes, and the envelope parses to the same payload. */ @Test - void declaredLengthMatchesParses() throws OwidException { - Owid owid = Owid.fromByteArray( - envelope(PAYLOAD.length, PAYLOAD, SIGNATURE)); + void declaredLengthMatchesParses() { + Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + envelope(PAYLOAD.length, PAYLOAD, SIGNATURE))); assertArrayEquals(PAYLOAD, owid.getPayload(), "should read the payload back unchanged"); assertArrayEquals(SIGNATURE, owid.getSignature(), @@ -126,11 +125,11 @@ void declaredLengthMatchesParses() throws OwidException { * application rather than format parsing. */ @Test - void matchingOneMebibytePayloadParses() throws OwidException { + void matchingOneMebibytePayloadParses() { byte[] payload = filled(1024 * 1024, (byte) 0x5A); - Owid owid = Owid.fromByteArray( - envelope(payload.length, payload, SIGNATURE)); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + envelope(payload.length, payload, SIGNATURE))); assertEquals(payload.length, owid.getPayloadLength()); assertArrayEquals(payload, owid.getPayload()); @@ -145,8 +144,9 @@ void matchingOneMebibytePayloadParses() throws OwidException { void libraryOutputParses() throws OwidException { Crypto crypto = Crypto.generate(); Creator creator = Creator.create(DOMAIN, crypto); - Owid original = creator.signBytes(PAYLOAD); - Owid parsed = Owid.fromByteArray(original.asByteArray()); + Owid original = creator.createBytes(PAYLOAD); + Owid parsed = ParseAssert.parsed( + Owid.tryParseBytes(original.asByteArray())); assertArrayEquals(PAYLOAD, parsed.getPayload(), "should read the payload the library wrote"); assertEquals(original, parsed, "should parse to an equal OWID"); @@ -163,8 +163,8 @@ void declaredLengthOffByOneRefused() { int[] declaredLengths = {PAYLOAD.length - 1, PAYLOAD.length + 1}; for (int declared : declaredLengths) { byte[] bytes = envelope(declared, PAYLOAD, SIGNATURE); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a declared length of " + declared); + ParseAssert.failed(Owid.tryParseBytes(bytes), + OwidParseStatus.BYTE_COUNT_MISMATCH); } } @@ -176,20 +176,23 @@ void declaredLengthOffByOneRefused() { void trailingByteAfterSignatureRefused() { byte[] bytes = envelope(PAYLOAD.length, PAYLOAD, SIGNATURE); byte[] longer = Arrays.copyOf(bytes, bytes.length + 1); - assertThrows(OwidException.class, () -> Owid.fromByteArray(longer), - "should refuse a byte after the signature"); + ParseAssert.failed(Owid.tryParseBytes(longer), + OwidParseStatus.BYTE_COUNT_MISMATCH); } /** - * A short signature is refused. The declared payload length is right - * for the payload, but the bytes after it are fewer than a signature. + * A short signature is refused as a byte count disagreement. The declared + * payload length is right for the payload, but the bytes after it are + * fewer than a signature, and what the reader can say for certain is that + * the declared payload cannot leave exactly the 64 bytes the version + * requires. */ @Test void shortSignatureRefused() { byte[] bytes = envelope(PAYLOAD.length, PAYLOAD, filled(SIGNATURE_LENGTH - 1, (byte) 0x99)); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a 63 byte signature"); + ParseAssert.failed(Owid.tryParseBytes(bytes), + OwidParseStatus.BYTE_COUNT_MISMATCH); } /** @@ -211,9 +214,9 @@ void mismatchedLargeDeclarationRefusedWithoutAllocating() { for (long declared : declaredLengths) { byte[] bytes = envelope(declared, new byte[0], new byte[0]); long before = allocatedBytes(); - assertThrows(OwidException.class, () -> Owid.fromByteArray(bytes), - "should refuse a declared length of " + declared); + OwidParseResult result = Owid.tryParseBytes(bytes); long allocated = allocatedBytes() - before; + ParseAssert.failed(result, OwidParseStatus.BYTE_COUNT_MISMATCH); assertTrue(allocated < ALLOCATION_BOUND, "declared " + declared + " allocated " + allocated + " bytes"); } @@ -225,8 +228,9 @@ void mismatchedLargeDeclarationRefusedWithoutAllocating() { * valid envelope. */ @Test - void emptyPayloadParses() throws OwidException { - Owid owid = Owid.fromByteArray(envelope(0, new byte[0], SIGNATURE)); + void emptyPayloadParses() { + Owid owid = ParseAssert.parsed( + Owid.tryParseBytes(envelope(0, new byte[0], SIGNATURE))); assertEquals(0, owid.getPayload().length, "should read an empty payload"); assertArrayEquals(SIGNATURE, owid.getSignature(), diff --git a/src/test/java/com/swancommunity/owid/SignatureStatusTest.java b/src/test/java/com/swancommunity/owid/SignatureStatusTest.java new file mode 100644 index 0000000..cafa40f --- /dev/null +++ b/src/test/java/com/swancommunity/owid/SignatureStatusTest.java @@ -0,0 +1,187 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.util.Collections; +import java.util.List; +import org.junit.jupiter.api.Test; + +/** + * Asking whether a signature is genuine has to keep "does not match" apart + * from "could not check". + * + *

A key that cannot be obtained or cannot be decoded leaves the signature + * unjudged. Reporting that as invalid would tell a caller an identifier had + * been tampered with when all that happened is an outage, and a caller acting + * on it would reject good identifiers.

+ */ +class SignatureStatusTest { + + private static final List NONE = Collections.emptyList(); + + private static Crypto crypto() throws OwidException { + return Crypto.generate(); + } + + /** A genuine signature is the only thing that reports valid. */ + @Test + void genuineSignatureIsValid() throws OwidException { + Crypto crypto = crypto(); + Owid owid = Creator.create("example.com", crypto) + .createString("payload"); + + OwidVerificationResult result = owid.verifyDetailed(crypto, NONE); + + assertTrue(result.isValid(), "a genuine signature should be valid"); + assertEquals(OwidSignatureStatus.SIGNATURE_VALID, result.getStatus(), + "should report the signature as valid"); + } + + /** The same question answered from the public key PEM. */ + @Test + void genuineSignatureIsValidThroughPem() throws OwidException { + Crypto crypto = crypto(); + Owid owid = Creator.create("example.com", crypto) + .createString("payload"); + + OwidVerificationResult result = owid.verifyDetailedWithPublicKey( + crypto.publicKeyPem(), NONE); + + assertEquals(OwidSignatureStatus.SIGNATURE_VALID, result.getStatus(), + "should report the signature as valid"); + } + + /** + * A signature checked against the wrong key does not match. This is the + * one status that means the identifier should be distrusted. + */ + @Test + void wrongKeyIsSignatureInvalid() throws OwidException { + Owid owid = Creator.create("example.com", crypto()) + .createString("payload"); + + OwidVerificationResult result = + owid.verifyDetailed(crypto(), NONE); + + assertFalse(result.isValid(), "the signature should not be valid"); + assertEquals(OwidSignatureStatus.SIGNATURE_INVALID, result.getStatus(), + "a well formed signature that does not match is invalid"); + } + + /** + * No key at all leaves the signature unexamined, which is not the same as + * the signature being wrong. + */ + @Test + void noKeyIsKeyUnavailable() throws OwidException { + Owid owid = Creator.create("example.com", crypto()) + .createString("payload"); + + assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, + owid.verifyDetailed(null, NONE).getStatus(), + "a missing crypto instance should not judge the signature"); + assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, + owid.verifyDetailedWithPublicKey(null, NONE).getStatus(), + "a missing PEM should not judge the signature"); + assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, + owid.verifyDetailedWithPublicKey(" ", NONE).getStatus(), + "an empty PEM should not judge the signature"); + } + + /** + * Key material that arrived but cannot be used is the key's fault and not + * the identifier's. On 30 August 2026 the key end points served PEM a + * strict parser rejects and every offline verification failed while the + * keys and identifiers were both fine, which as an invalid signature + * would have read as an attack. + */ + @Test + void undecodableKeyIsInvalidKey() throws OwidException { + Owid owid = Creator.create("example.com", crypto()) + .createString("payload"); + + assertEquals(OwidSignatureStatus.INVALID_KEY, + owid.verifyDetailedWithPublicKey("not a PEM", NONE) + .getStatus(), + "material that is not a key should be reported as the key"); + assertEquals(OwidSignatureStatus.INVALID_KEY, + owid.verifyDetailedWithPublicKey( + "-----BEGIN PUBLIC KEY-----\nAAAA\n" + + "-----END PUBLIC KEY-----\n", NONE) + .getStatus(), + "a PEM whose body is not a key should be reported as the key"); + } + + /** + * A signature field that is not the length the version requires cannot be + * checked. The marker for an absent optional OWID is the one thing that + * reaches a verification surface with no signature at all. + */ + @Test + void missingSignatureIsInvalidSignatureLength() throws OwidException { + Owid marker = ParseAssert.parsed( + Owid.tryParseBytes(Owid.emptyByteArray())); + + assertEquals(OwidSignatureStatus.INVALID_SIGNATURE_LENGTH, + marker.verifyDetailed(crypto(), NONE).getStatus(), + "no signature is not the same as a signature that is wrong"); + } + + /** + * A field that cannot be encoded stops the check before any cryptography, + * and that is the library failing rather than the identifier being wrong. + * Only the library can produce this, because a caller cannot build an + * OWID with a domain of its own choosing. + */ + @Test + void unencodableFieldIsVerificationError() throws OwidException { + StringBuilder domain = new StringBuilder(); + while (domain.length() <= Io.MAXIMUM_DOMAIN_LENGTH) { + domain.append('a'); + } + Owid owid = new Owid(Version.current(), domain.toString(), + Io.baseDate(), new byte[0], + Envelope.filled(Owid.SIGNATURE_LENGTH, (byte) 1)); + + assertEquals(OwidSignatureStatus.VERIFICATION_ERROR, + owid.verifyDetailed(crypto(), NONE).getStatus(), + "a field that cannot be encoded is not an invalid signature"); + } + + /** + * The boolean surfaces still answer the same question for callers that + * only need yes or no, and still raise for a key they cannot import + * rather than answering no. + */ + @Test + void booleanSurfacesKeepTheirBehaviour() throws OwidException { + Crypto crypto = crypto(); + Owid owid = Creator.create("example.com", crypto) + .createString("payload"); + + assertTrue(owid.verifyWithCrypto(crypto, NONE), + "a genuine signature should verify"); + assertTrue(owid.verifyWithPublicKey(crypto.publicKeyPem(), NONE), + "a genuine signature should verify through the PEM"); + assertFalse(owid.verifyWithCrypto(crypto(), NONE), + "a signature checked against another key should not verify"); + } +} diff --git a/src/test/java/com/swancommunity/owid/WireVectorsTest.java b/src/test/java/com/swancommunity/owid/WireVectorsTest.java index 863fd3b..719008c 100644 --- a/src/test/java/com/swancommunity/owid/WireVectorsTest.java +++ b/src/test/java/com/swancommunity/owid/WireVectorsTest.java @@ -54,10 +54,24 @@ private static byte[] decode(String value) { return Base64.getMimeDecoder().decode(value); } + /** + * Each vector also reads straight from its unpadded base 64 form, so the + * library's own decoder is shown to accept what the vectors carry. + */ + @Test + void vectorsReadFromUnpaddedBase64() { + for (String vector : new String[] {CREATOR, SUPPLIER, BAD}) { + Owid owid = ParseAssert.parsed(Owid.tryParse(vector)); + assertArrayEquals(decode(vector), + assertDoesNotThrow(owid::asByteArray), + "should read the same bytes from the encoded form"); + } + } + @Test void creatorRoundTripsByteExact() throws OwidException { byte[] original = decode(CREATOR); - Owid owid = Owid.fromByteArray(original); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); assertEquals("51db.uk", owid.getDomain(), "should read the domain"); assertEquals(Version.VERSION2, owid.getVersion(), "should read version 2"); @@ -70,7 +84,7 @@ void creatorRoundTripsByteExact() throws OwidException { @Test void supplierRoundTripsByteExact() throws OwidException { byte[] original = decode(SUPPLIER); - Owid owid = Owid.fromByteArray(original); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); assertEquals("pop-up.swan-demo.uk", owid.getDomain(), "should read the domain"); assertArrayEquals(new byte[] {0x01, 0x03}, owid.getPayload(), @@ -86,8 +100,7 @@ void supplierRoundTripsByteExact() throws OwidException { @Test void badParsesAndRoundTrips() throws OwidException { byte[] original = decode(BAD); - Owid owid = assertDoesNotThrow(() -> Owid.fromByteArray(original), - "the bad fixture should still parse"); + Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); assertEquals("badssp.swan-demo.uk", owid.getDomain(), "should read the domain"); assertArrayEquals(original, owid.asByteArray(), From a8375db27c4f1469cc04c946019a5c585e3ea182 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 00:14:40 +0100 Subject: [PATCH 2/5] Name the reading surface for Java and refuse the empty marker Three points from reviewing the four ports together. tryParse and tryParseBytes are C# spellings. Reading is now Owid.parse, overloaded on the encoded string and on the raw bytes, which is how Java says the same thing. verifyDetailed and verifyDetailedWithPublicKey become verify, overloaded on the Crypto and on the public key PEM, so the name says what the method answers rather than how much detail it carries. Both overloads mean a caller passing a literal null has to say which one it means, which the README records. createString and createBytes keep their names, being the house convention already set by Crypto.signByteArray and verifyByteArray rather than a translation of anything. The single byte zero marker for an absent optional OWID is now refused by the whole buffer read as UNSUPPORTED_VERSION. It stands for the absence of an identifier rather than for one, carrying no domain, no date and no signature, so handing one back put an OWID in a caller's hands that nothing had ever signed, which is the state the construction boundary exists to prevent, and it could never verify. Framed reading, where the marker means an absent node inside a stream, is unaffected, and this library has no framed reader. Owid.emptyByteArray still writes the marker for embedding in someone else's framed array and now says that reading it back refuses it. A buffer with no bytes in it stays MISSING_INPUT rather than UNEXPECTED_END, because nothing supplied is not the same as data that stopped part way through a field. That was already the behaviour and now carries the reason beside it. Every member of both status vocabularies now either has a test or carries a comment saying why it cannot be reached. INVALID_INPUT_TYPE is refused by the compiler, IMPLEMENTATION_CAPACITY_EXCEEDED needs more than a Java array can hold on either surface, and MALFORMED_ENVELOPE is a backstop with no path to it while the byte count rule holds. INVALID_SIGNATURE_LENGTH is reachable only from inside the package, since reading and creation both settle the signature at 64 bytes, so it is tested there and the reason is recorded on the member. --- README.md | 46 ++++--- .../java/com/swancommunity/owid/Owid.java | 31 +++-- .../com/swancommunity/owid/OwidException.java | 2 +- .../swancommunity/owid/OwidParseResult.java | 2 +- .../swancommunity/owid/OwidParseStatus.java | 11 ++ .../com/swancommunity/owid/OwidReader.java | 25 ++-- .../owid/OwidSignatureStatus.java | 17 ++- .../java/com/swancommunity/owid/Version.java | 10 +- .../ConstructionBoundaryTest.java | 2 +- .../owidconsumer/ReadmeExampleTest.java | 2 +- .../com/swancommunity/owid/CreatorTest.java | 4 +- .../swancommunity/owid/DomainLengthTest.java | 14 +-- .../com/swancommunity/owid/FixturesTest.java | 12 +- .../java/com/swancommunity/owid/IoTest.java | 10 +- .../swancommunity/owid/ParseContractTest.java | 114 +++++++++++++----- .../swancommunity/owid/PayloadLengthTest.java | 16 +-- .../owid/SignatureStatusTest.java | 49 +++++--- .../swancommunity/owid/WireVectorsTest.java | 8 +- 18 files changed, 248 insertions(+), 127 deletions(-) diff --git a/README.md b/README.md index 6efe78a..722ab4d 100644 --- a/README.md +++ b/README.md @@ -107,7 +107,7 @@ String encoded = owid.asBase64(); // Later, or elsewhere, read it back. Reading answers rather than throwing, // because whatever arrives from outside may not be an OWID at all. -OwidParseResult result = Owid.tryParse(encoded); +OwidParseResult result = Owid.parse(encoded); if (result.isSuccess()) { Owid copy = result.getValue(); String publicPem = crypto.publicKeyPem(); @@ -135,7 +135,8 @@ party.verifyWithCrypto(crypto, Collections.emptyList()); // false An OWID is read from whatever a caller was handed, which on a public end point means anything at all, so malformed data is an ordinary outcome rather -than an exceptional one. `Owid.tryParse` and `Owid.tryParseBytes` therefore +than an exceptional one. `Owid.parse`, overloaded on the encoded string +and on the raw bytes, therefore report three facts every time. | Fact | Where | @@ -158,14 +159,15 @@ thing whichever language read the bytes. | `INVALID_DOMAIN_ENCODING` | The domain is unterminated, or longer than the published maximum. | | `BYTE_COUNT_MISMATCH` | The declared payload count disagrees with the bytes present. | | `IMPLEMENTATION_CAPACITY_EXCEEDED` | Larger than this runtime can hold. Not reachable from the byte array surface, because a Java array cannot exceed `Integer.MAX_VALUE` bytes and so can never agree with a larger declaration. | -| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. | +| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. Every failure this reader can meet is classified by one of the rows above, so this is a backstop with no path to it while the byte count rule holds. | Reading and verifying are separate questions, and reading fetches no key and performs no cryptography. Bytes that are a well formed OWID read successfully -even when the signature does not match, and only `verifyDetailed` then +even when the signature does not match, and only `verify` then reports that it does not. -`verifyDetailed` keeps "does not match" apart from "could not check", because +`verify` returns an `OwidVerificationResult` and keeps "does not match" +apart from "could not check", because a key that cannot be obtained or cannot be decoded leaves the signature unjudged, and reporting that as invalid would read as an attack rather than as the outage it is. @@ -174,10 +176,10 @@ as the outage it is. |--------|---------| | `SIGNATURE_VALID` | Genuine for this data and this key. | | `SIGNATURE_INVALID` | Well formed and does not match. The only status that means the identifier should be distrusted. | -| `INVALID_SIGNATURE_LENGTH` | A signature field of the wrong length reached the check. | +| `INVALID_SIGNATURE_LENGTH` | A signature field of the wrong length reached the check. A consumer cannot produce one, because reading and creation both settle the signature at 64 bytes. | | `KEY_UNAVAILABLE` | No key was supplied, or the one supplied cannot verify. | | `INVALID_KEY` | Key material arrived and cannot be decoded or used. | -| `IMPLEMENTATION_CAPACITY_EXCEEDED` | More work than this runtime can hold. | +| `IMPLEMENTATION_CAPACITY_EXCEEDED` | More work than this runtime can hold, which needs an OWID and its chain to approach the two gigabyte limit of a Java array. | | `VERIFICATION_ERROR` | The check could not be completed for a reason that is not the identifier's fault. | ## How an OWID comes into being @@ -186,8 +188,7 @@ An OWID is only worth anything because it is signed, so a caller cannot build one. There is no public constructor and no setter, and an instance reaches calling code by exactly two routes. -1. A successful read of a complete serialized OWID, through `Owid.tryParse` - or `Owid.tryParseBytes`. +1. A successful read of a complete serialized OWID, through `Owid.parse`. 2. A creator signing one into existence, through `createString` or `createBytes`. @@ -207,8 +208,8 @@ copies, because a Java byte array is mutable. | Before | After | |--------|-------| -| `Owid.fromBase64(value)` | `Owid.tryParse(value)` | -| `Owid.fromByteArray(buffer)` | `Owid.tryParseBytes(buffer)` | +| `Owid.fromBase64(value)` | `Owid.parse(value)` | +| `Owid.fromByteArray(buffer)` | `Owid.parse(buffer)` | | `new Owid()`, then `setPayload`, then `creator.sign(owid)` | `creator.createBytes(payload)` | | `creator.signString(value)` | `creator.createString(value)` | | `creator.signBytes(value)` | `creator.createBytes(value)` | @@ -216,6 +217,10 @@ copies, because a Java byte array is mutable. | `owid.setVersion`, `setDomain`, `setDate`, `setPayload` | no replacement, the state is read only | | `Version.fromByte(b)` | no replacement, an unknown version byte is `UNSUPPORTED_VERSION` from a read | +`Owid.parse` is overloaded on the input type, so a caller passing a literal +`null` has to say which one it means, as in `Owid.parse((String) null)`. The +same is true of `verify`. + The parse surfaces do not throw for malformed data, so a caller that wrapped the old ones in `try`/`catch` reads the status instead. `OwidException` is still raised for the caller's own mistakes, such as an invalid creator @@ -225,8 +230,8 @@ domain, a null payload, or a field that cannot be serialized. - `Owid` holds the version, domain, date to the minute in UTC, payload bytes, and signature bytes, all read only. - - `Owid.tryParse` and `Owid.tryParseBytes` read a signed OWID and report - why rather than throwing. + - `Owid.parse` reads a signed OWID, from the encoded string or from the + raw bytes, and reports why rather than throwing. - `asBase64` and `asByteArray` serialize a signed OWID. - `payloadAsString` decodes the payload as UTF-8. `payloadAsPrintable` returns zero padded lower case hexadecimal with no separator. @@ -234,9 +239,9 @@ domain, a null payload, or a field that cannot be serialized. reports the payload size without copying it. - `verifyWithCrypto` and `verifyWithPublicKey` return whether the signature, covering this OWID and any others provided, is valid. - - `verifyDetailed` and `verifyDetailedWithPublicKey` answer the same - question with a status, keeping a key that could not be used apart from a - signature that does not match. + - `verify`, taking either the `Crypto` or the public key PEM, answers the + same question with a status, keeping a key that could not be used apart + from a signature that does not match. - `ageMinutes` returns the minutes elapsed since creation. - `Crypto` holds the keys. - `Crypto.generate` creates a new P-256 key pair. @@ -285,8 +290,13 @@ ASN.1 DER. The data covered by the signature is this OWID without its signature, followed by the complete bytes, including signature, of each other OWID in the order provided when signing. -An empty OWID is written as the single byte `0x00`. It marks an absent -optional OWID inside a larger byte array. +An absent optional OWID is written as the single byte `0x00`, which +`Owid.emptyByteArray` produces, and it marks the absence of a node inside a +larger framed byte array. `Owid.parse` refuses it with +`UNSUPPORTED_VERSION`, because a whole buffer holding nothing but the marker +holds no OWID, and handing one back would put an OWID in a caller's hands +that nothing had ever signed. Only a framed reader, which this library does +not have, can make sense of the marker. Base 64 decoding accepts the standard alphabet with or without the trailing padding, and skips line breaks and spaces. Anything else in the string is diff --git a/src/main/java/com/swancommunity/owid/Owid.java b/src/main/java/com/swancommunity/owid/Owid.java index 5ac1bfb..1d181f0 100644 --- a/src/main/java/com/swancommunity/owid/Owid.java +++ b/src/main/java/com/swancommunity/owid/Owid.java @@ -33,8 +33,8 @@ * *

An OWID is only worth anything because it is signed, so a caller cannot * build one. An instance reaches calling code by one of two routes, being - * {@link #tryParse(String)} or {@link #tryParseBytes(byte[])} reading bytes - * that were already a complete OWID, or {@link Creator#createBytes(byte[])} + * {@link #parse(String)} or {@link #parse(byte[])} reading bytes that were + * already a complete OWID, or {@link Creator#createBytes(byte[])} * and its companions signing one into existence. There is deliberately no way * to assemble a half made one, because an unsigned OWID is indistinguishable * from a signed one to the code downstream of it and the difference only @@ -106,13 +106,13 @@ public final class Owid { * *

A successful read says the bytes are a structurally valid OWID. It * says nothing about whether the signature is genuine, which is a - * separate question answered by {@link #verifyDetailed(Crypto, List)}.

+ * separate question answered by {@link #verify(Crypto, List)}.

* * @param value the base 64 encoded OWID, which may be null * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and * the reason the string is not an OWID */ - public static OwidParseResult tryParse(String value) { + public static OwidParseResult parse(String value) { if (value == null || value.isEmpty()) { return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); } @@ -134,7 +134,7 @@ public static OwidParseResult tryParse(String value) { * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and * the reason the bytes are not an OWID */ - public static OwidParseResult tryParseBytes(byte[] buffer) { + public static OwidParseResult parse(byte[] buffer) { return OwidReader.read(buffer); } @@ -169,8 +169,14 @@ void toBuffer(ByteArrayOutputStream buffer) throws OwidException { } /** - * Writes an empty OWID marker. Used to indicate optional OWIDs in byte - * arrays. + * Writes the marker for an absent optional OWID, being the single byte + * zero, for embedding in a larger framed byte array. + * + *

Reading it back through {@link #parse(byte[])} reports + * {@link OwidParseStatus#UNSUPPORTED_VERSION}, because the marker stands + * for the absence of an identifier rather than for one, and a whole + * buffer holding nothing but the marker holds no OWID. Only a framed + * reader, which this library does not have, can make sense of it.

* * @return a single byte array holding the empty marker */ @@ -387,8 +393,7 @@ public boolean verifyWithPublicKey(String publicPem, List others) * in the same order as when signed * @return the outcome of the check */ - public OwidVerificationResult verifyDetailed(Crypto crypto, - List others) { + public OwidVerificationResult verify(Crypto crypto, List others) { if (crypto == null || crypto.canVerify() == false) { return OwidVerificationResult.of( OwidSignatureStatus.KEY_UNAVAILABLE); @@ -420,8 +425,8 @@ public OwidVerificationResult verifyDetailed(Crypto crypto, } /** - * The same question as {@link #verifyDetailed(Crypto, List)}, starting - * from the public key in SPKI PEM form. + * The same question as {@link #verify(Crypto, List)}, starting from the + * public key in SPKI PEM form. * *

Key material that cannot be decoded reports * {@link OwidSignatureStatus#INVALID_KEY}, because the fault is in the @@ -433,7 +438,7 @@ public OwidVerificationResult verifyDetailed(Crypto crypto, * one, in the same order as when signed * @return the outcome of the check */ - public OwidVerificationResult verifyDetailedWithPublicKey(String publicPem, + public OwidVerificationResult verify(String publicPem, List others) { if (publicPem == null || publicPem.trim().isEmpty()) { return OwidVerificationResult.of( @@ -446,7 +451,7 @@ public OwidVerificationResult verifyDetailedWithPublicKey(String publicPem, return OwidVerificationResult.of( OwidSignatureStatus.INVALID_KEY); } - return verifyDetailed(crypto, others); + return verify(crypto, others); } /** diff --git a/src/main/java/com/swancommunity/owid/OwidException.java b/src/main/java/com/swancommunity/owid/OwidException.java index 107c456..4094767 100644 --- a/src/main/java/com/swancommunity/owid/OwidException.java +++ b/src/main/java/com/swancommunity/owid/OwidException.java @@ -23,7 +23,7 @@ * *

Reading a serialized OWID does not raise this. Data that arrived from * outside is expected to be malformed sometimes, so - * {@link Owid#tryParse(String)} and {@link Owid#tryParseBytes(byte[])} report + * {@link Owid#parse(String)} and {@link Owid#parse(byte[])} report * an {@link OwidParseStatus} instead. What remains here is the caller's own * mistakes, such as an invalid creator domain or a field that cannot be * serialized, and failures of the cryptography.

diff --git a/src/main/java/com/swancommunity/owid/OwidParseResult.java b/src/main/java/com/swancommunity/owid/OwidParseResult.java index dd9174a..9cd24f6 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseResult.java +++ b/src/main/java/com/swancommunity/owid/OwidParseResult.java @@ -58,7 +58,7 @@ static OwidParseResult failed(OwidParseStatus status) { * Whether the bytes were a complete, structurally valid OWID. This says * nothing about whether the signature is genuine, which is a separate * question answered by - * {@link Owid#verifyDetailed(Crypto, java.util.List)}. + * {@link Owid#verify(Crypto, java.util.List)}. * * @return true when the read produced an OWID */ diff --git a/src/main/java/com/swancommunity/owid/OwidParseStatus.java b/src/main/java/com/swancommunity/owid/OwidParseStatus.java index 833baee..19e7f31 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseStatus.java +++ b/src/main/java/com/swancommunity/owid/OwidParseStatus.java @@ -91,6 +91,17 @@ public enum OwidParseStatus { * The envelope is malformed in a way none of the others describes. A * fallback for the genuinely unclassified, not a substitute for naming a * failure that is already understood. + * + *

Not reachable in Java today, and so not covered by a test. Every + * failure this reader can meet is classified by one of the members above, + * and the one place that still reports this is the check that the + * envelope ended where the buffer did, which cannot fire while the + * declared payload count has already been required to leave exactly the + * signature. That check is kept as a backstop rather than removed, + * because a future change to the count arithmetic would otherwise start + * accepting bytes after the signature in silence. Loosening the count + * rule during a deliberate check of the tests made this status appear, so + * the backstop does work.

*/ MALFORMED_ENVELOPE } diff --git a/src/main/java/com/swancommunity/owid/OwidReader.java b/src/main/java/com/swancommunity/owid/OwidReader.java index c36588a..d4cb153 100644 --- a/src/main/java/com/swancommunity/owid/OwidReader.java +++ b/src/main/java/com/swancommunity/owid/OwidReader.java @@ -46,29 +46,30 @@ private OwidReader() { /** Reads one complete OWID occupying the whole of the buffer. */ static OwidParseResult read(byte[] buffer) { + // Nothing supplied is not the same as data that stopped part way + // through a field, so a buffer with no bytes in it is reported as the + // absence it is rather than as a truncation. if (buffer == null || buffer.length == 0) { return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); } int total = buffer.length; Version version = Version.forByte(buffer[0] & 0xFF); - if (version == null) { + if (version == null || version == Version.EMPTY) { + // The empty marker, being the single byte zero, is refused here + // along with the versions this implementation does not know. It + // stands for an absent node inside a framed stream rather than + // for an identifier, so what it carries is no domain, no date and + // no signature, and handing one back would put an OWID in a + // caller's hands that nothing had ever signed. That is the state + // the construction boundary exists to prevent, and it could never + // verify. A framed reader, which this library does not have, + // would still read the marker as the absence it means. return OwidParseResult.failed( OwidParseStatus.UNSUPPORTED_VERSION); } int at = 1; - if (version == Version.EMPTY) { - // The marker for an absent optional OWID is the version byte and - // nothing else, so anything after it belongs to no field. - if (at != total) { - return OwidParseResult.failed( - OwidParseStatus.MALFORMED_ENVELOPE); - } - return OwidParseResult.parsed(new Owid( - version, "", Io.baseDate(), new byte[0], new byte[0])); - } - // The domain, terminated by a zero byte and no longer than the // maximum published for a domain name. The walk stops at that // maximum rather than at the end of the buffer, so a buffer whose diff --git a/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java b/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java index f027025..e24153e 100644 --- a/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java +++ b/src/main/java/com/swancommunity/owid/OwidSignatureStatus.java @@ -51,6 +51,12 @@ public enum OwidSignatureStatus { * {@link OwidParseStatus#UNEXPECTED_END} or * {@link OwidParseStatus#BYTE_COUNT_MISMATCH} instead, because there the * envelope never formed. + * + *

A consumer of this library cannot produce it, because both routes an + * OWID arrives by settle the signature at the length the version + * requires. It is kept, and tested from inside the library, because the + * status is part of the cross language vocabulary and other surfaces can + * be handed a signature field on its own.

*/ INVALID_SIGNATURE_LENGTH, @@ -67,9 +73,14 @@ public enum OwidSignatureStatus { INVALID_KEY, /** - * The work required is more than this runtime can hold. Reaching it - * needs an OWID and its chain to approach the two gigabyte limit of a - * Java array. + * The work required is more than this runtime can hold. + * + *

Not covered by a test, because reaching it needs an OWID and its + * chain to approach the two gigabyte limit of a Java array, which cannot + * be built in a test suite that has to run on an ordinary machine. The + * path to it is real, being the overflow guard on the serialized length, + * which raises a distinct exception so this status does not have to be + * told apart from {@link #VERIFICATION_ERROR} by reading a message.

*/ IMPLEMENTATION_CAPACITY_EXCEEDED, diff --git a/src/main/java/com/swancommunity/owid/Version.java b/src/main/java/com/swancommunity/owid/Version.java index aa5e01c..4a7caed 100644 --- a/src/main/java/com/swancommunity/owid/Version.java +++ b/src/main/java/com/swancommunity/owid/Version.java @@ -26,7 +26,15 @@ */ public enum Version { - /** Marker used to indicate an optional OWID that is not present. */ + /** + * Marker used to indicate an optional OWID that is not present, inside a + * larger framed byte array. + * + *

No OWID carries this version. A whole buffer holding nothing but the + * marker holds no identifier, so {@link Owid#parse(byte[])} refuses it as + * {@link OwidParseStatus#UNSUPPORTED_VERSION} rather than handing back + * something nothing has ever signed.

+ */ EMPTY(0), /** diff --git a/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java b/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java index c17af3b..0a40473 100644 --- a/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java +++ b/src/test/java/com/example/owidconsumer/ConstructionBoundaryTest.java @@ -161,7 +161,7 @@ void aLibraryUserCanStillDoEverything() throws OwidException { "party".getBytes(StandardCharsets.UTF_8), Collections.singletonList(root)); - OwidParseResult result = Owid.tryParse(party.asBase64()); + OwidParseResult result = Owid.parse(party.asBase64()); assertEquals(OwidParseStatus.PARSED, result.getStatus(), "the created OWID should read back"); Owid copy = result.getValue(); diff --git a/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java index 222fa98..06b0555 100644 --- a/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java +++ b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java @@ -57,7 +57,7 @@ void createSerializeReadBackAndVerify() throws OwidException { // Later, or elsewhere, read it back. Reading answers rather than // throwing, because whatever arrives from outside may not be an OWID // at all. - OwidParseResult result = Owid.tryParse(encoded); + OwidParseResult result = Owid.parse(encoded); if (result.isSuccess()) { Owid copy = result.getValue(); String publicPem = crypto.publicKeyPem(); diff --git a/src/test/java/com/swancommunity/owid/CreatorTest.java b/src/test/java/com/swancommunity/owid/CreatorTest.java index f5c3dd5..2662a91 100644 --- a/src/test/java/com/swancommunity/owid/CreatorTest.java +++ b/src/test/java/com/swancommunity/owid/CreatorTest.java @@ -66,7 +66,7 @@ void signAndSelfVerifyThroughPem() throws OwidException { Creator creator = Creator.create("example.com", crypto); Owid owid = creator.createString("payload"); String encoded = owid.asBase64(); - Owid copy = ParseAssert.parsed(Owid.tryParse(encoded)); + Owid copy = ParseAssert.parsed(Owid.parse(encoded)); assertTrue(copy.verifyWithPublicKey(crypto.publicKeyPem(), Collections.emptyList()), "the decoded OWID should verify"); } @@ -78,7 +78,7 @@ void tamperedSignedOwidFails() throws OwidException { Owid owid = creator.createBytes(new byte[] {1, 2, 3}); byte[] bytes = owid.asByteArray(); bytes[bytes.length - 1] ^= 0x01; - Owid tampered = ParseAssert.parsed(Owid.tryParseBytes(bytes)); + Owid tampered = ParseAssert.parsed(Owid.parse(bytes)); assertFalse(tampered.verifyWithCrypto(crypto, Collections.emptyList()), "a tampered signature should not verify"); } diff --git a/src/test/java/com/swancommunity/owid/DomainLengthTest.java b/src/test/java/com/swancommunity/owid/DomainLengthTest.java index 6f2ef7d..f25ab6e 100644 --- a/src/test/java/com/swancommunity/owid/DomainLengthTest.java +++ b/src/test/java/com/swancommunity/owid/DomainLengthTest.java @@ -140,7 +140,7 @@ void maximumLengthDomainParses() throws OwidException { String domain = domainOfLength(MAXIMUM); byte[] bytes = envelope(ascii(domain)); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(bytes)); + Owid owid = ParseAssert.parsed(Owid.parse(bytes)); assertEquals(MAXIMUM, owid.getDomain().length(), "should read a domain of the published maximum length"); @@ -150,7 +150,7 @@ void maximumLengthDomainParses() throws OwidException { assertArrayEquals(bytes, owid.asByteArray(), "should write the same bytes back out"); assertEquals(owid, - ParseAssert.parsed(Owid.tryParseBytes(owid.asByteArray())), + ParseAssert.parsed(Owid.parse(owid.asByteArray())), "should parse its own output to an equal OWID"); } @@ -162,7 +162,7 @@ void maximumLengthDomainParses() throws OwidException { void overMaximumLengthDomainRefused() { byte[] bytes = envelope(ascii(domainOfLength(MAXIMUM + 1))); - ParseAssert.failed(Owid.tryParseBytes(bytes), + ParseAssert.failed(Owid.parse(bytes), OwidParseStatus.INVALID_DOMAIN_ENCODING); } @@ -176,7 +176,7 @@ void missingTerminatorRefused() { byte[] bytes = filled(64 * 1024, (byte) 'a'); bytes[0] = Version.VERSION3.asByte(); - ParseAssert.failed(Owid.tryParseBytes(bytes), + ParseAssert.failed(Owid.parse(bytes), OwidParseStatus.INVALID_DOMAIN_ENCODING); } @@ -192,7 +192,7 @@ void hostileDomainRefusedWithoutAllocating() { byte[] bytes = envelope(filled(HOSTILE_DOMAIN_LENGTH, (byte) 'a')); long before = allocatedBytes(); - OwidParseResult result = Owid.tryParseBytes(bytes); + OwidParseResult result = Owid.parse(bytes); long allocated = allocatedBytes() - before; ParseAssert.failed(result, OwidParseStatus.INVALID_DOMAIN_ENCODING); @@ -214,7 +214,7 @@ void maximumLengthDomainWritten() throws OwidException { Owid signed = creator.createBytes(PAYLOAD); Owid parsed = ParseAssert.parsed( - Owid.tryParseBytes(signed.asByteArray())); + Owid.parse(signed.asByteArray())); assertEquals(MAXIMUM, parsed.getDomain().length(), "should write and read a domain of the published maximum"); @@ -315,7 +315,7 @@ void libraryOutputParses() throws OwidException { Owid original = creator.createBytes(PAYLOAD); Owid parsed = ParseAssert.parsed( - Owid.tryParseBytes(original.asByteArray())); + Owid.parse(original.asByteArray())); assertEquals("51d.es", parsed.getDomain(), "should read the domain the library wrote"); diff --git a/src/test/java/com/swancommunity/owid/FixturesTest.java b/src/test/java/com/swancommunity/owid/FixturesTest.java index 17a7a3b..9b94cb5 100644 --- a/src/test/java/com/swancommunity/owid/FixturesTest.java +++ b/src/test/java/com/swancommunity/owid/FixturesTest.java @@ -133,22 +133,22 @@ private void runFixtures(Fixtures fixtures) throws OwidException { Crypto crypto = Crypto.newVerifyOnly(fixtures.spki()); List none = Collections.emptyList(); - Owid simple = ParseAssert.parsed(Owid.tryParse(fixtures.simple())); + Owid simple = ParseAssert.parsed(Owid.parse(fixtures.simple())); assertTrue(simple.verifyWithCrypto(crypto, none), "simple should verify"); assertTrue(simple.verifyWithPublicKey(fixtures.spki(), none), "simple should verify by public key PEM"); - Owid utf8 = ParseAssert.parsed(Owid.tryParse(fixtures.utf8())); + Owid utf8 = ParseAssert.parsed(Owid.parse(fixtures.utf8())); assertTrue(utf8.verifyWithCrypto(crypto, none), "utf8 should verify"); org.junit.jupiter.api.Assertions.assertEquals(UTF8_TEXT, utf8.payloadAsString(), "utf8 payload text should match"); - Owid root = ParseAssert.parsed(Owid.tryParse(fixtures.chainRoot())); + Owid root = ParseAssert.parsed(Owid.parse(fixtures.chainRoot())); assertTrue(root.verifyWithCrypto(crypto, none), "chain root should verify alone"); - Owid party = ParseAssert.parsed(Owid.tryParse(fixtures.chainParty())); + Owid party = ParseAssert.parsed(Owid.parse(fixtures.chainParty())); assertTrue(party.verifyWithCrypto(crypto, Collections.singletonList(root)), "chain party should verify with the root as the other"); assertFalse(party.verifyWithCrypto(crypto, none), @@ -158,13 +158,13 @@ private void runFixtures(Fixtures fixtures) throws OwidException { for (String encoded : new String[] {fixtures.simple(), fixtures.utf8(), fixtures.chainRoot()}) { byte[] tampered = flipLastByte(Base64.getMimeDecoder().decode(encoded)); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(tampered)); + Owid owid = ParseAssert.parsed(Owid.parse(tampered)); assertFalse(owid.verifyWithCrypto(crypto, none), "a flipped signature byte should break verification"); } byte[] tamperedParty = flipLastByte(Base64.getMimeDecoder().decode(fixtures.chainParty())); - Owid party2 = ParseAssert.parsed(Owid.tryParseBytes(tamperedParty)); + Owid party2 = ParseAssert.parsed(Owid.parse(tamperedParty)); assertFalse(party2.verifyWithCrypto(crypto, Collections.singletonList(root)), "a flipped party signature byte should break verification"); } diff --git a/src/test/java/com/swancommunity/owid/IoTest.java b/src/test/java/com/swancommunity/owid/IoTest.java index 236295c..8660953 100644 --- a/src/test/java/com/swancommunity/owid/IoTest.java +++ b/src/test/java/com/swancommunity/owid/IoTest.java @@ -53,7 +53,7 @@ void dateRoundtripVersion2() throws OwidException { assertArrayEquals(expected.toByteArray(), buffer.toByteArray(), "should write the minute count little endian"); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( Envelope.version3(Envelope.DOMAIN, minutes, PAYLOAD.length, PAYLOAD, Envelope.signature()))); assertEquals(date, owid.getDate(), @@ -70,7 +70,7 @@ void dateRoundtripVersion1() throws OwidException { assertArrayEquals(new byte[] {0x30, 0x39}, buffer.toByteArray(), "should write the hour count big endian"); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( Envelope.version1(Envelope.DOMAIN, 12_345L, PAYLOAD.length, PAYLOAD, Envelope.signature()))); assertEquals(date, owid.getDate(), "should keep hour granularity"); @@ -92,7 +92,7 @@ void stringRoundtrip() throws OwidException { byte[] bytes = buffer.toByteArray(); assertEquals(0, bytes[bytes.length - 1], "should be null terminated"); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( Envelope.version3("example.com", 1000L, PAYLOAD.length, PAYLOAD, Envelope.signature()))); assertEquals("example.com", owid.getDomain(), @@ -116,7 +116,7 @@ void uint32LittleEndian() { // The same four bytes read back through the date field, which is the // one place a whole unsigned 32 bit value reaches the reader. - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( Envelope.version3(Envelope.DOMAIN, 0x0A242B01L, PAYLOAD.length, PAYLOAD, Envelope.signature()))); assertEquals(Io.baseDate().plus(Duration.ofMinutes(0x0A242B01L)), @@ -134,7 +134,7 @@ void signatureRoundtrip() throws OwidException { assertArrayEquals(signature, buffer.toByteArray(), "should write the signature bytes unchanged"); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( Envelope.version3(Envelope.DOMAIN, 1000L, PAYLOAD.length, PAYLOAD, signature))); assertArrayEquals(signature, owid.getSignature(), diff --git a/src/test/java/com/swancommunity/owid/ParseContractTest.java b/src/test/java/com/swancommunity/owid/ParseContractTest.java index db1fd27..f666091 100644 --- a/src/test/java/com/swancommunity/owid/ParseContractTest.java +++ b/src/test/java/com/swancommunity/owid/ParseContractTest.java @@ -37,6 +37,17 @@ * when it did, and a named reason either way, and no expected failure throws. * {@link ParseAssert} checks all three on every case here, so a test cannot * pass while the result contradicts itself.

+ * + *

Every member of {@link OwidParseStatus} is exercised here, with the + * domain cases also covered in more depth by {@link DomainLengthTest} and the + * byte count cases by {@link PayloadLengthTest}. Two members are not, and + * cannot be, being {@link OwidParseStatus#INVALID_INPUT_TYPE}, which the + * compiler already refuses, and + * {@link OwidParseStatus#IMPLEMENTATION_CAPACITY_EXCEEDED}, which a Java byte + * array cannot reach. A third, + * {@link OwidParseStatus#MALFORMED_ENVELOPE}, is a backstop with no path to + * it while the byte count rule holds. The reason is recorded on each of those + * members as well.

*/ class ParseContractTest { @@ -51,7 +62,7 @@ private static byte[] wellFormed() { */ @Test void successReportsAllThreeFacts() { - OwidParseResult result = Owid.tryParseBytes(wellFormed()); + OwidParseResult result = Owid.parse(wellFormed()); assertTrue(result.isSuccess(), "should report success"); assertNotNull(result.getValue(), "should hand back the OWID"); @@ -65,7 +76,7 @@ void successReportsAllThreeFacts() { @Test void emptyPayloadParses() { Owid owid = ParseAssert.parsed( - Owid.tryParseBytes(Envelope.version3(new byte[0]))); + Owid.parse(Envelope.version3(new byte[0]))); assertEquals(0, owid.getPayloadLength(), "should read an empty payload"); @@ -82,7 +93,7 @@ void oneMebibytePayloadParsesFromBase64() { String encoded = Base64.getEncoder().encodeToString( Envelope.version3(payload)); - Owid owid = ParseAssert.parsed(Owid.tryParse(encoded)); + Owid owid = ParseAssert.parsed(Owid.parse(encoded)); assertEquals(payload.length, owid.getPayloadLength(), "should read the whole payload"); @@ -93,11 +104,12 @@ void oneMebibytePayloadParsesFromBase64() { /** Nothing to read is its own answer, on both surfaces. */ @Test void absentInputIsMissingInput() { - ParseAssert.failed(Owid.tryParse(null), OwidParseStatus.MISSING_INPUT); - ParseAssert.failed(Owid.tryParse(""), OwidParseStatus.MISSING_INPUT); - ParseAssert.failed(Owid.tryParseBytes(null), + ParseAssert.failed(Owid.parse((String) null), + OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.parse(""), OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.parse((byte[]) null), OwidParseStatus.MISSING_INPUT); - ParseAssert.failed(Owid.tryParseBytes(new byte[0]), + ParseAssert.failed(Owid.parse(new byte[0]), OwidParseStatus.MISSING_INPUT); } @@ -117,7 +129,7 @@ void invalidBase64IsReported() { }; for (String value : values) { OwidParseResult result = assertDoesNotThrow( - () -> Owid.tryParse(value), + () -> Owid.parse(value), "should not throw for a value that is not base 64"); ParseAssert.failed(result, OwidParseStatus.INVALID_BASE64); } @@ -132,8 +144,8 @@ void unpaddedBase64IsAccepted() { String padded = Base64.getEncoder().encodeToString(wellFormed()); String unpadded = padded.replace("=", ""); - Owid fromPadded = ParseAssert.parsed(Owid.tryParse(padded)); - Owid fromUnpadded = ParseAssert.parsed(Owid.tryParse(unpadded)); + Owid fromPadded = ParseAssert.parsed(Owid.parse(padded)); + Owid fromUnpadded = ParseAssert.parsed(Owid.parse(unpadded)); assertEquals(fromPadded, fromUnpadded, "padding should make no difference to what is read"); @@ -145,7 +157,7 @@ void unknownVersionIsReported() { byte[] bytes = wellFormed(); bytes[0] = 0x04; - ParseAssert.failed(Owid.tryParseBytes(bytes), + ParseAssert.failed(Owid.parse(bytes), OwidParseStatus.UNSUPPORTED_VERSION); } @@ -160,33 +172,77 @@ void truncatedFieldsAreUnexpectedEnd() { // Inside the domain, with no terminator reached. ParseAssert.failed( - Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd - 2)), + Owid.parse(Arrays.copyOf(complete, domainEnd - 2)), OwidParseStatus.UNEXPECTED_END); // Inside the date, two of its four bytes present. ParseAssert.failed( - Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd + 2)), + Owid.parse(Arrays.copyOf(complete, domainEnd + 2)), OwidParseStatus.UNEXPECTED_END); // Inside the payload length field, two of its four bytes present. ParseAssert.failed( - Owid.tryParseBytes(Arrays.copyOf(complete, domainEnd + 6)), + Owid.parse(Arrays.copyOf(complete, domainEnd + 6)), OwidParseStatus.UNEXPECTED_END); } /** - * The marker for an absent optional OWID is the single version byte, so - * it reads, and anything after it belongs to no field. + * A domain that never terminates, or runs past the published maximum + * before it does, is a domain that cannot be valid rather than data that + * merely stopped. {@link DomainLengthTest} covers the bound itself and + * the cost of refusing a hostile field. */ @Test - void emptyMarkerParsesAndTrailingBytesDoNot() { - Owid owid = ParseAssert.parsed( - Owid.tryParseBytes(Owid.emptyByteArray())); - assertEquals(Version.EMPTY, owid.getVersion(), - "should read the empty marker"); + void badDomainIsInvalidDomainEncoding() { + // Terminated, but longer than a domain name is allowed to be. + StringBuilder tooLong = new StringBuilder(); + while (tooLong.length() <= Io.MAXIMUM_DOMAIN_LENGTH) { + tooLong.append('a'); + } + ParseAssert.failed( + Owid.parse(Envelope.version3(tooLong.toString(), 1000L, 0, + new byte[0], Envelope.signature())), + OwidParseStatus.INVALID_DOMAIN_ENCODING); + + // Never terminated, in a buffer long enough that the walk has to stop + // itself rather than run out of bytes. + byte[] unterminated = Envelope.filled(64 * 1024, (byte) 'a'); + unterminated[0] = Version.VERSION3.asByte(); + ParseAssert.failed(Owid.parse(unterminated), + OwidParseStatus.INVALID_DOMAIN_ENCODING); + } + + /** + * A declared payload count that disagrees with the bytes present is + * refused before anything is sized by it. {@link PayloadLengthTest} + * covers the counts in every direction and proves nothing is allocated. + */ + @Test + void disagreeingByteCountIsByteCountMismatch() { + byte[] complete = wellFormed(); + byte[] longer = Arrays.copyOf(complete, complete.length + 1); - ParseAssert.failed(Owid.tryParseBytes(new byte[] {0, 1}), - OwidParseStatus.MALFORMED_ENVELOPE); + ParseAssert.failed(Owid.parse(longer), + OwidParseStatus.BYTE_COUNT_MISMATCH); + } + + /** + * The marker for an absent optional OWID is refused by the whole buffer + * read. + * + *

It stands for the absence of an identifier rather than for one, and + * it carries no domain, no date and no signature, so handing one back + * would put an OWID in a caller's hands that nothing had ever signed. + * That is the state the construction boundary exists to prevent, and it + * could never verify. A framed reader, which this library does not have, + * would still read the marker as the absence it means.

+ */ + @Test + void emptyMarkerIsRefused() { + ParseAssert.failed(Owid.parse(Owid.emptyByteArray()), + OwidParseStatus.UNSUPPORTED_VERSION); + ParseAssert.failed(Owid.parse(new byte[] {0, 1}), + OwidParseStatus.UNSUPPORTED_VERSION); } /** @@ -205,10 +261,10 @@ void structurallyValidWithWrongSignatureParsesThenFailsVerification() // only the signature stops describing the contents. bytes[bytes.length - Owid.SIGNATURE_LENGTH - 1] ^= 0x01; - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(bytes)); + Owid owid = ParseAssert.parsed(Owid.parse(bytes)); OwidVerificationResult verification = - owid.verifyDetailed(crypto, Collections.emptyList()); + owid.verify(crypto, Collections.emptyList()); assertEquals(OwidSignatureStatus.SIGNATURE_INVALID, verification.getStatus(), "the signature should be reported as not matching"); @@ -227,7 +283,7 @@ void structurallyValidWithWrongSignatureParsesThenFailsVerification() void readingTakesNoKeyAndNoCrypto() { int checked = 0; for (Method method : Owid.class.getDeclaredMethods()) { - if (method.getName().startsWith("tryParse") == false) { + if (method.getName().equals("parse") == false) { continue; } checked++; @@ -270,7 +326,7 @@ void malformedInputNeverThrows() { byte[] input = bytes; OwidParseResult result = assertDoesNotThrow( - () -> Owid.tryParseBytes(input), + () -> Owid.parse(input), "reading should never throw for malformed bytes"); assertEquals(result.isSuccess(), result.getValue() != null, "the value should be present exactly when it worked"); @@ -280,7 +336,7 @@ void malformedInputNeverThrows() { String encoded = Base64.getEncoder().encodeToString(input); OwidParseResult fromText = assertDoesNotThrow( - () -> Owid.tryParse(encoded), + () -> Owid.parse(encoded), "reading should never throw for malformed text"); assertEquals(fromText.isSuccess(), fromText.getValue() != null, "the value should be present exactly when it worked"); @@ -295,7 +351,7 @@ void malformedInputNeverThrows() { void failureCarriesNoneOfTheInput() { String secret = "cGFzc3dvcmRwYXNzd29yZHBhc3N3b3Jk"; - OwidParseResult result = Owid.tryParse(secret); + OwidParseResult result = Owid.parse(secret); ParseAssert.failed(result, OwidParseStatus.UNSUPPORTED_VERSION); assertEquals(OwidParseStatus.UNSUPPORTED_VERSION.name(), diff --git a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java index f580b85..a04d887 100644 --- a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java +++ b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java @@ -110,7 +110,7 @@ private static long allocatedBytes() { */ @Test void declaredLengthMatchesParses() { - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( envelope(PAYLOAD.length, PAYLOAD, SIGNATURE))); assertArrayEquals(PAYLOAD, owid.getPayload(), "should read the payload back unchanged"); @@ -128,7 +128,7 @@ void declaredLengthMatchesParses() { void matchingOneMebibytePayloadParses() { byte[] payload = filled(1024 * 1024, (byte) 0x5A); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes( + Owid owid = ParseAssert.parsed(Owid.parse( envelope(payload.length, payload, SIGNATURE))); assertEquals(payload.length, owid.getPayloadLength()); @@ -146,7 +146,7 @@ void libraryOutputParses() throws OwidException { Creator creator = Creator.create(DOMAIN, crypto); Owid original = creator.createBytes(PAYLOAD); Owid parsed = ParseAssert.parsed( - Owid.tryParseBytes(original.asByteArray())); + Owid.parse(original.asByteArray())); assertArrayEquals(PAYLOAD, parsed.getPayload(), "should read the payload the library wrote"); assertEquals(original, parsed, "should parse to an equal OWID"); @@ -163,7 +163,7 @@ void declaredLengthOffByOneRefused() { int[] declaredLengths = {PAYLOAD.length - 1, PAYLOAD.length + 1}; for (int declared : declaredLengths) { byte[] bytes = envelope(declared, PAYLOAD, SIGNATURE); - ParseAssert.failed(Owid.tryParseBytes(bytes), + ParseAssert.failed(Owid.parse(bytes), OwidParseStatus.BYTE_COUNT_MISMATCH); } } @@ -176,7 +176,7 @@ void declaredLengthOffByOneRefused() { void trailingByteAfterSignatureRefused() { byte[] bytes = envelope(PAYLOAD.length, PAYLOAD, SIGNATURE); byte[] longer = Arrays.copyOf(bytes, bytes.length + 1); - ParseAssert.failed(Owid.tryParseBytes(longer), + ParseAssert.failed(Owid.parse(longer), OwidParseStatus.BYTE_COUNT_MISMATCH); } @@ -191,7 +191,7 @@ void trailingByteAfterSignatureRefused() { void shortSignatureRefused() { byte[] bytes = envelope(PAYLOAD.length, PAYLOAD, filled(SIGNATURE_LENGTH - 1, (byte) 0x99)); - ParseAssert.failed(Owid.tryParseBytes(bytes), + ParseAssert.failed(Owid.parse(bytes), OwidParseStatus.BYTE_COUNT_MISMATCH); } @@ -214,7 +214,7 @@ void mismatchedLargeDeclarationRefusedWithoutAllocating() { for (long declared : declaredLengths) { byte[] bytes = envelope(declared, new byte[0], new byte[0]); long before = allocatedBytes(); - OwidParseResult result = Owid.tryParseBytes(bytes); + OwidParseResult result = Owid.parse(bytes); long allocated = allocatedBytes() - before; ParseAssert.failed(result, OwidParseStatus.BYTE_COUNT_MISMATCH); assertTrue(allocated < ALLOCATION_BOUND, "declared " + declared @@ -230,7 +230,7 @@ void mismatchedLargeDeclarationRefusedWithoutAllocating() { @Test void emptyPayloadParses() { Owid owid = ParseAssert.parsed( - Owid.tryParseBytes(envelope(0, new byte[0], SIGNATURE))); + Owid.parse(envelope(0, new byte[0], SIGNATURE))); assertEquals(0, owid.getPayload().length, "should read an empty payload"); assertArrayEquals(SIGNATURE, owid.getSignature(), diff --git a/src/test/java/com/swancommunity/owid/SignatureStatusTest.java b/src/test/java/com/swancommunity/owid/SignatureStatusTest.java index cafa40f..581442b 100644 --- a/src/test/java/com/swancommunity/owid/SignatureStatusTest.java +++ b/src/test/java/com/swancommunity/owid/SignatureStatusTest.java @@ -32,6 +32,12 @@ * unjudged. Reporting that as invalid would tell a caller an identifier had * been tampered with when all that happened is an outage, and a caller acting * on it would reject good identifiers.

+ * + *

Every member of {@link OwidSignatureStatus} is exercised here except + * {@link OwidSignatureStatus#IMPLEMENTATION_CAPACITY_EXCEEDED}, which needs + * an OWID and its chain to approach the two gigabyte limit of a Java array + * and so cannot be built in a suite that has to run on an ordinary machine. + * The reason is recorded on the member itself as well.

*/ class SignatureStatusTest { @@ -48,7 +54,7 @@ void genuineSignatureIsValid() throws OwidException { Owid owid = Creator.create("example.com", crypto) .createString("payload"); - OwidVerificationResult result = owid.verifyDetailed(crypto, NONE); + OwidVerificationResult result = owid.verify(crypto, NONE); assertTrue(result.isValid(), "a genuine signature should be valid"); assertEquals(OwidSignatureStatus.SIGNATURE_VALID, result.getStatus(), @@ -62,7 +68,7 @@ void genuineSignatureIsValidThroughPem() throws OwidException { Owid owid = Creator.create("example.com", crypto) .createString("payload"); - OwidVerificationResult result = owid.verifyDetailedWithPublicKey( + OwidVerificationResult result = owid.verify( crypto.publicKeyPem(), NONE); assertEquals(OwidSignatureStatus.SIGNATURE_VALID, result.getStatus(), @@ -79,7 +85,7 @@ void wrongKeyIsSignatureInvalid() throws OwidException { .createString("payload"); OwidVerificationResult result = - owid.verifyDetailed(crypto(), NONE); + owid.verify(crypto(), NONE); assertFalse(result.isValid(), "the signature should not be valid"); assertEquals(OwidSignatureStatus.SIGNATURE_INVALID, result.getStatus(), @@ -96,13 +102,13 @@ void noKeyIsKeyUnavailable() throws OwidException { .createString("payload"); assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, - owid.verifyDetailed(null, NONE).getStatus(), + owid.verify((Crypto) null, NONE).getStatus(), "a missing crypto instance should not judge the signature"); assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, - owid.verifyDetailedWithPublicKey(null, NONE).getStatus(), + owid.verify((String) null, NONE).getStatus(), "a missing PEM should not judge the signature"); assertEquals(OwidSignatureStatus.KEY_UNAVAILABLE, - owid.verifyDetailedWithPublicKey(" ", NONE).getStatus(), + owid.verify(" ", NONE).getStatus(), "an empty PEM should not judge the signature"); } @@ -119,11 +125,11 @@ void undecodableKeyIsInvalidKey() throws OwidException { .createString("payload"); assertEquals(OwidSignatureStatus.INVALID_KEY, - owid.verifyDetailedWithPublicKey("not a PEM", NONE) + owid.verify("not a PEM", NONE) .getStatus(), "material that is not a key should be reported as the key"); assertEquals(OwidSignatureStatus.INVALID_KEY, - owid.verifyDetailedWithPublicKey( + owid.verify( "-----BEGIN PUBLIC KEY-----\nAAAA\n" + "-----END PUBLIC KEY-----\n", NONE) .getStatus(), @@ -132,17 +138,30 @@ void undecodableKeyIsInvalidKey() throws OwidException { /** * A signature field that is not the length the version requires cannot be - * checked. The marker for an absent optional OWID is the one thing that - * reaches a verification surface with no signature at all. + * checked, and saying so is not the same as saying the signature is + * wrong. + * + *

The OWID is built here through the package private constructor, + * because a consumer cannot produce one: both routes an OWID arrives by, + * being a read and a creator, settle the signature at 64 bytes. The + * status is part of the cross language vocabulary and other surfaces can + * be handed a signature field on its own, so the branch is exercised from + * inside the package where it can be reached.

*/ @Test - void missingSignatureIsInvalidSignatureLength() throws OwidException { - Owid marker = ParseAssert.parsed( - Owid.tryParseBytes(Owid.emptyByteArray())); + void wrongLengthSignatureIsInvalidSignatureLength() throws OwidException { + Owid noSignature = new Owid(Version.current(), "example.com", + Io.baseDate(), new byte[0], new byte[0]); + Owid shortSignature = new Owid(Version.current(), "example.com", + Io.baseDate(), new byte[0], + Envelope.filled(Owid.SIGNATURE_LENGTH - 1, (byte) 1)); assertEquals(OwidSignatureStatus.INVALID_SIGNATURE_LENGTH, - marker.verifyDetailed(crypto(), NONE).getStatus(), + noSignature.verify(crypto(), NONE).getStatus(), "no signature is not the same as a signature that is wrong"); + assertEquals(OwidSignatureStatus.INVALID_SIGNATURE_LENGTH, + shortSignature.verify(crypto(), NONE).getStatus(), + "a 63 byte signature is not a signature that is wrong"); } /** @@ -162,7 +181,7 @@ void unencodableFieldIsVerificationError() throws OwidException { Envelope.filled(Owid.SIGNATURE_LENGTH, (byte) 1)); assertEquals(OwidSignatureStatus.VERIFICATION_ERROR, - owid.verifyDetailed(crypto(), NONE).getStatus(), + owid.verify(crypto(), NONE).getStatus(), "a field that cannot be encoded is not an invalid signature"); } diff --git a/src/test/java/com/swancommunity/owid/WireVectorsTest.java b/src/test/java/com/swancommunity/owid/WireVectorsTest.java index 719008c..917afd8 100644 --- a/src/test/java/com/swancommunity/owid/WireVectorsTest.java +++ b/src/test/java/com/swancommunity/owid/WireVectorsTest.java @@ -61,7 +61,7 @@ private static byte[] decode(String value) { @Test void vectorsReadFromUnpaddedBase64() { for (String vector : new String[] {CREATOR, SUPPLIER, BAD}) { - Owid owid = ParseAssert.parsed(Owid.tryParse(vector)); + Owid owid = ParseAssert.parsed(Owid.parse(vector)); assertArrayEquals(decode(vector), assertDoesNotThrow(owid::asByteArray), "should read the same bytes from the encoded form"); @@ -71,7 +71,7 @@ void vectorsReadFromUnpaddedBase64() { @Test void creatorRoundTripsByteExact() throws OwidException { byte[] original = decode(CREATOR); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); + Owid owid = ParseAssert.parsed(Owid.parse(original)); assertEquals("51db.uk", owid.getDomain(), "should read the domain"); assertEquals(Version.VERSION2, owid.getVersion(), "should read version 2"); @@ -84,7 +84,7 @@ void creatorRoundTripsByteExact() throws OwidException { @Test void supplierRoundTripsByteExact() throws OwidException { byte[] original = decode(SUPPLIER); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); + Owid owid = ParseAssert.parsed(Owid.parse(original)); assertEquals("pop-up.swan-demo.uk", owid.getDomain(), "should read the domain"); assertArrayEquals(new byte[] {0x01, 0x03}, owid.getPayload(), @@ -100,7 +100,7 @@ void supplierRoundTripsByteExact() throws OwidException { @Test void badParsesAndRoundTrips() throws OwidException { byte[] original = decode(BAD); - Owid owid = ParseAssert.parsed(Owid.tryParseBytes(original)); + Owid owid = ParseAssert.parsed(Owid.parse(original)); assertEquals("badssp.swan-demo.uk", owid.getDomain(), "should read the domain"); assertArrayEquals(original, owid.asByteArray(), From 4aa28c9516028b8daa37cd89533cf06fa4c2f929 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 07:45:09 +0100 Subject: [PATCH 3/5] Read one OWID out of something longer Every port gets a public framed read, and Java had none since the only reader that did it was package private and went with the throwing parser. Owid.parse takes a ByteBuffer, which is how Java says read one item and move along. It reads one envelope from where the buffer is positioned, moves the buffer to the first byte after it, and leaves whatever follows for the next read. OwidParseResult.getByteCount reports the same distance for a caller that would rather do the arithmetic itself, and the whole buffer read reports it too, where it is the length of the buffer. The two contracts differ in one place. A whole buffer holds one envelope and nothing else, so the declared payload has to leave exactly the signature and a byte after it is a byte count mismatch. A frame only requires the declared payload and the signature to be present, because what follows is the next frame rather than rubbish. A frame that runs past the bytes supplied is reported as a truncation rather than as a byte count disagreement, since a caller reading from a source that is still arriving needs to know whether to wait for more bytes or to give up on these, and those are different answers. Nothing is consumed by a failed read, so the buffer is left at the start of the frame that failed and the caller decides what to do with it. The empty marker is refused by the framed read as well, for the reason it is refused everywhere else, being that handing one back puts an OWID in a caller's hands that nothing has ever signed. A caller walking a stream that carries markers has to skip them itself, there being no status in the shared vocabulary that means an absent node. That is worth settling across the ports. Buffers with no array a caller may reach, being direct and read only ones, are read from a copy of what remains. Wrapped arrays, which is the ordinary case, are read in place. The status coverage was rechecked against the new surface and is unchanged, with the reasons on the three untestable members widened to cover both contracts. A parse result now asserts its byte count in every test, because the check that a failed read consumes nothing turned out to measure nothing on its own, the arithmetic being zero either way. --- README.md | 66 +++- .../java/com/swancommunity/owid/Owid.java | 106 ++++++- .../swancommunity/owid/OwidParseResult.java | 34 ++- .../swancommunity/owid/OwidParseStatus.java | 44 ++- .../com/swancommunity/owid/OwidReader.java | 77 +++-- .../owidconsumer/ReadmeExampleTest.java | 37 +++ .../swancommunity/owid/FramedReadTest.java | 286 ++++++++++++++++++ .../com/swancommunity/owid/ParseAssert.java | 7 + .../swancommunity/owid/ParseContractTest.java | 17 +- .../swancommunity/owid/PayloadLengthTest.java | 14 + 10 files changed, 614 insertions(+), 74 deletions(-) create mode 100644 src/test/java/com/swancommunity/owid/FramedReadTest.java diff --git a/README.md b/README.md index 722ab4d..482fc4a 100644 --- a/README.md +++ b/README.md @@ -135,9 +135,8 @@ party.verifyWithCrypto(crypto, Collections.emptyList()); // false An OWID is read from whatever a caller was handed, which on a public end point means anything at all, so malformed data is an ordinary outcome rather -than an exceptional one. `Owid.parse`, overloaded on the encoded string -and on the raw bytes, therefore -report three facts every time. +than an exceptional one. `Owid.parse`, overloaded on the encoded string, the +raw bytes and a `ByteBuffer`, therefore reports three facts every time. | Fact | Where | |------|-------| @@ -182,13 +181,52 @@ as the outage it is. | `IMPLEMENTATION_CAPACITY_EXCEEDED` | More work than this runtime can hold, which needs an OWID and its chain to approach the two gigabyte limit of a Java array. | | `VERIFICATION_ERROR` | The check could not be completed for a reason that is not the identifier's fault. | +## Reading one OWID out of something longer + +`Owid.parse(ByteBuffer)` is the framed read, for input that carries an OWID +inside something longer, such as a tree of them or a record with other fields +around it. It differs from the whole buffer read in one place. A whole buffer +has to end where the envelope does, so a byte after the signature belongs to +no field and is `BYTE_COUNT_MISMATCH`, whereas a frame only requires the +declared payload and the signature to be present and says nothing about what +follows, because what follows is the next frame rather than rubbish. + +```java +ByteBuffer buffer = ByteBuffer.wrap(bytes); +while (buffer.hasRemaining()) { + OwidParseResult result = Owid.parse(buffer); + if (result.isSuccess() == false) { + // buffer is still at the start of the frame that failed, and + // result.getStatus() says why. + break; + } + use(result.getValue()); +} +``` + +On success the buffer moves on to the first byte after the envelope, so +calling `parse` again reads the next one, and `getByteCount()` on the result +reports how far it moved. On failure the buffer is left exactly where it was +and nothing is consumed, because a half read frame leaves a caller somewhere +it cannot reason about, so what to do with a bad frame is the caller's to +decide. + +`UNEXPECTED_END` from the framed read means the frame runs past the bytes +supplied, so a caller reading from a source that is still arriving can wait +for more and read again from the same position. + +Buffers that carry no array a caller may reach, being direct and read only +ones, are read from a copy of what remains rather than in place. Callers +wrapping an array, which is the ordinary case, are read without any copy. + ## How an OWID comes into being An OWID is only worth anything because it is signed, so a caller cannot build one. There is no public constructor and no setter, and an instance reaches calling code by exactly two routes. -1. A successful read of a complete serialized OWID, through `Owid.parse`. +1. A successful read of a complete serialized OWID, through any `Owid.parse` + overload. 2. A creator signing one into existence, through `createString` or `createBytes`. @@ -230,8 +268,10 @@ domain, a null payload, or a field that cannot be serialized. - `Owid` holds the version, domain, date to the minute in UTC, payload bytes, and signature bytes, all read only. - - `Owid.parse` reads a signed OWID, from the encoded string or from the - raw bytes, and reports why rather than throwing. + - `Owid.parse` reads a signed OWID, from the encoded string, from the raw + bytes, or from a `ByteBuffer` holding one frame of something longer, and + reports why rather than throwing. `getByteCount` on the result says how + many bytes the envelope occupied. - `asBase64` and `asByteArray` serialize a signed OWID. - `payloadAsString` decodes the payload as UTF-8. `payloadAsPrintable` returns zero padded lower case hexadecimal with no separator. @@ -292,11 +332,11 @@ OWID in the order provided when signing. An absent optional OWID is written as the single byte `0x00`, which `Owid.emptyByteArray` produces, and it marks the absence of a node inside a -larger framed byte array. `Owid.parse` refuses it with -`UNSUPPORTED_VERSION`, because a whole buffer holding nothing but the marker -holds no OWID, and handing one back would put an OWID in a caller's hands -that nothing had ever signed. Only a framed reader, which this library does -not have, can make sense of the marker. +larger framed byte array. Every `Owid.parse` overload refuses it with +`UNSUPPORTED_VERSION`, the framed one included, because handing one back +would put an OWID in a caller's hands that nothing had ever signed. A caller +walking a stream that carries markers therefore has to skip them itself, +there being no status in the shared vocabulary that means an absent node. Base 64 decoding accepts the standard alphabet with or without the trailing padding, and skips line breaks and spaces. Anything else in the string is @@ -315,8 +355,8 @@ cross language signed fixtures including the chained case, confirm that a flipped signature byte fails verification, and cover the binary write helpers, the crypto, the creator, and the end point helpers. They also cover the parse contract, being every status the reading surfaces report together -with a run of malformed buffers that must never throw, and the construction -boundary, which is checked from a package outside the library because a check +with a run of malformed buffers that must never throw, the framed read and +what it consumes, and the construction boundary, which is checked from a package outside the library because a check made from inside it would measure nothing. ## License diff --git a/src/main/java/com/swancommunity/owid/Owid.java b/src/main/java/com/swancommunity/owid/Owid.java index 1d181f0..471039b 100644 --- a/src/main/java/com/swancommunity/owid/Owid.java +++ b/src/main/java/com/swancommunity/owid/Owid.java @@ -17,6 +17,8 @@ package com.swancommunity.owid; import java.io.ByteArrayOutputStream; +import java.nio.Buffer; +import java.nio.ByteBuffer; import java.nio.charset.StandardCharsets; import java.time.Duration; import java.time.Instant; @@ -33,8 +35,9 @@ * *

An OWID is only worth anything because it is signed, so a caller cannot * build one. An instance reaches calling code by one of two routes, being - * {@link #parse(String)} or {@link #parse(byte[])} reading bytes that were - * already a complete OWID, or {@link Creator#createBytes(byte[])} + * {@link #parse(String)}, {@link #parse(byte[])} or + * {@link #parse(ByteBuffer)} reading bytes that were already a complete + * OWID, or {@link Creator#createBytes(byte[])} * and its companions signing one into existence. There is deliberately no way * to assemble a half made one, because an unsigned OWID is indistinguishable * from a signed one to the code downstream of it and the difference only @@ -120,22 +123,101 @@ public static OwidParseResult parse(String value) { if (buffer == null) { return OwidParseResult.failed(OwidParseStatus.INVALID_BASE64); } - return OwidReader.read(buffer); + return OwidReader.read(buffer, 0, buffer.length, false); } /** * Reads a complete OWID from a buffer holding exactly one. * - *

The buffer must be one whole OWID and nothing else. Bytes after the - * envelope are refused, because this library has no framed reader and so - * there is nothing else they could belong to.

+ *

The buffer must be one whole OWID and nothing else, so bytes after + * the envelope are refused as {@link OwidParseStatus#BYTE_COUNT_MISMATCH} + * because on this surface there is nothing else they could belong to. To + * read one envelope out of something longer, and leave what follows for + * the next read, use {@link #parse(ByteBuffer)}.

* * @param buffer the serialized OWID bytes, which may be null * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and * the reason the bytes are not an OWID */ public static OwidParseResult parse(byte[] buffer) { - return OwidReader.read(buffer); + if (buffer == null) { + return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); + } + return OwidReader.read(buffer, 0, buffer.length, false); + } + + /** + * Reads one OWID from where the buffer is positioned, leaving whatever + * follows for the next read. + * + *

This is the framed read, for input that carries an OWID inside + * something longer, such as a tree of them or a record with other fields + * around it. It differs from {@link #parse(byte[])} in one place only. + * A whole buffer has to end where the envelope does, so a byte after the + * signature belongs to no field, whereas here the declared payload and + * the signature only have to be present and what follows is the next + * frame rather than rubbish.

+ * + *

On success the buffer is moved on to the first byte after the + * envelope, so calling this again reads the next one, and + * {@link OwidParseResult#getByteCount()} reports how far it moved. On + * failure the buffer is left exactly where it was and nothing is + * consumed, because a half read frame leaves a caller somewhere it cannot + * reason about, so what to do with a bad frame is the caller's to + * decide.

+ * + * 
+     * ByteBuffer buffer = ByteBuffer.wrap(bytes);
+     * while (buffer.hasRemaining()) {
+     *     OwidParseResult result = Owid.parse(buffer);
+     *     if (result.isSuccess() == false) {
+     *         break;
+     *     }
+     *     use(result.getValue());
+     * }
+     *
+ * + *

{@link OwidParseStatus#UNEXPECTED_END} here means the frame runs + * past the bytes supplied, so a caller reading from a growing source can + * wait for more and read again from the same position.

+ * + * @param buffer the bytes to read from, which may be null + * @return the OWID and {@link OwidParseStatus#PARSED}, or no value and + * the reason the bytes are not an OWID + */ + public static OwidParseResult parse(ByteBuffer buffer) { + if (buffer == null || buffer.hasRemaining() == false) { + return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); + } + OwidParseResult result; + if (buffer.hasArray()) { + int base = buffer.arrayOffset(); + result = OwidReader.read(buffer.array(), + base + buffer.position(), base + buffer.limit(), true); + } else { + // A direct or read only buffer has no array to walk, so the bytes + // are taken a copy of. Ordinary callers wrap an array and never + // reach this, and the copy is of what remains rather than of the + // envelope, because how long the envelope is cannot be known + // until it has been read. + byte[] remaining = new byte[buffer.remaining()]; + ByteBuffer view = buffer.duplicate(); + view.get(remaining); + result = OwidReader.read(remaining, 0, remaining.length, true); + } + if (result.isSuccess()) { + // Only a successful read moves the buffer on. A failed read + // reports consuming nothing, so the arithmetic alone would leave + // the buffer where it was, and this says so outright rather than + // resting on that. + // + // Buffer.position is called rather than ByteBuffer.position + // because the covariant override arrived in Java 9 and this + // library is built for 8. + ((Buffer) buffer).position( + buffer.position() + result.getByteCount()); + } + return result; } /** @@ -172,11 +254,13 @@ void toBuffer(ByteArrayOutputStream buffer) throws OwidException { * Writes the marker for an absent optional OWID, being the single byte * zero, for embedding in a larger framed byte array. * - *

Reading it back through {@link #parse(byte[])} reports + *

Every reading surface here, framed included, refuses it as * {@link OwidParseStatus#UNSUPPORTED_VERSION}, because the marker stands - * for the absence of an identifier rather than for one, and a whole - * buffer holding nothing but the marker holds no OWID. Only a framed - * reader, which this library does not have, can make sense of it.

+ * for the absence of an identifier rather than for one, and handing back + * an OWID with no domain, no date and no signature would put one in a + * caller's hands that nothing had ever signed. A caller walking a stream + * that carries markers therefore has to skip them itself, there being no + * status in the shared vocabulary that means an absent node.

* * @return a single byte array holding the empty marker */ diff --git a/src/main/java/com/swancommunity/owid/OwidParseResult.java b/src/main/java/com/swancommunity/owid/OwidParseResult.java index 9cd24f6..48705d1 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseResult.java +++ b/src/main/java/com/swancommunity/owid/OwidParseResult.java @@ -39,19 +39,26 @@ public final class OwidParseResult { private final OwidParseStatus status; - private OwidParseResult(Owid value, OwidParseStatus status) { + private final int byteCount; + + private OwidParseResult(Owid value, OwidParseStatus status, + int byteCount) { this.value = value; this.status = status; + this.byteCount = byteCount; } - /** The result of a read that produced the OWID given. */ - static OwidParseResult parsed(Owid value) { - return new OwidParseResult(value, OwidParseStatus.PARSED); + /** + * The result of a read that produced the OWID given, occupying the number + * of bytes given. + */ + static OwidParseResult parsed(Owid value, int byteCount) { + return new OwidParseResult(value, OwidParseStatus.PARSED, byteCount); } /** The result of a read that failed for the reason given. */ static OwidParseResult failed(OwidParseStatus status) { - return new OwidParseResult(null, status); + return new OwidParseResult(null, status, 0); } /** @@ -87,6 +94,23 @@ public OwidParseStatus getStatus() { return status; } + /** + * How many bytes the envelope occupied, or zero when the read failed. + * + *

This is what a caller reading one frame after another needs in order + * to find the next one. {@link Owid#parse(java.nio.ByteBuffer)} moves the + * buffer along by this much itself, so a caller using that surface does + * not have to. Reading a whole buffer this is the length of the buffer, + * because there the envelope is the whole of it.

+ * + *

Zero on failure, since a read that failed consumed nothing.

+ * + * @return the length of the envelope in bytes, or zero + */ + public int getByteCount() { + return byteCount; + } + /** * The status name on its own. The input is deliberately absent, because * a parse failure is often logged and the bytes came from outside. diff --git a/src/main/java/com/swancommunity/owid/OwidParseStatus.java b/src/main/java/com/swancommunity/owid/OwidParseStatus.java index 19e7f31..8e0e0cc 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseStatus.java +++ b/src/main/java/com/swancommunity/owid/OwidParseStatus.java @@ -60,6 +60,11 @@ public enum OwidParseStatus { * The data stopped in the middle of a field. Different from * {@link #BYTE_COUNT_MISMATCH}, which is a declaration disagreeing with * data that is all present. + * + *

Reading one frame out of something longer, this also covers a frame + * whose declared payload and signature run past the bytes supplied, so a + * caller reading from a source that is still arriving can wait for more + * and read again from the same place.

*/ UNEXPECTED_END, @@ -74,16 +79,26 @@ public enum OwidParseStatus { * present. Checked before anything is sized by the declaration, so a * sender cannot make a reader allocate by claiming a large payload it * did not send. + * + *

Only the whole buffer read reports this, because only there does the + * envelope have to end where the input does. Reading one frame out of + * something longer, bytes after the signature are the next frame rather + * than a disagreement, and a frame that runs past the input is + * {@link #UNEXPECTED_END}.

*/ BYTE_COUNT_MISMATCH, /** * The envelope is structurally consistent but larger than this runtime * can hold. Deliberately apart from the data being wrong, because the - * same bytes may be readable elsewhere. A Java byte array cannot hold - * more than {@link Integer#MAX_VALUE} bytes, so a declaration larger - * than that can never agree with the bytes present and this status is - * not reachable from the byte array surface. + * same bytes may be readable elsewhere. + * + *

Not reachable in Java, and so not covered by a test. A Java byte + * array cannot hold more than {@link Integer#MAX_VALUE} bytes, so a + * declaration larger than that can neither equal the bytes present, which + * is what the whole buffer read requires, nor be covered by them, which + * is what the framed read requires. The guard is kept so a future change + * to that arithmetic cannot silently truncate the declaration.

*/ IMPLEMENTATION_CAPACITY_EXCEEDED, @@ -92,16 +107,17 @@ public enum OwidParseStatus { * fallback for the genuinely unclassified, not a substitute for naming a * failure that is already understood. * - *

Not reachable in Java today, and so not covered by a test. Every - * failure this reader can meet is classified by one of the members above, - * and the one place that still reports this is the check that the - * envelope ended where the buffer did, which cannot fire while the - * declared payload count has already been required to leave exactly the - * signature. That check is kept as a backstop rather than removed, - * because a future change to the count arithmetic would otherwise start - * accepting bytes after the signature in silence. Loosening the count - * rule during a deliberate check of the tests made this status appear, so - * the backstop does work.

+ *

Not reachable in Java today, on either reading surface, and so not + * covered by a test. Every failure these readers can meet is classified + * by one of the members above. The one place that still reports this is + * the check that the envelope ended where the input did, which the framed + * read does not apply at all and which cannot fire on the whole buffer + * read while the declared payload count has already been required to + * leave exactly the signature. That check is kept as a backstop rather + * than removed, because a future change to the count arithmetic would + * otherwise start accepting bytes after the signature in silence. + * Loosening the count rule during a deliberate check of the tests made + * this status appear, so the backstop does work.

*/ MALFORMED_ENVELOPE } diff --git a/src/main/java/com/swancommunity/owid/OwidReader.java b/src/main/java/com/swancommunity/owid/OwidReader.java index d4cb153..c4ed0fe 100644 --- a/src/main/java/com/swancommunity/owid/OwidReader.java +++ b/src/main/java/com/swancommunity/owid/OwidReader.java @@ -35,26 +35,36 @@ * would still be built and unwound, so the cost would remain and only the * surface would look different.

* - *

This is the exact buffer contract, meaning the envelope has to end - * where the buffer does. The library has no framed reader, so there is - * nothing that later bytes could belong to.

+ *

The same walk serves both reading contracts, which differ in one place + * only. A whole buffer holds one envelope and nothing else, so the declared + * payload has to leave exactly the signature at the end and a byte after it + * belongs to no field. A frame is one envelope inside something longer, so + * the declared payload and the signature only have to be present, and what + * follows is the next frame rather than rubbish.

*/ final class OwidReader { private OwidReader() { } - /** Reads one complete OWID occupying the whole of the buffer. */ - static OwidParseResult read(byte[] buffer) { + /** + * Reads one OWID from the region of the buffer between the two offsets. + * + * @param framed false when the region holds one envelope and nothing + * else, so the envelope has to end where the region does, + * and true when the envelope is one frame inside something + * longer and whatever follows is the next frame + */ + static OwidParseResult read(byte[] buffer, int from, int total, + boolean framed) { // Nothing supplied is not the same as data that stopped part way - // through a field, so a buffer with no bytes in it is reported as the + // through a field, so a region with no bytes in it is reported as the // absence it is rather than as a truncation. - if (buffer == null || buffer.length == 0) { + if (buffer == null || total - from <= 0) { return OwidParseResult.failed(OwidParseStatus.MISSING_INPUT); } - int total = buffer.length; - Version version = Version.forByte(buffer[0] & 0xFF); + Version version = Version.forByte(buffer[from] & 0xFF); if (version == null || version == Version.EMPTY) { // The empty marker, being the single byte zero, is refused here // along with the versions this implementation does not know. It @@ -63,12 +73,14 @@ static OwidParseResult read(byte[] buffer) { // no signature, and handing one back would put an OWID in a // caller's hands that nothing had ever signed. That is the state // the construction boundary exists to prevent, and it could never - // verify. A framed reader, which this library does not have, - // would still read the marker as the absence it means. + // verify. The framed read refuses it for the same reason, so a + // caller walking a stream that carries markers has to skip them + // itself, there being no status in the shared vocabulary that + // means an absent node. return OwidParseResult.failed( OwidParseStatus.UNSUPPORTED_VERSION); } - int at = 1; + int at = from + 1; // The domain, terminated by a zero byte and no longer than the // maximum published for a domain name. The walk stops at that @@ -135,14 +147,25 @@ static OwidParseResult read(byte[] buffer) { // rather than wrapping, and a negative count can never equal a // declaration. // - // The disagreement is the finding even when the buffer also stopped - // early. What a reader can say for certain is that the declared - // payload cannot leave exactly the signature the version requires, - // and that is true whichever way the bytes fall short. Reporting it - // as a truncation instead would name a different fault for the same - // evidence. + // Reading a whole buffer, the disagreement is the finding even when + // the buffer also stopped early. What a reader can say for certain is + // that the declared payload cannot leave exactly the signature the + // version requires, and that is true whichever way the bytes fall + // short. Reporting it as a truncation instead would name a different + // fault for the same evidence. + // + // Reading a frame, only a shortfall is a finding, since a longer + // input is the next frame rather than a disagreement. A frame that + // runs past what is here is reported as a truncation, because a + // caller walking a stream needs to know whether to wait for more + // bytes or to give up on these, and those are different answers. long present = (long) (total - at) - Owid.SIGNATURE_LENGTH; - if (present != declared) { + if (framed) { + if (present < declared) { + return OwidParseResult.failed( + OwidParseStatus.UNEXPECTED_END); + } + } else if (present != declared) { return OwidParseResult.failed( OwidParseStatus.BYTE_COUNT_MISMATCH); } @@ -150,10 +173,10 @@ static OwidParseResult read(byte[] buffer) { // The bytes are all here. Whether this runtime can hold them in one // array is a separate question with a different answer, because the // same envelope may be readable elsewhere. A Java byte array cannot - // exceed Integer.MAX_VALUE, so the count above can never agree with - // a larger declaration and this cannot fire today. It is kept so a - // future change to that arithmetic cannot silently truncate the cast - // below. + // exceed Integer.MAX_VALUE, so neither contract above can be + // satisfied by a larger declaration and this cannot fire today. It is + // kept so a future change to that arithmetic cannot silently truncate + // the cast below. if (declared > Integer.MAX_VALUE) { return OwidParseResult.failed( OwidParseStatus.IMPLEMENTATION_CAPACITY_EXCEEDED); @@ -168,16 +191,18 @@ static OwidParseResult read(byte[] buffer) { System.arraycopy(buffer, at, signature, 0, Owid.SIGNATURE_LENGTH); at += Owid.SIGNATURE_LENGTH; - if (at != total) { + if (framed == false && at != total) { // Unreachable while the count check above holds, and kept so // that a future change to that arithmetic cannot silently start - // accepting bytes after the signature. + // accepting bytes after the signature. A frame says nothing about + // what follows it, so the check does not apply there. return OwidParseResult.failed( OwidParseStatus.MALFORMED_ENVELOPE); } return OwidParseResult.parsed( - new Owid(version, domain, date, payload, signature)); + new Owid(version, domain, date, payload, signature), + at - from); } /** diff --git a/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java index 06b0555..06e72ad 100644 --- a/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java +++ b/src/test/java/com/example/owidconsumer/ReadmeExampleTest.java @@ -25,7 +25,10 @@ import com.swancommunity.owid.Owid; import com.swancommunity.owid.OwidException; import com.swancommunity.owid.OwidParseResult; +import java.nio.ByteBuffer; +import java.util.ArrayList; import java.util.Collections; +import java.util.List; import org.junit.jupiter.api.Test; /** @@ -76,6 +79,40 @@ void createSerializeReadBackAndVerify() throws OwidException { } } + /** + * The framed loop from the README, reading two OWIDs written one after + * the other into the same array. + */ + @Test + void readingOneOwidAfterAnother() throws OwidException { + Crypto crypto = Crypto.generate(); + Creator creator = Creator.create("example.com", crypto); + byte[] one = creator.createString("one").asByteArray(); + byte[] two = creator.createString("two").asByteArray(); + byte[] bytes = new byte[one.length + two.length]; + System.arraycopy(one, 0, bytes, 0, one.length); + System.arraycopy(two, 0, bytes, one.length, two.length); + + List payloads = new ArrayList(); + + ByteBuffer buffer = ByteBuffer.wrap(bytes); + while (buffer.hasRemaining()) { + OwidParseResult result = Owid.parse(buffer); + if (result.isSuccess() == false) { + // buffer is still at the start of the frame that failed, and + // result.getStatus() says why. + break; + } + payloads.add(result.getValue().payloadAsString()); + } + + assertEquals(2, payloads.size(), "should have read both OWIDs"); + assertEquals("one", payloads.get(0), "should read the first payload"); + assertEquals("two", payloads.get(1), "should read the second payload"); + assertFalse(buffer.hasRemaining(), + "the two OWIDs should account for every byte"); + } + @Test void chainingCoversTheOtherOwids() throws OwidException { Crypto crypto = Crypto.generate(); diff --git a/src/test/java/com/swancommunity/owid/FramedReadTest.java b/src/test/java/com/swancommunity/owid/FramedReadTest.java new file mode 100644 index 0000000..81c2cfe --- /dev/null +++ b/src/test/java/com/swancommunity/owid/FramedReadTest.java @@ -0,0 +1,286 @@ +/* **************************************************************************** + * Copyright 2026 51 Degrees Mobile Experts Limited (51degrees.com) + * + * Licensed under the Apache License, Version 2.0 (the "License"); you may not + * use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT + * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the + * License for the specific language governing permissions and limitations + * under the License. + * ***************************************************************************/ + +package com.swancommunity.owid; + +import static org.junit.jupiter.api.Assertions.assertArrayEquals; +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import java.io.ByteArrayOutputStream; +import java.nio.Buffer; +import java.nio.ByteBuffer; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; +import org.junit.jupiter.api.Test; + +/** + * Reading one OWID out of something longer, and leaving what follows alone. + * + *

The framed read differs from the whole buffer read in one place. A whole + * buffer has to end where the envelope does, so a byte after the signature + * belongs to no field, whereas a frame only requires the declared payload and + * the signature to be present and says nothing about what follows, because + * what follows is the next frame rather than rubbish.

+ */ +class FramedReadTest { + + /** The two envelopes used throughout, with payloads that differ. */ + private static final byte[] FIRST = + Envelope.version3("first.example", 1000L, 3, + new byte[] {1, 2, 3}, Envelope.signature()); + + private static final byte[] SECOND = + Envelope.version3("second.example", 2000L, 5, + new byte[] {4, 5, 6, 7, 8}, Envelope.signature()); + + private static byte[] concatenated() { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + stream.write(FIRST, 0, FIRST.length); + stream.write(SECOND, 0, SECOND.length); + return stream.toByteArray(); + } + + /** + * Two complete envelopes one after the other read as two OWIDs from the + * same input, and between them they account for every byte. + */ + @Test + void twoEnvelopesReadOneAfterTheOther() { + ByteBuffer buffer = ByteBuffer.wrap(concatenated()); + + List read = new ArrayList(); + while (buffer.hasRemaining()) { + read.add(ParseAssert.parsed(Owid.parse(buffer))); + } + + assertEquals(2, read.size(), "should have read both envelopes"); + assertEquals("first.example", read.get(0).getDomain(), + "should read the first domain"); + assertArrayEquals(new byte[] {1, 2, 3}, read.get(0).getPayload(), + "should read the first payload"); + assertEquals("second.example", read.get(1).getDomain(), + "should read the second domain"); + assertArrayEquals(new byte[] {4, 5, 6, 7, 8}, + read.get(1).getPayload(), "should read the second payload"); + assertFalse(buffer.hasRemaining(), + "the two envelopes should account for every byte"); + } + + /** + * Each read reports how far it moved, and moves the buffer by exactly + * that much, so a caller can find the next frame either way. + */ + @Test + void eachReadReportsAndConsumesTheEnvelopeLength() { + ByteBuffer buffer = ByteBuffer.wrap(concatenated()); + + OwidParseResult first = Owid.parse(buffer); + ParseAssert.parsed(first); + assertEquals(FIRST.length, first.getByteCount(), + "should report the length of the first envelope"); + assertEquals(FIRST.length, buffer.position(), + "should move on by the length of the first envelope"); + + OwidParseResult second = Owid.parse(buffer); + ParseAssert.parsed(second); + assertEquals(SECOND.length, second.getByteCount(), + "should report the length of the second envelope"); + assertEquals(FIRST.length + SECOND.length, buffer.position(), + "should move on by the length of the second envelope"); + } + + /** + * The same two envelopes handed to the whole buffer read are refused, + * because there nothing else could own the bytes after the first + * signature. + */ + @Test + void theSameTwoEnvelopesAreRefusedByTheWholeBufferRead() { + ParseAssert.failed(Owid.parse(concatenated()), + OwidParseStatus.BYTE_COUNT_MISMATCH); + } + + /** + * An envelope cut short before its signature is refused, and nothing is + * consumed, so the caller is left where it started and decides what to do + * with the bytes itself. + * + *

The status is a truncation rather than a byte count disagreement, + * because a caller reading from a source that is still arriving needs to + * know whether to wait for more bytes or to give up on these.

+ */ + @Test + void truncatedFrameIsRefusedAndConsumesNothing() { + byte[] cutShort = Arrays.copyOf(FIRST, FIRST.length - 1); + ByteBuffer buffer = ByteBuffer.wrap(cutShort); + + ParseAssert.failed(Owid.parse(buffer), OwidParseStatus.UNEXPECTED_END); + + assertEquals(0, buffer.position(), + "a failed read should leave the buffer where it was"); + assertEquals(cutShort.length, buffer.remaining(), + "a failed read should consume nothing"); + } + + /** + * A frame that is good but is followed by a bad one leaves the good one + * read and the buffer sitting at the start of the bad one, which is what + * lets a caller decide what to do about it. + */ + @Test + void aBadFrameLeavesThePositionAtItsStart() { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + stream.write(FIRST, 0, FIRST.length); + byte[] rubbish = {0x63, 0x63, 0x63}; + stream.write(rubbish, 0, rubbish.length); + ByteBuffer buffer = ByteBuffer.wrap(stream.toByteArray()); + + ParseAssert.parsed(Owid.parse(buffer)); + assertEquals(FIRST.length, buffer.position(), + "should have read the good frame"); + + ParseAssert.failed(Owid.parse(buffer), + OwidParseStatus.UNSUPPORTED_VERSION); + assertEquals(FIRST.length, buffer.position(), + "should leave the buffer at the start of the bad frame"); + } + + /** + * The framed read reports the same vocabulary as everything else, and it + * is reachable through the public surface rather than only from inside. + */ + @Test + void framedFailuresUseTheSharedVocabulary() { + // A version this implementation does not know. + byte[] badVersion = FIRST.clone(); + badVersion[0] = 0x04; + ParseAssert.failed(Owid.parse(ByteBuffer.wrap(badVersion)), + OwidParseStatus.UNSUPPORTED_VERSION); + + // A domain that runs past the published maximum before terminating. + StringBuilder tooLong = new StringBuilder(); + while (tooLong.length() <= Io.MAXIMUM_DOMAIN_LENGTH) { + tooLong.append('a'); + } + ParseAssert.failed( + Owid.parse(ByteBuffer.wrap( + Envelope.version3(tooLong.toString(), 1000L, 0, + new byte[0], Envelope.signature()))), + OwidParseStatus.INVALID_DOMAIN_ENCODING); + + // Nothing left to read. + ParseAssert.failed(Owid.parse(ByteBuffer.wrap(new byte[0])), + OwidParseStatus.MISSING_INPUT); + ParseAssert.failed(Owid.parse((ByteBuffer) null), + OwidParseStatus.MISSING_INPUT); + + // A declaration far larger than the bytes supplied. + // PayloadLengthTest measures that nothing is sized by it. + ParseAssert.failed( + Owid.parse(ByteBuffer.wrap( + Envelope.version3(Envelope.DOMAIN, 1000L, 0xFFFFFFFFL, + new byte[0], Envelope.signature()))), + OwidParseStatus.UNEXPECTED_END); + + // The marker for an absent node is refused here too, because handing + // one back would be an OWID nothing had ever signed. + ParseAssert.failed(Owid.parse(ByteBuffer.wrap(Owid.emptyByteArray())), + OwidParseStatus.UNSUPPORTED_VERSION); + } + + /** + * A frame read from a buffer that starts part way through an array, which + * is what a caller slicing a larger record hands over. + */ + @Test + void aBufferThatStartsPartWayThroughAnArrayReads() { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + byte[] prefix = {(byte) 0xEE, (byte) 0xEE}; + stream.write(prefix, 0, prefix.length); + stream.write(FIRST, 0, FIRST.length); + ByteBuffer buffer = ByteBuffer.wrap(stream.toByteArray()); + ((Buffer) buffer).position(prefix.length); + ByteBuffer sliced = buffer.slice(); + + Owid owid = ParseAssert.parsed(Owid.parse(sliced)); + + assertEquals("first.example", owid.getDomain(), + "should read the frame from where the slice starts"); + assertFalse(sliced.hasRemaining(), + "should have consumed the whole slice"); + } + + /** + * A direct buffer has no array to walk, so the bytes are taken a copy of. + * The answer has to be the same one. + */ + @Test + void aDirectBufferReadsTheSame() { + byte[] bytes = concatenated(); + ByteBuffer direct = ByteBuffer.allocateDirect(bytes.length); + direct.put(bytes); + ((Buffer) direct).flip(); + assertFalse(direct.hasArray(), + "the test needs a buffer with no array behind it"); + + Owid first = ParseAssert.parsed(Owid.parse(direct)); + assertEquals(FIRST.length, direct.position(), + "should move a direct buffer on as well"); + Owid second = ParseAssert.parsed(Owid.parse(direct)); + + assertEquals("first.example", first.getDomain(), + "should read the first domain from a direct buffer"); + assertEquals("second.example", second.getDomain(), + "should read the second domain from a direct buffer"); + assertFalse(direct.hasRemaining(), + "should have consumed the whole direct buffer"); + } + + /** + * A read only buffer has no array a caller may reach either, and must + * read the same way and stay read only. + */ + @Test + void aReadOnlyBufferReadsTheSame() { + ByteBuffer readOnly = ByteBuffer.wrap(FIRST).asReadOnlyBuffer(); + assertTrue(readOnly.isReadOnly(), "the buffer should be read only"); + + Owid owid = ParseAssert.parsed(Owid.parse(readOnly)); + + assertEquals("first.example", owid.getDomain(), + "should read from a read only buffer"); + assertFalse(readOnly.hasRemaining(), + "should have consumed the whole buffer"); + } + + /** + * The whole buffer read reports the length of the envelope too, which + * there is the whole of the buffer. + */ + @Test + void theWholeBufferReadAlsoReportsTheEnvelopeLength() { + OwidParseResult result = Owid.parse(FIRST); + + ParseAssert.parsed(result); + assertEquals(FIRST.length, result.getByteCount(), + "the envelope should be the whole of the buffer"); + assertEquals(0, Owid.parse(new byte[] {0x04}).getByteCount(), + "a read that failed should report consuming nothing"); + } +} diff --git a/src/test/java/com/swancommunity/owid/ParseAssert.java b/src/test/java/com/swancommunity/owid/ParseAssert.java index 858bfba..0db6408 100644 --- a/src/test/java/com/swancommunity/owid/ParseAssert.java +++ b/src/test/java/com/swancommunity/owid/ParseAssert.java @@ -46,6 +46,8 @@ static Owid parsed(OwidParseResult result) { "a successful read should report PARSED"); assertNotNull(result.getValue(), "a successful read should hand back the OWID"); + assertTrue(result.getByteCount() > 0, + "a successful read should report the bytes it consumed"); return result.getValue(); } @@ -61,5 +63,10 @@ static void failed(OwidParseResult result, OwidParseStatus expected) { "should report the reason the input is not an OWID"); assertNull(result.getValue(), "a failed read should hand back no OWID"); + // The framed read moves the buffer on by this much, so a failure + // reporting anything other than nothing would leave a caller part way + // through a frame it could not reason about. + assertEquals(0, result.getByteCount(), + "a failed read should report consuming nothing"); } } diff --git a/src/test/java/com/swancommunity/owid/ParseContractTest.java b/src/test/java/com/swancommunity/owid/ParseContractTest.java index f666091..110de81 100644 --- a/src/test/java/com/swancommunity/owid/ParseContractTest.java +++ b/src/test/java/com/swancommunity/owid/ParseContractTest.java @@ -24,6 +24,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import java.lang.reflect.Method; +import java.nio.ByteBuffer; import java.util.Arrays; import java.util.Base64; import java.util.Collections; @@ -38,16 +39,21 @@ * {@link ParseAssert} checks all three on every case here, so a test cannot * pass while the result contradicts itself.

* + *

These are the whole buffer surfaces, being the encoded string and the + * byte array. The framed surface, which reads one envelope out of something + * longer, is covered by {@link FramedReadTest}.

+ * *

Every member of {@link OwidParseStatus} is exercised here, with the * domain cases also covered in more depth by {@link DomainLengthTest} and the * byte count cases by {@link PayloadLengthTest}. Two members are not, and * cannot be, being {@link OwidParseStatus#INVALID_INPUT_TYPE}, which the * compiler already refuses, and * {@link OwidParseStatus#IMPLEMENTATION_CAPACITY_EXCEEDED}, which a Java byte - * array cannot reach. A third, + * array cannot reach on either surface. A third, * {@link OwidParseStatus#MALFORMED_ENVELOPE}, is a backstop with no path to - * it while the byte count rule holds. The reason is recorded on each of those - * members as well.

+ * it while the byte count rule holds, and the framed surface does not apply + * that check at all. The reason is recorded on each of those members as + * well.

*/ class ParseContractTest { @@ -289,13 +295,14 @@ void readingTakesNoKeyAndNoCrypto() { checked++; for (Class parameter : method.getParameterTypes()) { assertTrue(parameter == String.class - || parameter == byte[].class, + || parameter == byte[].class + || parameter == ByteBuffer.class, "reading should take only the data to read, but " + method.getName() + " takes " + parameter.getName()); } } - assertEquals(2, checked, "should have checked both read surfaces"); + assertEquals(3, checked, "should have checked every read surface"); } /** diff --git a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java index a04d887..e6f9b70 100644 --- a/src/test/java/com/swancommunity/owid/PayloadLengthTest.java +++ b/src/test/java/com/swancommunity/owid/PayloadLengthTest.java @@ -23,6 +23,7 @@ import java.io.ByteArrayOutputStream; import java.lang.management.ManagementFactory; import java.lang.management.ThreadMXBean; +import java.nio.ByteBuffer; import java.nio.charset.StandardCharsets; import java.util.Arrays; import java.util.Collections; @@ -219,6 +220,19 @@ void mismatchedLargeDeclarationRefusedWithoutAllocating() { ParseAssert.failed(result, OwidParseStatus.BYTE_COUNT_MISMATCH); assertTrue(allocated < ALLOCATION_BOUND, "declared " + declared + " allocated " + allocated + " bytes"); + + // The framed read is handed the same claim, since it sizes the + // payload from the declaration too and a sender picks the number + // there as well. + ByteBuffer framed = ByteBuffer.wrap(bytes); + before = allocatedBytes(); + OwidParseResult framedResult = Owid.parse(framed); + allocated = allocatedBytes() - before; + ParseAssert.failed(framedResult, OwidParseStatus.UNEXPECTED_END); + assertTrue(allocated < ALLOCATION_BOUND, "framed declared " + + declared + " allocated " + allocated + " bytes"); + assertEquals(0, framed.position(), + "a refused frame should consume nothing"); } } From 4ce7192294ce94f14791f7ea75169c007da79d2f Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 07:57:57 +0100 Subject: [PATCH 4/5] Tell a caller when a node is deliberately absent The marker for an absent optional OWID, being the single byte zero, was reported as an unsupported version, which was not accurate. Version zero is supported and it means something, it simply is not an OWID. Reading it now reports the new OwidParseStatus.ABSENT_NODE. It still hands back no value, which is the part that matters, because the marker carries no domain, no date and no signature and returning an OWID for it would put one in a caller's hands that nothing had ever signed. Only what the caller is told has changed. Reading one frame out of something longer the marker is consumed, so a caller walking a run of frames steps over a node that is deliberately not there and reads the one after it. Reading a whole buffer the marker has to be the whole of it, and bytes after it belong to no field, which is MALFORMED_ENVELOPE. That gives MALFORMED_ENVELOPE a path to it for the first time, so it moves out of the list of members that carry a comment instead of a test, leaving two that genuinely cannot be reached. A ByteBuffer now moves on by whatever the read occupied rather than only when it succeeded, being the envelope for a success, the single byte for an absent node and nothing at all for a failure. The three cases are one rule, and a failed read still leaves the buffer at the start of the frame that failed. Also records, on the members and in the README, that reporting a short frame as UNEXPECTED_END rather than as a byte count disagreement is the settled rule across every implementation rather than a choice made here, and that BYTE_COUNT_MISMATCH means a declaration disagreeing with data that is all present, which only the whole buffer contract can meet. --- README.md | 33 ++++++++---- .../java/com/swancommunity/owid/Owid.java | 37 +++++++------ .../swancommunity/owid/OwidParseResult.java | 27 +++++++--- .../swancommunity/owid/OwidParseStatus.java | 54 ++++++++++++++----- .../com/swancommunity/owid/OwidReader.java | 35 +++++++----- .../java/com/swancommunity/owid/Version.java | 9 ++-- .../swancommunity/owid/FramedReadTest.java | 46 ++++++++++++++-- .../com/swancommunity/owid/ParseAssert.java | 17 ++++++ .../swancommunity/owid/ParseContractTest.java | 34 ++++++------ 9 files changed, 205 insertions(+), 87 deletions(-) diff --git a/README.md b/README.md index 482fc4a..9298fd2 100644 --- a/README.md +++ b/README.md @@ -153,12 +153,13 @@ thing whichever language read the bytes. | `MISSING_INPUT` | Nothing was supplied to read. | | `INVALID_INPUT_TYPE` | Not reachable in Java, where the compiler already refuses anything that is not a string or a byte array. | | `INVALID_BASE64` | The string is not base 64, so there are no bytes to read. | -| `UNSUPPORTED_VERSION` | The first byte names a version this library does not know. | -| `UNEXPECTED_END` | The data stopped in the middle of a field. | +| `UNSUPPORTED_VERSION` | The first byte names a version this library does not know. Version zero is known, and is `ABSENT_NODE` below. | +| `UNEXPECTED_END` | The data stopped in the middle of a field. Reading a frame, this also covers a frame running past the bytes supplied. | | `INVALID_DOMAIN_ENCODING` | The domain is unterminated, or longer than the published maximum. | -| `BYTE_COUNT_MISMATCH` | The declared payload count disagrees with the bytes present. | +| `BYTE_COUNT_MISMATCH` | The declared payload count disagrees with the bytes present. Only the whole buffer read reports it. | | `IMPLEMENTATION_CAPACITY_EXCEEDED` | Larger than this runtime can hold. Not reachable from the byte array surface, because a Java array cannot exceed `Integer.MAX_VALUE` bytes and so can never agree with a larger declaration. | -| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. Every failure this reader can meet is classified by one of the rows above, so this is a backstop with no path to it while the byte count rule holds. | +| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. Reached by an absent node marker followed by bytes on the whole buffer read, where the marker has to be the whole of it. | +| `ABSENT_NODE` | The bytes are the marker for an absent optional OWID. Not a fault and not an OWID, so no value is handed back. | Reading and verifying are separate questions, and reading fetches no key and performs no cryptography. Bytes that are a well formed OWID read successfully @@ -213,7 +214,15 @@ decide. `UNEXPECTED_END` from the framed read means the frame runs past the bytes supplied, so a caller reading from a source that is still arriving can wait -for more and read again from the same position. +for more and read again from the same position. That is the settled rule +across every implementation rather than a choice this one made, because +knowing whether to wait for more bytes or to give up on these is what a +caller of a framed read most needs to be told. + +An absent node marker in the middle of a run of frames reports `ABSENT_NODE` +and consumes its single byte, so the loop above steps over it and reads the +frame after it. It hands back no OWID, because the marker carries no +signature. Buffers that carry no array a caller may reach, being direct and read only ones, are read from a copy of what remains rather than in place. Callers @@ -253,7 +262,7 @@ copies, because a Java byte array is mutable. | `creator.signBytes(value)` | `creator.createBytes(value)` | | `new Owid()`, then `creator.signWithOthers(owid, others)` | `creator.createBytes(payload, others)` | | `owid.setVersion`, `setDomain`, `setDate`, `setPayload` | no replacement, the state is read only | -| `Version.fromByte(b)` | no replacement, an unknown version byte is `UNSUPPORTED_VERSION` from a read | +| `Version.fromByte(b)` | no replacement, an unknown version byte is `UNSUPPORTED_VERSION` from a read, and version zero is `ABSENT_NODE` | `Owid.parse` is overloaded on the input type, so a caller passing a literal `null` has to say which one it means, as in `Owid.parse((String) null)`. The @@ -332,11 +341,13 @@ OWID in the order provided when signing. An absent optional OWID is written as the single byte `0x00`, which `Owid.emptyByteArray` produces, and it marks the absence of a node inside a -larger framed byte array. Every `Owid.parse` overload refuses it with -`UNSUPPORTED_VERSION`, the framed one included, because handing one back -would put an OWID in a caller's hands that nothing had ever signed. A caller -walking a stream that carries markers therefore has to skip them itself, -there being no status in the shared vocabulary that means an absent node. +larger framed byte array. Reading it reports `ABSENT_NODE` and hands back no +OWID, because the marker carries no signature and returning one would put an +OWID in a caller's hands that nothing had ever signed. It is not a fault +either, since version zero is supported and what it means is that a node is +missing. Read as a frame the marker is consumed, so a caller steps over the +absent node and reads the frame after it. Read as a whole buffer the marker +has to be the whole of it, and bytes after it are `MALFORMED_ENVELOPE`. Base 64 decoding accepts the standard alphabet with or without the trailing padding, and skips line breaks and spaces. Anything else in the string is diff --git a/src/main/java/com/swancommunity/owid/Owid.java b/src/main/java/com/swancommunity/owid/Owid.java index 471039b..694e290 100644 --- a/src/main/java/com/swancommunity/owid/Owid.java +++ b/src/main/java/com/swancommunity/owid/Owid.java @@ -205,18 +205,15 @@ public static OwidParseResult parse(ByteBuffer buffer) { view.get(remaining); result = OwidReader.read(remaining, 0, remaining.length, true); } - if (result.isSuccess()) { - // Only a successful read moves the buffer on. A failed read - // reports consuming nothing, so the arithmetic alone would leave - // the buffer where it was, and this says so outright rather than - // resting on that. - // - // Buffer.position is called rather than ByteBuffer.position - // because the covariant override arrived in Java 9 and this - // library is built for 8. - ((Buffer) buffer).position( - buffer.position() + result.getByteCount()); - } + // The buffer moves on by exactly what the read occupied, which is the + // envelope for a success, the single byte for an absent node, and + // nothing at all for a failure. A failed read therefore leaves the + // buffer at the start of the frame that failed. + // + // Buffer.position is called rather than ByteBuffer.position because + // the covariant override arrived in Java 9 and this library is built + // for 8. + ((Buffer) buffer).position(buffer.position() + result.getByteCount()); return result; } @@ -254,13 +251,15 @@ void toBuffer(ByteArrayOutputStream buffer) throws OwidException { * Writes the marker for an absent optional OWID, being the single byte * zero, for embedding in a larger framed byte array. * - *

Every reading surface here, framed included, refuses it as - * {@link OwidParseStatus#UNSUPPORTED_VERSION}, because the marker stands - * for the absence of an identifier rather than for one, and handing back - * an OWID with no domain, no date and no signature would put one in a - * caller's hands that nothing had ever signed. A caller walking a stream - * that carries markers therefore has to skip them itself, there being no - * status in the shared vocabulary that means an absent node.

+ *

Reading it back reports {@link OwidParseStatus#ABSENT_NODE} and + * hands back no value, because the marker stands for the absence of an + * identifier rather than for one, and an OWID with no domain, no date and + * no signature would be one nothing had ever signed. Read as a frame, + * through {@link #parse(ByteBuffer)}, the marker is consumed, so a caller + * walking a run of frames steps over the absent node and reads the frame + * after it. Read as a whole buffer the marker has to be the whole of it, + * and bytes after it are + * {@link OwidParseStatus#MALFORMED_ENVELOPE}.

* * @return a single byte array holding the empty marker */ diff --git a/src/main/java/com/swancommunity/owid/OwidParseResult.java b/src/main/java/com/swancommunity/owid/OwidParseResult.java index 48705d1..20c8525 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseResult.java +++ b/src/main/java/com/swancommunity/owid/OwidParseResult.java @@ -27,7 +27,7 @@ *

The three move together. When {@link #isSuccess()} is true the value is * not null and the status is {@link OwidParseStatus#PARSED}, and when it is * false the value is null and the status names one of the expected - * problems.

+ * problems, or says the bytes were the marker for an absent node.

* *

A result carries no text taken from the input. The bytes came from * outside, so putting them in a message would mean logging whatever an @@ -61,6 +61,16 @@ static OwidParseResult failed(OwidParseStatus status) { return new OwidParseResult(null, status, 0); } + /** + * The result of reading the marker for an absent node, which hands back + * no OWID but does occupy the bytes given, so a caller reading one frame + * after another steps over the absent node and reads the one after it. + */ + static OwidParseResult absentNode(int byteCount) { + return new OwidParseResult( + null, OwidParseStatus.ABSENT_NODE, byteCount); + } + /** * Whether the bytes were a complete, structurally valid OWID. This says * nothing about whether the signature is genuine, which is a separate @@ -74,9 +84,10 @@ public boolean isSuccess() { } /** - * The OWID that was read, or null when the read failed. Callers should + * The OWID that was read, or null when there was none. Callers should * test {@link #isSuccess()} first rather than testing this for null, - * because the status also says which of the expected problems it was. + * because the status also says which of the expected problems it was, or + * that the bytes were the marker for an absent node. * * @return the OWID on success, otherwise null */ @@ -95,7 +106,7 @@ public OwidParseStatus getStatus() { } /** - * How many bytes the envelope occupied, or zero when the read failed. + * How many bytes the read occupied. * *

This is what a caller reading one frame after another needs in order * to find the next one. {@link Owid#parse(java.nio.ByteBuffer)} moves the @@ -103,9 +114,13 @@ public OwidParseStatus getStatus() { * not have to. Reading a whole buffer this is the length of the buffer, * because there the envelope is the whole of it.

* - *

Zero on failure, since a read that failed consumed nothing.

+ *

Three cases. The length of the envelope on success, one byte for + * {@link OwidParseStatus#ABSENT_NODE} so that a caller steps over the + * marker and reads the frame after it, and zero for every failure, since + * a read that failed consumed nothing and left the caller where it + * started.

* - * @return the length of the envelope in bytes, or zero + * @return the bytes the read occupied, or zero */ public int getByteCount() { return byteCount; diff --git a/src/main/java/com/swancommunity/owid/OwidParseStatus.java b/src/main/java/com/swancommunity/owid/OwidParseStatus.java index 8e0e0cc..7dd0cf1 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseStatus.java +++ b/src/main/java/com/swancommunity/owid/OwidParseStatus.java @@ -64,7 +64,10 @@ public enum OwidParseStatus { *

Reading one frame out of something longer, this also covers a frame * whose declared payload and signature run past the bytes supplied, so a * caller reading from a source that is still arriving can wait for more - * and read again from the same place.

+ * and read again from the same place. That is the settled rule across + * every implementation, not a choice this one made, because knowing + * whether to wait for more bytes or to give up on these is the thing a + * caller of a framed read most needs to be told.

*/ UNEXPECTED_END, @@ -84,7 +87,9 @@ public enum OwidParseStatus { * envelope have to end where the input does. Reading one frame out of * something longer, bytes after the signature are the next frame rather * than a disagreement, and a frame that runs past the input is - * {@link #UNEXPECTED_END}.

+ * {@link #UNEXPECTED_END}. Every implementation draws the line in the + * same place, so this status means a declaration disagreeing with data + * that is all present, and nothing else.

*/ BYTE_COUNT_MISMATCH, @@ -107,17 +112,38 @@ public enum OwidParseStatus { * fallback for the genuinely unclassified, not a substitute for naming a * failure that is already understood. * - *

Not reachable in Java today, on either reading surface, and so not - * covered by a test. Every failure these readers can meet is classified - * by one of the members above. The one place that still reports this is - * the check that the envelope ended where the input did, which the framed - * read does not apply at all and which cannot fire on the whole buffer - * read while the declared payload count has already been required to - * leave exactly the signature. That check is kept as a backstop rather - * than removed, because a future change to the count arithmetic would - * otherwise start accepting bytes after the signature in silence. - * Loosening the count rule during a deliberate check of the tests made - * this status appear, so the backstop does work.

+ *

One thing reaches it, being the marker for an absent node followed + * by bytes on the whole buffer contract, where the marker has to be the + * whole of the buffer and what follows belongs to no field.

+ * + *

The other place that reports it is the check that an envelope ended + * where the input did, which the framed read does not apply at all and + * which cannot fire on the whole buffer read while the declared payload + * count has already been required to leave exactly the signature. That + * check is kept as a backstop rather than removed, because a future + * change to the count arithmetic would otherwise start accepting bytes + * after the signature in silence. Loosening the count rule during a + * deliberate check of the tests made it fire, so the backstop does + * work.

+ */ + MALFORMED_ENVELOPE, + + /** + * The bytes are the marker for an absent optional OWID, being the single + * byte zero, so there is deliberately no identifier here. + * + *

Not a failure and not an OWID. Version zero is supported and it + * means something, which is why this is not + * {@link #UNSUPPORTED_VERSION}, but what it means is that a node is + * missing, so no value is handed back. The marker carries no domain, no + * date and no signature, and returning an OWID for it would put one in a + * caller's hands that nothing had ever signed.

+ * + *

Reading one frame out of something longer, the marker is consumed, + * so a caller walking a run of frames can step over an absent node and + * read the one after it. Reading a whole buffer the marker has to be the + * whole of it, and bytes after it are + * {@link #MALFORMED_ENVELOPE}.

*/ - MALFORMED_ENVELOPE + ABSENT_NODE } diff --git a/src/main/java/com/swancommunity/owid/OwidReader.java b/src/main/java/com/swancommunity/owid/OwidReader.java index c4ed0fe..06e161f 100644 --- a/src/main/java/com/swancommunity/owid/OwidReader.java +++ b/src/main/java/com/swancommunity/owid/OwidReader.java @@ -65,18 +65,29 @@ static OwidParseResult read(byte[] buffer, int from, int total, } Version version = Version.forByte(buffer[from] & 0xFF); - if (version == null || version == Version.EMPTY) { - // The empty marker, being the single byte zero, is refused here - // along with the versions this implementation does not know. It - // stands for an absent node inside a framed stream rather than - // for an identifier, so what it carries is no domain, no date and - // no signature, and handing one back would put an OWID in a - // caller's hands that nothing had ever signed. That is the state - // the construction boundary exists to prevent, and it could never - // verify. The framed read refuses it for the same reason, so a - // caller walking a stream that carries markers has to skip them - // itself, there being no status in the shared vocabulary that - // means an absent node. + if (version == Version.EMPTY) { + // The marker for an absent node, being the single byte zero. It + // is not an OWID and no value is handed back, because it carries + // no domain, no date and no signature and returning one would put + // an OWID in a caller's hands that nothing had ever signed. It is + // not a fault either, since version zero is supported and it + // means a node is missing, which is why the caller is told that + // rather than told the version is unknown. + // + // Reading a frame the marker is consumed, so a caller walking a + // run of frames steps over the absent node and reads the one + // after it. Reading a whole buffer the marker has to be the whole + // of it, and bytes after it belong to no field. + if (framed == false && from + 1 != total) { + return OwidParseResult.failed( + OwidParseStatus.MALFORMED_ENVELOPE); + } + return OwidParseResult.absentNode(1); + } + if (version == null) { + // A version byte this implementation does not know, which is a + // different thing from the marker above, where the version is + // known and says a node is missing. return OwidParseResult.failed( OwidParseStatus.UNSUPPORTED_VERSION); } diff --git a/src/main/java/com/swancommunity/owid/Version.java b/src/main/java/com/swancommunity/owid/Version.java index 4a7caed..85f77ff 100644 --- a/src/main/java/com/swancommunity/owid/Version.java +++ b/src/main/java/com/swancommunity/owid/Version.java @@ -30,10 +30,11 @@ public enum Version { * Marker used to indicate an optional OWID that is not present, inside a * larger framed byte array. * - *

No OWID carries this version. A whole buffer holding nothing but the - * marker holds no identifier, so {@link Owid#parse(byte[])} refuses it as - * {@link OwidParseStatus#UNSUPPORTED_VERSION} rather than handing back - * something nothing has ever signed.

+ *

No OWID carries this version, so reading the marker hands back no + * value and reports {@link OwidParseStatus#ABSENT_NODE}, being the + * absence of a node rather than a fault. Reading one frame out of + * something longer the marker is consumed, so a caller steps over the + * absent node and reads the frame after it.

*/ EMPTY(0), diff --git a/src/test/java/com/swancommunity/owid/FramedReadTest.java b/src/test/java/com/swancommunity/owid/FramedReadTest.java index 81c2cfe..cc0465d 100644 --- a/src/test/java/com/swancommunity/owid/FramedReadTest.java +++ b/src/test/java/com/swancommunity/owid/FramedReadTest.java @@ -198,10 +198,48 @@ void framedFailuresUseTheSharedVocabulary() { new byte[0], Envelope.signature()))), OwidParseStatus.UNEXPECTED_END); - // The marker for an absent node is refused here too, because handing - // one back would be an OWID nothing had ever signed. - ParseAssert.failed(Owid.parse(ByteBuffer.wrap(Owid.emptyByteArray())), - OwidParseStatus.UNSUPPORTED_VERSION); + } + + /** + * The marker for an absent node hands back no OWID, says so in its own + * words, and takes the one byte it is, so a caller walking a run of + * frames can step over a node that is deliberately not there. + */ + @Test + void anAbsentNodeIsSteppedOverAndTheNextFrameRead() { + ByteArrayOutputStream stream = new ByteArrayOutputStream(); + byte[] marker = Owid.emptyByteArray(); + stream.write(marker, 0, marker.length); + stream.write(FIRST, 0, FIRST.length); + ByteBuffer buffer = ByteBuffer.wrap(stream.toByteArray()); + + ParseAssert.absentNode(Owid.parse(buffer)); + assertEquals(marker.length, buffer.position(), + "should have stepped over the marker and nothing more"); + + Owid owid = ParseAssert.parsed(Owid.parse(buffer)); + + assertEquals("first.example", owid.getDomain(), + "should read the frame that follows the absent node"); + assertFalse(buffer.hasRemaining(), + "the marker and the frame should account for every byte"); + } + + /** + * A marker on its own, and a run of them, read as absent nodes rather + * than as anything wrong. + */ + @Test + void aRunOfAbsentNodesReadsOneAtATime() { + ByteBuffer buffer = ByteBuffer.wrap(new byte[] {0, 0, 0}); + + int absent = 0; + while (buffer.hasRemaining()) { + ParseAssert.absentNode(Owid.parse(buffer)); + absent++; + } + + assertEquals(3, absent, "should have read three absent nodes"); } /** diff --git a/src/test/java/com/swancommunity/owid/ParseAssert.java b/src/test/java/com/swancommunity/owid/ParseAssert.java index 0db6408..0076b8a 100644 --- a/src/test/java/com/swancommunity/owid/ParseAssert.java +++ b/src/test/java/com/swancommunity/owid/ParseAssert.java @@ -69,4 +69,21 @@ static void failed(OwidParseResult result, OwidParseStatus expected) { assertEquals(0, result.getByteCount(), "a failed read should report consuming nothing"); } + + /** + * Asserts the bytes were the marker for an absent node, that nothing was + * handed back for it, and that it occupied the single byte a caller has + * to step over to reach the next frame. + */ + static void absentNode(OwidParseResult result) { + assertNotNull(result, "a read should always report a result"); + assertFalse(result.isSuccess(), + "the marker for an absent node is not an OWID"); + assertEquals(OwidParseStatus.ABSENT_NODE, result.getStatus(), + "should report the absence of a node"); + assertNull(result.getValue(), + "the marker should hand back no OWID"); + assertEquals(1, result.getByteCount(), + "the marker should occupy the one byte it is"); + } } diff --git a/src/test/java/com/swancommunity/owid/ParseContractTest.java b/src/test/java/com/swancommunity/owid/ParseContractTest.java index 110de81..81ef33d 100644 --- a/src/test/java/com/swancommunity/owid/ParseContractTest.java +++ b/src/test/java/com/swancommunity/owid/ParseContractTest.java @@ -49,11 +49,8 @@ * cannot be, being {@link OwidParseStatus#INVALID_INPUT_TYPE}, which the * compiler already refuses, and * {@link OwidParseStatus#IMPLEMENTATION_CAPACITY_EXCEEDED}, which a Java byte - * array cannot reach on either surface. A third, - * {@link OwidParseStatus#MALFORMED_ENVELOPE}, is a backstop with no path to - * it while the byte count rule holds, and the framed surface does not apply - * that check at all. The reason is recorded on each of those members as - * well.

+ * array cannot reach on either surface. The reason is recorded on each of + * those members as well.

*/ class ParseContractTest { @@ -233,22 +230,25 @@ void disagreeingByteCountIsByteCountMismatch() { } /** - * The marker for an absent optional OWID is refused by the whole buffer - * read. + * The marker for an absent optional OWID hands back no OWID, which is the + * thing that matters, and says what it is rather than calling it a fault. * - *

It stands for the absence of an identifier rather than for one, and - * it carries no domain, no date and no signature, so handing one back - * would put an OWID in a caller's hands that nothing had ever signed. - * That is the state the construction boundary exists to prevent, and it - * could never verify. A framed reader, which this library does not have, - * would still read the marker as the absence it means.

+ *

It carries no domain, no date and no signature, so handing one back + * would put an OWID in a caller's hands that nothing had ever signed, + * which is the state the construction boundary exists to prevent. Version + * zero is supported and it means something, though, so the caller is told + * that a node is absent rather than that the version is unknown.

+ * + *

Reading a whole buffer the marker has to be the whole of it, so + * bytes after it belong to no field.

*/ @Test - void emptyMarkerIsRefused() { - ParseAssert.failed(Owid.parse(Owid.emptyByteArray()), - OwidParseStatus.UNSUPPORTED_VERSION); + void emptyMarkerIsAnAbsentNodeAndNotAnOwid() { + ParseAssert.absentNode(Owid.parse(Owid.emptyByteArray())); + ParseAssert.absentNode(Owid.parse("AA==")); + ParseAssert.failed(Owid.parse(new byte[] {0, 1}), - OwidParseStatus.UNSUPPORTED_VERSION); + OwidParseStatus.MALFORMED_ENVELOPE); } /** From 5d8facf841ca36474823f1788d10128a88f6c6f0 Mon Sep 17 00:00:00 2001 From: James Rosewell Date: Mon, 31 Aug 2026 08:44:16 +0100 Subject: [PATCH 5/5] Report an absent node whatever follows the marker The whole buffer read answered MALFORMED_ENVELOPE when the single byte marker for an absent node was followed by more bytes, while the Rust, PHP and JavaScript ports all answer ABSENT_NODE for the same input. The version byte settles the question on its own, because nothing after it can turn the value into an OWID, so this port now gives the same answer as the others on both reading contracts and still hands back no value. Checked by reading the same seven buffers through every port, being an empty buffer, the marker alone, the marker followed by bytes, a valid envelope, a valid envelope with one byte after it, one short by a byte, and an unknown version. Java was the only port that differed, and only on the third of them. All four now agree line for line. MALFORMED_ENVELOPE goes back to being unreachable, so its comment, the status table in the README and the note on the parse contract tests say so again. The README also said the domain had no encoded maximum, which stopped being true when the 253 character bound went in on read, on the creator and on the write helpers. It now records the bound and says that both reading contracts refuse an oversized declaration, naming BYTE_COUNT_MISMATCH for the whole buffer and UNEXPECTED_END for a frame. mvn test passes 99 tests, 0 failures. --- README.md | 27 ++++++++++------ .../java/com/swancommunity/owid/Owid.java | 12 +++---- .../swancommunity/owid/OwidParseResult.java | 4 +-- .../swancommunity/owid/OwidParseStatus.java | 32 ++++++++----------- .../com/swancommunity/owid/OwidReader.java | 14 ++++---- .../swancommunity/owid/ParseContractTest.java | 19 ++++++----- 6 files changed, 57 insertions(+), 51 deletions(-) diff --git a/README.md b/README.md index 9298fd2..0a88d7e 100644 --- a/README.md +++ b/README.md @@ -36,17 +36,25 @@ creates, signs, serializes, and verifies OWIDs. The OWID wire format stores the payload length as an unsigned 32 bit value, so a payload from zero through 4,294,967,295 bytes is structurally valid. The -format defines no smaller payload limit. The null-terminated domain has no -separate encoded maximum, so the protocol alone is not an application input -limit for the complete envelope. +format defines no smaller payload limit. The null-terminated domain is capped +at 253 characters, so that field is at most 254 bytes with its terminator, +which leaves the payload as the only part of the envelope the protocol leaves +open ended, so the protocol alone is not an application input limit for the +complete envelope. This library validates that the declared payload length agrees with the bytes present before it sizes or copies the payload. A large declaration without the corresponding bytes is malformed, and is reported as -`BYTE_COUNT_MISMATCH` without allocating the declared size. A matching large +`BYTE_COUNT_MISMATCH` on the whole buffer read, or as `UNEXPECTED_END` on the +framed read, without allocating the declared size either way. A matching large payload is not malformed merely because it is large, and parsing work and memory use scale with the bytes actually present. +The 253 character maximum binds this library on both sides. A buffer whose +domain field runs past it is refused when it is read, and a domain longer than +it is refused when a `Creator` is built and again when the domain is written, +so the library will not emit an OWID that it would then refuse to read. + The in-memory APIs remain subject to Java's signed `int` array indexing, address-space and available-memory limits, so a single Java byte array cannot represent every value allowed by the unsigned wire field. Applications @@ -158,8 +166,8 @@ thing whichever language read the bytes. | `INVALID_DOMAIN_ENCODING` | The domain is unterminated, or longer than the published maximum. | | `BYTE_COUNT_MISMATCH` | The declared payload count disagrees with the bytes present. Only the whole buffer read reports it. | | `IMPLEMENTATION_CAPACITY_EXCEEDED` | Larger than this runtime can hold. Not reachable from the byte array surface, because a Java array cannot exceed `Integer.MAX_VALUE` bytes and so can never agree with a larger declaration. | -| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. Reached by an absent node marker followed by bytes on the whole buffer read, where the marker has to be the whole of it. | -| `ABSENT_NODE` | The bytes are the marker for an absent optional OWID. Not a fault and not an OWID, so no value is handed back. | +| `MALFORMED_ENVELOPE` | Malformed in a way none of the others describes. Nothing reaches it, because the byte count rule already refuses everything it would catch, and it is kept as a backstop so a later change to that arithmetic cannot start accepting bytes after the signature in silence. | +| `ABSENT_NODE` | The bytes are the marker for an absent optional OWID, on both reading contracts. Not a fault and not an OWID, so no value is handed back. | Reading and verifying are separate questions, and reading fetches no key and performs no cryptography. Bytes that are a well formed OWID read successfully @@ -345,9 +353,10 @@ larger framed byte array. Reading it reports `ABSENT_NODE` and hands back no OWID, because the marker carries no signature and returning one would put an OWID in a caller's hands that nothing had ever signed. It is not a fault either, since version zero is supported and what it means is that a node is -missing. Read as a frame the marker is consumed, so a caller steps over the -absent node and reads the frame after it. Read as a whole buffer the marker -has to be the whole of it, and bytes after it are `MALFORMED_ENVELOPE`. +missing. The first byte settles this on both reading contracts, because +nothing after it can turn the value into an OWID. Read as a frame the marker +is also consumed, so a caller steps over the absent node and reads the frame +after it. Base 64 decoding accepts the standard alphabet with or without the trailing padding, and skips line breaks and spaces. Anything else in the string is diff --git a/src/main/java/com/swancommunity/owid/Owid.java b/src/main/java/com/swancommunity/owid/Owid.java index 694e290..51f06cd 100644 --- a/src/main/java/com/swancommunity/owid/Owid.java +++ b/src/main/java/com/swancommunity/owid/Owid.java @@ -254,12 +254,12 @@ void toBuffer(ByteArrayOutputStream buffer) throws OwidException { *

Reading it back reports {@link OwidParseStatus#ABSENT_NODE} and * hands back no value, because the marker stands for the absence of an * identifier rather than for one, and an OWID with no domain, no date and - * no signature would be one nothing had ever signed. Read as a frame, - * through {@link #parse(ByteBuffer)}, the marker is consumed, so a caller - * walking a run of frames steps over the absent node and reads the frame - * after it. Read as a whole buffer the marker has to be the whole of it, - * and bytes after it are - * {@link OwidParseStatus#MALFORMED_ENVELOPE}.

+ * no signature would be one nothing had ever signed. The first byte + * settles that on both reading contracts, since nothing after it can + * turn the value into an OWID. Read as a frame, through + * {@link #parse(ByteBuffer)}, the marker is consumed, so a caller walking + * a run of frames steps over the absent node and reads the frame after + * it.

* * @return a single byte array holding the empty marker */ diff --git a/src/main/java/com/swancommunity/owid/OwidParseResult.java b/src/main/java/com/swancommunity/owid/OwidParseResult.java index 20c8525..8c1863c 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseResult.java +++ b/src/main/java/com/swancommunity/owid/OwidParseResult.java @@ -111,8 +111,8 @@ public OwidParseStatus getStatus() { *

This is what a caller reading one frame after another needs in order * to find the next one. {@link Owid#parse(java.nio.ByteBuffer)} moves the * buffer along by this much itself, so a caller using that surface does - * not have to. Reading a whole buffer this is the length of the buffer, - * because there the envelope is the whole of it.

+ * not have to. On a successful whole buffer read this is the length of + * the buffer, because there the envelope is the whole of it.

* *

Three cases. The length of the envelope on success, one byte for * {@link OwidParseStatus#ABSENT_NODE} so that a caller steps over the diff --git a/src/main/java/com/swancommunity/owid/OwidParseStatus.java b/src/main/java/com/swancommunity/owid/OwidParseStatus.java index 7dd0cf1..06f7ac7 100644 --- a/src/main/java/com/swancommunity/owid/OwidParseStatus.java +++ b/src/main/java/com/swancommunity/owid/OwidParseStatus.java @@ -112,19 +112,15 @@ public enum OwidParseStatus { * fallback for the genuinely unclassified, not a substitute for naming a * failure that is already understood. * - *

One thing reaches it, being the marker for an absent node followed - * by bytes on the whole buffer contract, where the marker has to be the - * whole of the buffer and what follows belongs to no field.

- * - *

The other place that reports it is the check that an envelope ended - * where the input did, which the framed read does not apply at all and - * which cannot fire on the whole buffer read while the declared payload - * count has already been required to leave exactly the signature. That - * check is kept as a backstop rather than removed, because a future - * change to the count arithmetic would otherwise start accepting bytes - * after the signature in silence. Loosening the count rule during a - * deliberate check of the tests made it fire, so the backstop does - * work.

+ *

Nothing produces one today, so no test can. The single place that + * reports it is the check that an envelope ended where the input did, + * which the framed read does not apply at all and which cannot fire on + * the whole buffer read while the declared payload count has already + * been required to leave exactly the signature. That check is kept as a + * backstop rather than removed, because a future change to the count + * arithmetic would otherwise start accepting bytes after the signature + * in silence. Loosening the count rule during a deliberate check of the + * tests made it fire, so the backstop does work.

*/ MALFORMED_ENVELOPE, @@ -139,11 +135,11 @@ public enum OwidParseStatus { * date and no signature, and returning an OWID for it would put one in a * caller's hands that nothing had ever signed.

* - *

Reading one frame out of something longer, the marker is consumed, - * so a caller walking a run of frames can step over an absent node and - * read the one after it. Reading a whole buffer the marker has to be the - * whole of it, and bytes after it are - * {@link #MALFORMED_ENVELOPE}.

+ *

The first byte settles this on both reading contracts, because + * nothing after it can turn the value into an OWID. Reading one frame + * out of something longer, the marker is consumed, so a caller walking a + * run of frames can step over an absent node and read the one after + * it.

*/ ABSENT_NODE } diff --git a/src/main/java/com/swancommunity/owid/OwidReader.java b/src/main/java/com/swancommunity/owid/OwidReader.java index 06e161f..d4d9633 100644 --- a/src/main/java/com/swancommunity/owid/OwidReader.java +++ b/src/main/java/com/swancommunity/owid/OwidReader.java @@ -74,14 +74,12 @@ static OwidParseResult read(byte[] buffer, int from, int total, // means a node is missing, which is why the caller is told that // rather than told the version is unknown. // - // Reading a frame the marker is consumed, so a caller walking a - // run of frames steps over the absent node and reads the one - // after it. Reading a whole buffer the marker has to be the whole - // of it, and bytes after it belong to no field. - if (framed == false && from + 1 != total) { - return OwidParseResult.failed( - OwidParseStatus.MALFORMED_ENVELOPE); - } + // The first byte settles this on both contracts, because nothing + // after it can turn the value into an OWID, so a whole buffer + // that begins with the marker is reported as an absent node + // whatever else it carries. Reading a frame the marker is also + // consumed, so a caller walking a run of frames steps over the + // absent node and reads the one after it. return OwidParseResult.absentNode(1); } if (version == null) { diff --git a/src/test/java/com/swancommunity/owid/ParseContractTest.java b/src/test/java/com/swancommunity/owid/ParseContractTest.java index 81ef33d..930cec3 100644 --- a/src/test/java/com/swancommunity/owid/ParseContractTest.java +++ b/src/test/java/com/swancommunity/owid/ParseContractTest.java @@ -45,12 +45,14 @@ * *

Every member of {@link OwidParseStatus} is exercised here, with the * domain cases also covered in more depth by {@link DomainLengthTest} and the - * byte count cases by {@link PayloadLengthTest}. Two members are not, and + * byte count cases by {@link PayloadLengthTest}. Three members are not, and * cannot be, being {@link OwidParseStatus#INVALID_INPUT_TYPE}, which the - * compiler already refuses, and + * compiler already refuses, * {@link OwidParseStatus#IMPLEMENTATION_CAPACITY_EXCEEDED}, which a Java byte - * array cannot reach on either surface. The reason is recorded on each of - * those members as well.

+ * array cannot reach on either surface, and + * {@link OwidParseStatus#MALFORMED_ENVELOPE}, which the byte count rule + * already makes unreachable and which is kept only as a backstop. The reason + * is recorded on each of those members as well.

*/ class ParseContractTest { @@ -239,16 +241,17 @@ void disagreeingByteCountIsByteCountMismatch() { * zero is supported and it means something, though, so the caller is told * that a node is absent rather than that the version is unknown.

* - *

Reading a whole buffer the marker has to be the whole of it, so - * bytes after it belong to no field.

+ *

The first byte settles this on both reading contracts, because + * nothing after it can turn the value into an OWID, so a whole buffer + * beginning with the marker is an absent node whatever else it carries. + * That is the answer every OWID implementation gives.

*/ @Test void emptyMarkerIsAnAbsentNodeAndNotAnOwid() { ParseAssert.absentNode(Owid.parse(Owid.emptyByteArray())); ParseAssert.absentNode(Owid.parse("AA==")); - ParseAssert.failed(Owid.parse(new byte[] {0, 1}), - OwidParseStatus.MALFORMED_ENVELOPE); + ParseAssert.absentNode(Owid.parse(new byte[] {0, 1})); } /**