Harden 51Did parsing to answer with a reason instead of throwing - #118
Merged
Conversation
The OWID library removed its throwing parse factories, public constructors and setters on branch harden/parse-without-throwing, so an OWID now only comes from a successful read or from a creator that signs it. The submodule points at that branch tip, 5d8facf, in the SWAN-community repository for now, because the 51Degrees fork does not yet carry the commit. Both the URL and the pin move back to the fork at the merged commit once the branch is on main.
FodId.tryFromBase64 and FodId.tryFromByteArray answer with a FodIdParseResult that reports whether the read worked, the 51Did only when it did, and a FodIdParseStatus either way. The status vocabulary is the OWID library's own, mirrored member for member under the same names so an envelope fault is reported exactly as the envelope reader found it, plus PAYLOAD_TOO_SHORT for a payload that cannot hold the five byte header and INVALID_TYPE_PAYLOAD_LENGTH for one shorter than its type's minimum. Nothing puts an upper bound on a payload, because the bytes past the value are a creator context section whose shape belongs to the cloud. The throwing readers fromBase64, fromByteArray and fromOwid make the same read and throw the exception types they always did, so there is one walk of the payload and not two. fromOwid no longer copies the envelope, because the library only hands out immutable ones. Reading never checks the signature, and verifyDetailed says why a signature does or does not verify, keeping a key that could not be used apart from a signature that did not match. The tests cover the three facts on every result, the type minimums, longer domains and longer context sections, every OWID status carried across unchanged, and the throwing readers agreeing with the non-throwing ones. The test factory signs through the library's creator and reads hand-written envelopes back through the library's parse.
DidClient.verify(String) and DidClient.redeem(String, ...) now read the value with FodId.tryFromBase64 after the existing length guard and before any key is fetched or the cloud is called, throwing the IllegalArgumentException they already documented with the parse status in the message. Malformed input therefore costs no network round trip and no use, and the value is still sent to the cloud exactly as the caller gave it. verifySignatureDetailed uses the library's verification result rather than catching an exception per candidate key, so a key that cannot be read and a signature that does not match are both simply the next candidate to try. The tests prove no transport call happens for malformed input, that a longer identifier still reaches the cloud, that a tampered signature reads and then verifies as invalid, and that a key list that cannot be fetched is an error and never a false.
The hardened OWID library has no public constructor, so the example asks the creator to stamp and sign the payload in one step instead of building an envelope and signing it afterwards.
The README now explains the non-throwing readers and the three facts every result reports, what each status means and which failures are data results rather than exceptions, the per type lower bounds and the absence of any upper bound, that the client's length guard is client policy and not a format limit, and a before and after for a caller who reached the removed OWID API through this package.
The owid-java submodule now records 694f63332734132f9e432b9bf1e9ecb734af1b1c, the head of main on the 51Degrees/owid-java fork, which carries the squash merged hardening commit 802f7e4 and has the same tree. The .gitmodules entry returns to ../owid-java.git on branch main, as it was on this repository's main, so the temporary SWAN-community URL, branch and pin at 5d8facf are gone.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
The OWID library this package compiles in (SWAN-community/owid-java, branch
harden/parse-without-throwing, PR #5) has been hardened so that anOwidonly ever comes from a successful non-throwing read or from aCreatorthat signs it. The throwing factoriesOwid.fromBase64andOwid.fromByteArray, the public constructors and the setters are gone.pipeline.didcalled the removed factories, so this PR adapts the package to the hardened surface and gives 51Did reading the same shape.The 51Did reading contract is now:
Owid.parse.PAYLOAD_TOO_SHORT. Random then needs the header plus 16 GUID bytes, and Probabilistic and HashedEmail need the header plus a 32 byte hash, or the read reportsINVALID_TYPE_PAYLOAD_LENGTH. Reserved keeps its documented best-effort behaviour.Public API
Added, all in
fiftyone.pipeline.did:FodId.tryFromBase64(String)andFodId.tryFromByteArray(byte[]), returningFodIdParseResult.FodIdParseResultwithisSuccess(),getValue()(null on failure, never a half-read 51Did) andgetStatus()(PARSEDon success).FodIdParseStatus, the OWIDOwidParseStatusvocabulary mirrored member for member plusPAYLOAD_TOO_SHORTandINVALID_TYPE_PAYLOAD_LENGTH, withfromOwid(OwidParseStatus)mapping by name and refusing anything the mirror does not know. A test asserts every OWID member is mirrored.FodId.verifyDetailed(String publicPem)returning the library'sOwidVerificationResult, so "does not match" (SIGNATURE_INVALID) stays apart from "could not check" (KEY_UNAVAILABLE,INVALID_KEY).Kept, with the same exception types as before:
FodId.fromBase64,FodId.fromByteArray,FodId.fromOwid,FodId.verify(String). The throwing readers delegate to the same read as the non-throwing ones, so there is one walk of the payload.fromOwidstill declaresOwidExceptionso existing callers compile, but no longer copies the envelope, because the library only hands out immutable ones.Changed in
DidClient:verify(String)andredeem(String, String, String)read the value after the existing length guard and before any key fetch or cloud call, and refuse a value that does not read with theIllegalArgumentExceptionthey already documented, now naming the status.verifySignatureDetaileduses the library's verification result instead of catching an exception per candidate key. The 4096 character guard is untouched and stays client policy, not a format limit.Removed from callers in this repository: every use of
Owid.fromBase64,Owid.fromByteArray,new Owid(...),Creator.sign(Owid),Owid.setDateandOwid.setPayload, which were inFodId, the test factory, two tests and thepipeline.developer-examples.fodidexample. A repository-wide grep finds no other use.Before and after for a caller
Before:
After:
Code that built an envelope with
new Owid(domain, date, payload)and thencreator.sign(owid)now asks the creator for one withcreator.createBytes(payload). The README carries the same migration note.The OWID pin
The
owid-javasubmodule moves from7a7f303to694f6333, the head ofmainon the 51Degrees/owid-java fork, which carries the squash merged hardening commit802f7e4and has the same tree as that commit..gitmodulespoints at../owid-java.giton branchmain, as it did on this repository's main.pipeline.did/pom.xmlcompiles the submodule source straight into the module withbuild-helper-maven-plugin, so the build really compiles the hardened source and nothing comes from a cached artefact.Tests
pipeline.didbefore: 99 tests (2 skipped, the live tests that need a resource key). After: 125 tests, 0 failures, 2 skipped. The newFodIdParseTestsclass holds the non-throwing contract and every existing test still passes.pipeline.developer-examples.fodid: 9 tests before and after, 0 failures.mvn install -DskipTests, thenmvn surefire:test): all 25 modules built, and 587 tests ran across the repository with 0 failures and 7 skipped (the skips are pre-existing, being the live tests that need a resource key). Onlypipeline.didandpipeline.developer-examples.fodidreference the 51Did package.Neutralisation
Each new check was undone in turn, the
pipeline.didsuite run, and the check restored. Every neutralisation was caught, and the suite was green again after restoring.PAYLOAD_TOO_SHORT)DidClientTests.verify_ShortPayloadStringIsRefusedBeforeTransport,FodIdParseTests.tryFromBase64_ShorterThanHeader_PayloadTooShort,FodIdTests.constructor_PayloadEmpty_ThrowsINVALID_TYPE_PAYLOAD_LENGTH)DidClientTests.redeem_MalformedStringIsRefusedBeforeTransport,FodIdParseTests.throwingReaders_ThrowTheDocumentedTypesForTheSameInputs,FodIdParseTests.tryFromBase64_ProbabilisticOneByteShort_InvalidTypePayloadLength,FodIdParseTests.tryFromBase64_RandomOneByteShort_InvalidTypePayloadLength,FodIdParseTests.tryFromByteArray_HashedEmailOneByteShort_InvalidTypePayloadLength,FodIdTests.constructor_HashedEmailPayloadOneByteShort_Throws,FodIdTests.constructor_PayloadOneByteShort_Throws,FodIdTests.constructor_RandomPayloadOneByteShort_ThrowsMALFORMED_ENVELOPE)DidClientTests.verify_MalformedStringIsRefusedBeforeTransport,FodIdParseTests.throwingReaders_ThrowTheDocumentedTypesForTheSameInputs,FodIdParseTests.tryFromBase64_InvalidBase64_ReportsTheOwidStatus,FodIdParseTests.tryFromByteArray_DeclarationMismatch_PropagatedUnchanged,FodIdParseTests.tryFromByteArray_OtherEnvelopeFaults_PropagatedUnchanged,FodIdParseTests.tryFrom_NullOrEmpty_MissingInputensureReadsAs51Didcalls removed)DidClientTests.redeem_MalformedStringIsRefusedBeforeTransport,DidClientTests.verify_MalformedStringIsRefusedBeforeTransport,DidClientTests.verify_ShortPayloadStringIsRefusedBeforeTransportDidClientTests.verifySignature_EarlierNeighbourWithinToleranceAfterBoundary,DidClientTests.verifySignature_FalseWithAPublishedKeyFromAnotherPeriod,DidClientTests.verifySignature_FalseWithTheWrongKey,DidClientTests.verifySignature_LaterNeighbourWithinToleranceBeforeBoundary,DidClientTests.verifySignature_TamperedSignatureIsInvalidNotAnErrorChecked with no issue
DidClientis unchanged, still runs before reading, key retrieval and transport, and is still tested separately from the reader (verify_OverLongStringIsRefusedBeforeTransport,verify_OverLongObjectIsRefusedBeforeTransport,redeem_OverLongInputsAreRefusedBeforeTransport).var, and the module's Animal Sniffer check against the Java 8 signature passed as part of the build.FodIdbefore the OWID reader sees it, as before.What remains
DidClientLiveTestsare skipped without a resource key, as before.Produced with AI assistance under James Rosewell's direction and needs human review.