Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This module holds the developer examples for the 51Did package

| Program | What it shows |
| --- | --- |
| `Main` | Reads a 51Did offline. Builds a sample identifier in process, parses it back with `FodId` and shows that the value is stable while the envelope changes on every issue. Needs no cloud access. |
| `Main` | Reads a 51Did offline. Builds a sample identifier in process, parses it back with `FodId` and shows that the match key is stable while the envelope changes on every issue. Needs no cloud access. |
| `CreatorContextDemoServer` | Serves a small web page that creates a 51Did in the browser, verifies it from the browser, and redeems the encrypted creator context result on this server with `DidClient`, which is the only place the licence key lives. |

## Creator context
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@
* back with {@link FodId} and prints the three payload fields.
* <p>
* It also demonstrates the headline use case: a 51Did is re-issued fresh on
* every call (the envelope, hence the base64, changes), but the value (the
* Hash) is stable. <b>Compare values, never envelopes.</b>
* every call (the envelope, hence the base64, changes), but the match key
* is stable. <b>Compare match keys, never envelopes.</b>
*/
public class Main {

Expand All @@ -62,28 +62,29 @@ public void run() throws Exception {
System.out.println(" Flags : 0x"
+ Integer.toHexString(fodId.getFlags()));
System.out.println(" LicenseId : " + fodId.getLicenseId());
System.out.println(" Hash : " + toHex(fodId.getHash()));
System.out.println(" Match key : " + toHex(fodId.getMatchKey()));
System.out.println(" Verifies : "
+ fodId.verify(crypto.publicKeyPem()));

// Issue the SAME payload again: a separate envelope, same value.
// Issue the SAME payload again: a separate envelope, same match
// key.
FodId reissued = FodId.fromBase64(issue(creator, payload));
boolean sameEnvelope =
fodId.asBase64().equals(reissued.asBase64());
boolean sameValue =
Arrays.equals(fodId.getHash(), reissued.getHash());
boolean sameMatchKey =
Arrays.equals(fodId.getMatchKey(), reissued.getMatchKey());

System.out.println();
System.out.println("Same payload, re-issued:");
System.out.println(" Same envelope (base64) : " + sameEnvelope);
System.out.println(" Same value (Hash) : " + sameValue);
System.out.println(" Same match key : " + sameMatchKey);

// The reader's whole purpose: the value is the stable, comparable
// part while the envelope is not.
if (sameEnvelope || !sameValue) {
// The reader's whole purpose: the match key is the stable,
// comparable part while the envelope is not.
if (sameEnvelope || !sameMatchKey) {
throw new IllegalStateException(
"Expected a different envelope but the same value "
+ "across reissues.");
"Expected a different envelope but the same match "
+ "key across reissues.");
}
}

Expand All @@ -99,7 +100,7 @@ private String issue(Creator creator, byte[] payload)

/**
* A canonical 37-byte Probabilistic payload: flags 0x00, License Id
* 0x12345678 (little-endian) and a 32-byte value 0x20..0x3F.
* 0x12345678 (little-endian) and a 32-byte match key 0x20..0x3F.
*/
private byte[] samplePayload() {
byte[] payload = new byte[FodId.PAYLOAD_LENGTH];
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,8 +72,8 @@ public void init() throws Exception {
/**
* The 51Did example is fully offline, so unlike the cloud examples it must
* complete without throwing. {@code run()} also self-checks the
* value-stable / envelope-changes invariant and throws if it does not
* hold.
* invariant that the match key is stable while the envelope changes,
* and throws if it does not hold.
*/
@Test
public void FodId_Example_Test() throws Exception {
Expand Down Expand Up @@ -241,7 +241,7 @@ private static String keyList(Crypto crypto) {

/**
* A canonical 37-byte Probabilistic payload: flags 0x00, License Id
* 0x12345678 (little-endian) and a 32-byte value 0x20..0x3F.
* 0x12345678 (little-endian) and a 32-byte match key 0x20..0x3F.
*/
private static byte[] samplePayload() {
byte[] payload = new byte[FodId.PAYLOAD_LENGTH];
Expand Down
29 changes: 16 additions & 13 deletions pipeline.did/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,26 +13,27 @@ A 51Did is described at three levels, and the wording is deliberate.
the version, domain, date, payload and signature. It changes byte-for-byte
every time the cloud issues one, even for the same inputs, because the date
and signature change on each call.
- The **value** is the stable, comparable part of the payload after the Flags
and License Id: a 32-byte SHA-256 for Probabilistic and HashedEmail
- The **match key** is the stable, comparable part of the payload after the
Flags and License Id, a 32-byte SHA-256 for Probabilistic and HashedEmail
identifiers, or 16 GUID bytes for Random. Two 51Dids for the same inputs
share the same value even though their envelopes differ.
share the same match key even though their envelopes differ.

**Comparing two 51Dids means comparing their values, never their envelopes.**
**Comparing two 51Dids means comparing their match keys, never their
envelopes.**

## Payload layout

The header is shared by every identifier type; bits 6-7 of Flags select the
type and the length of the value that follows.
The header is shared by every identifier type. Bits 6-7 of Flags select the
type and the length of the match key that follows.

| Offset | Length | Field | Type |
|-------:|-------:|------------|-------------------------------------------------|
| 0 | 1 | Flags | uint8: bits 0-2 usage, bits 6-7 identifier type |
| 1 | 4 | LicenseId | uint32 (little-endian) |
| 5 | 16/32 | Value | SHA-256 (Probabilistic, HashedEmail) or GUID (Random) |
| 5 | 16/32 | Match key | SHA-256 (Probabilistic, HashedEmail) or GUID (Random) |
| after | any | Context | Optional creator context section, readable only by 51Degrees |

| Bits 7-6 | `IdType` | Value length | Minimum payload |
| Bits 7-6 | `IdType` | Match key length | Minimum payload |
|---------:|-----------------|-------------:|----------------:|
| `00` | `PROBABILISTIC` | 32 | 37 |
| `01` | `RANDOM` | 16 | 21 |
Expand Down Expand Up @@ -172,7 +173,7 @@ FodId fodId = FodId.fromBase64(base64FromCloudService);
int flags = fodId.getFlags();
IdType type = fodId.getType(); // PROBABILISTIC / RANDOM / HASHED_EMAIL
long licenseId = fodId.getLicenseId();
byte[] hash = fodId.getHash(); // SHA-256 or GUID bytes, see type
byte[] matchKey = fodId.getMatchKey(); // SHA-256 or GUID bytes, see type

// Delegated OWID-level fields and operations.
String domain = fodId.getDomain();
Expand All @@ -189,11 +190,13 @@ FodId a = FodId.fromBase64(idprobglobalA);
FodId b = FodId.fromBase64(idprobglobalB);

// The envelope (date, signature, base64) differs across reissues.
// The value inside the payload is stable - this is what you compare:
boolean sameValue = java.util.Arrays.equals(a.getHash(), b.getHash());
// The match key inside the payload is stable - this is what you compare:
boolean sameMatchKey = java.util.Arrays.equals(a.getMatchKey(), b.getMatchKey());
```

Use `getHash()` as the cache / dedup key.
Use `getMatchKey()` as the cache / dedup key. `getHash()` remains as a
deprecated alias of `getMatchKey()`, returning the same bytes, and will be
removed in a future release.

## Verifying on your server

Expand Down Expand Up @@ -342,4 +345,4 @@ signed envelopes there is no longer a copy made inside it.
- **No creation of new 51Dids.** This is a reader and a verifier; new 51Dids
are issued by the 51Degrees cloud / on-premise hashing engines.
- **No upper bound on a payload.** The cloud owns the shape of anything past
the value, and this package does not second-guess it.
the match key, and this package does not second-guess it.
4 changes: 2 additions & 2 deletions pipeline.did/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,8 @@
<name>51Degrees :: Pipeline :: 51Did</name>
<description>Strongly typed reader for the 51Did (51Degrees Identifier)
value returned by the 51Degrees Cloud service. Parses the OWID
envelope and exposes the Flags, License Id and value (Hash) plus the
identifier type. Compare values, never envelopes.</description>
envelope and exposes the Flags, License Id and match key plus the
identifier type. Compare match keys, never envelopes.</description>
<url>https://51degrees.com?utm_source=maven&amp;utm_medium=package&amp;utm_campaign=pipeline-java&amp;utm_content=pipeline.did-pom.xml&amp;utm_term=url</url>

<properties>
Expand Down
98 changes: 59 additions & 39 deletions pipeline.did/src/main/java/fiftyone/pipeline/did/FodId.java
Original file line number Diff line number Diff line change
Expand Up @@ -42,25 +42,26 @@
* A 51Did is described at three levels, and the wording here is deliberate.
* The <b>51Did</b> is the identifier as a whole. The <b>envelope</b> is the
* signed {@link Owid} that carries it (version, domain, date, payload,
* signature), re-issued fresh on every call. The <b>value</b> is the stable,
* comparable part of the payload after the Flags and License Id, exposed as
* {@link #getHash()}. Two 51Dids for the same inputs share the same value even
* though their envelopes differ on every issue. <b>Compare values, never
* envelopes.</b>
* signature), re-issued fresh on every call. The <b>match key</b> is the
* stable, comparable part of the payload after the Flags and License Id,
* exposed as {@link #getMatchKey()}. Two 51Dids for the same inputs share the
* same match key even though their envelopes differ on every issue.
* <b>Compare match keys, never envelopes.</b>
* <p>
* Payload layout. The header (offsets 0-4) is shared by every identifier type;
* bits 6-7 of Flags select the {@link IdType} and the length of the value that
* follows:
* Payload layout. The header (offsets 0-4) is shared by every identifier type.
* Bits 6-7 of Flags select the {@link IdType} and the length of the match key
* that follows:
* <ul>
* <li>offset 0, length 1: Flags (bits 0-2 usage, bits 6-7 type)</li>
* <li>offset 1, length 4: License Id (uint32, little-endian)</li>
* <li>offset 5: value - 32-byte SHA-256 (Probabilistic, HashedEmail) or
* 16 GUID bytes (Random)</li>
* <li>after the value, optionally: a creator context section, which binds
* the identifier to the browser and connection it was created on.
* Only 51Degrees can read it, so this reader exposes it only as the
* part of {@link #getPayload()} beyond the value. Its lengths belong
* to the cloud, so this reader puts no upper bound on a payload.</li>
* <li>offset 5: match key - 32-byte SHA-256 (Probabilistic, HashedEmail)
* or 16 GUID bytes (Random)</li>
* <li>after the match key, optionally: a creator context section, which
* binds the identifier to the browser and connection it was created
* on. Only 51Degrees can read it, so this reader exposes it only as
* the part of {@link #getPayload()} beyond the match key. Its lengths
* belong to the cloud, so this reader puts no upper bound on a
* payload.</li>
* </ul>
* <p>
* Reading and verifying are two separate steps. {@link #tryFromBase64(String)}
Expand Down Expand Up @@ -95,10 +96,10 @@ public final class FodId {
/** Byte length of the License Id field. */
public static final int LICENSE_ID_LENGTH = 4;

/** Byte offset of the value (Hash) field within the payload. */
/** Byte offset of the match key field within the payload. */
public static final int HASH_OFFSET = 5;

/** Byte length of the SHA-256 value. */
/** Byte length of the match key field (SHA-256). */
public static final int HASH_LENGTH = 32;

/**
Expand All @@ -107,7 +108,7 @@ public final class FodId {
*/
public static final int HEADER_LENGTH = HASH_OFFSET;

/** Byte length of the GUID value carried by Random identifiers. */
/** Byte length of the GUID match key carried by Random identifiers. */
public static final int GUID_LENGTH = 16;

/**
Expand All @@ -118,7 +119,7 @@ public final class FodId {

/**
* Minimum byte length of a Probabilistic or HashedEmail 51Did payload
* (Flags + License Id + Hash). Random payloads are shorter - see
* (Flags + License Id + match key). Random payloads are shorter - see
* {@link #RANDOM_PAYLOAD_LENGTH}.
*/
public static final int PAYLOAD_LENGTH = HASH_OFFSET + HASH_LENGTH;
Expand All @@ -132,18 +133,18 @@ public final class FodId {
private final Owid owid;
private final int flags;
private final long licenseId;
private final byte[] hash;
private final byte[] matchKey;

/**
* Built only by {@link #read(Owid)} once the payload has passed the
* 51Did rules, so an instance never exists for a payload that failed
* them.
*/
private FodId(Owid owid, int flags, long licenseId, byte[] hash) {
private FodId(Owid owid, int flags, long licenseId, byte[] matchKey) {
this.owid = owid;
this.flags = flags;
this.licenseId = licenseId;
this.hash = hash;
this.matchKey = matchKey;
}

// ----- Reading without throwing -----
Expand Down Expand Up @@ -205,29 +206,31 @@ private static FodIdParseResult read(OwidParseResult envelope) {
* The rules are lower bounds only. The header must be present before the
* type can be read, and the type then sets the least the payload can
* hold. Anything longer is accepted as it stands, because the bytes past
* the value are a creator context section whose shape the cloud judges.
* the match key are a creator context section whose shape the cloud
* judges.
*/
private static FodIdParseResult read(Owid owid) {
byte[] payload = owid.getPayload();
if (payload.length < HEADER_LENGTH) {
return FodIdParseResult.failed(FodIdParseStatus.PAYLOAD_TOO_SHORT);
}
int flags = payload[FLAGS_OFFSET] & 0xFF;
int valueLength;
int matchKeyLength;
switch (IdType.fromFlags(flags)) {
case RANDOM:
valueLength = GUID_LENGTH;
matchKeyLength = GUID_LENGTH;
break;
case RESERVED:
// Not yet assigned, so read best-effort. The header fields
// are unpacked and whatever follows is exposed as the value.
valueLength = payload.length - HEADER_LENGTH;
// are unpacked and whatever follows is exposed as the
// match key.
matchKeyLength = payload.length - HEADER_LENGTH;
break;
default:
valueLength = HASH_LENGTH;
matchKeyLength = HASH_LENGTH;
break;
}
if (payload.length < HEADER_LENGTH + valueLength) {
if (payload.length < HEADER_LENGTH + matchKeyLength) {
return FodIdParseResult.failed(
FodIdParseStatus.INVALID_TYPE_PAYLOAD_LENGTH);
}
Expand All @@ -238,12 +241,13 @@ private static FodIdParseResult read(Owid owid) {
| ((payload[LICENSE_ID_OFFSET + 1] & 0xFFL) << 8)
| ((payload[LICENSE_ID_OFFSET + 2] & 0xFFL) << 16)
| ((payload[LICENSE_ID_OFFSET + 3] & 0xFFL) << 24);
// The value is copied out so that mutating the array a caller gets
// back from getHash() can never reach the envelope's own bytes.
byte[] hash = Arrays.copyOfRange(
payload, HASH_OFFSET, HASH_OFFSET + valueLength);
// The match key is copied out so that mutating the array a caller
// gets back from getMatchKey() can never reach the envelope's own
// bytes.
byte[] matchKey = Arrays.copyOfRange(
payload, HASH_OFFSET, HASH_OFFSET + matchKeyLength);
return FodIdParseResult.parsed(
new FodId(owid, flags, licenseId, hash));
new FodId(owid, flags, licenseId, matchKey));
}

// ----- Reading with exceptions -----
Expand Down Expand Up @@ -396,15 +400,31 @@ public long getLicenseId() {
}

/**
* Returns the value bytes from the payload (a 32-byte SHA-256 for
* Returns the match key from the payload (a 32-byte SHA-256 for
* Probabilistic and HashedEmail identifiers, or 16 GUID bytes for Random).
* This is the stable, comparable part of the envelope - use it as the
* cache / dedup key.
* The match key is the stable, comparable part of the envelope. Two
* 51Dids for the same inputs share the same match key even though their
* envelopes (date, signature) differ on every issue, so use the match
* key as the cache / dedup key.
*
* @return a defensive copy of the value bytes
* @return a defensive copy of the match key bytes
*/
public byte[] getMatchKey() {
return matchKey.clone();
}

/**
* Deprecated alias for {@link #getMatchKey()}. The stable, comparable
* part of a 51Did is now called the match key, mirroring the Model Terms
* for Marketing vocabulary. This alias will be removed in a future
* release.
*
* @return the same bytes as {@link #getMatchKey()}
* @deprecated renamed to {@link #getMatchKey()}
*/
@Deprecated
public byte[] getHash() {
return hash.clone();
return getMatchKey();
}

/** @return the OWID version. */
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -110,8 +110,8 @@ public enum FodIdParseStatus {
* The envelope read and the header names a type, but the payload is
* shorter than the minimum for that type. Random needs the header plus
* 16 GUID bytes, and Probabilistic and HashedEmail need the header plus
* a 32 byte hash. A longer payload is never refused here, because
* anything past the value is a creator context section whose lengths
* a 32 byte match key. A longer payload is never refused here, because
* anything past the match key is a creator context section whose lengths
* belong to the cloud.
*/
INVALID_TYPE_PAYLOAD_LENGTH;
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,11 +30,11 @@
* {@link fiftyone.pipeline.did.FodIdParseResult} instead of throwing, whose
* {@link fiftyone.pipeline.did.FodIdParseStatus} names why an input is not
* a 51Did, and the {@code from} readers make the same read and throw. A
* 51Did exposes the three payload fields (Flags, License Id and the value
* Hash) and the identifier {@link fiftyone.pipeline.did.IdType}, and
* 51Did exposes the three payload fields (Flags, License Id and the match
* key) and the identifier {@link fiftyone.pipeline.did.IdType}, and
* delegates OWID-level concerns to the envelope it holds. Reading never
* checks the signature. Compare 51Dids by their value ({@code getHash()}),
* never by their envelopes.
* checks the signature. Compare 51Dids by their match key
* ({@code getMatchKey()}), never by their envelopes.
* <p>
* {@link fiftyone.pipeline.did.DidClient} is what a server uses against the
* 51Degrees cloud: it fetches and holds the published signing keys, verifies
Expand Down
Loading
Loading