fix: Closes a few divergences Rust <> TypeScript

.gitignore

@@ -28,8 +28,10 @@ coverage
 # Rust cross-validation harness build artifacts
 tests/rust-validation/target
 
-# Lockfile: intentionally absent until @blockchaincommons/dcbor-parse is
-# published. Local verification resolves it through the workspace link, which
-# would bake an unpublishable path into bun.lock. A real lockfile is committed
-# at the first release.
+# Lockfile: intentionally absent until @blockchaincommons/dcbor-parse
+# 1.0.0-beta.2 is on npm (the registry has beta.1, and this package needs the
+# newer one). Until then no lockfile can be resolved from the registry, and a
+# local install resolves that dependency through the workspace link, which
+# would bake an unpublishable path into bun.lock. Commit a real lockfile, and
+# switch the workflows to `bun install --frozen-lockfile`, once it publishes.
 bun.lock

.size-limit.json

@@ -2,7 +2,7 @@
   {
     "name": "ESM entry (import *), minified + gzipped",
     "path": "dist/index.mjs",
-    "limit": "19 kB",
+    "limit": "25 kB",
     "ignore": [
       "@blockchaincommons/components",
       "@blockchaincommons/dcbor",
@@ -28,7 +28,7 @@
   {
     "name": "patterns subpath, minified + gzipped",
     "path": "dist/patterns.mjs",
-    "limit": "13 kB",
+    "limit": "19 kB",
     "ignore": [
       "@blockchaincommons/components",
       "@blockchaincommons/dcbor",

CODEOWNERS

@@ -1,3 +1,2 @@
 # These owners will be the default owners for everything in this repo.
-
-*  @ChristopherA
+* @ChristopherA @leonardocustodio

MIGRATION.md

@@ -1,15 +1,6 @@
 # Migrating to the redesigned `@blockchaincommons/dcbor-pattern`
 
-The pattern language, its canonical display strings, the paths and captures
-a match yields (and their order) and the formatted output are unchanged;
-this was proven against a frozen pre-redesign bundle
-(`tests/differential.test.ts`, 8 961 golden vectors) and against
-`bc-dcbor-pattern-rust` 0.11.1 (`tests/rust-validation`: 8 551 of 8 961
-vectors byte-identical, the rest the documented classes in
-`RUST_DIVERGENCES.md`). Matching bugs were fixed towards the reference on
-the way (see §5). §0 lists the changes of the current tree; §1 onwards
-describes the earlier move from the `Result`-based API.
-
+`@blockchaincommons/dcbor-pattern` is the successor to `@bcts/dcbor-pattern`.
 
 ## 0. Checklist for the current API
 
@@ -21,8 +12,10 @@ records each change as a named row. Work through the list:
 - [ ] `@blockchaincommons/dcbor-pattern/internal` is now
   `@blockchaincommons/dcbor-pattern/patterns` (same names).
 - [ ] `@blockchaincommons/dcbor-pattern/lexer` is gone. Languages that embed
-  dCBOR patterns use `parsePatternPrefix`, which reports how much of the
+  dCBOR patterns use `parsePatternPartial`, which reports how much of the
   text it consumed.
+- [ ] `not(p)` is `notMatching(p)` and `nullPattern()` is `nullValue()`;
+  `bool(v)` keeps its name.
 - [ ] `ParseResult<T>` is `DcborResult<T, DcborPatternError>`; `Result`,
   `Ok`, `Err`, `ParseFailure` and `failureMessage` are gone.
 - [ ] `error.details.span` is `error.span`; `error.details` is a
@@ -51,20 +44,40 @@ records each change as a named row. Work through the list:
   a `TypeError`; invalid constructor values (`number(NaN)`, `capture("a b",
   …)`, `digestPrefix` over 32 bytes, an inverted `numberRange`, an invalid
   `dateIso8601`) are `RangeError`s.
-- [ ] `parsePattern` and `parsePatternPrefix` take `{ maxDepth }` (500 by
-  default) and reject deeper text with `NestingTooDeep`.
+- [ ] `parsePattern` and `parsePatternPartial` take an optional
+  `{ maxDepth }` and reject deeper text with `NestingTooDeep`; there is no
+  default limit.
+- [ ] A number literal beyond the double range (`1e400`) parses as the
+  infinity pattern; before it was `InvalidNumberFormat`. An inverted range
+  (`5...1`) parses and matches nothing; before it threw a `RangeError`.
+- [ ] A map pattern whose keys match and whose values do not reports the
+  captures of the constraints it satisfied, with no paths; before it
+  reported none.
 - [ ] Number display no longer uses an exponent (`1e21` displays as
   `1000000000000000000000`); integers beyond 2⁵³ round-trip exactly.
 - [ ] Trailing whitespace after a pattern is consumed by
-  `parsePatternPrefix` and reported in `length`.
+  `parsePatternPartial` and reported in `length`.
 - [ ] `search` de-duplicates its paths; captures inside repeats and groups
   follow the reference's dispatch (see `RUST_DIVERGENCES.md`).
+- [ ] Regexes are read as the reference's dialect and translated: `\w`,
+  `\d`, `\s`, `\b` are Unicode-aware, `.` matches any code point but `\n`,
+  `(?m)` anchors at `\n` only, `a**` and `(?U)` are accepted, `\p{…}` takes
+  loose names, classes nest and take set operations. A byte regex runs in
+  Unicode mode over the bytes (`h'/é/'` matches `c3a9`) unless it starts
+  with `(?-u)`; before, it ran over one character per byte. Lookaround and
+  backreferences stay rejected.
+- [ ] Rejections carry the reference's variants and spans: `UnexpectedToken`
+  names the token met, `UnexpectedEndOfInput` or `ExpectedClose…` the end
+  of the source; an unterminated literal spans its opening delimiter;
+  unrecognised text spans a keyword's prefix or one character;
+  `InvalidCaptureGroupName` is no longer raised (`@(1)` is
+  `UnrecognizedToken`).
 
 ## 1. Entries
 
 | Entry | Contents |
 |---|---|
-| root | `parsePattern`/`tryParsePattern` (and the `…Prefix` forms), the constructors, `paths`, `matches`, `display`, `pathsWithCaptures`, `Interval`, `Quantifier`, `Reluctance`, `DcborPatternError` |
+| root | `parsePattern`/`tryParsePattern` (and the `…Partial` forms), the constructors, `paths`, `matches`, `display`, `pathsWithCaptures`, `Interval`, `Quantifier`, `Reluctance`, `DcborPatternError` |
 | `/format` | `formatPaths`, `formatPath`, `FormatPathsOptions`, `PathElementFormat` |
 | `/patterns` | the per-kind pattern types, constructors, matchers and displayers, for languages that embed dCBOR patterns |
 
@@ -76,7 +89,7 @@ parsers and matchers) are no longer re-exported.
 | Before | After |
 |---|---|
 | `parse(src): Result<Pattern>` | `parsePattern(src): Pattern` (throws `DcborPatternError`) or `tryParsePattern(src): DcborResult<Pattern, DcborPatternError>` |
-| `parsePartial(src): Result<[Pattern, number]>` | `parsePatternPrefix(src): { pattern, length }` or `tryParsePatternPrefix(src)` |
+| `parsePartial(src): Result<[Pattern, number]>` | `parsePatternPartial(src): { pattern, length }` or `tryParsePatternPartial(src)` |
 | `Result`, `Ok`, `Err`, `unwrap`, `unwrapOr`, `map`, `errorToString`, `adjustSpan`, the bare `Error` union, `PatternError { errorType }` | `DcborPatternError { code, details, span, fullMessage(source) }`, `DcborPatternErrorCode`, `DcborResult<T, E> = { ok: true, value } \| { ok: false, error: E }` |
 | `result.error.type` / `result.error.span` | `result.error.code` / `result.error.span` |
 
@@ -90,12 +103,12 @@ Spans stay UTF-16 code-unit offsets.
 | `patternMatches(p, cbor)` / `matches` | `matches(p, cbor)` |
 | `patternDisplay(p)` | `display(p)` |
 | `patternPathsWithCaptures(p, cbor)` / `pathsWithCaptures` / `pathsWithCapturesDirect` | `pathsWithCaptures(p, cbor)` → `{ paths, captures: Map<string, Path[]> }` |
-| `nullPattern()` / `bool(v)` | `nullValue()` / `boolean(v)` |
-| `Lexer.new(src)` | `parsePatternPrefix(src)` |
+| `nullPattern()` / `not(p)` | `nullValue()` / `notMatching(p)` |
+| `Lexer.new(src)` | `parsePatternPartial(src)` |
 
 Every other constructor (`any`, `number`, `numberRange`, `text`, `textRegex`,
 `byteString`, `date…`, `digest…`, `knownValue…`, `anyArray`, `anyMap`,
-`anyTagged`, `and`, `or`, `not`, `capture`, `search`, `sequence`, `repeat`,
+`anyTagged`, `and`, `or`, `bool`, `capture`, `search`, `sequence`, `repeat`,
 `group`, …) keeps its name.
 
 ## 4. Formatting (`/format`)

README.md

@@ -81,26 +81,36 @@ if (!rejected.ok) {
 }
 
 try {
-  parsePattern("@(1)");
+  parsePattern("(1 2)");
 } catch (e) {
-  if (DcborPatternError.isDcborPatternError(e)) console.log(e.fullMessage("@(1)"));
-  // line 1: Invalid capture group name ''
-  // @(1)
-  // ^
+  if (DcborPatternError.isDcborPatternError(e)) console.log(e.fullMessage("(1 2)"));
+  // line 1: Unexpected token `2`
+  // (1 2)
+  //    ^
 }
 ```
 
+Errors name the token the parser met (`UnexpectedToken`, or
+`UnrecognizedToken` for text no token starts) or the end of the source
+(`UnexpectedEndOfInput`, or `ExpectedCloseParen`, `ExpectedCloseBracket`,
+`ExpectedCloseBrace` and `ExpectedColon` when it ends inside a capture,
+`search(…)`, `tagged(…)`, an array or a map), with the reference's variants
+and spans: an unterminated literal spans its opening delimiter, and
+unrecognised text spans what a single-pass scanner reads before giving up.
+
 Spans are UTF-16 code-unit offsets; `spanToByteOffsets(source, span)`
 converts one to UTF-8 byte offsets. A wrong argument type (a haystack that
 is not a `Cbor`, a pattern that is not a `Pattern`, a non-string source) is
 a `TypeError`; a wrong value (`number(NaN)`, an inverted range, a capture
 name that is not an identifier, a `maxDepth` that is not a positive
 integer) is a `RangeError`.
 
-`parsePatternPrefix` parses the pattern at the start of a longer text and
+`parsePatternPartial` parses the pattern at the start of a longer text and
 reports how much it consumed, for languages that embed dCBOR patterns.
-Nesting is limited to 500 levels by default (`ParseOptions.maxDepth`);
-deeper text is rejected with `NestingTooDeep`.
+Nesting is not limited by default, as in the reference: text nested a few
+thousand levels deep exhausts the engine's stack with a `RangeError`.
+`ParseOptions.maxDepth` sets a limit when one is wanted; deeper text is
+then rejected with `NestingTooDeep`.
 
 ### Entries
 
@@ -130,7 +140,7 @@ Whitespace between tokens is ignored. Keywords are case-sensitive.
 | `/regex/` | text the regex matches |
 | `bstr` | any byte string |
 | `h'0a0b'` | those bytes |
-| `h'/regex/'` | a byte string the regex matches, one character per byte (`\xNN` names a byte) |
+| `h'/regex/'` | a byte string the regex matches, read as UTF-8 (or as raw bytes under `(?-u)`) |
 | `date` | any date (tag 1) |
 | `date'2024-01-01'` | that date (ISO-8601, a bare date or a full timestamp) |
 | `date'2024-01-01...2024-12-31'`, `date'2024-01-01...'`, `date'...2024-12-31'` | a date in the range, from the date on, or up to the date |
@@ -171,25 +181,36 @@ pattern.
 
 ### Regex dialect
 
-A regex inside a pattern is written in the dialect of the pattern language,
-which is shared with every implementation, and translated to a JavaScript
-regex when the pattern is parsed:
-
-- A text regex runs over code points; a byte regex runs over one character
-  per byte, so `\xNN` and `[\x00-\x7f]` name bytes.
-- Leading `(?i)`, `(?m)`, `(?s)`, `(?x)` groups set the flags; `(?P<name>…)`
-  is a named group; `\x{hhhh}` is a code point; `\pL` and `\p{Greek}` are
-  Unicode classes; `\A` and `\z` anchor at the ends; `[[:alpha:]]` and the
-  other POSIX classes are accepted.
-- Possessive quantifiers (`a*+`) are accepted and run greedy.
-- Lookaround, backreferences and `(?U)` are not part of the dialect and are
-  rejected with `InvalidRegex`. `\w`, `\d` and `\b` are ASCII, as in
-  JavaScript.
-
-A constructor also accepts a `RegExp`: its `i`, `m` and `s` flags become
-inline flags, `u` is implied, and `g` or `y` are refused.
-
-Runnable examples live in the [`examples/`](https://github.com/BlockchainCommons/bc-dcbor-pattern-ts/tree/master/examples) directory.
+A regex inside a pattern (`/…/`, `'/…/'`, `date'/…/'`, `tagged(/…/, p)`,
+`h'/…/'`, `digest'/…/'`) is written in the dialect every implementation of
+the pattern language shares, and translated to a JavaScript regex that
+matches the same strings when the pattern is parsed:
+
+- `\w`, `\d`, `\s` and `\b` are Unicode-aware; `.` matches any code point
+  but `\n`; `(?m)` anchors at `\n` only; `(?s)`, `(?R)`, `(?U)` (the greed
+  swap), `(?x)` (verbose mode) and scoped `(?i:…)` work as in the dialect;
+  `a**` is nested repetition; `\a`, `\x{…}`, `\u`, `\U`, `\b{start}` and
+  the escapable punctuation are accepted; `\p{…}` takes loose names
+  (`\p{greek}`, `\p{sc=Greek}`, `\p{any}`); classes nest and take `&&`,
+  `--`, `~~` and `[:alpha:]`.
+- A text regex runs over the text's code points. A byte regex runs in
+  Unicode mode over the bytes: `h'/é/'` matches the UTF-8 bytes of `é`,
+  `\xff` is U+00FF and a byte that is not part of a valid sequence matches
+  nothing; under a leading `(?-u)` it runs over the raw bytes with ASCII
+  classes and `\xNN` names byte NN.
+- Lookaround, backreferences, `\Z`, octal escapes, `{,n}` and more than
+  250 nested groups are rejected with `InvalidRegex`, as the dialect
+  rejects them.
+- Not translated: the Unicode property tables the engine lacks (`Age`,
+  `gcb`, `wb`, `sb`), and Unicode mode changing inside a byte regex
+  (`a(?-u:\xff)`); these are `InvalidRegex` here and accepted by the
+  reference. The regex runs on JavaScript's backtracking engine, so a
+  pathological regex can take exponential time where the reference's
+  engine is linear.
+
+The translation is proven against the reference's engine by a differential
+corpus of regex sources over text and byte subjects, replayed by
+`tests/rust-validation`.
 
 ## Status - Beta
 
@@ -219,7 +240,7 @@ To build and work on this library, you'll need the following tools:
 This `bc-dcbor-pattern-ts` project is either derived from or was inspired by:
 
 - [BlockchainCommons/bc-dcbor-pattern-rust](https://github.com/BlockchainCommons/bc-dcbor-pattern-rust) - The reference Rust implementation, by [Wolf McNally](https://github.com/wolfmcnally).
-- [paritytech/bcts](https://github.com/paritytech/bcts) - A TypeScript port covering many Blockchain Commons' implementations, by [Parity Technologies](https://github.com/paritytech).
+- [paritytech/bcts](https://github.com/paritytech/bcts) - A TypeScript port of many Blockchain Commons' specs, by [Parity Technologies](https://github.com/paritytech).
 
 ## Financial Support