Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 48 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

# bytecodec

Typed JavaScript and TypeScript byte utilities for base64, base64url, UTF-8 strings, JSON, gzip, concatenation, comparison, and byte-source normalization. The package ships tree-shakeable ESM plus CommonJS entry points and keeps the same API across Node, Bun, Deno, browsers, and edge runtimes.
Typed JavaScript and TypeScript byte utilities for base64, base64url, hex, Z85, UTF-8 strings, JSON, gzip, concatenation, comparison, and byte-source normalization. The package ships tree-shakeable ESM plus CommonJS entry points and keeps the same API across Node, Bun, Deno, browsers, and edge runtimes.

## Compatibility

Expand All @@ -17,7 +17,7 @@ Typed JavaScript and TypeScript byte utilities for base64, base64url, UTF-8 stri

## Goals

- Developer-friendly API for base64, base64url, UTF-8, JSON, gzip, concat, equality, and byte normalization.
- Developer-friendly API for base64, base64url, hex, Z85, UTF-8, JSON, gzip, concat, equality, and byte normalization.
- No runtime dependencies or bundler shims.
- Tree-shakeable ESM by default with CommonJS compatibility and no side effects.
- Returns copies for safety when normalizing inputs.
Expand Down Expand Up @@ -82,6 +82,28 @@ const encoded = toBase64UrlString(bytes) // string of base64url chars
const decoded = fromBase64UrlString(encoded) // Uint8Array
```

### Hex

```js
import { toHex, fromHex } from '@sovereignbase/bytecodec'

const bytes = new Uint8Array([222, 173, 190, 239])
const encoded = toHex(bytes) // "deadbeef"
const decoded = fromHex(encoded) // Uint8Array
```

### Z85

```js
import { toZ85String, fromZ85String } from '@sovereignbase/bytecodec'

const bytes = new Uint8Array([0x86, 0x4f, 0xd2, 0x6f, 0xb5, 0x59, 0xf7, 0x5b])
const encoded = toZ85String(bytes) // "HelloWorld"
const decoded = fromZ85String(encoded) // Uint8Array
```

Z85 encodes 4 input bytes into 5 output characters, so `toZ85String()` requires a byte length divisible by 4 and `fromZ85String()` requires a string length divisible by 5.

### UTF-8 strings

```js
Expand Down Expand Up @@ -162,7 +184,7 @@ Uses `TextEncoder`, `TextDecoder`, `btoa`, and `atob`. Gzip uses `CompressionStr

### Validation & errors

Validation failures throw `BytecodecError` instances with a `code` string, for example `BASE64URL_INVALID_LENGTH`, `BASE64_DECODER_UNAVAILABLE`, `UTF8_DECODER_UNAVAILABLE`, and `GZIP_COMPRESSION_UNAVAILABLE`. Messages are prefixed with `{@sovereignbase/bytecodec}`.
Validation failures throw `BytecodecError` instances with a `code` string, for example `BASE64URL_INVALID_LENGTH`, `HEX_INVALID_CHARACTER`, `Z85_INVALID_BLOCK`, `BASE64_DECODER_UNAVAILABLE`, `UTF8_DECODER_UNAVAILABLE`, and `GZIP_COMPRESSION_UNAVAILABLE`. Messages are prefixed with `{@sovereignbase/bytecodec}`.

### Safety / copying semantics

Expand All @@ -172,8 +194,8 @@ Validation failures throw `BytecodecError` instances with a `code` string, for e

`npm test` covers:

- 53 unit tests
- 4 integration tests
- 68 unit tests
- 6 integration tests
- Node E2E: ESM and CommonJS
- Bun E2E: ESM and CommonJS
- Deno E2E: ESM
Expand All @@ -183,26 +205,30 @@ Validation failures throw `BytecodecError` instances with a `code` string, for e

## Benchmarks

Latest local `npm run bench` run on 2026-03-19 with Node `v22.14.0 (win32 x64)`:
Latest local `npm run bench` run on 2026-03-23 with Node `v22.14.0 (win32 x64)`:

| Benchmark | Result |
| ---------------- | ------------------------- |
| base64 encode | 1,717,210 ops/s (29.1 ms) |
| base64 decode | 2,326,783 ops/s (21.5 ms) |
| base64url encode | 768,469 ops/s (65.1 ms) |
| base64url decode | 1,173,307 ops/s (42.6 ms) |
| utf8 encode | 1,479,264 ops/s (33.8 ms) |
| utf8 decode | 4,109,139 ops/s (12.2 ms) |
| json encode | 353,666 ops/s (56.6 ms) |
| json decode | 513,064 ops/s (39.0 ms) |
| concat 3 buffers | 664,735 ops/s (75.2 ms) |
| toUint8Array | 4,721,669 ops/s (42.4 ms) |
| toArrayBuffer | 751,732 ops/s (266.1 ms) |
| toBufferSource | 8,952,992 ops/s (22.3 ms) |
| equals same | 3,766,379 ops/s (53.1 ms) |
| equals diff | 4,285,463 ops/s (46.7 ms) |
| gzip compress | 3,118 ops/s (128.3 ms) |
| gzip decompress | 5,070 ops/s (78.9 ms) |
| base64 encode | 1,391,126 ops/s (35.9 ms) |
| base64 decode | 2,089,279 ops/s (23.9 ms) |
| base64url encode | 697,088 ops/s (71.7 ms) |
| base64url decode | 1,095,554 ops/s (45.6 ms) |
| hex encode | 1,053,832 ops/s (47.4 ms) |
| hex decode | 1,027,413 ops/s (48.7 ms) |
| z85 encode | 244,928 ops/s (204.1 ms) |
| z85 decode | 1,596,730 ops/s (31.3 ms) |
| utf8 encode | 1,537,199 ops/s (32.5 ms) |
| utf8 decode | 3,481,143 ops/s (14.4 ms) |
| json encode | 681,747 ops/s (29.3 ms) |
| json decode | 989,746 ops/s (20.2 ms) |
| concat 3 buffers | 846,612 ops/s (59.1 ms) |
| toUint8Array | 9,396,818 ops/s (21.3 ms) |
| toArrayBuffer | 884,096 ops/s (226.2 ms) |
| toBufferSource | 9,279,881 ops/s (21.6 ms) |
| equals same | 3,932,572 ops/s (50.9 ms) |
| equals diff | 4,060,534 ops/s (49.3 ms) |
| gzip compress | 4,126 ops/s (96.9 ms) |
| gzip decompress | 5,550 ops/s (72.1 ms) |

Command: `npm run bench`

Expand Down
10 changes: 10 additions & 0 deletions benchmark/bench.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,20 @@ import {
fromBase64String,
fromBase64UrlString,
fromCompressed,
fromHex,
fromJSON,
fromString,
fromZ85String,
toArrayBuffer,
toBase64String,
toBase64UrlString,
toBufferSource,
toCompressed,
toHex,
toJSON,
toString,
toUint8Array,
toZ85String,
} from '../dist/index.js'

function formatOps(iterations, durationMs) {
Expand Down Expand Up @@ -51,12 +55,18 @@ const sampleJson = { ok: true, count: 42, note: '@sovereignbase/bytecodec' }
const sampleJsonBytes = fromJSON(sampleJson)
const base64 = toBase64String(sampleBytes)
const base64Url = toBase64UrlString(sampleBytes)
const hex = toHex(sampleBytes)
const z85 = toZ85String(sampleBytes)
const compressed = await toCompressed(sampleBytes)

bench('base64 encode', 50000, () => toBase64String(sampleBytes))
bench('base64 decode', 50000, () => fromBase64String(base64))
bench('base64url encode', 50000, () => toBase64UrlString(sampleBytes))
bench('base64url decode', 50000, () => fromBase64UrlString(base64Url))
bench('hex encode', 50000, () => toHex(sampleBytes))
bench('hex decode', 50000, () => fromHex(hex))
bench('z85 encode', 50000, () => toZ85String(sampleBytes))
bench('z85 decode', 50000, () => fromZ85String(z85))
bench('utf8 encode', 50000, () => fromString(sampleText))
bench('utf8 decode', 50000, () => toString(sampleTextBytes))
bench('json encode', 20000, () => fromJSON(sampleJson))
Expand Down
2 changes: 1 addition & 1 deletion jsr.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"$schema": "https://jsr.io/schema/config-file.v1.json",
"name": "@sovereignbase/bytecodec",
"version": "1.3.2",
"version": "1.3.3",
"exports": "./src/index.ts",
"publish": {
"include": [
Expand Down
7 changes: 6 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "@sovereignbase/bytecodec",
"version": "1.3.3",
"description": "JS/TS Runtime-Agnostic byte toolkit for UTF-8 strings, base64, base64url, JSON, normalization, compression, concatenation, and comparison.",
"description": "JS/TS runtime-agnostic byte toolkit for UTF-8, base64, base64url, hex, Z85, JSON, normalization, compression, concatenation, and comparison.",
"keywords": [
"base64url",
"base64",
Expand All @@ -15,9 +15,14 @@
"web",
"node",
"utf8",
"string",
"text",
"json",
"equals",
"hex",
"hexadecimal",
"z85",
"zeromq",
"compression",
"gzip",
"binary",
Expand Down
7 changes: 7 additions & 0 deletions src/.errors/class.ts
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,18 @@ export type BytecodecErrorCode =
| 'CONCAT_NORMALIZE_FAILED'
| 'GZIP_COMPRESSION_UNAVAILABLE'
| 'GZIP_DECOMPRESSION_UNAVAILABLE'
| 'HEX_INPUT_EXPECTED'
| 'HEX_INVALID_CHARACTER'
| 'HEX_INVALID_LENGTH'
| 'JSON_PARSE_FAILED'
| 'JSON_STRINGIFY_FAILED'
| 'STRING_INPUT_EXPECTED'
| 'UTF8_DECODER_UNAVAILABLE'
| 'UTF8_ENCODER_UNAVAILABLE'
| 'Z85_INPUT_EXPECTED'
| 'Z85_INVALID_BLOCK'
| 'Z85_INVALID_CHARACTER'
| 'Z85_INVALID_LENGTH'

export class BytecodecError extends Error {
readonly code: BytecodecErrorCode
Expand Down
38 changes: 34 additions & 4 deletions src/.helpers/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,39 @@ export async function importNodeBuiltin<T = unknown>(
specifier: string
): Promise<T> {
// Keep neutral bundles from rewriting node: specifiers for non-Node runtimes.
const importer = new Function(
'specifier',
'return import(specifier)'
) as (value: string) => Promise<T>
const importer = new Function('specifier', 'return import(specifier)') as (
value: string
) => Promise<T>
return importer(specifier)
}

export const HEX_PAIRS = Array.from({ length: 256 }, (_, value) =>
value.toString(16).padStart(2, '0')
)

export const HEX_VALUES = (() => {
const table = new Int16Array(128).fill(-1)

for (let index = 0; index < 10; index++)
table['0'.charCodeAt(0) + index] = index

for (let index = 0; index < 6; index++) {
table['A'.charCodeAt(0) + index] = index + 10
table['a'.charCodeAt(0) + index] = index + 10
}

return table
})()

export const Z85_CHARS =
'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ.-:+=^!/*?&<>()[]{}@%$#'

export const Z85_VALUES = (() => {
const table = new Int16Array(128).fill(-1)

for (let i = 0; i < Z85_CHARS.length; i++) {
table[Z85_CHARS.charCodeAt(i)] = i
}

return table
})()
59 changes: 59 additions & 0 deletions src/fromHex/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/*
* Copyright 2026 Sovereignbase
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { BytecodecError } from '../.errors/class.js'
import { HEX_VALUES } from '../.helpers/index.js'

/**
* Decodes a hexadecimal string into a new `Uint8Array`.
*
* @param hex The hexadecimal string to decode.
* @returns A new `Uint8Array` containing the decoded bytes.
*/
export function fromHex(hex: string): Uint8Array {
if (typeof hex !== 'string')
throw new BytecodecError(
'HEX_INPUT_EXPECTED',
'fromHex expects a string input'
)

if (hex.length % 2 !== 0)
throw new BytecodecError(
'HEX_INVALID_LENGTH',
'Hex string must have an even length'
)

const bytes = new Uint8Array(hex.length / 2)

for (let offset = 0; offset < hex.length; offset += 2) {
const highCode = hex.charCodeAt(offset)
const lowCode = hex.charCodeAt(offset + 1)
const highNibble = highCode < 128 ? HEX_VALUES[highCode] : -1
const lowNibble = lowCode < 128 ? HEX_VALUES[lowCode] : -1

if (highNibble === -1 || lowNibble === -1)
throw new BytecodecError(
'HEX_INVALID_CHARACTER',
`Invalid hex character at index ${
highNibble === -1 ? offset : offset + 1
}`
)

bytes[offset / 2] = (highNibble << 4) | lowNibble
}

return bytes
}
72 changes: 72 additions & 0 deletions src/fromZ85String/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
/*
* Copyright 2026 Sovereignbase
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

import { BytecodecError } from '../.errors/class.js'
import { Z85_VALUES } from '../.helpers/index.js'

/**
* Decodes a Z85 string into a new `Uint8Array`.
*
* @param z85String The Z85 string to decode.
* @returns A new `Uint8Array` containing the decoded bytes.
*/
export function fromZ85String(z85String: string): Uint8Array {
if (typeof z85String !== 'string')
throw new BytecodecError(
'Z85_INPUT_EXPECTED',
'fromZ85String expects a string input'
)

if (z85String.length % 5 !== 0)
throw new BytecodecError(
'Z85_INVALID_LENGTH',
'Z85 string length must be divisible by 5'
)

const bytes = new Uint8Array((z85String.length / 5) * 4)
let byteOffset = 0

for (let blockOffset = 0; blockOffset < z85String.length; blockOffset += 5) {
let value = 0

for (let digitOffset = 0; digitOffset < 5; digitOffset++) {
const stringOffset = blockOffset + digitOffset
const code = z85String.charCodeAt(stringOffset)
const digit = code < 128 ? Z85_VALUES[code] : -1

if (digit === -1)
throw new BytecodecError(
'Z85_INVALID_CHARACTER',
`Invalid Z85 character at index ${stringOffset}`
)

value = value * 85 + digit
}

if (value > 0xffffffff)
throw new BytecodecError(
'Z85_INVALID_BLOCK',
`Invalid Z85 block at index ${blockOffset}`
)

bytes[byteOffset++] = value >>> 24
bytes[byteOffset++] = (value >>> 16) & 0xff
bytes[byteOffset++] = (value >>> 8) & 0xff
bytes[byteOffset++] = value & 0xff
}

return bytes
}
Loading