Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/rebuild-from-openapi.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@automators/datamaker": major
---

Rebuild the SDK from the API's OpenAPI document. Types are generated rather than hand-written, and the client covers projects, templates, sets, keymaps, masking policies and plans instead of only generate/export.
22 changes: 13 additions & 9 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,22 @@ jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
with:
version: 8
- uses: actions/setup-node@v3
- uses: actions/checkout@v4
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 20.x
cache: "pnpm"

- run: pnpm install --frozen-lockfile
- run: pnpm run lint && pnpm run build

# The types in src/generated are produced from spec/openapi.json. Stale
# or hand-edited types mean the SDK describes an API that has moved on,
# which is exactly how this package fell 2.5 years behind. Fail here
# rather than discover it from a customer.
- name: Generated types match the spec
run: pnpm run generate:check

- run: pnpm run lint
- run: pnpm run test
env:
DATAMAKER_API_KEY: ${{ secrets.DATAMAKER_API_KEY }}
DEV_ACCOUNT_API: ${{secrets.DEV_ACCOUNT_API}}
- run: pnpm run build
12 changes: 7 additions & 5 deletions .github/workflows/publish.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,13 @@ jobs:
if: ${{ github.event.workflow_run.conclusion == 'success' }}
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: pnpm/action-setup@v2
with:
version: 8
- uses: actions/setup-node@v3
- uses: actions/checkout@v4
# pnpm comes from `packageManager` in package.json, so the release uses
# the same version as CI and as the lockfile. Pinning it here separately
# is how this workflow ended up on pnpm 8 against a v9 lockfile, which
# `--frozen-lockfile` refuses - and only at release time.
- uses: pnpm/action-setup@v4
- uses: actions/setup-node@v4
with:
node-version: 20.x
cache: "pnpm"
Expand Down
122 changes: 76 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,63 +1,93 @@
# DataMaker
# @automators/datamaker

[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://opensource.org/licenses/MIT)
The official TypeScript / Node.js client for the [DataMaker](https://datamaker.automators.com) API.

## What is it?
> **Note on naming.** The unscoped `datamaker` package on npm is an unrelated project by another author. This package is the official one and is always **`@automators/datamaker`**.

The official Node.js / Typescript library for the datamaker API. Datamaker assists with generating realistic relational data for testing and development purposes.

## Installation

```sh
```bash
npm install @automators/datamaker
```

## Quick start
Node 18+, and it also runs on Deno, Bun, browsers and edge runtimes. Zero runtime dependencies: it uses the platform `fetch`.

Basic example:
## Usage

```ts
import { DataMaker, Template } from "@automators/datamaker";
import { DataMaker } from "@automators/datamaker";

const dm = new DataMaker({ apiKey: process.env.DATAMAKER_API_KEY });

const sets = await dm.sets.list();

const datamaker = new DataMaker({
apiKey: `YOUR_API_KEY`,
const saved = await dm.sets.save({
name: "golden customers",
data: [{ id: 1, name: "Ada Lovelace" }],
});

const generateData = async () => {
const template = {
name: "basic template",
quantity: 2,
fields: [
{
name: "first_name",
type: "First Name",
},
{
name: "last_name",
type: "Last Name",
},
{
name: "email",
type: "Derived",
options: {
value: "{{first_name}}.{{last_name}}@automators.com",
},
},
],
} satisfies Template;

const data = await datamaker.generate(template);
const result = await data.json();
console.log(result);
};

generateData();
const { mappings, missing } = await dm.keymaps.lookup({
mapName: "sap-material-migration",
object: "Material",
oldKeys: ["OLD-1", "OLD-2"],
});
```

`apiKey` falls back to `DATAMAKER_API_KEY` and `baseURL` to `DATAMAKER_API_URL` - the same variables `datamaker-py` reads, so one configured machine works with both SDKs.

## Resources

| Client | Methods |
| --- | --- |
| `dm.projects` | `list` `get` `create` `update` `delete` |
| `dm.templates` | `list` `get` `create` `update` `delete` |
| `dm.sets` | `list` `get` `create` `update` `delete` `save` |
| `dm.keymaps` | `list` `put` `lookup` `entries` `delete` |
| `dm.maskingPolicies` | `list` `get` `create` `update` `delete` |
| `dm.plans` | `list` `get` `update` `delete` |

Method names mirror `datamaker-py`, so one set of docs covers both.

For anything not yet wrapped, `dm.http` is the typed transport:

```ts
const rows = await dm.http.get<MyShape>("/scenarios", { query: { projectId } });
```

## Development & Contibutions
## Errors

See the [contributing.md](/CONTRIBUTING.md) guide for details on how to contribute to this project.
A non-2xx throws `DataMakerError`, carrying the status and the parsed body rather than a flattened string, because callers branch on both:

## License
```ts
import { DataMakerError } from "@automators/datamaker";

try {
await dm.sets.delete(id);
} catch (error) {
if (error instanceof DataMakerError && error.status === 409) {
// the set is locked - unlock it first
}
}
```

`401` means the key is wrong, `403` that it is right but lacks a permission, `409` that the resource is locked.

## Types are generated, not written

Every type comes from the API's own OpenAPI document, generated into `src/generated/schema.ts`:

```bash
pnpm generate # regenerate from spec/openapi.json
pnpm generate:check # fail if the committed types are stale (CI runs this)
```

This is deliberate. The previous release of this package sat at v0.2.0 for two and a half years while the API moved on, because its types were hand-copied and nothing ever failed when they diverged. Do not hand-edit `src/generated/`.

## Development

```bash
pnpm install
pnpm test
pnpm lint
pnpm build
```

[MIT](https://github.com/automator-com/datamaker-core/blob/main/LICENSE)
The tests never reach a real API: they inject a `fetch` stub and assert on the request the SDK builds.
62 changes: 31 additions & 31 deletions examples/basic.ts
Original file line number Diff line number Diff line change
@@ -1,34 +1,34 @@
import { DataMaker } from "../src/index";
import { Template } from "../src/template";
/**
* The shortest useful thing: save a set, then read it back.
*
* Run with: DATAMAKER_API_KEY=... npx tsx examples/basic.ts
*/
import { DataMaker, DataMakerError } from "../src/index.js";

const datamaker = new DataMaker({});
const dm = new DataMaker();

const generateData = async () => {
const template = {
name: "basic template",
quantity: 2,
fields: [
{
name: "first_name",
type: "First Name",
},
{
name: "last_name",
type: "Last Name",
},
{
name: "email",
type: "Derived",
options: {
value: "{{first_name}}.{{last_name}}@automators.com",
},
},
],
} satisfies Template;

const data = await datamaker.generate(template);
const result = await data.json();
console.log(result);
};
const saved = await dm.sets.save({
name: `example-${Date.now()}`,
description: "Written by examples/basic.ts",
data: [
{ id: 1, name: "Ada Lovelace", email: "ada@example.com" },
{ id: 2, name: "Alan Turing", email: "alan@example.com" },
],
});

generateData();
console.log(`saved set ${saved.id} with ${saved.rowCount} rows`);

// The detail endpoint carries `createdByName`, which the list does not.
const detail = await dm.sets.get(saved.id);
console.log(`created by: ${detail.createdByName ?? "an API key, not a user"}`);

try {
await dm.sets.delete(saved.id);
console.log("cleaned up");
} catch (error) {
if (error instanceof DataMakerError && error.status === 409) {
console.log("the set is locked; unlock it before deleting");
} else {
throw error;
}
}
36 changes: 0 additions & 36 deletions examples/exportToApi.ts

This file was deleted.

17 changes: 0 additions & 17 deletions examples/exportToDB.ts

This file was deleted.

14 changes: 0 additions & 14 deletions examples/generateFromTemplateId.ts

This file was deleted.

32 changes: 32 additions & 0 deletions examples/keymaps.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
/**
* KeyMaps: record the keys a migration minted, then translate in bulk.
*
* Run with: DATAMAKER_API_KEY=... npx tsx examples/keymaps.ts
*/
import { DataMaker } from "../src/index.js";

const dm = new DataMaker();
const mapName = "sap-material-migration";

await dm.keymaps.put({
mapName,
object: "Material",
entries: [
{ oldKey: "OLD-1", newKey: "NEW-1" },
{ oldKey: "OLD-2", newKey: "NEW-2" },
],
});

// `missing` is the point: it separates "no mapping yet" from "not asked for",
// which a plain record of results could not express.
const { mappings, missing } = await dm.keymaps.lookup({
mapName,
object: "Material",
oldKeys: ["OLD-1", "OLD-2", "OLD-3"],
});

console.log("resolved:", mappings);
console.log("not mapped yet:", missing);

const page = await dm.keymaps.entries(mapName, { page: 1, pageSize: 50 });
console.log(`${page.entries.length} of ${page.total} entries`);
Loading
Loading