📝 Rename the Ingestion docs section to API Sync

.changeset/api-sync-docs-rename.md

@@ -0,0 +1,6 @@
+---
+"chkit": patch
+"@chkit/plugin-ingest": patch
+---
+
+Point README documentation links at the renamed API Sync section (`/api-sync/`).

README.md

@@ -8,9 +8,9 @@
 
 chkit is an open-source CLI for ClickHouse. Review migration SQL before applying it. Keep table definitions and API readers in your repository, alongside the code that uses them. Run the CLI from the terminal or CI.
 
-**TypeScript:** schemas, migrations, and ingestion. **Python:** schemas and migrations through [chkit-py](https://chkit.obsessiondb.com/python/overview/).
+**TypeScript:** schemas, migrations, and API sync. **Python:** schemas and migrations through [chkit-py](https://chkit.obsessiondb.com/python/overview/).
 
-[Get started](https://chkit.obsessiondb.com/getting-started/) · [Build a data source](https://chkit.obsessiondb.com/ingestion/quickstart/) · [Documentation](https://chkit.obsessiondb.com)
+[Get started](https://chkit.obsessiondb.com/getting-started/) · [Build a data source](https://chkit.obsessiondb.com/api-sync/quickstart/) · [Documentation](https://chkit.obsessiondb.com)
 
 > **Beta:** the public API is still evolving. Keep the CLI, core, and plugins on matching versions.
 
@@ -117,7 +117,7 @@ export const content = definePipeline({ id: 'content', streams: [postStream] })
 bunx chkit ingest run --tag pipeline:content
 ```
 
-The default loader writes the rows. For production sources, add [pagination](https://chkit.obsessiondb.com/ingestion/readers/) and [incremental reads](https://chkit.obsessiondb.com/ingestion/incremental-syncs/) when the provider supports them.
+The default loader writes the rows. For production sources, add [pagination](https://chkit.obsessiondb.com/api-sync/readers/) and [incremental reads](https://chkit.obsessiondb.com/api-sync/incremental-syncs/) when the provider supports them.
 
 ### 4. Query through a view
 
@@ -149,7 +149,7 @@ Expected result for the demo dataset:
 | 2 | 10 |
 | 3 | 10 |
 
-If the final shape may change, [retain raw records and transform in ClickHouse](https://chkit.obsessiondb.com/ingestion/destinations/) instead of mapping every field up front.
+If the final shape may change, [retain raw records and transform in ClickHouse](https://chkit.obsessiondb.com/api-sync/destinations/) instead of mapping every field up front.
 
 ### 5. Evolve the model
 
@@ -177,15 +177,15 @@ bunx chkit ingest list
 bunx chkit ingest run --tag pipeline:content
 ```
 
-Use `check` in CI for migration state and schema drift. Schedule ingestion through cron, CI, or another job runner, with one ingestion process per target. Incremental sources resume from committed state; this full-sync demo reads the dataset again. See [scheduling and recovery](https://chkit.obsessiondb.com/ingestion/operations/).
+Use `check` in CI for migration state and schema drift. Schedule ingestion through cron, CI, or another job runner, with one ingestion process per target. Incremental sources resume from committed state; this full-sync demo reads the dataset again. See [scheduling and recovery](https://chkit.obsessiondb.com/api-sync/operations/).
 
 ## Set up chkit for your project
 
 | Goal | Start here |
 |---|---|
 | Manage a new schema | [Getting started](https://chkit.obsessiondb.com/getting-started/) |
 | Adopt an existing database | [Pull a live schema](https://chkit.obsessiondb.com/plugins/pull/) |
-| Implement an API source | [Ingestion quickstart](https://chkit.obsessiondb.com/ingestion/quickstart/) |
+| Implement an API source | [API sync quickstart](https://chkit.obsessiondb.com/api-sync/quickstart/) |
 | Generate application types | [TypeScript codegen](https://chkit.obsessiondb.com/plugins/codegen/) |
 | Recompute stored data | [SQL backfills](https://chkit.obsessiondb.com/plugins/backfill/) |
 | Work with a coding agent | [Agent skills](https://chkit.obsessiondb.com/ai-agents/) |
@@ -208,7 +208,7 @@ See the [CLI reference](https://chkit.obsessiondb.com/cli/overview/) for command
 
 ## Python
 
-Install [`chkit-py`](https://pypi.org/project/chkit-py/) (`pip install chkit-py`) to define schemas and run migrations, drift detection, and CI checks with Python config and schema files. API ingestion requires TypeScript. The Python source is in [`chkit_python/`](chkit_python).
+Install [`chkit-py`](https://pypi.org/project/chkit-py/) (`pip install chkit-py`) to define schemas and run migrations, drift detection, and CI checks with Python config and schema files. API sync requires TypeScript. The Python source is in [`chkit_python/`](chkit_python).
 
 ## Documentation
 

apps/docs/astro.config.mjs

@@ -75,27 +75,27 @@ export default defineConfig({
 					autogenerate: { directory: 'schema' },
 				},
 				{
-					label: 'Ingestion',
+					label: 'API Sync',
 					items: [
-						{ label: 'Overview', slug: 'ingestion' },
-						{ label: 'Quickstart', slug: 'ingestion/quickstart' },
-						{ label: 'Authoring skill', slug: 'ingestion/agent-skill' },
+						{ label: 'Overview', slug: 'api-sync' },
+						{ label: 'Quickstart', slug: 'api-sync/quickstart' },
+						{ label: 'Authoring skill', slug: 'api-sync/agent-skill' },
 						{
 							label: 'Build a source',
 							collapsed: true,
 							items: [
-								{ slug: 'ingestion/readers' },
-								{ slug: 'ingestion/destinations' },
-								{ slug: 'ingestion/incremental-syncs' },
-								{ slug: 'ingestion/loading' },
+								{ slug: 'api-sync/readers' },
+								{ slug: 'api-sync/destinations' },
+								{ slug: 'api-sync/incremental-syncs' },
+								{ slug: 'api-sync/loading' },
 							],
 						},
 						{
 							label: 'Run and verify',
 							collapsed: true,
 							items: [
-								{ slug: 'ingestion/operations' },
-								{ slug: 'ingestion/testing' },
+								{ slug: 'api-sync/operations' },
+								{ slug: 'api-sync/testing' },
 							],
 						},
 					],

apps/docs/functions/_middleware.ts

@@ -1,6 +1,14 @@
 export const onRequest: PagesFunction<{ ASSETS: Fetcher }> = async (context) => {
 	const url = new URL(context.request.url);
 
+	// The Ingestion section moved to /api-sync/. Keep old links (READMEs,
+	// installed agent skills, search results) working, including .md URLs.
+	const moved = url.pathname.match(/^\/ingestion(\/.*|\.md)?$/);
+	if (moved) {
+		url.pathname = `/api-sync${moved[1] ?? '/'}`;
+		return Response.redirect(url.toString(), 301);
+	}
+
 	// Serve the raw Markdown asset behind /_raw/ as text/markdown.
 	async function serveRaw(slug: string): Promise<Response | null> {
 		const assetUrl = new URL(`/_raw/${slug}.md`, url.origin);

apps/docs/src/components/Footer.astro

@@ -41,7 +41,7 @@ const repo = 'https://github.com/obsessiondb/chkit';
 					<li><a href="/cli/overview/">CLI reference</a></li>
 					<li><a href="/configuration/overview/">Configuration</a></li>
 					<li><a href="/schema/dsl-reference/">Schema</a></li>
-					<li><a href="/ingestion/">Ingestion</a></li>
+					<li><a href="/api-sync/">API Sync</a></li>
 				</ul>
 			</div>
 			<div class="chk-footer-col">

apps/docs/src/components/Hero.astro

@@ -7,7 +7,7 @@
 const isHome = Astro.locals.starlightRoute.slug === '';
 ---
 
 {isHome ? (
 <div class="hero chk-home-hero">
   <div class="landing-copy">
     <p class="eyebrow">Open-source ClickHouse CLI</p>
@@ -22,7 +22,7 @@
       ))}
     </div>
     <a class="example-link" href="#from-a-schema-to-a-working-data-sync">See the six-step example <span aria-hidden="true">↓</span></a>
-    <p class="language-scope"><strong>TypeScript</strong> schema + ingestion <span aria-hidden="true">/</span> <a href="/python/overview/"><strong>Python</strong> schema + migrations</a></p>
+    <p class="language-scope"><strong>TypeScript</strong> schema + API sync <span aria-hidden="true">/</span> <a href="/python/overview/"><strong>Python</strong> schema + migrations</a></p>
     <p class="chk-backed"><span>Built by</span> <a href="https://obsessiondb.com">ObsessionDB</a><span aria-hidden="true">·</span><span>Open source · MIT</span></p>
   </div>
 

apps/docs/src/content/docs/ai-agents.md

@@ -23,7 +23,7 @@ Append `.md` to a documentation URL to read Markdown, such as [`/ai-agents.md`](
 
 ## What chkit is
 
-Use chkit to define ClickHouse schemas in TypeScript or Python, generate migration SQL, and check the live database for drift. Use the ingestion plugin to load API data with TypeScript readers.
+Use chkit to define ClickHouse schemas in TypeScript or Python, generate migration SQL, and check the live database for drift. Use the ingest plugin to sync API data with TypeScript readers.
 
 Run chkit through shell commands. Install the agent skills for command and authoring guidance.
 
@@ -56,15 +56,15 @@ chkit skills add obsessiondb/chkit --skill chkit
 
 The skill installs into the project's agent directory (for example `.claude/skills/chkit/` or `.agents/skills/chkit/`). On an interactive `chkit init`, chkit also detects the active agent and offers to install the skill automatically.
 
-### Authoring ingestion sources
+### Authoring API sync sources
 
-For TypeScript API ingestion, install the focused authoring skill:
+For TypeScript API sync, install the focused authoring skill:
 
 ```sh
 chkit skills add obsessiondb/chkit --skill chkit-ingestion
 ```
 
-It guides decisions about raw versus shaped data, transformations, pagination, incremental state, and loaders, then links to the relevant docs. Start with the [ingestion quickstart](/ingestion/quickstart/); each guide explains when to use its alternatives. Ingestion requires a direct `clickhouse` connection; the workbench executor alone is insufficient. See [skill installation and usage](/ingestion/agent-skill/).
+It guides decisions about raw versus shaped data, transformations, pagination, incremental state, and loaders, then links to the relevant docs. Start with the [API sync quickstart](/api-sync/quickstart/); each guide explains when to use its alternatives. API sync requires a direct `clickhouse` connection; the workbench executor alone is insufficient. See [skill installation and usage](/api-sync/agent-skill/).
 
 ## Step 3: Scaffold based on the answers
 
@@ -148,7 +148,7 @@ In TypeScript, plugins are npm packages registered in the `plugins` array of `cl
 | Adopt chkit on an **existing** ClickHouse database | [`@chkit/plugin-pull`](/plugins/pull/) | Introspects the live database into local schema files so the user starts from real tables, not a blank example. |
 | Generate **typed row models**: TypeScript types (and optional Zod schemas), or Pydantic models in Python: from the schema | [`@chkit/plugin-codegen`](/plugins/codegen/) | Keeps application row types in sync with the schema definitions. |
 | **Backfill** historical data into materialized views | [`@chkit/plugin-backfill`](/plugins/backfill/) | Time-windowed loads with checkpoints, for large or resumable backfills. |
-| **Ingest application API data** into ClickHouse | [`@chkit/plugin-ingest`](/ingestion/) | TypeScript only; finite pulls with journaled checkpoints and an external scheduler. |
+| **Ingest application API data** into ClickHouse | [`@chkit/plugin-ingest`](/api-sync/) | TypeScript only; finite pulls with journaled checkpoints and an external scheduler. |
 | Deploy to **ObsessionDB** | [`@chkit/plugin-obsessiondb`](/obsessiondb/overview/) | ObsessionDB connection and engine configuration; rewrites `Shared` engines when targeting non-ObsessionDB ClickHouse. |
 
 Install plugins for the project's stated requirements.

apps/docs/src/content/docs/ingestion/agent-skill.md → apps/docs/src/content/docs/api-sync/agent-skill.md

@@ -1,6 +1,6 @@
 ---
-title: Ingestion authoring skill
-description: Install a concise agent skill for implementing new chkit ingestion sources.
+title: API sync authoring skill
+description: Install a concise agent skill for implementing new chkit API sync sources.
 ---
 
 Install `chkit-ingestion` to give your coding agent source-authoring instructions and links to the relevant guides.
@@ -19,7 +19,7 @@ The TypeScript CLI also provides a pass-through:
 chkit skills add obsessiondb/chkit --skill chkit-ingestion
 ```
 
-Choose the agent in the installer. Install `@chkit/plugin-ingest` and configure credentials through the [quickstart](/ingestion/quickstart/) before running a sync. The separate `chkit` skill covers schema and migration workflows.
+Choose the agent in the installer. Install `@chkit/plugin-ingest` and configure credentials through the [quickstart](/api-sync/quickstart/) before running a sync. The separate `chkit` skill covers schema and migration workflows.
 
 For a local checkout containing the skill, use:
 
@@ -48,6 +48,6 @@ The skill covers the authoring steps and requirements for bounded readers, check
 
 ## Related pages
 
-- [Destinations and transformations](/ingestion/destinations/): choose storage and mapping for the source.
-- [Quickstart](/ingestion/quickstart/): a working first source.
+- [Destinations and transformations](/api-sync/destinations/): choose storage and mapping for the source.
+- [Quickstart](/api-sync/quickstart/): a working first source.
 - [For AI agents](/ai-agents/): general chkit setup and schema workflows.

apps/docs/src/content/docs/ingestion/destinations.md → apps/docs/src/content/docs/api-sync/destinations.md

@@ -108,7 +108,7 @@ Changing the mapping affects future writes. Updating older rows requires replayi
 
 Decide what one row represents alongside its stored shape. A reader can fetch a root object and its children, but chkit does not discover relationships or load child objects automatically.
 
-**Embed children when the application consumes a complete object.** For example, retain a ticket and its comments together for document retrieval. This denormalized shape keeps its context in one record, at the cost of rebuilding that record when a child changes. Fetch all required child pages before yielding the root; see [nested loading](/ingestion/readers/#parent-records-and-child-collections).
+**Embed children when the application consumes a complete object.** For example, retain a ticket and its comments together for document retrieval. This denormalized shape keeps its context in one record, at the cost of rebuilding that record when a child changes. Fetch all required child pages before yielding the root; see [nested loading](/api-sync/readers/#parent-records-and-child-collections).
 
 **Use separate entity tables when children need independent queries or updates.** For a warehouse, `tickets`, `ticket_comments`, and `customers` can have their own columns and stable keys, with `ticket_id` and `customer_id` linking them. Each stream has one destination; define separate streams for separately loaded tables, or derive them in ClickHouse from retained raw objects.
 
@@ -200,7 +200,7 @@ A separate stream can read retained raw tables, assemble documents in `read`, an
 
 ## Related pages
 
-- [Readers and pagination](/ingestion/readers/): fetch root objects and their children.
-- [Loading and batching](/ingestion/loading/): start with the default loader.
+- [Readers and pagination](/api-sync/readers/): fetch root objects and their children.
+- [Loading and batching](/api-sync/loading/): start with the default loader.
 - [Schema DSL](/schema/dsl-reference/): table and view definitions.
-- [Scheduling and recovery](/ingestion/operations/#backfill-source-data): source rereads versus SQL backfills.
+- [Scheduling and recovery](/api-sync/operations/#backfill-source-data): source rereads versus SQL backfills.

apps/docs/src/content/docs/ingestion/incremental-syncs.md → apps/docs/src/content/docs/api-sync/incremental-syncs.md

@@ -7,17 +7,17 @@ Use an incremental strategy to select the records to fetch and the state to comm
 
 ## Start with a full sync
 
-For a small, bounded dataset, omit `incremental` as in the [quickstart](/ingestion/quickstart/). Each run reads the source again with no bookmark. Keep this approach when the provider has no reliable change filter and the cost of a complete read is acceptable.
+For a small, bounded dataset, omit `incremental` as in the [quickstart](/api-sync/quickstart/). Each run reads the source again with no bookmark. Keep this approach when the provider has no reliable change filter and the cost of a complete read is acceptable.
 
-A full sync does not replace the destination snapshot or detect missing records. See [deletion handling](/ingestion/destinations/#handle-deleted-records) before treating the destination as a complete current-state mirror.
+A full sync does not replace the destination snapshot or detect missing records. See [deletion handling](/api-sync/destinations/#handle-deleted-records) before treating the destination as a complete current-state mirror.
 
 For expensive full reads, use the provider's change mechanism: a timestamp window for time filters, or durable provider state for a resumable change feed. Pagination tokens alone do not establish either contract.
 
 ## Timestamp windows
 
 `start` is the first run's lower bound. Later runs start at the last watermark minus `overlapMs` (default `0`). The upper bound is a fixed execution cutoff, or the explicit backfill upper bound. Choose overlap based on provider indexing delays and late updates; reconcile repeated records in the destination.
 
-This complete source module uses the client from [Readers and pagination](/ingestion/readers/#a-reusable-page-client). Save it as `src/sources/helpdesk.ts` and re-export its table and pipeline from the project entry:
+This complete source module uses the client from [Readers and pagination](/api-sync/readers/#a-reusable-page-client). Save it as `src/sources/helpdesk.ts` and re-export its table and pipeline from the project entry:
 
 ```ts
 import { definePipeline, defineStream, paginate, rawRows, rawTable, timestampWindow } from '@chkit/plugin-ingest'
@@ -126,6 +126,6 @@ The built-in full-sync and cursor strategies do not interpret `--from` / `--to`.
 
 ## Related pages
 
-- [Readers and pagination](/ingestion/readers/): provider contracts and bounded fetching.
-- [Loading and batching](/ingestion/loading/#batch-identity): checkpoint state versus batch identity.
-- [Scheduling and recovery](/ingestion/operations/#backfill-source-data): isolated historical runs.
+- [Readers and pagination](/api-sync/readers/): provider contracts and bounded fetching.
+- [Loading and batching](/api-sync/loading/#batch-identity): checkpoint state versus batch identity.
+- [Scheduling and recovery](/api-sync/operations/#backfill-source-data): isolated historical runs.

apps/docs/src/content/docs/api-sync/index.md

@@ -0,0 +1,34 @@
+---
+title: API Sync
+description: Read application APIs into ClickHouse with TypeScript streams and durable checkpoints.
+---
+
+chkit API sync runs finite pulls from application APIs, loads rows into schema-managed ClickHouse tables, and records progress after writes succeed.
+
+## How it fits together
+
+Start with **API → reader → raw table → SQL view** when the final shape may change. For an established schema, map records in the reader and load typed tables. A loader writes the rows; a checkpoint records where the next execution should resume.
+
+- A **stream** owns a stable ID, destination, reader, and incremental strategy.
+- A **pipeline** groups streams, tags, concurrency limits, and retry defaults. It has no durable state and does not order dependent streams.
+- A **run** executes a selection of streams once. Cron, CI, or another scheduler starts the next run.
+
+Use TypeScript and a direct `clickhouse` connection, including for ObsessionDB databases. The workbench executor does not support API sync. Create destination tables through schema migrations; ingestion creates its journal and writes data.
+
+## Start
+
+- [Quickstart](/api-sync/quickstart/): ingest a small public API and query the result.
+- [Install the authoring skill](/api-sync/agent-skill/): give a coding agent the authoring workflow and relevant documentation.
+
+## Build
+
+- [Readers and pagination](/api-sync/readers/): provider requests, credentials, SDKs, and bounded pages.
+- [Destinations and transformations](/api-sync/destinations/): raw or shaped storage, related objects, current state, and history.
+- [Incremental syncs](/api-sync/incremental-syncs/): full syncs, timestamp windows, and provider state.
+- [Loading and batching](/api-sync/loading/): use the default loader and tune it when needed.
+
+## Operate
+
+- [Scheduling and recovery](/api-sync/operations/): tags, retries, budgets, backfills, and monitoring.
+- [Test a source](/api-sync/testing/): exercise checkpoints and recovery without a live database.
+- [Plugin reference](/plugins/ingest/): configuration and command summary.