Skip to content

📝 Add chkit ingest to the CLI reference - #210

Open
KeKs0r wants to merge 1 commit into
mainfrom
docs/cli-ingest-reference
Open

KeKs0r wants to merge 1 commit into
mainfrom
docs/cli-ingest-reference

Conversation

@KeKs0r

@KeKs0r KeKs0r commented Sep 24, 2026

Copy link
Copy Markdown
Member

Summary

  • Add cli/ingest.md covering run, list, and status: flags, stream selection, backfills, exit codes, and JSON output, derived from packages/plugin-ingest/src/plugin.ts.
  • Point the CLI overview's chkit ingest row at the new page (it previously linked to the plugin page's Commands section) and cross-link it from the Ingest plugin page.
  • Move chkit plugin to sidebar order 12 so chkit ingest sits before it.

Docs-only; no changeset.

Test plan

  • turbo run typecheck lint build --filter=./apps/docs passes; 48 raw pages, cli/ingest.md listed in llms.txt
  • All internal links and anchors across the docs resolve

Vex-Session: session-ad7ddb262a78682dcf732e38
@composalagent

composalagent Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Read the agent transcripts for this pull request on Composal → https://composal.ai/obsession/chkit/pulls/210/transcripts

The plugin-page-only approach was not sufficient for the requested CLI documentation, so ingest now has a dedicated CLI reference page rather than relying on an indirect link.

Skills used

  • documentation-authoring
  • pr-creation

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-24T19:54:26.072105Z d4e351a PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d4e351ac18

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


### Prerequisites

The ingest plugin must be registered in `plugins` and the config must set `entry` to a module that exports at least one `definePipeline(...)` value. Without an exported pipeline, every subcommand fails with exit code 2.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Document schema globs as valid pipeline sources

The entry requirement excludes a supported configuration: resolveConfig maps either entry or legacy schema globs into config.schema (packages/core/src/model.ts:56-65), and loadGraph deliberately imports every module from that array (packages/plugin-ingest/src/plugin.ts:219-223). A project whose schema file exports a pipeline can therefore use all three commands without setting entry; describing entry as mandatory may cause users to perform an unnecessary, potentially disruptive config migration.

Useful? React with 👍 / 👎.


### `status`

Reads the journal and prints the committed checkpoint of each selected stream's scheduled namespace. Streams that have never committed progress show `(no checkpoint)`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Disclose that status may create the journal table

For a target where the ingestion journal does not yet exist, status is not only a read: it calls journal.ensure() before querying, and that method executes CREATE TABLE IF NOT EXISTS (packages/plugin-ingest/src/plugin.ts:138-146, packages/plugin-ingest/src/journal.ts:57-60). A direct ClickHouse connection with only SELECT privileges can therefore fail, so the reference should state that status needs permission to create the journal rather than presenting it solely as reading checkpoints.

Useful? React with 👍 / 👎.

|------|---------|
| 0 | Success. For `run`, every selected stream succeeded |
| 1 | Error, or a `run` with any stream that did not succeed |
| 2 | Configuration error: no exported pipeline, no matching stream, invalid flags, or missing direct connection |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Report parser-level flag errors as exit code 1

For an unknown flag or a missing flag value, such as chkit ingest run --bogus or chkit ingest run --from, direct dispatch catches UnknownFlagError/MissingFlagValueError and explicitly sets exit code 1 (packages/cli/src/runtime/command-dispatch.ts:137-150); the chkit plugin ingest ... forwarder likewise returns 1 (packages/cli/src/commands/plugin.ts:115-127). Only semantic validation performed inside the ingest command, such as an invalid timestamp, returns 2, so grouping all invalid flags under code 2 can make automation handle CLI usage errors incorrectly.

Useful? React with 👍 / 👎.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant