Skip to content

auto-docs: Update RPCN connector docs#428

Open
vbotbuildovich wants to merge 1 commit into
mainfrom
auto-docs/update-rpcn-connector-docs
Open

auto-docs: Update RPCN connector docs#428
vbotbuildovich wants to merge 1 commit into
mainfrom
auto-docs/update-rpcn-connector-docs

Conversation

@vbotbuildovich
Copy link
Copy Markdown
Contributor

Redpanda Connect Documentation Update

OSS Version: 4.91.0 → 4.92.0
Cloud Version: 4.92.0

Summary

  • 30 new fields across 5 connectors

📝 Action Items for Writers

  • Review generated documentation
Detailed Changes (click to expand)

New Fields

config/redpanda:

  • acks
  • max_buffered_records
  • max_buffered_bytes
  • max_in_flight_requests
  • record_retries
  • record_delivery_timeout

outputs/kafka_franz:

  • acks
  • max_buffered_records
  • max_buffered_bytes
  • max_in_flight_requests
  • record_retries
  • record_delivery_timeout

outputs/redpanda:

  • acks
  • max_buffered_records
  • max_buffered_bytes
  • max_in_flight_requests
  • record_retries
  • record_delivery_timeout

outputs/redpanda_migrator:

  • acks
  • max_buffered_records
  • max_buffered_bytes
  • max_in_flight_requests
  • record_retries
  • record_delivery_timeout

tracers/redpanda:

  • acks
  • max_buffered_records
  • max_buffered_bytes
  • max_in_flight_requests
  • record_retries
  • record_delivery_timeout

🔍 Cloud Support Gap Analysis

74 connectors available in OSS but not in cloud:

buffers:

  • sqlite (stable)

caches:

  • couchbase (stable)
  • file (stable)

inputs:

  • amqp_1 (stable)
  • beanstalkd (stable)
  • cassandra (stable)
  • cockroachdb_changefeed (stable)
  • csv (stable)
  • discord (stable)
  • dynamic (stable)
  • file (stable)
  • hdfs (stable)
  • nanomsg (stable)
  • nats_stream (stable)
  • nsq (stable)
  • ockam_kafka (stable)
  • parquet (stable)
  • pulsar (stable)
  • socket (stable)
  • socket_server (stable)
  • stdin (stable)
  • subprocess (beta)
  • twitter_search (experimental)
  • websocket (stable)

outputs:

  • amqp_1 (stable)
  • beanstalkd (stable)
  • cassandra (stable)
  • couchbase (stable)
  • cypher (stable)
  • discord (stable)
  • dynamic (stable)
  • elasticsearch_v9 (stable)
  • file (stable)
  • hdfs (stable)
  • http_server (stable)
  • nanomsg (stable)
  • nats_stream (stable)
  • nsq (stable)
  • ockam_kafka (stable)
  • pulsar (stable)
  • pusher (stable)
  • socket (stable)
  • sql (deprecated)
  • stdout (stable)
  • subprocess (beta)
  • websocket (stable)

processors:

  • awk (stable)
  • command (stable)
  • couchbase (stable)
  • crash (beta)
  • grok (stable)
  • javascript (stable)
  • msgpack (stable)
  • ollama_chat (stable)
  • ollama_embeddings (stable)
  • ollama_moderation (stable)
  • parquet (deprecated)
  • protobuf (stable)
  • redpanda_data_transform (stable)
  • sentry_capture (stable)
  • sql (deprecated)
  • subprocess (stable)
  • wasm (stable)

metrics:

  • aws_cloudwatch (stable)
  • influxdb (stable)
  • json_api (stable)
  • logger (beta)
  • statsd (stable)

tracers:

  • jaeger (stable)

bloblang-functions:

  • count (deprecated)
  • env (stable)
  • file (stable)
  • file_rel (stable)
  • hostname (stable)

Generated: 2026-05-15T10:46:53.537Z


📚 Review Guide

For help reviewing this content, see our Review Guide.

🤖 Generation Details

  • Workflow Run: 25913553346
  • Triggered By: workflow_dispatch
  • Generated At: 2026-05-15 10:47:10 UTC
  • Node Version: 24
📋 Generation Output (last 200 lines)
... (output truncated, showing last 200 of 329 lines)


<!-- PR_SUMMARY_START -->

## Redpanda Connect Documentation Update

**OSS Version:** 4.91.0 → 4.92.0
**Cloud Version:** 4.92.0

### Summary

- **30** new fields across 5 connectors

### 📝 Action Items for Writers

- [ ] Review generated documentation

<details>
<summary><strong>Detailed Changes</strong> (click to expand)</summary>

#### New Fields

**config/redpanda:**
- `acks`
- `max_buffered_records`
- `max_buffered_bytes`
- `max_in_flight_requests`
- `record_retries`
- `record_delivery_timeout`

**outputs/kafka_franz:**
- `acks`
- `max_buffered_records`
- `max_buffered_bytes`
- `max_in_flight_requests`
- `record_retries`
- `record_delivery_timeout`

**outputs/redpanda:**
- `acks`
- `max_buffered_records`
- `max_buffered_bytes`
- `max_in_flight_requests`
- `record_retries`
- `record_delivery_timeout`

**outputs/redpanda_migrator:**
- `acks`
- `max_buffered_records`
- `max_buffered_bytes`
- `max_in_flight_requests`
- `record_retries`
- `record_delivery_timeout`

**tracers/redpanda:**
- `acks`
- `max_buffered_records`
- `max_buffered_bytes`
- `max_in_flight_requests`
- `record_retries`
- `record_delivery_timeout`

#### 🔍 Cloud Support Gap Analysis

**74 connectors available in OSS but not in cloud:**

**buffers:**
- sqlite (stable)

**caches:**
- couchbase (stable)
- file (stable)

**inputs:**
- amqp_1 (stable)
- beanstalkd (stable)
- cassandra (stable)
- cockroachdb_changefeed (stable)
- csv (stable)
- discord (stable)
- dynamic (stable)
- file (stable)
- hdfs (stable)
- nanomsg (stable)
- nats_stream (stable)
- nsq (stable)
- ockam_kafka (stable)
- parquet (stable)
- pulsar (stable)
- socket (stable)
- socket_server (stable)
- stdin (stable)
- subprocess (beta)
- twitter_search (experimental)
- websocket (stable)

**outputs:**
- amqp_1 (stable)
- beanstalkd (stable)
- cassandra (stable)
- couchbase (stable)
- cypher (stable)
- discord (stable)
- dynamic (stable)
- elasticsearch_v9 (stable)
- file (stable)
- hdfs (stable)
- http_server (stable)
- nanomsg (stable)
- nats_stream (stable)
- nsq (stable)
- ockam_kafka (stable)
- pulsar (stable)
- pusher (stable)
- socket (stable)
- sql (deprecated)
- stdout (stable)
- subprocess (beta)
- websocket (stable)

**processors:**
- awk (stable)
- command (stable)
- couchbase (stable)
- crash (beta)
- grok (stable)
- javascript (stable)
- msgpack (stable)
- ollama_chat (stable)
- ollama_embeddings (stable)
- ollama_moderation (stable)
- parquet (deprecated)
- protobuf (stable)
- redpanda_data_transform (stable)
- sentry_capture (stable)
- sql (deprecated)
- subprocess (stable)
- wasm (stable)

**metrics:**
- aws_cloudwatch (stable)
- influxdb (stable)
- json_api (stable)
- logger (beta)
- statsd (stable)

**tracers:**
- jaeger (stable)

**bloblang-functions:**
- count (deprecated)
- env (stable)
- file (stable)
- file_rel (stable)
- hostname (stable)

</details>

---

*Generated: 2026-05-15T10:46:53.537Z*

<!-- PR_SUMMARY_END -->

Done: Updated Antora version: 4.92.0
Generation Report:
   • Partial files: 1154
  • Fields partials: 278 total
    – modules/components/partials/fields/buffers/memory.adoc
    – modules/components/partials/fields/buffers/sqlite.adoc
    – modules/components/partials/fields/buffers/system_window.adoc
    – modules/components/partials/fields/caches/aws_dynamodb.adoc
    – modules/components/partials/fields/caches/aws_s3.adoc
    – modules/components/partials/fields/caches/couchbase.adoc
    – modules/components/partials/fields/caches/file.adoc
    – modules/components/partials/fields/caches/gcp_cloud_storage.adoc
    – modules/components/partials/fields/caches/lru.adoc
    – modules/components/partials/fields/caches/memcached.adoc
    … plus 268 more

  • Examples partials: 642 total
    – modules/components/partials/examples/buffers/sqlite.adoc
    – modules/components/partials/examples/buffers/system_window.adoc
    – modules/components/partials/examples/caches/multilevel.adoc
    – modules/components/partials/examples/inputs/aws_dynamodb_cdc.adoc
    – modules/components/partials/examples/inputs/azure_cosmosdb.adoc
    – modules/components/partials/examples/inputs/cassandra.adoc
    – modules/components/partials/examples/inputs/file.adoc
    – modules/components/partials/examples/inputs/gcp_bigquery_select.adoc
    – modules/components/partials/examples/inputs/generate.adoc
    – modules/components/partials/examples/inputs/http_client.adoc
    … plus 632 more

   • Full drafts:   0
  • Draft files: 0 total

Done: whats-new.adoc updated with Version 4.92.0

📄 Summary:
   • Run time: 2026-05-15T10:46:53.537Z
   • Version used: 4.92.0

Full logs available in workflow run


This PR was automatically generated. Please review the changes and merge when ready.

Auto-generated by workflow run 25913553346
@vbotbuildovich vbotbuildovich requested a review from a team as a code owner May 15, 2026 10:47
@vbotbuildovich vbotbuildovich added auto-docs documentation Improvements or additions to documentation labels May 15, 2026
@netlify
Copy link
Copy Markdown

netlify Bot commented May 15, 2026

Deploy Preview for redpanda-connect ready!

Name Link
🔨 Latest commit 5ba2759
🔍 Latest deploy log https://app.netlify.com/projects/redpanda-connect/deploys/6a06f9b8e2c7b200084f421a
😎 Deploy Preview https://deploy-preview-428--redpanda-connect.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 15, 2026

Review Change Stack

📝 Walkthrough

Walkthrough

This PR updates the Redpanda Connect documentation site to release version 4.92.0. The release introduces six new Kafka producer configuration fields (acks, max_buffered_records, max_buffered_bytes, max_in_flight_requests, record_retries, record_delivery_timeout) across multiple components, adds validation rules requiring idempotent_write to pair with acks: "all", enhances Oracle CDC checkpoint documentation with per-PDB behavior details, extends S3 canned ACL options, and removes deprecated components (tigerbeetle_cdc, zmq4, ffi). Configuration examples and field documentation are updated across all affected outputs and tracers, and a v4.92.0 entry is added to the release notes.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • redpanda-data/rp-connect-docs#411: Both PRs are part of the same generated docs release workflow and touch the same connector schema/manifests—e.g., both remove tigerbeetle_cdc/zmq4 (and ffi) in the generated modules/components/attachments/connect-*.json while also bumping antora.yml's latest-connect-version.
  • redpanda-data/rp-connect-docs#337: Both PRs touch the Redpanda tracer documentation; the retrieved PR introduces the Redpanda tracer, and this PR extends that tracer configuration with new Kafka producer reliability/buffering/retry fields.
  • redpanda-data/rp-connect-docs#357: Both PRs modify the same Antora site attribute (antora.ymllatest-connect-version) and add/update corresponding "What's New" release entries with version-specific field documentation.

Suggested reviewers

  • JakeSCahill
  • prakhargarg105
  • kbatuigas
  • rockwotj
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title 'auto-docs: Update RPCN connector docs' accurately summarizes the main change: updating Redpanda Connect documentation to version 4.92.0 with new field support.
Description check ✅ Passed The description is comprehensive and directly related to the changeset, detailing version updates (4.91.0 → 4.92.0), new fields added (30 across 5 connectors), and providing detailed lists of changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch auto-docs/update-rpcn-connector-docs

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
docs-data/connect-4.92.0.json (1)

36570-36694: 💤 Low value

Optional: Case-sensitivity edge case in component linter.

The field-level linter for acks performs case-insensitive validation (this.string().lowercase()), but the component-level linter at Line 36694 compares the raw value: this.acks.or("all") != "all". If a user sets acks: "ALL", it passes field validation but would fail the idempotent_write check.

Since documented options are lowercase and defaults work correctly, this is unlikely to cause issues in practice. Consider verifying if the underlying system normalizes values before component-level validation runs.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs-data/connect-4.92.0.json` around lines 36570 - 36694, The
component-level linter compares a raw acks value (`this.acks.or("all") !=
"all"`) which can mismatch the field-level case-insensitive check
(`this.string().lowercase()`); update the component linter to normalize before
comparing—e.g. use `this.acks.or("all").lowercase() != "all"` (or otherwise call
the same normalization function) in the root linter expression so values like
"ALL" are treated equivalently to "all" when evaluating `idempotent_write`
requirements.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@modules/components/attachments/connect-4.92.0.json`:
- Line 36730: The object-level linter compares this.acks.or("all") to "all"
case-sensitively, causing mismatches for values like "ALL"; update the "linter"
expression so it normalizes the acks value (e.g., lowercasing) before the
idempotency check and any other acks comparisons, and then use the normalized
value in the condition that references this.idempotent_write and this.acks (the
expression containing this.idempotent_write == true && this.acks.or("all") !=
"all"); apply the same normalization wherever acks is compared in this file (and
the other noted ranges).

In `@modules/get-started/pages/whats-new.adoc`:
- Around line 118-119: The table entry for record_delivery_timeout is truncated;
update the description for the record_delivery_timeout field to be a complete
sentence mentioning the unit (milliseconds) and clarifying when it is evaluated
(e.g., "record_delivery_timeout — timeout in ms; evaluated before writing a
request or after receiving a produce response"). Edit the whats-new.adoc entry
for record_delivery_timeout to replace the incomplete "ms`" fragment with the
full explanatory sentence so the release notes table reads clearly.

---

Nitpick comments:
In `@docs-data/connect-4.92.0.json`:
- Around line 36570-36694: The component-level linter compares a raw acks value
(`this.acks.or("all") != "all"`) which can mismatch the field-level
case-insensitive check (`this.string().lowercase()`); update the component
linter to normalize before comparing—e.g. use `this.acks.or("all").lowercase()
!= "all"` (or otherwise call the same normalization function) in the root linter
expression so values like "ALL" are treated equivalently to "all" when
evaluating `idempotent_write` requirements.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 892f9a07-2b2f-49d4-a551-52eb7f807e59

📥 Commits

Reviewing files that changed from the base of the PR and between eb5b720 and 5ba2759.

⛔ Files ignored due to path filters (1)
  • package-lock.json is excluded by !**/package-lock.json
📒 Files selected for processing (18)
  • antora.yml
  • docs-data/connect-4.92.0.json
  • docs-data/connect-diff-4.90.3_to_4.91.0.json
  • docs-data/connect-diff-4.91.0_to_4.92.0.json
  • modules/components/attachments/connect-4.92.0.json
  • modules/components/examples/advanced/outputs/kafka_franz.yaml
  • modules/components/examples/advanced/outputs/ockam_kafka.yaml
  • modules/components/examples/advanced/outputs/redpanda.yaml
  • modules/components/examples/advanced/outputs/redpanda_migrator.yaml
  • modules/components/examples/advanced/tracers/redpanda.yaml
  • modules/components/examples/common/outputs/ockam_kafka.yaml
  • modules/components/partials/fields/inputs/oracledb_cdc.adoc
  • modules/components/partials/fields/outputs/kafka_franz.adoc
  • modules/components/partials/fields/outputs/ockam_kafka.adoc
  • modules/components/partials/fields/outputs/redpanda.adoc
  • modules/components/partials/fields/outputs/redpanda_migrator.adoc
  • modules/components/partials/fields/tracers/redpanda.adoc
  • modules/get-started/pages/whats-new.adoc
💤 Files with no reviewable changes (1)
  • docs-data/connect-diff-4.90.3_to_4.91.0.json

}
],
"linter": "root = match {\n this.partitioner == \"manual\" && this.partition.or(\"\") == \"\" => \"a partition must be specified when the partitioner is set to manual\"\n this.partitioner != \"manual\" && this.partition.or(\"\") != \"\" => \"a partition cannot be specified unless the partitioner is set to manual\"\n this.timestamp.or(\"\") != \"\" && this.timestamp_ms.or(\"\") != \"\" => \"both timestamp and timestamp_ms cannot be specified simultaneously\"\n}"
"linter": "root = match {\n this.partitioner == \"manual\" && this.partition.or(\"\") == \"\" => \"a partition must be specified when the partitioner is set to manual\"\n this.partitioner != \"manual\" && this.partition.or(\"\") != \"\" => \"a partition cannot be specified unless the partitioner is set to manual\"\n this.timestamp.or(\"\") != \"\" && this.timestamp_ms.or(\"\") != \"\" => \"both timestamp and timestamp_ms cannot be specified simultaneously\"\n this.idempotent_write == true && this.acks.or(\"all\") != \"all\" => \"idempotent_write requires acks to be set to all\"\n}"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Normalize acks before the idempotency guard comparison.

The field-level validator accepts case-insensitive values, but these object-level checks are case-sensitive, so values like ALL can pass one check and fail the other.

🔧 Proposed fix
- this.idempotent_write == true && this.acks.or("all") != "all" => "idempotent_write requires acks to be set to all"
+ this.idempotent_write == true && this.acks.or("all").string().lowercase() != "all" => "idempotent_write requires acks to be set to all"

Also applies to: 43988-43988, 44203-44203

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/components/attachments/connect-4.92.0.json` at line 36730, The
object-level linter compares this.acks.or("all") to "all" case-sensitively,
causing mismatches for values like "ALL"; update the "linter" expression so it
normalizes the acks value (e.g., lowercasing) before the idempotency check and
any other acks comparisons, and then use the normalized value in the condition
that references this.idempotent_write and this.acks (the expression containing
this.idempotent_write == true && this.acks.or("all") != "all"); apply the same
normalization wherever acks is compared in this file (and the other noted
ranges).

Comment on lines +118 to +119
|record_delivery_timeout
|ms`. This is evaluated before writing a request or after a produce response.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix truncated record_delivery_timeout description text.

Line 119 starts mid-sentence (`ms``), so the field description is incomplete and unclear in the release notes table.

✏️ Proposed patch
 |record_delivery_timeout
-|ms`. This is evaluated before writing a request or after a produce response.
+|The maximum time a record may be retained by the client before failing (`0` disables the timeout). This is evaluated before writing a request or after a produce response.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
|record_delivery_timeout
|ms`. This is evaluated before writing a request or after a produce response.
|record_delivery_timeout
|The maximum time a record may be retained by the client before failing (`0` disables the timeout). This is evaluated before writing a request or after a produce response.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@modules/get-started/pages/whats-new.adoc` around lines 118 - 119, The table
entry for record_delivery_timeout is truncated; update the description for the
record_delivery_timeout field to be a complete sentence mentioning the unit
(milliseconds) and clarifying when it is evaluated (e.g.,
"record_delivery_timeout — timeout in ms; evaluated before writing a request or
after receiving a produce response"). Edit the whats-new.adoc entry for
record_delivery_timeout to replace the incomplete "ms`" fragment with the full
explanatory sentence so the release notes table reads clearly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-docs documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants