Skip to content

Docs/configuration walkthrough revamp#84

Merged
dleshchev merged 4 commits into
mainfrom
docs/configuration-walkthrough-revamp
May 14, 2026
Merged

Docs/configuration walkthrough revamp#84
dleshchev merged 4 commits into
mainfrom
docs/configuration-walkthrough-revamp

Conversation

@RamyaGuru
Copy link
Copy Markdown
Collaborator

Summary

Revises docs/tutorials/configuration-walkthrough.md to address review feedback from #66 and bring the page back in sync with the current example YAMLs. Adds a CI gate that catches the drift class this PR fixes,
so the base walkthrough can't silently fall out of sync with examples/daqiri_bench_raw_tx_rx.yaml again.

Changes

Decision tree (#choosing-an-example-config):

  • New prose intro "Choosing the appropriate DAQIRI backend for your setup" before the questions, framing the DPDK / RoCE / sockets choice by hardware and topology (with a
    ConnectX link). Addresses the review comment on #66.
  • Original Q1 ("baseline throughput") restored and lightly trimmed to remove redundancy with the new prose section. Q2–Q5 unchanged.

Annotated walkthrough:

  • Base TX+RX section realigned with the current examples/daqiri_bench_raw_tx_rx.yaml: stream_type: "raw" and loopback: "" added, legacy manager: removed, single-segment RX restored, buf_size: 8064,
    payload_size: 8000, minimal bench_rx. Roughly two dozen annotations renumbered.
  • New ### Header-data split (HDS) section — diff-only walkthrough of daqiri_bench_raw_tx_rx_hds.yaml (new CPU memory regions, chained memory_regions: per queue, ipv4_len flow rule, matched
    bench_tx.payload_size).
  • New ### Packet reordering on the GPU section — diff-only walkthrough of daqiri_bench_raw_tx_rx_reorder_seq_1024.yaml (dedicated Reorder_RX_GPU region with size math, batch_size / timeout_us, full
    reorder_configs: block walked key by key, TX-side seqno injection).
  • Annotation style standardized to a two-tier scheme: scalar keys get compact value cards (key · type · required/default · supported values); section keys keep free-prose explanations.
  • hl_lines added to every HDS and reorder snippet so the feature-defining values stand out visually, matching the treatment in the base walkthrough.

CI / drift guard:

  • scripts/check_doc_refs.py gains a base_walkthrough_drift() check that extracts the ```yaml block under ### Base TX+RX config, strips `# (N)!` annotation markers, parses both it and
    `examples/daqiri_bench_raw_tx_rx.yaml` with `yaml.safe_load`, and fails with a unified diff if the parsed dicts don't match. PyYAML is already transitively available via `mkdocs-material`; no new CI dependency.
  • HDS and reorder snippets are intentionally fragments and are not strict-checked — the pairing rule for those lives in .claude/rules/docs-sync.md.

Reviewing

Preview locally with mkdocs serve:

git fetch origin
git checkout docs/configuration-walkthrough-revamp

# First time only:
python3 -m venv .venv
source .venv/bin/activate
pip install mkdocs-material

mkdocs serve

Then open http://127.0.0.1:8000/daqiri/tutorials/configuration-walkthrough/. That's the only page modified by this PR.

RamyaGuru and others added 3 commits May 13, 2026 16:27
- Add "Choosing the appropriate DAQIRI backend" prose intro before
  the example-config questions
- Realign the annotated base walkthrough with the current
  daqiri_bench_raw_tx_rx.yaml (stream_type, loopback, single-segment
  RX, buf_size 8064, payload_size 8000, minimal bench_rx)
- Add diff-style walkthrough sections for HDS and GPU packet
  reordering, with feature-defining lines highlighted
- Standardize scalar annotations as two-tier value cards
  (key, type, required/default, supported values)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>
- check_doc_refs.py: parse the ```yaml block under "### Base TX+RX
  config" in configuration-walkthrough.md (stripping # (N)! annotation
  markers), parse examples/daqiri_bench_raw_tx_rx.yaml, and fail with
  a unified diff if the two structures don't match. PyYAML is already
  transitively available via mkdocs-material; no new CI dependency
  needed. Only the base walkthrough is checked; the HDS and reorder
  diff snippets are intentional fragments.
- docs-sync rule: add an "Annotated walkthrough pairing" bullet
  listing the three YAMLs reproduced in the walkthrough section and
  noting which one is CI-enforced vs. review-by-hand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>
@RamyaGuru RamyaGuru requested a review from dleshchev May 14, 2026 15:48
@RamyaGuru RamyaGuru marked this pull request as ready for review May 14, 2026 15:50
@greptile-apps
Copy link
Copy Markdown

greptile-apps Bot commented May 14, 2026

Greptile Summary

This PR revamps docs/tutorials/configuration-walkthrough.md to match the current example YAMLs and adds two new diff-style sections (HDS and GPU reorder), then adds a CI drift-guard in scripts/check_doc_refs.py that structurally compares the embedded base walkthrough block against examples/daqiri_bench_raw_tx_rx.yaml after each commit.

  • Walkthrough realigned: base TX+RX section updated to match the live YAML (stream_type: "raw", loopback: "", buf_size: 8064, minimal bench_rx); annotations renumbered and standardised to a two-tier value-card / free-prose scheme; hl_lines added to HDS and reorder snippets.
  • Two new diff-only sections: Header-data split (HDS) and packet reordering on the GPU, each walking only the delta keys against the base config.
  • CI drift guard: base_walkthrough_drift() in check_doc_refs.py parses both sides with yaml.safe_load and fails with a unified diff on mismatch; the yaml import is kept local with a try/except ImportError guard (resolving the prior review comment).

Confidence Score: 5/5

Documentation and CI script changes only; no library code, no API surface, no build system touched.

All changes are documentation prose, annotation renumbering, and a new Python CI helper. The walkthrough YAML block was verified against examples/daqiri_bench_raw_tx_rx.yaml and matches field-for-field. The drift check correctly keeps the yaml import local with a fallback, as requested in the prior review. No correctness concerns were found that would affect the build or runtime behaviour.

No files require special attention. The minor diagnostic-output quality note on scripts/check_doc_refs.py and the residual stale snippet in docs/index.html are both non-blocking.

Important Files Changed

Filename Overview
docs/tutorials/configuration-walkthrough.md Major revamp: base TX+RX section realigned with the current YAML (stream_type, loopback, buf_size:8064, minimal bench_rx), annotations renumbered. Two new diff-only sections added for HDS and GPU reorder. Annotations standardised to a two-tier value-card / free-prose scheme. Content verified against examples/daqiri_bench_raw_tx_rx.yaml — all fields match, including the rq_q_0 queue name.
mkdocs.yml Single nav-title rename: "Configuration File" → "Configuration YAML Walkthrough". No path change; existing links to tutorials/configuration-walkthrough/ are unaffected.
.claude/rules/docs-sync.md Adds three annotated-walkthrough pairing rules (base full-file, HDS diff-only, reorder diff-only), documenting which YAMLs are governed by CI vs. hand review. Clean addition with no conflicts.
scripts/check_doc_refs.py Adds base_walkthrough_drift() check that extracts the annotated YAML block, strips annotation markers, and compares parsed dicts against the canonical example on disk. The yaml import is kept local with a try/except guard (addressing the previous review comment). One minor issue: the unified diff normalisation does not sort keys, which can produce noisy output.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["check_doc_refs main()"] --> B["known_binaries()"]
    A --> C["known_yamls()"]
    A --> D["decision_tree_coverage()"]
    A --> E["base_walkthrough_drift()"]

    E --> F{"PyYAML\ninstalled?"}
    F -- No --> G["Skip with warning\nother checks still run"]
    F -- Yes --> H["Regex-extract yaml block\nfrom Base TX+RX section"]
    H --> I["Strip annotation markers\nfrom extracted text"]
    I --> J["safe_load both sides\nwalkthrough vs source file"]
    J --> K{"Dicts equal?"}
    K -- Yes --> L["Pass"]
    K -- No --> M["yaml.safe_dump both sides\nbuild unified diff\nreturns error list"]

    style G fill:#f5a623,color:#000
    style L fill:#7ed321,color:#000
    style M fill:#d0021b,color:#fff
Loading

Reviews (2): Last reviewed commit: "Address Greptile review feedback on docs..." | Re-trigger Greptile

@RamyaGuru RamyaGuru marked this pull request as draft May 14, 2026 16:10
- scripts/check_doc_refs.py: move `import yaml` inside
  base_walkthrough_drift() so the script's pre-existing reference and
  coverage checks keep working in environments without PyYAML
  installed. Print a WARNING and skip the drift check when PyYAML is
  unavailable; make the success-summary string conditional on whether
  the drift check actually ran.
- mkdocs.yml: rename the tutorials nav entry from "Configuration File"
  to "Configuration YAML Walkthrough" to clarify it's a walkthrough
  of the YAML config (distinct from API Reference and Configuration).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Signed-off-by: Ramya Gurunathan <rgurunathan@nvidia.com>
@RamyaGuru RamyaGuru marked this pull request as ready for review May 14, 2026 16:56
@dleshchev dleshchev merged commit 5f9f7a4 into main May 14, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants