docs: clarify managed LLM start event ordering

[willkill07]

Overview

Clarifies the managed execution documentation after #125 so diagrams and notes state that LLM start events are emitted after request intercepts and sanitize-request guardrails, but before execution or stream execution intercepts run.

• I confirm this contribution is my own work, or I have the right to submit it under this project's license.
• I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

• Updates the middleware order sequence and detailed Mermaid flow to include explicit start-event and end-event emission points.
• Aligns FAQ and instrumentation notes with start-event emission before execution intercepts.
• Updates Rust LLM API comments so generated reference docs describe the managed LLM start-event ordering.

Where should the reviewer start?

Start with docs/about/concepts/middleware.md, which contains the diagram and ordering model. Then check crates/core/src/api/llm.rs for the generated API reference wording.

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

• Relates to fix: emit LLM start before execution intercepts #125

Summary by CodeRabbit

• Documentation
  • Clarified middleware execution lifecycle ordering and event timing across documentation and code comments.
  • Improved instrumentation guides to better specify when validations should occur relative to middleware stages.
  • Enhanced middleware pipeline documentation with clearer sequencing of request/response sanitization and event emission points.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Enterprise

Run ID: 6409323b-d48e-4259-8e66-01e4eeb02b6d

📥 Commits

Reviewing files that changed from the base of the PR and between 91ca10c and 78ac36a.

📒 Files selected for processing (6)

• crates/core/src/api/llm.rs
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md
• docs/instrument-applications/instrument-llm-call.md
• docs/instrument-applications/instrument-tool-call.md
• docs/resources/support-and-faqs.md

📜 Recent review details

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Check / Run

🧰 Additional context used

📓 Path-based instructions (30)

**/*.{md,rst,html,txt}

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)

**/*.{md,rst,html,txt}: Always spell NVIDIA in all caps. Do not use Nvidia, nvidia, nVidia, nVIDIA, or NV.
Use an NVIDIA before a noun because the name starts with an 'en' sound.
Do not add a registered trademark symbol after NVIDIA when referring to the company.
Use trademark symbols with product names only when the document type or legal guidance requires them.
Verify official capitalization, spacing, and hyphenation for product names.
Precede NVIDIA product names with NVIDIA on first mention when it is natural and accurate.
Do not rewrite product names for grammar or title-case rules.
Preserve third-party product names according to the owner's spelling.
Include the company name and full model qualifier on first use when it helps identify the model.
Preserve the official capitalization and punctuation of model names.
Use shorter family names only after the full name is established.
Spell out a term on first use and put the acronym in parentheses unless the acronym is widely understood by the intended audience.
Use the acronym on later mentions after it has been defined.
For long documents, reintroduce the full term if readers might lose context.
Form plurals of acronyms with s, not an apostrophe, such as GPUs.
In headings, common acronyms can remain abbreviated. Spell out the term in the first or second sentence of the body.
Common terms such as CPU, GPU, PC, API, and UI usually do not need to be spelled out for developer audiences.

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,rst,html}

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-brand-terminology.md)

Link the first mention of a product name when the destination helps the reader.

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,rst,txt}

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-guide.md)

**/*.{md,rst,txt}: Spell NVIDIA in all caps. Do not use Nvidia, nvidia, or NV.
Format commands, code elements, expressions, package names, file names, and paths as inline code.
Use descriptive link text. Avoid raw URLs and weak anchors such as 'here' or 'read more.'
Use title case consistently for technical documentation headings.
Introduce code blocks, lists, tables, and images with complete sentences.
Write procedures as imperative steps. Keep steps parallel and split long procedures into smaller tasks.
Prefer active voice, present tense, short sentences, contractions, and plain English.
Use can for possibility and reserve may for permission.
Use after for temporal relationships instead of once.
Prefer refer to over see when the wording points readers to another resource.
Avoid culture-specific idioms, unnecessary Latinisms, jokes, and marketing exaggeration in technical documentation.
Spell out months in body text, avoid ordinal dates, and use clear time zones.
Spell out whole numbers from zero through nine unless they are technical values, parameters, versions, or UI values.
Use numerals for 10 or greater and include commas in thousands.
Do not add trademark symbols to learning-oriented documentation unless the source, platform, or legal guidance explicitly requires them.
Do not add trademark symbols to NeMo Flow learning documentation by default.
Do not rewrite API names, package names, command flags, or code literals for style reasons.

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,markdown,rst}

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/assets/nvidia-style-technical-docs.md)

**/*.{md,markdown,rst}: Use title case consistently in technical documentation headings
Avoid quotation marks, ampersands, and exclamation marks in headings
Keep product, event, research, and whitepaper names in their official title case
Use title case for table headers
Do not force social-media sentence case into technical docs
Use monospace formatting for code elements, commands, parameters, package names, and expressions
Use monospace formatting for directories, file names, and paths
Use angle brackets inside monospace for variables inside paths, such as /home/<username>/.login
Use quotation marks for error messages and strings in documentation
Use bold formatting for UI buttons, menus, fields, and labels in documentation
Use angle brackets between UI labels for menu paths, such as File > Save As
Use italics for new terms on first use in documentation
Use italics for publication titles in documentation
Use plain text formatting for keyboard shortcuts in documentation
Prefer [NVIDIA/NeMo](link) format for GitHub repository references over generic phrases like 'the GitHub repo'
Introduce every code block with a complete sentence
Do not make a code block complete the grammar of the previous sentence
Do not continue a sentence after a code block
Use syntax highlighting when the format supports it for code blocks
Avoid the word 'snippet' unless the surrounding docs already use it as a term of art
Keep inline method, function, and class references consistent with nearby docs, omitting empty parentheses for prose readability when no call is shown
Use descriptive anchor text that matches the destination title when possible for links
Avoid raw URLs in running text in documentation
Avoid generic link anchors such as 'here,' 'this page,' and 'read more' in documentation
Include the acronym in link text if a linked term includes an acronym
Do not link long sentences or multiple sentences in documentation
Avoid links that pull readers away from a procedure unles...

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{html,md}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license headers in HTML and Markdown files using HTML comment syntax

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

docs/**/*.md

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Run ./scripts/build-docs.sh for documentation site changes

docs/**/*.md: Relevant getting-started or reference docs must be updated when examples change
Release-policy docs must point to GitHub Releases as the only release-history source of truth

docs/**/*.md: Use title case for headings in technical documentation
Introduce code blocks, tables, and lists with complete lead-in sentences in documentation

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.md

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Run Markdown link checking via lychee for README.md, CONTRIBUTING.md, and docs/ through pre-commit hooks

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,markdown,py,sh,bash,js,ts,java,cpp,go,rust}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep package names, repo references, and build commands current in documentation

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{RELEASING.md,CHANGELOG.md,docs/**/*.md}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep release-process and release-notes guidance in repo-maintainer docs such as RELEASING.md, not as user-facing docs pages or CHANGELOG.md

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,markdown,py,sh,bash}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Keep stable user-facing wrappers at scripts/ root in docs and examples; only point at namespaced helper paths when documenting internal maintenance work

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,markdown,py,sh,bash,js,ts,example}

📄 CodeRabbit inference engine (.agents/skills/contribute-docs/SKILL.md)

Example commands must match current package names and paths

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{scripts/*.sh,docs/**/*.md}

📄 CodeRabbit inference engine (.agents/skills/contribute-integration/SKILL.md)

Use root ./scripts/*.sh commands in docs and contributor guidance as documented, with implementations under scripts/third-party/

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{docs/**,examples/**,crates/adaptive/**,python/nemo_flow/**,go/nemo_flow/**,**/{example,component}.{ts,tsx,js,rs,py,go}}

📄 CodeRabbit inference engine (.agents/skills/maintain-optimizer/SKILL.md)

Any new adaptive component kind must have documentation, examples, and binding coverage across all supported languages

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{README*,CHANGELOG*,docs/**/*.{md,rst,txt},examples/**/*,*.md}

📄 CodeRabbit inference engine (.agents/skills/rename-surfaces/SKILL.md)

Update documentation, examples, and getting-started guides with new package/module/crate names after rename operations

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{md,txt,rst}

📄 CodeRabbit inference engine (.agents/skills/review-doc-style/SKILL.md)

**/*.{md,txt,rst}: Ensure commands, package names, file paths, and APIs in documentation are correct and not stale; flag incorrect or outdated information as blocking issues
Ensure examples and procedures in documentation will execute successfully with current APIs and commands
Use consistent user-facing terminology throughout documentation that matches current repo terminology
Capitalize NVIDIA correctly in all documentation and public-facing text
Format code, commands, paths, and filenames as inline code (monospace) in documentation
Use descriptive anchor text for links instead of bare URLs or weak labels like 'here' in documentation
Prefer active voice, present tense, short sentences, and plain English in documentation
Structure documentation procedures as imperative steps that are easy to scan and not too long for a single sequence
Prefer 'after' instead of 'once' for temporal references in documentation
Use 'can' instead of 'may' when describing possibility (rather than permission) in documentation
Avoid ambiguous numeric dates and ordinal dates in documentation body text

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{README.md,docs/**/*.md,examples/**/*.{js,ts,py,go,rs}}

📄 CodeRabbit inference engine (.agents/skills/maintain-packaging/SKILL.md)

Keep documentation and examples synchronized with current install, import, and build commands

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.{py,js,ts,tsx,go,rs,md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

Format changed files with the language-native formatter before the final lint/test pass

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• crates/core/src/api/llm.rs
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{README.md,CONTRIBUTING.md,docs/**/*.md}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

For docs-only changes, run targeted checks only if commands, package names, or examples changed. Use just docs for docs-site builds and just docs-linkcheck when links changed

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}

⚙️ CodeRabbit configuration file

{docs/**,README.md,CONTRIBUTING.md,RELEASING.md,SECURITY.md}: Review documentation for technical accuracy against the current API, command correctness, and consistency across language bindings.
Flag stale examples, missing SPDX headers where required, and instructions that no longer match CI or pre-commit behavior.

Files:

• docs/instrument-applications/instrument-tool-call.md
• docs/instrument-applications/instrument-llm-call.md
• docs/resources/support-and-faqs.md
• docs/about/concepts/middleware.md
• docs/instrument-applications/advanced-guide.md

**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-ffi-surface/SKILL.md)

**/*.rs: Run cargo fmt --all for FFI work as it is Rust work
Run just test-rust for FFI validation
Run cargo clippy --workspace --all-targets -- -D warnings to enforce warnings-as-errors linting

**/*.rs: Run cargo fmt --all for Rust code formatting
Run cargo clippy --workspace --all-targets -- -D warnings to enforce Rust linting with no warnings
Run just test-rust as the shared-runtime build/test wrapper for Rust changes

Use Rust snake_case naming convention for Rust code

**/*.rs: Any Rust change must run just test-rust
Any Rust change must run cargo fmt --all
Any Rust change must run cargo clippy --workspace --all-targets -- -D warnings

**/*.rs: Run cargo fmt --all for Rust code formatting when Node changes touch Rust files
Run cargo clippy --workspace --all-targets -- -D warnings to enforce strict linting when Rust files changed as part of Node work

**/*.rs: Always run just test-rust when any Rust code changes
Always run cargo fmt --all when any Rust code changes
Always run cargo clippy --workspace --all-targets -- -D warnings when any Rust code changes

If any Rust files changed as part of the Python work, also run cargo fmt --all, just test-rust, and cargo clippy --workspace --all-targets -- -D warnings

Files:

• crates/core/src/api/llm.rs

**/*.{rs,go,js,ts,tsx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Include SPDX license headers in all Rust, Go, JavaScript, and TypeScript source files using C-style comment syntax

Files:

• crates/core/src/api/llm.rs

**/*.{rs,py,go,js,ts,tsx}

📄 CodeRabbit inference engine (CONTRIBUTING.md)

Use SONAR_IGNORE_START / SONAR_IGNORE_END markers only for documented false positives that cannot be resolved in code; keep ignored blocks small, add explanatory comments, and require reviewer sign-off

Files:

• crates/core/src/api/llm.rs

crates/core/src/api/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/add-binding-feature/SKILL.md)

crates/core/src/api/**/*.rs: Implement public API changes first in crates/core/src/api/ and related core modules such as crates/core/src/api/runtime/, crates/core/src/codec/, or crates/core/src/json.rs
Use snake_case naming convention for Rust core API implementations

Files:

• crates/core/src/api/llm.rs

{crates/core/src/api/tool.rs,crates/core/src/api/llm.rs}

📄 CodeRabbit inference engine (.agents/skills/add-middleware/SKILL.md)

Wire middleware chain execution into the appropriate pipeline stage in crates/core/src/api/tool.rs or crates/core/src/api/llm.rs by calling the chain method at the correct execution point

Files:

• crates/core/src/api/llm.rs

**/*.{js,ts,tsx,jsx,py,rs,go,java,c,cpp,h,cc,cxx,cs,rb,php,swift,kt}

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changed files must be formatted with the language-native formatter

Files:

• crates/core/src/api/llm.rs

crates/{core,adaptive}/**

📄 CodeRabbit inference engine (.agents/skills/prepare-pr/SKILL.md)

Changes to crates/core or crates/adaptive must run the full language matrix

Files:

• crates/core/src/api/llm.rs

{crates/core,crates/adaptive}/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/test-wasm-binding/SKILL.md)

If the change touched shared runtime semantics in crates/core or crates/adaptive, also use validate-change

Files:

• crates/core/src/api/llm.rs

crates/core/**/*.rs

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

When crates/core changes, run the full validation matrix across Rust, Python, Go, Node.js, and WebAssembly

crates/core/**/*.rs: Use Json = serde_json::Value in Rust-facing runtime APIs where the existing code expects JSON payloads.
Use Result<T> with FlowError in core runtime paths. Keep errors explicit and binding-appropriate at the wrapper layer.

Files:

• crates/core/src/api/llm.rs

**/*.{rs,py,js,ts,tsx,go}

📄 CodeRabbit inference engine (.agents/skills/validate-change/SKILL.md)

During iteration, prefer uv run pre-commit run --files <changed files...> for targeted validation

Files:

• crates/core/src/api/llm.rs

crates/{core,adaptive}/**/*.rs

⚙️ CodeRabbit configuration file

crates/{core,adaptive}/**/*.rs: Review the Rust runtime for async correctness, scope isolation, middleware ordering, and event lifecycle regressions.
Pay close attention to task-local/thread-local scope propagation, callback lifetimes, stream finalization, and root_uuid isolation.
Public API changes should preserve existing behavior unless tests and docs show the intended migration path.

Files:

• crates/core/src/api/llm.rs

🔇 Additional comments (6)

crates/core/src/api/llm.rs (1)

462-465: LGTM!

Also applies to: 492-495, 595-597, 625-626

docs/about/concepts/middleware.md (1)

111-112: LGTM!

Also applies to: 134-134, 147-147, 149-150, 152-157, 184-184, 186-186, 199-200, 208-208, 212-216, 219-224, 226-226, 228-228, 232-233, 236-236, 240-240

docs/instrument-applications/advanced-guide.md (1)

210-217: LGTM!

docs/resources/support-and-faqs.md (1)

255-255: LGTM!

Also applies to: 257-259, 285-289

docs/instrument-applications/instrument-llm-call.md (1)

221-223: LGTM!

docs/instrument-applications/instrument-tool-call.md (1)

201-203: LGTM!

---

Walkthrough

Documentation updates clarify middleware execution ordering across the NeMo-Flow codebase. Changes consistently position request-sanitization and start-event emission before execution intercepts, with response-sanitization and end-event emission after callback execution. Updates span API function docs, conceptual middleware reference, and user integration guides with expanded diagrams showing observability event sequencing.

Changes

Middleware Execution Ordering Documentation

Layer / File(s)	Summary
API Function Documentation crates/core/src/api/llm.rs	Updated llm_call_execute and llm_stream_call_execute doc comments to clarify that start event emits before execution intercepts run, execution intercepts and callback follow, and end event emits after (without output on failure for non-streaming, with single end-event for streaming).
Middleware Ordering Reference Documentation docs/about/concepts/middleware.md	Expanded "Managed Execution Order" section with reworded header, updated sequence diagram to position event subscriber emission before execution-intercept wrapping, revised numbered steps to explicitly include sanitize-request and start-event timing, and reworked flowchart with dedicated observability nodes for start/end events and event subscribers with updated class mappings.
Integration & Validation Guides docs/instrument-applications/advanced-guide.md, docs/resources/support-and-faqs.md, docs/instrument-applications/instrument-llm-call.md, docs/instrument-applications/instrument-tool-call.md	Updated middleware execution order descriptions and validation checklists to specify that LLM/tool start input reflects request after both request intercepts and sanitize-request guardrails; clarified pipeline ordering to position start-event emission and request sanitization before execution intercepts, with response sanitization and end-event emission after callback execution.

Possibly related PRs

• NVIDIA/NeMo-Flow#125: Introduces LLM lifecycle ordering behavior (start event before execution intercepts) that this PR documents in API and user-facing guidance.

---

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	Title follows Conventional Commits format with 'docs' type and provides a clear, imperative summary of the documentation clarification.
Description check	✅ Passed	Description includes all required template sections with clear overview, detailed change summary, reviewer guidance, and issue reference.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}