refactor(indexer): split pipeline.rs (2374 lines) into 7-module per-concern tree#17
Conversation
src/indexer/pipeline.rs (2374 lines) → src/indexer/pipeline/{mod,embed,
context,python_modules,resolve,index_files,tests}.rs. mod.rs (237 lines)
keeps the public entry points (run_full_index, ensure_file_indexed,
run_incremental_index{,_cached}) + IndexStats/IndexResult/ProgressFn +
collect_dirty_node_ids glue. The Phase-0..3 orchestrator stays whole in
index_files.rs (827 lines) — its phases share local transaction/atomics/
batch_parsed/name_to_ids state that splitting would have to thread back
in via large arg lists.
Per-concern submodules:
embed.rs (71) — embed_and_store_batch + sequential fallback
context.rs (197) — categorize_edges + format_route_from_metadata
+ regenerate_context_strings + repair_null_*
python_modules.rs (73) — build_python_module_map + resolve_python_*
resolve.rs (204) — refine_ambiguous_targets + resolve_pending_calls
index_files.rs (827) — Phase 0..3 orchestrator + FileIndexed
tests.rs (884) — all #[cfg(test)] tests
Public surface preserved (`crate::indexer::pipeline::{run_full_index,
ensure_file_indexed, run_incremental_index, run_incremental_index_cached,
embed_and_store_batch, repair_null_context_strings, IndexStats,
IndexResult, ProgressFn}`). External callers in cli.rs / mcp/server /
tests / benches / claude-plugin all keep their imports unchanged.
Verification:
- cargo check: clean
- cargo +1.95.0 clippy --no-default-features -- -D warnings: clean
- cargo +1.95.0 clippy --all-targets -- -D warnings: clean
- cargo test --release: 292 lib + 6 + 44 + 19 + 6 + 54 = 421 tests, 0 failed
(1 pre-existing #[ignore])
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Warning Rate limit exceeded
You’ve run out of usage credits. Purchase more in the billing tab. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (8)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…/ pipeline) Pure refactor release — three biggest source files (8049 lines as monoliths) decomposed into 26 per-concern submodules. Zero behavior change, public surface preserved across all three splits. - src/storage/queries.rs (2892 lines) → src/storage/queries/ (10 files, #15) - src/parser/relations.rs (2783 lines) → src/parser/relations/ (9 files, #16) - src/indexer/pipeline.rs (2374 lines) → src/indexer/pipeline/ (7 files, #17) External callers in cli.rs / mcp/server / tests / benches / claude-plugin keep their imports unchanged. Two orchestrator-style functions kept whole deliberately (walk_for_relations ~650 lines, index_files ~770 lines) — their arms/phases share local state that splitting would either duplicate or thread back via large arg lists. Verification: cargo check + cargo +1.95.0 clippy (no-default-features + all-targets, -D warnings) + cargo test --release (292 lib + 129 integration = 421 tests, 0 failed). Pre-merge CI green on all three PRs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Summary
src/indexer/pipeline.rs(2374 lines) intosrc/indexer/pipeline/{mod,embed,context,python_modules,resolve,index_files,tests}.rs. Completes the 三巨头 split started with refactor(storage): split queries.rs into 10-module per-domain tree #15 (queries.rs) and refactor(parser): split relations.rs (2783 lines) into 9-module per-concern tree #16 (relations.rs).mod.rs(237 lines) keeps public entry points + structs + glue. The Phase-0..3 orchestrator stays whole inindex_files.rs(827 lines) — its phases share local transaction/atomics/batch_parsed/name_to_ids state that splitting would have to thread back via large arg lists.mod.rsembed.rsembed_and_store_batch+ sequential fallback pathcontext.rscategorize_edges+format_route_from_metadata+regenerate_context_strings+repair_null_context_stringspython_modules.rsbuild_python_module_map+resolve_python_module_targetsresolve.rsrefine_ambiguous_targets+resolve_pending_callsindex_files.rsFileIndexedtests.rs#[cfg(test)]testsPublic surface preserved (
crate::indexer::pipeline::{run_full_index, ensure_file_indexed, run_incremental_index, run_incremental_index_cached, embed_and_store_batch, repair_null_context_strings, IndexStats, IndexResult, ProgressFn}). External callers incli.rs/mcp/server/tests//benches/need no edits.Submodule items use
pub(super);embed_and_store_batchandrepair_null_context_stringsgetpub use'd at mod.rs to keep their existing public path.Test plan
cargo check— cleancargo +1.95.0 clippy --no-default-features -- -D warnings— cleancargo +1.95.0 clippy --all-targets -- -D warnings— cleancargo test --release— 292 lib + 129 integration = 421 tests, 0 failed (1 pre-existing#[ignore])🤖 Generated with Claude Code