Skip to content

fix: make grep truncation exhaustively recoverable - #905

Open
kvandre12-commits wants to merge 3 commits into
mpfaffenberger:mainfrom
kvandre12-commits:fix/exhaustive-grep-pagination
Open

fix: make grep truncation exhaustively recoverable#905
kvandre12-commits wants to merge 3 commits into
mpfaffenberger:mainfrom
kvandre12-commits:fix/exhaustive-grep-pagination

Conversation

@kvandre12-commits

@kvandre12-commits kvandre12-commits commented Sep 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #904. That PR made the 50-match cap visible through truncated; this PR makes the remaining results recoverable without changing query semantics.

  • add next_offset to bounded grep results
  • accept offset on the registered grep tool and both local/backend implementations
  • sort local ripgrep output by path so repeated calls traverse deterministically
  • keep context rows outside the 50-real-match page budget
  • define completeness precisely: repeat the same query/directory with next_offset until it is null

Why

truncated=True prevents a false completeness claim, but “narrow the search” changes the query and cannot prove that every result from the original search was observed. Offset pagination preserves the bounded response while allowing exhaustive traversal of the unchanged search field.

Verification

  • focused grep/backend/rendering/i18n suite: 287 passed in 5.68s
  • changed Python files: ruff format --check clean
  • scoped import autofix: ruff check --fix --select I clean
  • git diff --check clean

Direct 441-match A/B receipt against current origin/main

Observation Current main (#904) Candidate
First response 50, truncated=True 50, truncated=True, next_offset=50
Continuation call rejected (TypeError) accepted
Page sizes 50 only 50 x 8 + 41
Total recovered 50/441 441/441
Omissions 391 0
Duplicates 0 0
Exact deterministic order not applicable yes
Terminal state no continuation truncated=False, next_offset=None

Environment notes

Verification ran on Android/Termux with Python 3.14.6. Ruff 0.15.13 (the repository CI pin) cannot build on Android; the available Ruff 0.16.3 reports pre-existing full-tree findings, so Linux CI remains authoritative for the pinned lint job. A broad non-browser test run reached platform-specific failures unrelated to grep; all directly affected coverage passes in the focused 287-test run.

…agination

# Conflicts:
#	code_puppy/tools/file_operations.py
#	code_puppy/tools/tools_content.py
#	tests/tools/test_grep_live_behavior.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant