Skip to content

fix: validate batch_size and empty-query benchmarks - #25

Closed
HarshRajSinghania wants to merge 1 commit into
AgentPostmortem:mainfrom
HarshRajSinghania:fix/ingest-batch-and-empty-benchmark
Closed

HarshRajSinghania wants to merge 1 commit into
AgentPostmortem:mainfrom
HarshRajSinghania:fix/ingest-batch-and-empty-benchmark

Conversation

@HarshRajSinghania

Copy link
Copy Markdown
Contributor

Summary

Guard two unhandled boundary cases in ingestion and the benchmark harness.

Motivation

Both issues are open, unassigned, labeled help wanted / good first issue, with explicit acceptance criteria. No open PRs existed for them.

Implementation

  • batched() now raises ValueError("batch_size must be >= 1") when size < 1.
  • run_benchmark treats empty per-mode recall lists as 0.0 instead of dividing by zero. Mode results are still emitted with n_queries=0 and qps=0.0. Index recall for non-dense / empty cases remains NaN as before.

Testing

Ran locally:

PYTHONPATH=src python -m pytest tests -q

Result: 34 passed.

New coverage:

  • tests/test_ingest.py: batched([1,2,3], 0) and batch_size=-1 raise the specified error; ingest_documents(..., batch_size=0) does the same.
  • tests/test_benchmark.py::test_benchmark_zero_queries_returns_empty_modes: empty query set returns modes with zeroed metrics.

Fixes #22
Fixes #23

Reject batch_size < 1 with ValueError("batch_size must be >= 1") instead of a raw range() error.
Return zeroed ModeResult metrics when a dataset has no queries instead of dividing by zero.
@royalpinto007

Copy link
Copy Markdown
Member

Thanks! Merging #26 for this one since it also guards ingest_documents directly (fail-fast) with lighter mock-based tests. Appreciate the work.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

run_benchmark ZeroDivisionErrors on a zero-query dataset ingest_documents(batch_size=0) dies with raw range() error

2 participants