Conversation
Add explicit math.isfinite() check in parse_token_int so float('inf'),
float('-inf'), and float('nan') are rejected before reaching Pydantic
int coercion. Previously these values were only handled incidentally via
float.is_integer() returning False.
- Add math.isfinite guard in parse_token_int (models.py)
- Update parse_token_int docstring to document IEEE special float rule
- Add parametrized test for _extract_output_tokens with special floats
- Extend _EQUIVALENCE_CASES cross-check with inf, -inf, and nan
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR clarifies and hardens token-count parsing by explicitly rejecting IEEE 754 special float values (inf, -inf, nan) in parse_token_int, ensuring the contract is documented and guarded against future refactors.
Changes:
- Added an explicit
math.isfinite()guard inparse_token_intbefore thefloat.is_integer()check. - Documented the special-float rejection rule in
parse_token_int’s docstring. - Added regression tests covering
inf,-inf, andnan, and extended equivalence-case coverage to include them.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
src/copilot_usage/models.py |
Adds explicit finite-float validation and documents special-float handling in parse_token_int. |
tests/copilot_usage/test_parser.py |
Adds targeted tests for special floats and extends fast-path vs model-path equivalence cases. |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Quality Gate — Auto-approved
Low-impact defensive fix with good test coverage. The math.isfinite() guard makes parse_token_int's rejection of IEEE 754 special floats (inf, -inf, nan) explicit rather than relying on the incidental behavior of float.is_integer(). No behavioral change for real-world inputs.
Evaluated:
import mathplaced correctly in stdlib imports- Guard positioned correctly before
is_integer()inside theisinstance(raw, float)block - Docstring accurately updated
- Parametrized tests cover all three IEEE 754 specials
- Equivalence cross-check cases extended
- All 8 CI checks green
- Complies with CODING_GUIDELINES.md (typing, testing, formatting)
microsasa
deleted the
fix/875-sanitize-ieee-special-floats-cbae788755389aaf
branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #875
Problem
parse_token_intinmodels.pyonly rejectedfloat('inf')andfloat('nan')incidentally —float.is_integer()happens to returnFalsefor both, but this was undocumented and fragile. The issue requested an explicit guard so the contract is clear and future refactors don't silently regress.Changes
Validator fix (
src/copilot_usage/models.py)import mathand an explicitmath.isfinite(raw)guard inparse_token_intbefore theis_integer()checkNoneruleTests (
tests/copilot_usage/test_parser.py)test_returns_none_for_ieee_special_floatstoTestExtractOutputTokenscoveringinf,nan,-inf_EQUIVALENCE_CASESwithfloat('inf'),float('-inf'), andfloat('nan')so the cross-check between_extract_output_tokensandAssistantMessageDatacovers these valuesPre-existing tests
test_models.py::TestSanitizeNonNumericTokens::test_special_float_maps_to_zeroalready existed and continues to passVerification
make check V=1passes: lint ✅, pyright ✅, bandit ✅, unit tests (99% coverage) ✅, e2e tests ✅