Repository navigation
Conversation
run_function took the return type from the IR and read only i32 and i64, as signed. It now takes an optional ctypes type, so a caller can read an unsigned or narrow result the way C would see it. Without one, it does what it did before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
hypothesis joins the test extra. conftest.py registers three profiles, chosen with HYPOTHESIS_PROFILE: - dev (the default): 100 examples. - ci: 100 examples, derandomized, so a CI failure always reproduces. - thorough: 3000 examples, for bug hunting. None of them has a deadline: an example compiles a program twice. CI runs the suite with the ci profile. The example database (.hypothesis/) is ignored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/property generates integer programs, prints each as PythonBPF and as C, and runs both on the host. clang's result is the specification: - clang compiles the C (-O0 -fwrapv) to a native object. - PythonBPF's IR is retargeted to the host. - Both are JIT-run in a worker process, so a miscompile that traps cannot end the pytest run. cints.py models C's promotions, usual arithmetic conversions and operators. It discards programs that would be undefined in C, and it is checked against clang on every example, so a model bug is never blamed on the compiler. Three properties: - test_int_expressions: random nested expressions over every ctypes integer type and c_bool, with typed and untyped temporaries. The worked examples of integers.md are pinned. - test_int_operators: every binary operator on every pair of types. - test_int_conversions: every (source type, destination type, site) triple. The sites are constructor, declaration, reassignment, untyped local, global store, return, global initialiser and each augmented assignment. Deliberately breaking >>, %, /, the usual arithmetic conversions, promotion, sign extension or c_bool conversion fails a property each time, shrunk to a program of two or three lines. Divergences found and not yet fixed are in known_gaps.py, each pinned as a strict xfail in test_known_gaps.py: - negated-literal-rank: -2147483648 is ranked int, not long long. - untyped-local-unary-or-compare: `t = -x`, `t = not x` and `t = a < b` give "Undefined variable". - shift-result-type: a shift is typed by both operands, not by its promoted left operand. This includes <<= and >>=. - untyped-copy-keeps-narrow-type: `t = a` keeps a's narrow type, where integers.md says an undeclared local is 64 bits. - The global_init site: a negative @bpfglobal initialiser becomes 0, and c_bool(2) becomes 0. tests/README.md documents the suite and the plan for what comes next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review of the first version of the suite found four holes. - The conversion sites returned the converted value, which shows its bits but not the type it was stored as. The untyped_local site passed over a real bug that way. Each site now reads the value back through an expression whose result depends on its sign and width. - With the type drawn at random, 25 examples per site left a single type with about 3. A deliberately broken c_bool conversion went undetected under some seeds. The conversion property is now parametrized by destination type and the operator property by left operand type, a sixteenth of the profile's examples each. - The global_init site was a strict xfail. Hypothesis shrank its expected failures, which took 115 of the property's 187 seconds. Its two bugs are now pinned in test_known_gaps.py, and the site steps around them. - Gaps are now stepped around by drawing only inputs that avoid them, not by assume(). Filtering tripped Hypothesis's filter_too_much health check. untyped-copy-keeps-narrow-type is renamed untyped-copy-keeps-source-type. `t = a` keeps a's type for any source that is not 64-bit, c_uint32 and c_int32 included, not only narrow ones. Under the ci profile, each of these deliberate compiler bugs fails a property: - >> always logical - % always unsigned - / always unsigned - equal-width mixed signs going signed - promotion keeping an unsigned sign - widening always by zero-extension - c_bool conversion by truncation Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
r41k0u
requested review from
varun-r-mallya
and
a balanced review from Copilot
and removed request for
Copilot and
varun-r-mallya
October 27, 2026 21:15
Copilot stopped reviewing on behalf of
r41k0u due to an error
October 27, 2026 21:16
There was a problem hiding this comment.
🟡 Changes recommended
The property tests can mask real failures and miss type combinations and a known divergence, while the harness can hang on timeouts and accumulate temporary files.
6 open issues
What changed in this PR
Adds Hypothesis property-based tests to strengthen PythonBPF’s integer conversion checks by comparing generated programs against clang and a reference model.
Changes:
- Generates tests for integer expressions, operators and conversion sites.
- Runs compiled programs in isolated workers and pins known divergences as strict expected failures.
- Adds test profiles, CI integration and documentation.
| File | Description |
|---|---|
tests/README.md |
Documents property tests, known gaps and execution profiles. |
tests/property/test_known_gaps.py |
Pins known compiler divergences as strict expected failures. |
tests/property/test_int_operators.py |
Tests binary operators across operand types and values. |
tests/property/test_int_expressions.py |
Tests generated expressions and documented examples. |
tests/property/test_int_conversions.py |
Tests conversion sites with value and type-sensitive probes. |
tests/property/programs.py |
Generates equivalent PythonBPF and C programs. |
tests/property/known_gaps.py |
Defines known-divergence predicates and examples. |
tests/property/harness.py |
Compiles, executes and compares generated programs. |
tests/property/cints.py |
Models integer types, conversions and operations. |
tests/property/__init__.py |
Establishes the property-test package. |
tests/framework/host_jit.py |
Supports explicit return types when executing host code. |
tests/conftest.py |
Registers development, CI and thorough Hypothesis profiles. |
pyproject.toml |
Adds Hypothesis to test dependencies. |
.gitignore |
Ignores Hypothesis-generated artifacts. |
.github/workflows/test.yml |
Selects the deterministic CI profile. |
Review effort: Balanced
Comment on lines
+83
to
+85
| return _pool.submit(fn, *args).result(timeout=60) | ||
| except BrokenProcessPool: | ||
| _pool = None |
Comment on lines
+109
to
+112
| def compile_pythonbpf(source: str) -> str: | ||
| stem = WORKDIR / f"p{next(_names)}" | ||
| py, ll = stem.with_suffix(".py"), stem.with_suffix(".ll") | ||
| py.write_text(source) |
Comment on lines
+58
to
+62
| lambda p: any( | ||
| isinstance(n, Neg) | ||
| and isinstance(n.operand, Lit) | ||
| and n.operand.value == 2**31 | ||
| for n in nodes(p) |
Comment on lines
+157
to
+160
| @pytest.mark.parametrize("dst", ALL_TYPES, ids=lambda t: t.name) | ||
| @pytest.mark.parametrize("site", SITES) | ||
| @settings(max_examples=max(5, settings.default.max_examples // 16)) | ||
| @given(data=st.data()) |
Comment on lines
+40
to
+41
| @settings(max_examples=max(5, settings.default.max_examples // 16)) | ||
| @given(right_ty=TYPES, data=st.data()) |
Comment on lines
+23
to
+24
| marks=pytest.mark.xfail( | ||
| strict=True, raises=AssertionError, reason=g.reason |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adding PBTs to the project to find more of those pesky type conversion edge cases easily, and make the type conversion logic more airtight.