Skip to content

Commit 0fb1caf

Browse files
Revise README for clarity and additional details
Updated README.md to enhance project description, installation instructions, and usage examples.
1 parent 1adf7d7 commit 0fb1caf

1 file changed

Lines changed: 154 additions & 48 deletions

File tree

README.md

Lines changed: 154 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,77 +1,183 @@
11
# Python-BPF
2-
<p align="center">
3-
<a href="https://www.python.org/downloads/release/python-3080/"><img src="https://img.shields.io/badge/python-3.8-blue.svg"></a>
4-
<a href="https://pypi.org/project/pythonbpf"><img src="https://badge.fury.io/py/pythonbpf.svg"></a>
5-
</p>
62

7-
This is an LLVM IR generator for eBPF programs in Python. We use llvmlite to generate LLVM IR from pure Python. This is then compiled to LLVM object files, which can be loaded into the kernel for execution. We do not rely on BCC to do our compilation.
3+
Python-BPF is an LLVM IR generator for eBPF programs written in Python. It uses [llvmlite](https://github.com/numba/llvmlite) to generate LLVM IR and then compiles to LLVM object files. These object files can be loaded into the kernel for execution. Unlike BCC, Python-BPF performs compilation without relying on its infrastructure.
84

9-
# DO NOT USE IN PRODUCTION. IN DEVELOPMENT.
5+
> **Note**: This project is under active development and not ready for production use.
106
11-
## Video Demo
12-
[Video demo for code under demo/](https://youtu.be/eMyLW8iWbks)
7+
---
138

14-
## Slide Deck
15-
[Slide deck explaining the project](https://docs.google.com/presentation/d/1DsWDIVrpJhM4RgOETO9VWqUtEHo3-c7XIWmNpi6sTSo/edit?usp=sharing)
9+
## Overview
1610

17-
## Installation
18-
- Have `clang` installed.
19-
- `pip install pythonbpf`
11+
* Generate eBPF programs directly from Python.
12+
* Compile to LLVM object files for kernel execution.
13+
* Built with `llvmlite` for IR generation.
14+
* Supports maps, helpers, and global definitions for BPF.
15+
* Companion project: [pylibbpf](https://github.com/pythonbpf/pylibbpf), which provides the bindings required for object loading and execution.
16+
17+
---
18+
19+
## Installation
20+
21+
Dependencies:
22+
23+
* `clang`
24+
* Python ≥ 3.8
25+
26+
Install via pip:
27+
28+
```bash
29+
pip install pythonbpf pylibbpf
30+
```
31+
32+
---
33+
34+
## Example Usage
2035

21-
## Usage
2236
```python
23-
# pythonbpf_example.py
24-
from pythonbpf import bpf, map, bpfglobal, section, compile
25-
from pythonbpf.helpers import bpf_ktime_get_ns
37+
import time
38+
from pythonbpf import bpf, map, section, bpfglobal, BPF
39+
from pythonbpf.helpers import pid
2640
from pythonbpf.maps import HashMap
41+
from pylibbpf import *
42+
from ctypes import c_void_p, c_int64, c_uint64, c_int32
43+
import matplotlib.pyplot as plt
2744

28-
from ctypes import c_void_p, c_int64, c_int32, c_uint64
45+
# This program attaches an eBPF tracepoint to sys_enter_clone,
46+
# counts per-PID clone syscalls, stores them in a hash map,
47+
# and then plots the distribution as a histogram using matplotlib.
48+
# It provides a quick view of process creation activity over 10 seconds.
2949

3050
@bpf
3151
@map
32-
def last() -> HashMap:
33-
return HashMap(key=c_uint64, value=c_uint64, max_entries=1)
52+
def hist() -> HashMap:
53+
return HashMap(key=c_int32, value=c_uint64, max_entries=4096)
3454

3555
@bpf
36-
@section("tracepoint/syscalls/sys_enter_execve")
37-
def hello(ctx: c_void_p) -> c_int32:
38-
print("entered")
39-
return c_int32(0)
40-
41-
@bpf
42-
@section("tracepoint/syscalls/sys_exit_execve")
43-
def hello_again(ctx: c_void_p) -> c_int64:
44-
print("exited")
45-
key = 0
46-
tsp = last().lookup(key)
47-
print(tsp)
48-
ts = bpf_ktime_get_ns()
56+
@section("tracepoint/syscalls/sys_enter_clone")
57+
def hello(ctx: c_void_p) -> c_int64:
58+
process_id = pid()
59+
one = 1
60+
prev = hist().lookup(process_id)
61+
if prev:
62+
previous_value = prev + 1
63+
print(f"count: {previous_value} with {process_id}")
64+
hist().update(process_id, previous_value)
65+
return c_int64(0)
66+
else:
67+
hist().update(process_id, one)
4968
return c_int64(0)
5069

70+
5171
@bpf
5272
@bpfglobal
5373
def LICENSE() -> str:
5474
return "GPL"
5575

56-
def some_normal_function():
57-
print("normal function")
5876

59-
# compiles and dumps object file in the same directory
60-
compile()
77+
b = BPF()
78+
b.load_and_attach()
79+
hist = BpfMap(b, hist)
80+
print("Recording")
81+
time.sleep(10)
82+
83+
counts = list(hist.values())
84+
85+
plt.hist(counts, bins=20)
86+
plt.xlabel("Clone calls per PID")
87+
plt.ylabel("Frequency")
88+
plt.title("Syscall clone counts")
89+
plt.show()
6190
```
62-
- Run `python pythonbpf_example.py` to get the compiled object file that can be then loaded into the kernel.
91+
---
92+
93+
## Architecture
94+
95+
Python-BPF provides a complete pipeline to write, compile, and load eBPF programs in Python:
96+
97+
1. **Python Source Code**
98+
99+
* Users write BPF programs in Python using decorators like `@bpf`, `@map`, `@section`, and `@bpfglobal`.
100+
* Maps (hash maps), helpers (e.g., `ktime`, `deref`), and tracepoints are defined using Python constructs, preserving a syntax close to standard Python.
101+
102+
2. **AST Generation**
103+
104+
* The Python `ast` module parses the source code into an Abstract Syntax Tree (AST).
105+
* Decorators and type annotations are captured to determine BPF maps, tracepoints, and global variables.
106+
107+
3. **LLVM IR Emission**
108+
109+
* The AST is transformed into LLVM Intermediate Representation (IR) using `llvmlite`.
110+
* IR captures BPF maps, control flow, assignments, and calls to helper functions.
111+
* Debug information is emitted for easier inspection.
112+
113+
4. **LLVM Object File Compilation**
114+
115+
* The LLVM IR (`.ll`) is compiled into a BPF target object file (`.o`) using `llc -march=bpf -O2`.
116+
* This produces a kernel-loadable ELF object file containing the BPF bytecode.
117+
118+
5. **libbpf Integration (via pylibbpf)**
119+
120+
* The compiled object file can be loaded into the kernel using `pylibbpf`.
121+
* Maps, tracepoints, and program sections are initialized, and helper functions are resolved.
122+
* Programs are attached to kernel hooks (e.g., syscalls) for execution.
123+
124+
6. **Execution in Kernel**
125+
126+
* The kernel executes the loaded eBPF program.
127+
* Hash maps, helpers, and global variables behave as defined in the Python source.
128+
* Output can be read via BPF maps, helper functions, or trace printing.
129+
130+
This architecture eliminates the need for embedding C code in Python, allowing full Python tooling support while generating true BPF object files ready for kernel execution.
131+
132+
---
63133

64134
## Development
65-
- Make a virtual environment and activate it using `python3 -m venv .venv && source .venv/bin/activate`.
66-
- Run `make install` to install the required dependencies.
67-
- Run `make` to see the compilation output of the example.
68-
- Run `check.sh` to check if generated object file passes through the verifier inside the examples directory.
69-
- Run `make` in the `examples/c-form` directory to modify the example C BPF program to check the actual LLVM IR generated by clang.
70135

71-
### Development Notes
72-
- Run ` ./check.sh check execve2.o;` in examples folder to check if the object code passes the verifier.
73-
- Run ` ./check.sh run execve2.o;` in examples folder to run the object code using `bpftool`.
136+
1. Create a virtual environment and activate it:
137+
138+
```bash
139+
python3 -m venv .venv
140+
source .venv/bin/activate
141+
```
142+
143+
2. Install dependencies:
144+
145+
```bash
146+
make install
147+
```
148+
149+
3. Build and test examples:
150+
151+
```bash
152+
make
153+
```
154+
155+
4. Verify an object file with the kernel verifier:
156+
157+
```bash
158+
./check.sh check execve2.o
159+
```
160+
161+
5. Run an object file using `bpftool`:
162+
163+
```bash
164+
./check.sh run execve2.o
165+
```
166+
167+
6. Explore LLVM IR output from clang in `examples/c-form` by running `make`.
168+
169+
---
170+
171+
## Resources
172+
173+
* [Video demonstration](https://youtu.be/eMyLW8iWbks)
174+
* [Slide deck](https://docs.google.com/presentation/d/1DsWDIVrpJhM4RgOETO9VWqUtEHo3-c7XIWmNpi6sTSo/edit?usp=sharing)
175+
176+
---
74177

75178
## Authors
76-
- [@r41k0u](https://github.com/r41k0u)
77-
- [@varun-r-mallya](https://github.com/varun-r-mallya)
179+
180+
* [@r41k0u](https://github.com/r41k0u)
181+
* [@varun-r-mallya](https://github.com/varun-r-mallya)
182+
183+
---

0 commit comments

Comments
 (0)