BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training

Reference implementation of BEP, a binary-native, gradient-free error-propagation algorithm for training multi-layer binary MLPs and binary RNNs (BEP-TT for the recurrent case). BEP is introduced in:

BEP: A Binary Error Propagation Algorithm for Binary Neural Networks Training. Colombo et al., ICLR 2026. https://openreview.net/forum?id=jxtCMoZIu8

BEP extends the prior work "Training Multi-Layer Binary Neural Networks With Random Local Binary Error Signals" (Colombo et al., 2025), which is the SotA baseline in the paper (see Baselines).

Repository layout

Path	Contents
src/	The BEP contribution: binary MLP trainer (train_mlp.py), binary RNN / BEP-TT trainers (train_rnn.py, train_rnn_kfold.py), and the layers/, subnet/, dataset/ packages.
baselines/qat_larq/	Isolated Quantization-Aware-Training baseline (TensorFlow 2.10 + Larq). Own pyproject.toml + uv.lock: a separate environment (see its README).
out/	Committed result CSVs (the data behind every figure/table).
fig/	Paper figures (PDF), regenerated by Visualize.ipynb from out/.
extract_features.py	One-off AlexNet feature extractor for the CIFAR-10 / Imagenette transfer-learning datasets.
Visualize.ipynb	Reproduces all figures/tables from the committed out/*.csv.

Installation

Requires Python 3.12 and an NVIDIA GPU. Install uv, then:

uv sync

⚠️ CUDA / cupy setup: read this before running (most common failure)

The trainers run on cupy-cuda12x (CUDA 12). If the host's system CUDA is not 12.x (e.g. CUDA 13), cupy cannot find the CUDA-12 runtime and you will see:

RuntimeError: CuPy failed to load libnvrtc.so.12: ... libnvrtc.so.12: cannot open shared object file
# or, on the first GPU op:
ImportError: libcublas.so.12: cannot open shared object file: No such file or directory

The CUDA-12 libraries are already installed inside the environment as pip nvidia-*-cu12 wheels; you just need them on the loader path. Source the helper (after uv sync, from an activated env or via uv run):

source scripts/cuda_env.sh      # prepends the pip nvidia-*-cu12 wheel dirs to LD_LIBRARY_PATH

or inline:

export LD_LIBRARY_PATH="$(python -c 'import os,nvidia; b=os.path.dirname(nvidia.__file__); print(":".join(os.path.join(b,d,"lib") for d in os.listdir(b) if os.path.isdir(os.path.join(b,d,"lib"))))')${LD_LIBRARY_PATH:+:$LD_LIBRARY_PATH}"

This is unnecessary if your system already provides CUDA 12.

Transfer-learning features (CIFAR-10 / Imagenette)

These datasets use pre-extracted AlexNet features. Regenerate them into .data/<dataset>tl/:

python extract_features.py --dataset cifar10       # -> .data/cifar10tl/{train,test}_{features,labels}.npy
python extract_features.py --dataset imagenette

FashionMNIST, MNIST (S-MNIST) and the UCR datasets download automatically on first use.

Reproducing the paper

Each scripts/run_*.sh launches the training sweep (architectures × gating ν × seeds) for one dataset; runs log to Weights & Biases. The committed out/*.csv are the aggregated results; figures reproduce offline from those CSVs via Visualize.ipynb. Regenerating the CSVs from training itself uses Weights & Biases.

Artifact	Command
Fig. 2: MLP test-acc vs #params (L=2/3)	bash scripts/run_mlp_{synthetic,fmnist,cifar10tl,imagenettetl}.sh
Fig. 3 / Fig. 8: binary RNN on S-MNIST (window length, gating ν, backward horizon)	bash scripts/run_rnn_smnist.sh
Table 1 / Table 3 / Fig. 4-7: RNN on 30 UCR datasets (3-fold CV) and MLP ablations	bash scripts/run_rnn_ucr_kfold.sh; MLP ablations from the run_mlp_*.sh sweeps
QAT baseline (Fig. 2 / Table 1)	see baselines/qat_larq/README.md (separate env)
Plot everything	open Visualize.ipynb

Single-configuration example (binary MLP on FashionMNIST):

python src/train_mlp.py --algo-layer stability --algo-perc cp+r \
    --dataset fmnist --binarize-dataset --layers "525_525_525" --group-size 105 \
    --backprop-first --freeze-last --pre-activation-threshold 0.25 \
    --rob 0.75 --prob-reinforcement 0.5 --prob-update 1.0 \
    --bs 100 --epochs 50 --seed 42 --n-runs 5

Baselines

• Colombo 2025 (SotA): the prior Random Local Binary Error Signals method. It is not reimplemented here; its curves ship as reference data in out/<dataset>/sota.csv. Code: https://github.com/AI-Tech-Research-Lab/BNN · DOI 10.1088/2632-2153/adf0c1.
• QAT: baselines/qat_larq/ (isolated TensorFlow + Larq env).

Citation

If you use this code, please cite both BEP (ICLR 2026) and Colombo et al. 2025.: see CITATION.bib.

Authors and Contacts

If you have questions, suggestions or problems, feel free to open an Issue. You can contact us at:

• Luca Colombo: luca2.colombo@polimi.it