Repository navigation
Add C weight initializers, LR schedules, and extra losses - #52
Merged
Merged
Conversation
…closes #46, closes #47) init.h/init.c: torch nn.init initializers (uniform, xavier uniform/normal, he uniform/normal with fan_in mode) over nir_rng_t, plus a torch fan-in/out helper for Linear/Affine/Conv weight shapes. schedule.h/schedule.c: stateless step-decay, cosine-annealing, and exponential LR schedules mirroring torch StepLR/CosineAnnealingLR/ ExponentialLR, keyed by the checkpoint step counter and applied through the SGD/Adam lr fields. loss.h/loss.c: mean-reduction MSE and softmax cross-entropy over spike counts (the decided spike-count form, matching the snnTorch sum-then-CE recipe and spikeforge's averaged-readout-plus-CE path). Tests: test_init (fans, stats, reproducibility, errors), test_init_graph (set_weights + forward), test_schedule (closed forms, boundaries, optimizer composition), test_loss (closed forms, CE equivalence). 39/39 ctest green in Debug, Release, and ASan/UBSan.
w4ffl35
force-pushed
the
fix/nir-init-loss-sched
branch
from
October 14, 2026 11:21
0f3f5d3 to
85007d6
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #45, closes #46, closes #47.
What
New training-side modules (headers in
include/nir_c_runtime/, sources insrc/, registered in both CMake files). No changes to backward ops, STDP, driver, or checkpoint-state code.#45 — weight initializers (
init.h/init.c)nir_init_uniform,nir_init_xavier_uniform/_normal,nir_init_he_uniform/_normal(torch kaimingfan_inmode), all drawing fromnir_rng_t.nir_init_fansimplements torch_calculate_fan_in_and_fan_outfor Linear/Affine[out, in]and Conv[out, in, *kernel]shapes.#46 — LR schedules (
schedule.h/schedule.c)nir_lr_step_decay,nir_lr_cosine,nir_lr_exponentialmirroring torch StepLR / CosineAnnealingLR / ExponentialLR withlast_epoch = step.base_lr; the counter is the checkpointstep(completed optimizer steps), so resume continues the curve. Header shows the per-epochopt->lrwrite for both SGD and Adam.t_max(rises again, no clamp) — documented.#47 — losses (
loss.h/loss.c)nir_mse_loss: mean-reduction MSE + gradient, mirroringF.mse_losson one sample; serves readout/membrane regression recipes.nir_spike_count_loss: the decided form is cross-entropy over spike counts (not MSE on counts), matching the snnTorch sum-over-time-then-CE recipe and spikeforge's averaged-readout-plus-cross_entropypath. Caller sums the T-step window; each step's spikes share their class's count gradient.Tests
test_init: fan values incl. Conv shapes, mean/variance within tolerance at N=20000, seeded bit-identical reproducibility, error shapes.test_init_graph: Xavier-init fc1 weights land vianir_graph_set_weights(read back byte-identical) and the graph runs forward cleanly.test_schedule: closed-form values incl. boundary steps (9/10, 19/20, T/2, T, 2T), SGD + Adam composition throughlr, error shapes.test_loss: closed-form MSE and spike-count value + gradient, spike-count bit-equalsnir_cross_entropyon the same vector, aliasing rules.-Wall -Wextra -Werror.