A small, didactic ML library in Ruby, built for learning. The goal is not to compete with production tools like scikit-learn; it is to see how ideas connect from scratch: vectors → losses → gradients → models → a tiny neural net. Every algorithm is written out explicitly so you can read exactly what happens inside — no black boxes, no magic imports, just plain code you can follow step by step.
Who this is for
- Developers who already know Ruby (or any language) and want ML intuition + mechanics.
- Anyone who asked: "What is actually happening inside
model.fit?"
What you need first
- Ruby and Bundler installed.
- Comfort reading loops, arrays, and a little math notation (sums, slopes—nothing heavy).
- Read The learning path once (order matters).
- Run Setup so tests pass on your machine.
- Work top to bottom in Learn by phase: run the snippets in
irbor small scripts. - Use Classical models (cheat sheet) as a memory aid, not as your first read.
- Open
model_stack_map.mdwhen you wonder which model uses which loss or layer.
| Step | Topic | Idea you take away |
|---|---|---|
| 1 | Vector / Matrix |
Data and parameters are structured numbers; shapes must match. |
| 2 | MSE + SGD + LinearRegression |
A loss measures error; gradient descent nudges weights to reduce it. |
| 3 | LogisticRegression |
Same training loop, but outputs probabilities for classification. |
| 4 | KNN / KMeans |
Not everything is "train with gradients"—some models memorize or cluster. |
| 5 | NeuralNetwork (XOR) |
Non-linearity and layers let you solve what a single line cannot. |
| 6 | save / load |
A model is also data you can serialize. |
| 7 | (Optional) Web demo | See training and state in a browser—nice capstone, not required to learn the core. |
flowchart LR
A[Vector Matrix] --> B[LinearRegression]
B --> C[LogisticRegression]
C --> D[KNN KMeans]
D --> E[NeuralNetwork XOR]
E --> F[save load]
bundle installRun the full test suite:
bundle exec rspec -I libWhy start here? Almost every model multiplies numbers in bulk. If shapes are wrong, the bug is usually here.
irb -I librequire "ruby_ml/vector"
v1 = RubyML::Vector.new([1, 2, 3])
v2 = RubyML::Vector.new([4, 5, 6])
v1.shape # => [3]
v1.transpose # => [[1, 2, 3]]
v1.add(v2).to_a # => [5, 7, 9]
v1.sub(v2).to_a # => [-3, -3, -3]
v1.dot(v2) # => 32
v1.mul_scalar(2).to_a # => [2, 4, 6]Vector × matrix multiplication
require "ruby_ml/vector"
vector = RubyML::Vector.new([2, 3])
matrix = [
[1, 4, 7],
[2, 5, 8]
]
vector.mul_matrix(matrix).to_a
# => [8, 23, 38]Basic broadcasting (vector + matrix)
require "ruby_ml/vector"
vector = RubyML::Vector.new([10, 20, 30])
matrix = [
[1, 2, 3],
[4, 5, 6]
]
# axis: :row -> add per column (vector length == number of columns)
vector.broadcast_add_to_matrix(matrix, axis: :row)
# => [[11, 22, 33], [14, 25, 36]]
# axis: :column -> add per row (vector length == number of rows)
RubyML::Vector.new([10, 20]).broadcast_add_to_matrix(matrix, axis: :column)
# => [[11, 12, 13], [24, 25, 26]]Matrix operations
require "ruby_ml/matrix"
require "ruby_ml/vector"
m1 = RubyML::Matrix.new([[1, 2, 3], [4, 5, 6]])
m2 = RubyML::Matrix.new([[10, 20, 30], [40, 50, 60]])
m1.shape # => [2, 3]
m1.transpose.to_a # => [[1, 4], [2, 5], [3, 6]]
m1.add(m2).to_a # => [[11, 22, 33], [44, 55, 66]]
m2.sub(m1).to_a # => [[9, 18, 27], [36, 45, 54]]
m1.mul_scalar(2).to_a # => [[2, 4, 6], [8, 10, 12]]Matrix multiply and dot
require "ruby_ml/matrix"
a = RubyML::Matrix.new([[1, 2, 3], [4, 5, 6]])
b = RubyML::Matrix.new([[7, 8], [9, 10], [11, 12]])
a.mul_matrix(b).to_a
# => [[58, 64], [139, 154]]
a.dot(b).to_a
# => [[58, 64], [139, 154]]Matrix + vector broadcasting
require "ruby_ml/matrix"
require "ruby_ml/vector"
matrix = RubyML::Matrix.new([[1, 2, 3], [4, 5, 6]])
# axis: :row -> vector length matches number of columns
matrix.broadcast_add_vector(RubyML::Vector.new([10, 20, 30]), axis: :row).to_a
# => [[11, 22, 33], [14, 25, 36]]
# axis: :column -> vector length matches number of rows
matrix.broadcast_add_vector([10, 20], axis: :column).to_a
# => [[11, 12, 13], [24, 25, 26]]What to notice: shape and row/column counts are enforced—good errors save you hours.
Idea: predict ŷ ≈ xᵀw + b. The library computes gradients by hand (no autograd) so you can read every step.
require "ruby_ml"
x = (0..5).map { |i| [i.to_f] }
y = x.map { |row| (3.0 * row[0]) + 2.0 } # known line: y = 3x + 2
model = RubyML::Models::LinearRegression.new(learning_rate: 0.01, epochs: 1500)
model.fit(x, y)
model.weights # ~ [3.0]
model.bias # ~ 2.0
model.loss_history.first > model.loss_history.last # loss should drop
model.predict([[6.0], [7.0]]) # ~ [20.0, 23.0]What to notice: loss_history tells a story—learning means the line fits the points better over time.
Logistic regression — same training pattern as linear, but the output is a probability between 0 and 1, then a class using a threshold.
require "ruby_ml"
x = [[0.0], [0.1], [0.2], [0.8], [0.9], [1.0]]
y = [0, 0, 0, 1, 1, 1]
model = RubyML::Models::LogisticRegression.new(
learning_rate: 0.5,
epochs: 2000,
threshold: 0.5
)
model.fit(x, y)
model.loss_history.first > model.loss_history.last # => true (downward trend)
model.predict_proba([[0.1], [0.9]]) # => [low_prob, high_prob]
model.predict([[0.1], [0.9]]) # => [0, 1]
model.score(x, y) # => accuracyk-NN — stores training points; prediction = vote or average among nearest neighbors (no gradient loop).
require "ruby_ml"
x_train = [[0.0], [0.1], [0.2], [0.8], [0.9], [1.0]]
y_train = [0, 0, 0, 1, 1, 1]
classifier = RubyML::Models::KNN.new(k: 3, task: :classification)
classifier.fit(x_train, y_train)
classifier.predict([[0.05], [0.95]]) # => [0, 1]
classifier.score(x_train, y_train) # => accuracy
x_reg = [[0.0], [1.0], [2.0], [3.0]]
y_reg = [0.0, 2.0, 4.0, 6.0]
regressor = RubyML::Models::KNN.new(k: 2, task: :regression)
regressor.fit(x_reg, y_reg)
regressor.predict([[1.5]]) # => [3.0] (neighbor average)
regressor.score(x_reg, y_reg) # => R²k-means — groups points into clusters by moving centroids until assignments stabilize.
require "ruby_ml"
x = [
[0.0, 0.0], [0.2, -0.1], [-0.1, 0.1],
[10.0, 10.0], [9.8, 10.2], [10.1, 9.9]
]
model = RubyML::Models::KMeans.new(k: 2, max_iter: 100, tol: 1e-6)
model.fit(x)
model.centroids # => learned centroids
model.labels # => cluster id per training row
model.predict([[0.0, 0.1], [10.2, 10.1]]) # => cluster ids
model.score(x) # => inertia (lower is better)What to notice: compare "optimize a loss with gradients" (logistic) vs "distance + rules" (k-NN, k-means).
Idea: XOR is not linearly separable with one straight boundary. A small MLP with hidden units can learn it. Backprop is written explicitly—trace Dense, activations, and CrossEntropy in the code.
require "ruby_ml"
x = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]
y = [0, 1, 1, 0]
model = RubyML::Models::NeuralNetwork.xor_default(learning_rate: 0.5)
model.fit(x, y, epochs: 8000)
model.loss_history.first > model.loss_history.last # => true (downward trend)
model.predict_proba(x) # => Nx1 probabilities
model.predict(x) # => [0, 1, 1, 0] (expected)
model.score(x, y) # => accuracyLinear classifier with a simple mistake-driven update. Good contrast with logistic regression's smooth loss.
require "ruby_ml"
# Logical OR (linearly separable)
x = [[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]]
y = [0, 1, 1, 1]
model = RubyML::Models::Perceptron.new(learning_rate: 0.1, epochs: 50)
model.fit(x, y)
model.predict(x) # => [0, 1, 1, 1]
model.score(x, y) # => 1.0Every model supports save(path) and .load(path) as JSON so you can treat a trained model like any other artifact.
require "ruby_ml"
x = [[0.0], [1.0], [2.0], [3.0]]
y = [1.0, 3.0, 5.0, 7.0]
model = RubyML::Models::LinearRegression.new(learning_rate: 0.01, epochs: 2000)
model.fit(x, y)
before = model.predict([[4.0]])
model.save("linear_model.json")
restored = RubyML::Models::LinearRegression.load("linear_model.json")
after = restored.predict([[4.0]])
before # ~= afterLets you train, inspect state, and predict through HTTP—useful to connect "library code" to "something running."
bundle exec rackup --port 4567 -o 0.0.0.0Open http://localhost:4567.
Main endpoints
GET /healthGET /grad— gradient descent visualization pagePOST /train/:modelGET /state/:modelPOST /predict/:model
Dynamic study mode (step-by-step training, adjustable learning_rate / threshold):
- Supported models:
linear_regression,logistic_regression - Session endpoints:
POST /session/start/:modelPOST /session/step/:idPOST /session/update/:idGET /session/state/:idPOST /session/predict/:idPOST /session/stop/:id
- Flow: start session → toggle
running(pause/resume) → advance step-by-step → adjust hyperparameters live
All models in the demo: linear_regression, logistic_regression, knn, kmeans, neural_network
| Model | Main job | Gradient-based? | Historical note |
|---|---|---|---|
| Linear regression | Predict continuous values | Yes | Legendre (1805), Gauss (1809) |
| Logistic regression | Binary classification + probability | Yes | Berkson (1944) |
| k-NN | Classify or regress by neighbors | No (lazy learning) | Fix & Hodges (1950s), Cover & Hart (1967) |
| k-means | Unsupervised clusters | No (assign / update) | Lloyd (1957), MacQueen (1967) |
| Perceptron | Binary linear boundary | Rule-based updates | Rosenblatt (1958) |
| Neural network | Non-linear patterns (e.g. XOR) | Yes | — |
Dependency map (which file uses which loss/layer): model_stack_map.md.
Full suite:
bundle exec rspec -I libTargeted runs:
# Vector / Matrix
bundle exec rspec -I lib spec/lib/ruby_ml/vector_spec.rb
bundle exec rspec -I lib spec/lib/ruby_ml/matrix_spec.rb
# Phase 2 — loss, optimizer, linear regression
bundle exec rspec -I lib spec/lib/ruby_ml/losses/mse_spec.rb spec/lib/ruby_ml/optimizers/sgd_spec.rb spec/lib/ruby_ml/models/linear_regression_spec.rb
# Phase 3 — logistic regression
bundle exec rspec -I lib spec/lib/ruby_ml/losses/binary_cross_entropy_spec.rb spec/lib/ruby_ml/models/logistic_regression_spec.rb
# k-NN
bundle exec rspec -I lib spec/lib/ruby_ml/models/knn_spec.rb
# k-means
bundle exec rspec -I lib spec/lib/ruby_ml/models/kmeans_spec.rbGreen tests mean the numerical behavior matches what the lesson expects—use them when you change code.
Quick reference for how library components couple together. Update this whenever you add a model, loss, optimizer, layer, or activation.
- file:
lib/ruby_ml/models/linear_regression.rb - loss:
RubyML::Losses::MSE - optimizer:
RubyML::Optimizers::SGD - layers / activations: none
How it works: computes predictions as ŷ = w·x + b, measures error with MSE, then nudges w and b in the direction that reduces the loss (gradient descent). Each epoch is one full pass over the training data.
- file:
lib/ruby_ml/models/logistic_regression.rb - loss:
RubyML::Losses::BinaryCrossEntropy - optimizer:
RubyML::Optimizers::SGD - layers / activations:
sigmoidlives inside the model (no separate module)
How it works: same gradient loop as linear regression, but squashes the raw score through a sigmoid so the output is always between 0 and 1. A threshold turns the probability into a class label.
- file:
lib/ruby_ml/models/knn.rb - loss: none
- optimizer: none
- layers / activations: none
How it works: no training phase at all — fit just stores the data. At predict time, it finds the k closest training points by Euclidean distance and returns the majority class (classification) or mean value (regression).
- file:
lib/ruby_ml/models/kmeans.rb - loss: none (minimises inertia internally)
- optimizer: none
- layers / activations: none
How it works: Lloyd's algorithm. Start with k random centroids, assign each point to its nearest centroid, recompute each centroid as the mean of its assigned points, repeat until centroids stop moving. No labels required.
- file:
lib/ruby_ml/models/perceptron.rb - loss: none (classification error drives the update directly)
- optimizer: none (internal perceptron rule)
- layers / activations: step function inside the model
How it works: for each misclassified point, add or subtract the input from the weights. No loss function, no gradient — pure mistake-driven correction. Only converges if the data is linearly separable.
- file:
lib/ruby_ml/models/neural_network.rb - layers:
RubyML::Layers::Dense - activations:
RubyML::Activations::ReLU,RubyML::Activations::Sigmoid - loss:
RubyML::Losses::CrossEntropy - optimizer:
RubyML::Optimizers::SGD
How it works: forward pass multiplies inputs through each Dense layer and activation in sequence. Backward pass propagates the loss gradient back through each layer (chain rule), and SGD adjusts the weights. The XOR default uses a 2 → 4 (ReLU) → 1 (sigmoid) architecture — the hidden layer introduces the non-linearity that makes XOR solvable.
- file:
lib/ruby_ml.rb - role: single entry point that aggregates all
requires for losses, optimizers, layers, activations, and models.