Implementing a Neural Network from Scratch

The objective of this repo is to implement a neural network from scratch (without using an LLm to write all the code for me!)

L-1

In this first lesson we are attempting to implement just 1 neuron. Once implemented and trained, a single neuron will act as a logistic regression, so it can solve problems that have 2 answers and are linearly separable, this is, on a dataset, we can clearly draw a line between the 2 "groups". Some examples:

• Logic Gates — AND, OR, NAND, NOR
• Email spam vs not spam (with simple features)
• Tumor malignant vs benign (with linearly related features)
• Pass vs fail based on study hours

To vizualize:

AND                    OR
x₂                     x₂
1 | O    X             1 | X    X
  |                      |
0 | O    O             0 | O    X
  |_________x₁           |_________x₁
    0    1                 0    1

In this two examples, we can clearly draw a line to separate the Xs and Os

This doesn't seem like much, but it's the building block for neural-networks and modern LLMs.

So for this first lesson, the objective will be to

🎯 Train a neuron to classify points above or below a line