Learning rate: 0.001 or 0.1?

Learning rate $\eta$ sets the step size each update. Too small — e.g. $\eta = 10^{-6}$ for 20 steps — loss might only move from $0.69$ to $0.68$, training crawls. Too large — $\eta = 1.0$ with gradients around $10$ — one jump sends weights into saturated sigmoid, gradients vanish or loss goes NaN.

On the course spam toy net, $\eta = 0.1$ often works for one NumPy neuron; $\eta = 0.01$ is safer before input scaling. Practice: try $0.1$, $0.01$, $0.001$ with the same seed, print loss every 5 steps.

$\eta$	Loss after 20 steps (example)
0.1	0.42 — healthy drop
1.0	NaN

There is no magic number for every net; there is a ritual of watching curves. The Deep Learning track shows both extremes before the learning-rate widget compares loss paths. When you move to larger nets later, schedulers and warmup help, but the same question remains: did the first 20 steps improve loss or blow up?