How does one gradient descent step work?

Gradient descent moves parameters downhill on the loss. For one weight $w$ , derivative $\partial L / \partial w$ and learning rate $\eta$ :

w_{\text{new}} = w_{\text{old}} - \eta \frac{\partial L}{\partial w}

Toy numbers: $w = 0.5$ , $\partial L / \partial w = 2.0$ , $\eta = 0.1$ . Update: $w_{\text{new}} = 0.5 - 0.1 \times 2.0 = 0.3$ . Loss usually drops if $\eta$ is not too large.

The minus sign trips beginners: gradient increases loss, so we step opposite. With many parameters, each $w_j$ and $b$ updates together after gradients on one example or a batch.

python

w, grad, eta = 0.5, 2.0, 0.1
w = w - eta * grad
assert w == 0.3

The Deep Learning track does this by hand on one spam email before wrapping a NumPy loop — so when loss explodes, you check $\eta$ or gradients first. Treat $\partial L/\partial w$ as “if I nudge $w$ a tiny bit, does loss go up or down?” — the update rule is just that question turned into arithmetic. On the course spreadsheet exercise, students recompute $w_{\text{new}}$ for the spam neuron before trusting w -= eta * grad in NumPy.