Loss: cross-entropy for classification

For binary label $y \in \{0, 1\}$ and predicted probability $p$, binary cross-entropy is:

$$L = -\bigl[y \ln p + (1-y)\ln(1-p)\bigr]$$

If $y = 1$ and the model outputs $p = 0.99$, $L \approx 0.01$ — nearly correct. Same label with $p = 0.01$ (confidently wrong) gives $L \approx 4.6$ — a loud penalty.

Versus $(1-p)^2$ when $y=1$: at $p=0.01$ the square is only $\approx 0.98$, not loud enough to fix weights. Log loss punishes overconfident mistakes harder.

Two spam emails ($y=1$):

$p$	$L$
0.9	0.11
0.1	2.30

Mean loss over both is $(0.11 + 2.30)/2 \approx 1.21$. That is how the Deep Learning track scores “how bad” a classifier is before you write backward. Pair cross-entropy with sigmoid outputs so every prediction lives in $(0,1)$ and the loss stays finite; mixing logits with MSE is legal but the course stays with the classic classification pair for the spam neuron. Watch the training log: a confident wrong $p=0.01$ on a positive label should spike loss faster than a gentle $p=0.4$ mistake.