One neuron computes a weighted sum

Scenario

The four vectors from the previous lesson are on the table: the advertisement is [2 1 1 0 0], the customer asking about an order is [0 0 0 1 1]. Chị Hà wants a simple rule. The phrases "giảm giá" and "miễn phí" should make an email more suspicious. "Đơn hàng" and "hóa đơn" should pull it towards a customer's mail. You need one number per email, and you need to know which way that number moves when you trust one phrase less.

This lesson stops at the sum. It does not squash the number into the range 0 to 1. That is the sigmoid in the next lesson.

Objectives

• Compute $z = w \cdot x + b$ for one email once you have the vector, the weights and the bias.
• Change one weight and say whether $z$ rises or falls, when that email's matching cell is greater than 0.
• Show that a cell equal to 0 means changing its weight does not change $z$.

Intuition

Each phrase gets a vote. A positive vote pulls the email towards "suspicious". A negative vote pulls it towards a customer's mail. You multiply how often the phrase appears by its vote, add those products, then add a suspicion you already hold when the email contains none of the phrases. That built-in suspicion is the bias.

If an email never says "miễn phí", the vote for "miễn phí" contributes nothing to that email's total, however large the vote is. There is nothing to multiply.

Core concepts

The neuron in this lesson does one calculation. The email vector is $x$, the weight vector is $w$, in the same five-cell order as the previous lesson. The bias is $b$.

The sign of $w_i$ is the direction of the pull. The size is how hard it pulls for each time the phrase appears. When $x_i = 0$, the term $w_i x_i$ is 0, even if $w_i$ changes.

The numbers in the code, in the order giảm giá, miễn phí, nhấp, đơn hàng, hóa đơn:

A negative $b$ means that before any phrase is seen, $z$ already leans towards ordinary mail. The words "nghi rác" in the printed output are only a temporary reading for $z > 0$. They are not a probability. The next lesson turns $z$ into a number in $(0, 1)$.

Worked example

Take email 1, $x = (2, 1, 1, 0, 0)$.

1. Giảm giá: $1.5 \times 2 = 3$.
2. Miễn phí: $1.0 \times 1 = 1$.
3. Nhấp: $0.8 \times 1 = 0.8$.
4. The other two cells are multiplied by 0, so they add 0.
5. Add the bias: $z = 3 + 1 + 0.8 - 0.5 = 4.3$.

Lower the "miễn phí" weight from $1.0$ to $0.2$. Email 1 contains "miễn phí" once, so the total falls by $0.8$: $z = 4.3 - 0.8 = 3.5$. Emails 2 and 4 do not contain "miễn phí", so their $z$ stays put.

The code in code/neuron.py uses the same four vectors as the previous lesson.

import numpy as np

VOCAB = ["giảm giá", "miễn phí", "nhấp", "đơn hàng", "hóa đơn"]

# Bốn vector của bài 01, mỗi hàng là một thư.
X = np.array([
    [2, 1, 1, 0, 0],
    [0, 0, 0, 1, 1],
    [0, 1, 1, 1, 0],
    [0, 0, 0, 0, 1],
])


def neuron(x, w, b):
    """Tổng có trọng số của đầu vào, cộng độ lệch."""
    return np.dot(w, x) + b


def report(title, w, b):
    print(title)
    for i, x in enumerate(X, start=1):
        z = neuron(x, w, b)
        verdict = "nghi rác" if z > 0 else "thư thường"
        print(f"  Thư {i}: z = {z:+.1f} -> {verdict}")


w = np.array([1.5, 1.0, 0.8, -1.2, -2.0])
b = -0.5
report("Trọng số ban đầu", w, b)

# Đổi đúng một trọng số: bớt tin vào từ "miễn phí".
w2 = w.copy()
w2[VOCAB.index("miễn phí")] = 0.2
report("Sau khi giảm trọng số của 'miễn phí' từ 1.0 xuống 0.2", w2, b)

Running it prints:

Trọng số ban đầu
  Thư 1: z = +4.3 -> nghi rác
  Thư 2: z = -3.7 -> thư thường
  Thư 3: z = +0.1 -> nghi rác
  Thư 4: z = -2.5 -> thư thường
Sau khi giảm trọng số của 'miễn phí' từ 1.0 xuống 0.2
  Thư 1: z = +3.5 -> nghi rác
  Thư 2: z = -3.7 -> thư thường
  Thư 3: z = -0.7 -> thư thường
  Thư 4: z = -2.5 -> thư thường

Email 3 starts at $+0.1$ and, after the weight change, sits at $-0.7$. The same email's $z$ changes sign only because its "miễn phí" cell is 1. The + in the printed line is how the format marks a positive number: it means $4.3$, not an extra addition.

Common pitfalls

• Forgetting the bias. Drop $b$ and email 4 has $z = -2.0$ instead of $-2.5$. An email with none of the phrases would give $z = 0$ without $b$, which leans neither way. $b = -0.5$ is the lean towards ordinary mail.
• Reading $z$ as a probability. $+4.3$ is not 430%, and $-3.7$ is not a rate. Only the sign and the size mean something in this lesson. The label "nghi rác" when $z > 0$ is a temporary threshold in the code, so you can see the direction of a change. The sigmoid will replace it.
• Changing one weight and expecting every email to move. Email 2 stays at $-3.7$ after the "miễn phí" weight drops, because that cell is 0 on email 2. If one email does not move, look at the matching cell before you decide the code is broken.

Summary

• The neuron in this lesson computes $z = w \cdot x + b$ and does not apply an activation.
• A positive weight pulls towards suspicious mail, a negative weight pulls towards a customer's mail, once per time the phrase appears.
• Changing one weight moves $z$ by the change in the weight times the number in that cell.
• A cell equal to 0 means its weight does not contribute to $z$.
• $z$ is not yet a probability.

Practice

Take the email you counted in the previous lesson, $x = (1, 1, 1, 0, 1)$, with the same $w$ and $b$ as in neuron.py.

1. Compute $z$ by hand. Then add this row to X and run the file to check.
2. Change the "hóa đơn" weight from $-2.0$ to $-1.0$ and leave the other weights. Does $z$ for this email rise or fall, and by how much?

Self-check: step 1 gives $z = 0.8$. Step 2 gives $z = 1.8$, a rise of $1.0$, because the "hóa đơn" cell is 1 and that weight increased by 1.