The Story: The first “neuron” — can a machine learn at all?

The Core Idea

Rosenblatt asked whether a machine could learn the way biological systems do — not just detect the world, but store and use that experience. His answer was the perceptron: a machine that takes visual input like light on a retina, passes it through a web of connected units that sum up weighted signals and fire if they cross a threshold, and finally outputs a classification — circle or rectangle, 0 or 1. Crucially, memory isn’t stored in one single place but distributed across the whole network of connections, just like the brain — and it gets stronger with experience.

Perceptron architecture

Key Concepts

1. Three-layer architecture

S-points (retina/input) → A-units (sum signals, fire if above threshold θ) → R-units (compete, winner suppresses all others and gives the final answer)

2. The learning rule

Active A-units gain value (V) when the correct response fires. Next time the same stimulus appears, those same A-units fire stronger → same R-unit wins more confidently. Neurons that fire together, wire together.

3. Three learning systems

  • Alpha (α): active cell gains +1 value per impulse, keeps it forever. Simple but unstable long term.
  • Beta (β): each source-set gets a constant rate of gain shared among active cells. Net value keeps growing → amplifies noise → worst performer.
  • Gamma (γ): active cells gain value at the expense of inactive cells in the same source-set. Total value stays constant (zero-sum). Most biologically realistic and best performer.

4. Linear separability

The perceptron can only draw one straight line through feature space. If classes are cleanly separated it works perfectly. If they are interleaved like XOR, no straight line can separate them and the perceptron completely fails. Real-world problems (image recognition, spam filtering, medical diagnosis) all have complex curved decision boundaries that a single straight line can never capture.

Linear separability vs XOR

5. Distributed memory

Knowledge is not stored in one place. It is spread across thousands of connections throughout the network. If you damage part of the system, performance degrades gradually — not catastrophically. Exactly like biological brains.

Implementation

Built from scratch in NumPy following the core ideas from the paper.

import numpy as np

class Perceptron:
    def __init__(self, input_size, learning_rate=0.1, threshold=0.5):
        self.weights = np.random.randn(input_size) * 0.01
        self.lr = learning_rate
        self.threshold = threshold

    def predict(self, x):
        total_sum = np.dot(self.weights, x)
        return 1 if total_sum >= self.threshold else 0

    def fit(self, X, y, epochs=100):
        for epoch in range(epochs):
            for x_i, y_i in zip(X, y):
                prediction = self.predict(x_i)
                error = y_i - prediction
                self.weights += self.lr * error * x_i

Full implementation (with training scripts): ml-papers-from-scratch / unit1-perceptron

Results

  • AND gate → 100% accuracy (linearly separable, perceptron nails it)
  • XOR gate → 50% accuracy (not linearly separable, perceptron fails — proving the core limitation)

What Problem Did It Leave Unsolved?

The perceptron can only draw a straight line through data. Real-world problems — spam filtering, medical diagnosis, image recognition — have complex curved decision boundaries that a single straight line can never capture. You cannot hand-write rules either: a 28×28 image has 784 pixels and 2^784 possible combinations — more than atoms in the observable universe. The perceptron also completely fails at relational reasoning (“which object is to the LEFT of the square?”) and any higher-order abstraction. These failures directly caused the first AI Winter after Minsky and Papert proved the XOR limitation in 1969 — and fixing it required multi-layer networks and backpropagation. That’s exactly what comes next in this roadmap.

Summary

Rosenblatt’s 1958 perceptron was the first serious attempt to make a machine learn like a brain. Instead of storing memory as photographs or hard-coded rules, it stores knowledge in the strength of connections between simple units — just like neurons. Light hits a retina, signals flow through association units that sum and threshold them, and a response unit fires when it wins a competition. With each correct answer reinforced, the active connections grow stronger — the system literally learns from experience. But it has one fatal flaw: it can only separate data with a straight line. The moment the real world gets complicated — and it always does — the perceptron fails. That limitation sent researchers on a 60-year journey that eventually produced deep learning.