Neurons and Networks
Today's Plan
Today you'll explore what's happening underneath the AI systems we've been discussing. You've seen how LLMs predict the next token and debated whether that counts as "understanding." Now you'll get hands-on with the building blocks: artificial neurons and neural networks.
Here's one way to think about what you'll see today: neural networks are mathematical functions. They take in numbers and compute an output — nothing more. But they are extraordinarily complicated mathematical functions, and the way they learn to compute the right output is fascinating.
Here's another thing worth knowing: the basic idea of an artificial neuron was inspired by how neurons work in our brains. Researchers studied biological neural networks and built simplified mathematical versions. Now, decades later, artificial neural networks are being used to study the brain itself — helping neuroscientists understand the very biological systems that inspired AI in the first place.
You'll experiment with interactive visualizations, generate questions about what you observe, and investigate those questions with different partners.
Log in to participate in this activity.
Log InExplore: Single Neuron
This activity involves working with a partner.
A Single Artificial Neuron
An artificial neuron is the simplest building block in a neural network. It takes some inputs, multiplies each one by a weight, adds them up with a bias, and passes the result through an activation function. That's it — but from this simple operation, surprisingly complex behavior can emerge.
The tool below lets you experiment with a single neuron. Try adjusting the sliders and see what happens.
Neuron Explorer
A neuron listens to signals from its neighbors. Each connection can amplify, dampen, or even reverse a signal. If the total is strong enough, the neuron fires and sends its own signal onward. Try adjusting the sliders to see when this neuron fires.
How strong is each incoming signal?
How much does each connection let through?
How eager is this neuron to fire?
Try These
Hover over a button to see what it does. Try "Your turn!" to test yourself!
Things to try:
- Use the preset challenges (AND gate, OR gate) — can you figure out what the neuron needs to do?
- What happens when you change the activation function? How does it change the neuron's behavior?
- Can you make the neuron fire for only specific input combinations?
- Check out the biological analogy — how does this compare to what we know about brain cells?
Explore: Digit Network
This activity involves working with a partner.
A Network of Neurons
Now let's see what happens when you connect many neurons together. The network below is trained to recognize handwritten digits (0-9). Each neuron does the same basic operation you just explored — weighted sum, bias, activation function — but together they can do something much more impressive.
Digit Recognition Network
A neural network that recognizes handwritten digits. 784 → 16 → 16 → 10 neurons.
Pre-trained model
Trained on 60,000 examples before you arrived
Hover over a neuron in the diagram to see what it responds to. Click to pin.
Things to try:
- Draw a few digits and see if the network classifies them correctly. What digits does it struggle with?
- Try the "Train" mode — watch the connections change color and thickness as the network learns from examples. Notice how the green and orange connections match the single neuron you just explored.
- Stop training partway and switch back to "Classify" — the network now uses your partially-trained model. Does it work? Try training longer and classify again.
- Use "Learning" mode to step through exactly how the network learns from a single example — where do the errors flow?
- Click on individual neurons to see what they're responding to — what patterns have they learned?
- What connections can you see between the single neuron you just explored and this full network?
Discussion: What Did You Notice?
Quick Check-In
Geoff will ask a few of you to share something you noticed or something that surprised you during the exploration.
Generate Questions
What Are You Curious About?
Based on what you just explored, what questions do you have about how neurons and neural networks work? Think about what surprised you, what you couldn't explain, or what you want to understand better.
Enter at least 2 questions below.
Log in to submit questions.
Question Review
Log in to view the question board.
Investigate
This activity involves working with a partner.
Investigate a Question
With your new partner, pick one of the questions above to investigate. Use the demos below to experiment and gather evidence. You should also feel free to ask an AI — the interactive tools are great for some questions, but others might benefit from a conversation with ChatGPT or Claude. Try to build an answer based on what you can observe and learn, not just what you think you know.
Neuron Explorer
A neuron listens to signals from its neighbors. Each connection can amplify, dampen, or even reverse a signal. If the total is strong enough, the neuron fires and sends its own signal onward. Try adjusting the sliders to see when this neuron fires.
How strong is each incoming signal?
How much does each connection let through?
How eager is this neuron to fire?
Try These
Hover over a button to see what it does. Try "Your turn!" to test yourself!
Digit Recognition Network
A neural network that recognizes handwritten digits. 784 → 16 → 16 → 10 neurons.
Pre-trained model
Trained on 60,000 examples before you arrived
Hover over a neuron in the diagram to see what it responds to. Click to pin.
Log in to submit a response.
Explain & Learn
This activity involves working with a partner.
Explain & Learn
You're with a new partner now. Take turns:
- Explain what question you investigated and what you found
- Listen to what your partner investigated and learned
- Discuss:
- If you investigated different questions: how do your findings connect? Did you discover related things?
- If you investigated the same question: did you try different things? Did you reach the same conclusions? What did one of you notice that the other missed?
Log in to submit a response.
Share Out
Wrap-Up
Looking Ahead
The artificial neurons and networks you explored today are the building blocks underneath the LLMs we've been discussing all semester. The same fundamental operations — weighted sums, activation functions, backpropagation — power everything from this digit recognizer to ChatGPT.
The difference is scale. The digit network you trained today has about 13,000 connections. A frontier model like GPT-4 or Claude has hundreds of billions of parameters — roughly 10 million times larger. The structure is also different in ways we'll explore later, but the core idea is the same: take in numbers, multiply by weights, add them up, apply a function, repeat.
Next time, we'll look at how these building blocks scale up into the systems you've been using.
Want to go deeper? The 3Blue1Brown series on neural networks is an excellent visual introduction that goes into much more mathematical detail — including the calculus behind backpropagation.
Feedback
Log in to submit feedback.