Neural Networks: From Data to Predictions

Recommended resources

The freely available resources this lesson will direct you to.

In our previous lesson, we established a clear map of the AI landscape, distinguishing between the broad field of Artificial Intelligence, the data-driven approach of Machine Learning, and the powerful, layered technique of Deep Learning. You now know that deep learning's magic comes from artificial neural networks.

Today, we will open up that "black box" and examine the engine inside. Our goal is to understand exactly how a neural network takes an input—like the pixels of an image or features of a user—and processes it to generate a prediction. This mechanical, step-by-step flow of information is known as forward propagation. Mastering this concept is the essential next step toward understanding how AI models work, and eventually, how you can build features powered by them.

The Simplest Predictive Model

Before we dive into a complex network with many layers, let's start with the most basic predictive model imaginable. It contains the fundamental DNA of a neural network. Imagine we want to predict a house's price based on its size. This is a classic regression problem.

The following interactive guide by Jay Alammar provides a brilliantly simple starting point. It will walk you through building a model with just one input, one weight, and one bias.

A Visual and Interactive Guide to the Basics of Neural Networks

This guide strips away all complexity to reveal the core components of a prediction: an input, a weight, and a bias. It establishes the foundational formula y = Wx + b in a very intuitive, visual way.

Please read the first two parts of the guide: Start from the beginning and read the section that introduces the house price example. This will show you how an input (area) is multiplied by a weight to produce a prediction. Continue to the next section, "Harder, Better, Faster, Stronger". Here, the author introduces the bias term to improve the model's flexibility. Focus on understanding how adding this bias allows the line to shift, better fitting the data. Don't worry about the "loss function" details for now; we'll cover that in a later lesson on training.

This simple model, prediction = weight * input + bias, is the fundamental calculation at the heart of every single neuron in a complex neural network.

Anatomy of a Neural Network

Now, let's scale up from that single calculation to a complete neural network. A network organizes these simple calculations into a powerful, layered structure.

A typical neural network consists of an input layer, one or more hidden layers, and an output layer. Data flows from left to right, with each neuron in a layer processing the outputs from the previous layer.

As shown in the diagram, a neural network has three types of layers:

Input Layer: This is where the data enters the network. Each neuron in this layer represents a single feature of your input data. For an image, this could be the brightness of each pixel. For a user, it could be their age, country, and number of purchases.
Hidden Layers: These are the intermediate layers where the real "thinking" happens. Each neuron in a hidden layer takes the outputs from the previous layer as its inputs. By combining and transforming these inputs, the network learns to recognize increasingly complex patterns.
Output Layer: This is the final layer that produces the prediction. The number of neurons here depends on the task. For a regression task like predicting a house price, it might be a single neuron. For a classification task (e.g., identifying a handwritten digit from 0-9), it would typically be ten neurons, one for each possible digit.

But why is this layered structure so effective? The key idea is that it allows the network to build a hierarchy of knowledge. The brilliant 3Blue1Brown video below explains this intuition using the classic example of recognizing handwritten digits.

But what is a neural network? Chapter 1, Deep learning

This video masterfully explains the conceptual basis for a layered architecture. It shows how a complex problem can be broken down into simpler patterns, with each layer of the network specializing in detecting features at a different level of abstraction.

Please watch the following segments: What are Neurons and Layers? This section introduces the core components: neurons as number-holders (activations) and their organization into input, hidden, and output layers. The Power of Layered Abstraction: This part explains the hope for what these layers do—how the first hidden layer might learn to detect simple edges from pixels, the next layer might combine edges into loops and lines, and the final layer might combine those patterns into complete digits.

Inside a Single Neuron

We've established the overall structure. Now let's zoom in on a single neuron in one of the hidden layers to see exactly what it computes. As we hinted at with the house price example, it performs a two-step process:

Calculate a Weighted Sum + Bias: The neuron receives outputs from all neurons in the previous layer. Each of these incoming connections has a weight, which signifies its importance. The neuron multiplies each input by its corresponding weight, sums them all up, and then adds a single bias value. This is the exact same z = w*x + b logic from our simple model, just with more inputs. The result is often called the pre-activation or logit.
Apply an Activation Function: The neuron then passes this result through a non-linear function called an activation function. This function "squishes" the output into a desired range (e.g., between 0 and 1) and, crucially, introduces non-linearity. Without this step, a multi-layered network would mathematically collapse into a single, simple linear model, no matter how many layers it had. It's the activation function that allows the network to learn complex, "squiggly" patterns in the data.

The following guide provides a clear mathematical breakdown of this process and explains the critical role of activation functions.

Forward Propagation in Neural Networks: A Complete Guide

This article from DataCamp formalizes the computation within a neuron and explains why activation functions are not just an optional extra, but a fundamental requirement for a network's learning capability.

Please read the following two sections: Navigate to "The single neuron: where it all begins". This section details the components and provides a concrete numerical example of the calculation z = w*x + b and the subsequent application of an activation function, a = f(z). Then, jump to the FAQ section at the end and read the answer to the question "Why do we need activation functions in forward propagation?". This gives a concise reason for their importance.

Forward Propagation: The Complete Flow

Now we can put everything together. Forward propagation is simply the process of performing these neuron calculations, layer by layer, from the input to the output.

The input data is fed into the input layer.
The activations from the input layer are passed to the first hidden layer.
Each neuron in the hidden layer calculates its own output (weighted sum + bias -> activation function).
The entire layer's outputs are then passed as inputs to the next layer.
This process repeats until the final output layer is reached, which produces the network's prediction.

The StatQuest video below provides an exceptionally clear, step-by-step walkthrough of this entire process, showing how the numbers flow and get transformed to create a final prediction.

The Essential Main Ideas of Neural Networks

This video visually traces the flow of a single input value through a simple neural network. It demystifies the process by showing exactly how the weights, biases, and activation functions at each stage combine to produce a final, complex output from simple building blocks.

Watch the core section of the video, the step-by-step demonstration, where Josh Starmer plugs a dosage value into the network and calculates the result at each node until he gets the final prediction.

Formalizing the Process for a Developer

As a developer, you can think of forward propagation as a deterministic algorithm. Given a set of inputs and fixed parameters (weights and biases), it will always produce the same output. The following resource formalizes this algorithm and introduces the use of matrix mathematics, which is how this process is implemented efficiently in code.

Given your background, you'll appreciate how a series of individual neuron calculations can be elegantly expressed as a sequence of matrix multiplications and vector additions. This is why libraries like NumPy, TensorFlow, and PyTorch are built around optimized matrix operations.

Forward Propagation in Neural Networks: A Complete Guide

This reading connects the conceptual flow to a practical, algorithmic implementation. The use of matrix notation and pseudocode will resonate with your developer background.

Please read the following sections from the DataCamp article we looked at earlier: Start with "From neurons to layers". This explains how the computations for a whole layer can be expressed efficiently using a single matrix multiplication. Continue to "Forward Propagation Through a Complete Network", which shows the sequence of matrix equations for a multi-layer network. Finally, study the "Forward propagation algorithm" section. The pseudocode here provides a clear, high-level blueprint for how you would implement this process in code.

Conclusion

In this lesson, we demystified the process of how a neural network makes a prediction. We've seen that what appears to be a "black box" is actually a structured and logical sequence of simple mathematical operations.

Here are the key takeaways:

Forward Propagation: This is the process of data flowing forward through the network from the input layer to the output layer to generate a prediction.
The Neuron's Job: Each neuron calculates a weighted sum of its inputs, adds a bias, and passes the result through an activation function.
Parameters: The weights and biases are the "dials and knobs" of the network. The process of "training" a network, which we'll cover later, is all about finding the optimal values for these parameters.
Activation Functions: These are essential for introducing non-linearity, which gives the network the power to model complex, real-world patterns.
Algorithmic View: The entire process can be viewed as a function or an algorithm, which is implemented efficiently using matrix and vector operations.

Now that you understand how a network makes a prediction, you might be asking: how does it learn the right weights and biases in the first place? To answer that, we first need to understand the different ways a machine can learn. In our next lesson, we will explore the three main paradigms of machine learning: supervised, unsupervised, and reinforcement learning. This will provide the context for understanding the training process that makes these networks so powerful.