The Building Blocks of AI: Data, Features, and Parameters

The Trinity of AI Behavior: Data, Features, and Parameters

At its heart, a trained AI model is a complex mathematical function that takes an input and produces an output (a prediction or decision). Its specific behavior is not manually programmed but is instead shaped by a powerful interplay of three elements:

1. Training Data: The vast collection of examples—images, text, or structured data—that the model learns from. It's the "textbook" for the AI.
2. Features: The specific, measurable characteristics of the data that the model is instructed to consider. These are the "key concepts" extracted from the textbook. For example, in an image of a cat, features could be pixel values, edge shapes, or textures.
3. Model Parameters: The internal variables of the model, primarily weights and biases, that are adjusted during training. This is the "knowledge" the model internalizes after studying the features in its training data.

Let's explore each of these components and see how they are inextricably linked.

1. Model Parameters: The Internal 'Wiring' of the AI

The direct determinants of a model's behavior are its parameters. In a neural network, these are the weights and biases associated with each connection between neurons. Think of them as thousands, millions, or even trillions of "dials" or "knobs" that are tuned during training. The final setting of these dials defines exactly how the model will respond to any new input.

To visualize this, let's watch a segment from the excellent 3Blue1Brown series on neural networks. This clip explains how weights and biases function as the core parameters that allow a network to recognize patterns.

As the video explains, each neuron's output is calculated from a weighted sum of the inputs from the previous layer, plus a bias.

• Weights: Determine the strength and sign (positive/excitatory or negative/inhibitory) of the connection between neurons. A higher weight means the input from one neuron has more influence on the next.
• Biases: Act as an offset, making it easier or harder for a neuron to activate. It allows the neuron to fire even if all its inputs are zero.

The collection of all weights and biases in a network constitutes its model parameters. For a formal definition and a look at the underlying mathematics, the following article is very helpful.

From a practical programming perspective, when you use a library like Scikit-learn (which you may encounter in your Python work), these parameters are accessible attributes of the trained model object, often named coefs_ (for weights) and intercepts_ (for biases), as noted in its documentation (LINK).

So, if parameters define the model's behavior, where do their specific values come from?

2. The Training Process: How Data Shapes Parameters

Model parameters are not set by hand. They are learned automatically from training data through a process of optimization. This process, often called the "training loop," is the heart of machine learning.

The core idea is:

1. Start with random parameter values.
2. Show the model a batch of training data and have it make predictions (this is the forward pass).
3. Compare the model's predictions to the true labels using a loss function, which calculates a single number representing the model's error or "lousiness."
4. Use an algorithm to calculate how each parameter contributed to that error. This process is called backpropagation.
5. Slightly adjust all parameters in the direction that will decrease the error. This update step is typically done using an optimizer like gradient descent.
6. Repeat this process millions of times with all the training data.

This next video brilliantly visualizes how training data, the cost function, and gradient descent work together to shape the model's parameters.

The entire process is a feedback loop where the training data guides the iterative refinement of the model's parameters. The image below provides a fantastic summary of this cycle.

The takeaway is simple but profound: the final values of a model's parameters are a direct function of the data it was trained on.

3. Features: The Lens Through Which the Model Sees Data

A model doesn't just ingest raw data; it processes specific features of that data. In simple cases, like the digit recognition example, the features might be the raw pixel values. However, the power of deep learning comes from its ability to learn a hierarchy of features.

Let's revisit the first video to see this concept in action.

In many real-world applications, especially outside of image or text processing, features are explicitly selected by the data scientists building the model. This process, called feature engineering, is a critical step that has massive implications for performance and fairness.

Consider the COMPAS algorithm from our previous lesson. The data scientists chose features like 'age', 'sex', and 'number of prior arrests' from the raw criminal history data. The model's parameters were then trained to find correlations between these specific features and the likelihood of re-offending. If 'number of prior arrests' is itself a result of biased policing, including it as a feature directly injects that societal bias into the model's learning process.

Synthesis: The Chain of Determinism

We can now see the full chain of how an AI's behavior is determined:

1. We start with Training Data, which contains inherent patterns, correlations, and, potentially, societal biases.
2. We (or the model itself) extract Features from this data. This choice of features acts as a lens, focusing the model's attention on certain aspects of the data while ignoring others.
3. Through the training process (e.g., gradient descent), the model adjusts its internal Parameters (weights and biases) to minimize its prediction error on the features of the training data.
4. The final, trained set of parameters is the model's decision-making logic. It represents the learned patterns, and it dictates the model's behavior and performance on all future data.

This trained logic results in a decision boundary—an implicit rule the model uses to classify new data. The image below visualizes this. The complex, curved line separating the two groups of points is the model's behavior, and that line is defined entirely by the parameters learned from the training data.

Conclusion

Today we have traced the path from data to decision. We've established that an AI's behavior is not magical or arbitrary; it is the deterministic outcome of a learning process guided by data.

Key Takeaways:

• An AI model's behavior is defined by its internal parameters (weights and biases).
• These parameters are learned automatically from training data through an optimization process (like gradient descent) that aims to minimize prediction error.
• The model learns patterns based on the specific features it is given from the data. The choice of features can be a critical point where bias is introduced.
• This interconnected system—Data → Features → Parameters → Behavior—is the technical pathway through which societal biases become encoded in algorithms and where legal risks are created.

Preview of the Next Lesson:

Now that we have a firm grasp of this fundamental mechanism, our next lesson will perform a deep dive into its most critical and legally relevant consequence: bias. We will move beyond simply saying "bias exists" to systematically identifying the different sources of bias in datasets and explaining how they propagate through the machine learning lifecycle we've outlined today.