Welcome to Module 8. In the previous module, you built a powerful, automated backend system using Celery for asynchronous tasks and periodic jobs. You now have the foundational infrastructure for handling long-running, complex operations without blocking the user experience. This is a critical skill set for the kind of scalable applications Infilect builds.
Now, we will pivot to the "intelligence" layer of such applications. This module focuses on integrating machine learning (ML) models into your Django backend, directly addressing the job requirement to "build software and backend systems that cater to machine learning models." We won't be building or training these models ourselves—that's the role of a data scientist—but as a backend engineer, you are responsible for taking their finished models and making them accessible, reliable, and scalable through APIs.
To begin, we must first understand the two fundamental phases of any ML model's lifecycle: training and inference. This conceptual lesson will explain the crucial differences between them and, most importantly, why this distinction has profound implications for the systems you will design.
The Two Lives of a Machine Learning Model
Every machine learning model essentially lives two distinct lives: a "student" life and a "professional" life.
- Training: This is the model's time in school. It learns from a vast amount of data, like a student studying textbooks and past exams.
- Inference: This is the model at work. After graduating, it uses its learned knowledge to make quick, effective decisions on new, unseen problems.
The IBM Technology video below provides an excellent overview of this concept using a clear, practical example of a spam detector.
AI Inference: The Secret to AI's Superpowers
This video, "AI Inference: The Secret to AI's Superpowers," clearly explains the two main stages of an AI model's life. It uses a simple spam filter analogy that makes the concepts easy to grasp.
Watch the introduction explaining the two primary stages. Then, understand the training phase, where the model computes relationships and stores them as "weights." Next, see how the inference phase uses this knowledge to interpret new data. Finally, watch the complete spam detector example, which walks through both training (learning from labeled emails) and inference (classifying a new email).
As the video illustrates, these two phases have entirely different goals, processes, and outputs. Let's look at them more closely.
Phase 1: Training (The Learning Phase)
The goal of training is to create a model. This process involves feeding a model algorithm an enormous, carefully prepared dataset where the correct answers are already known (so-called "labeled data").
For example, to train a model to recognize products on a retail shelf (a core task for a company like Infilect), data scientists would feed it hundreds of thousands of images, each one meticulously labeled with the products present (e.g., "Coca-Cola 12oz can," "Lays Classic chips").
The model iteratively processes this data, makes a prediction, checks its prediction against the correct label, and calculates its error. It then adjusts its internal configuration—a complex set of numbers called weights—to reduce this error. This cycle repeats millions of times. It's a computationally-intensive, time-consuming process that can take hours, days, or even weeks on powerful, specialized hardware.
The final output of the training phase is the trained model: a file containing the optimized weights that represent all the "knowledge" the model has learned.
Phase 2: Inference (The Prediction Phase)
The goal of inference is to use the trained model to make a prediction on new, unseen data. During inference, the model is frozen—it does not learn or change its weights. It simply takes a new input, performs a single, rapid calculation using its stored knowledge (the weights), and produces an output.
This is where you, the backend engineer, come in. Your job is to build a system that can:
- Load the trained model file.
- Expose it via an API endpoint.
- Receive a request with new data (e.g., an image of a new retail shelf).
- Pass this data to the model for inference.
- Return the model's prediction (e.g., a JSON object listing the products it identified) in the API response.
Why This Distinction is Critical for Backend Engineers
Understanding that training and inference are different is not just academic. It has major practical consequences for system design, cost, and performance. The Clarifai article "AI Model Training vs Inference" breaks this down brilliantly.
AI Model Training vs Inference: Key Differences Explained
This article section provides a systematic breakdown of the operational differences between training and inference across five key dimensions. This is the core of what you need to understand to build effective backend systems for ML.
Please read the entire section titled Key Differences. Pay close attention to the distinct requirements for each phase in terms of: Purpose and Data Flow Computational Demands Latency and Performance Cost and Energy Consumption Hardware Requirements
Let's summarize the crucial points from that article from a backend engineer's perspective:
| Dimension | Training | Inference | Your Responsibility as a Backend Engineer |
|---|---|---|---|
| Purpose | To learn patterns and create a model. | To apply patterns and make predictions. | You build systems for the prediction part. |
| Data Flow | Processes massive, batched datasets. | Processes single, real-time inputs. | Your API will handle one request at a time. |
| Latency | High latency is acceptable. It can run overnight. | Low latency is critical. Must respond in milliseconds. | Your system must be optimized for speed. |
| Compute | Extremely high (CPU/GPU for days). | Low per request, but continuous. | Must handle thousands of small, fast computations efficiently. |
| Cost | A large, one-time capital expense. | A smaller, but continuous operational expense. | This is crucial: Inference can account for 80-90% of the total lifetime cost of an AI system. Your design directly impacts the company's operational budget. |
| Hardware | Clusters of powerful, expensive GPUs/TPUs. | Diverse: CPUs, GPUs, specialized chips (NPUs), or even edge devices. | You'll deploy models to hardware optimized for cost-per-prediction and low latency, not raw training power. |
This table makes it clear why you can't just take a model and "run it." A backend system for inference must be designed with low latency and high throughput in mind. It needs to be scalable to handle fluctuating request volumes and cost-effective to prevent operational expenses from spiraling out of control. These are the engineering challenges you'll be solving.
Conclusion
In this lesson, you've learned the fundamental distinction between machine learning training and inference.
Key Takeaways:
- Training is the offline, resource-intensive process of teaching a model by showing it labeled data. Its output is a trained model file.
- Inference is the online, real-time process of using a trained model to make predictions on new, unseen data. The model does not change during inference.
- As a backend engineer, your role is to build systems for inference, which demands a focus on low latency, high throughput, and cost efficiency.
- The operational requirements for inference (speed, continuous operation, cost) are vastly different from those for training (raw power, batch processing).
You now have the conceptual framework needed to start building ML-powered services. In our next lesson, we'll take our first practical step: you will learn how to load a pre-trained model file into a Python application, preparing it to be served.