Welcome to the first lesson of your course on building production-grade predictive models. Given your background as an architect, you know that the success of any complex system hinges on a solid foundation. In machine learning, that foundation isn't code or infrastructure; it's the clear and precise framing of the problem you're trying to solve.
This initial module focuses on providing an "Architect's View" of a machine learning project. We'll begin with the most critical and often overlooked phase: translating a business ambition into a solvable machine learning task. By the end of this lesson, you will be able to frame a business problem as a predictive modeling task with clearly defined success metrics. This is the first and most important step towards being "delivery ready" for your upcoming project.
The Machine Learning Project Lifecycle
Before we dive in, let's look at the big picture. A machine learning project is a cyclical process, not a linear one. The diagram below illustrates the key stages, from initial problem framing to long-term monitoring and maintenance—a perspective that is central to your role as an architect.
As you can see, everything starts with "Problem Framing." A flawed understanding here will cascade through the entire lifecycle, leading to a model that, while technically correct, fails to deliver business value.
From Business Goal to Machine Learning Problem
Problem framing is a structured process of understanding a business need and re-articulating it in a way that a machine learning model can address. It's about moving from a vague objective like "improve customer retention" to a concrete task like "predict the probability of a customer churning in the next 30 days."
This process involves asking a series of critical questions to deconstruct the business problem.
Once you have a clear business problem statement, you can translate it into the language of machine learning. At its core, any supervised machine learning problem can be described as a triplet: (Data, Model, Objective Function).
To see how this works in practice, let's start with a foundational reading.
Stage 2. Project Scoping And Framing The Problem - Omniverse
This article by Gao Hongnan provides a structured approach to scoping an ML project. It will help you break down a business goal into the essential components of a machine learning problem.
Please read the introduction and the first example on e-commerce recommendations. Pay close attention to the table of "Key Components," which breaks down a problem into inputs, outputs, an objective function, and the type of ML task. This framework is fundamental.
Defining the Model's Task
After identifying the high-level business goal, the next step is to define precisely what you want the model to do and what kind of output it should produce. This decision has significant implications for data labeling, model selection, and how the model's output is consumed by your application.
Framing an ML problem | Machine Learning
This resource from Google for Developers offers a very practical guide on problem framing, written for practitioners.
Focus on the section Define the ideal outcome and the model's goal. The table provided gives clear examples of how to connect a high-level business objective to a concrete task for the model. This is the bridge between the business world and the technical implementation.
Classification vs. Regression: An Architectural Choice
The two most common types of predictive modeling tasks are classification (predicting a category, like "spam" or "not spam") and regression (predicting a number, like a house price). Choosing between them isn't always straightforward and can depend on how the prediction will be used. This is an architectural decision, not just a modeling one.
The following reading provides an excellent scenario that illustrates how your system's downstream behavior can influence this choice.
Framing an ML problem | Machine Learning
Let's continue with the Google article. This next part is crucial for making a foundational design choice that affects the entire system.
Read from "Identify the output you need" down to the end of the section on Regression. Pay special attention to the video caching example. It illustrates a key architectural consideration: how your app's behavior (fixed vs. dynamic thresholds) should influence the choice between a regression and a classification model.
When You Can't Measure What You Want: Proxy Labels
In an ideal world, your dataset would contain the exact label you want to predict (e.g., a is_useful flag on a recommended video). In reality, you often can't measure your ideal outcome directly. Instead, you must find a proxy label—an available feature in your data that serves as a stand-in for the true outcome.
For example, to predict if a video is useful, you might use was_liked or watch_time_percentage > 80% as proxies. The choice of a proxy label is a critical modeling decision that can introduce subtle biases. Click-through rate, a common proxy for engagement, can famously lead to models that favor "clickbait."
Framing an ML problem | Machine Learning
Often, the ideal thing we want to predict isn't directly measurable. This is where proxy labels come in.
Read the short section on Proxy labels. Note the examples of problematic proxies for "useful videos." Choosing a good proxy requires a mix of data availability, business intuition, and an awareness of potential unintended consequences.
Defining and Measuring Success
You've defined the business problem and translated it into a specific ML task. But how will you know if your project is a success? This is where we must draw a sharp line between model evaluation metrics and business success metrics.
- Model Evaluation Metrics (Offline Metrics): These are technical measures used during development to assess a model's performance on a dataset. Examples include Accuracy, Precision, Recall, or Root Mean Squared Error (RMSE). They tell you how "good" the model is at its specific task.
- Business Success Metrics (Online Metrics): These are the Key Performance Indicators (KPIs) that measure the project's impact on the business. They answer the question, "Did the model help us achieve our goal?" Examples include a 5% reduction in customer churn, a 10% increase in average order value, or improved user engagement.
Your MBA background provides you with the perfect lens for this. A model can have 99% accuracy but fail to move the needle on any business KPI, making the project a failure. Conversely, a model with modest accuracy might drive significant business value.
To help structure this thinking, the "Metrics Pyramid" is a useful mental model.
Understand Business Metrics for Data Analysts (Most Get This Wrong)
To effectively connect your model's performance to business value, it's essential to understand the hierarchy of business metrics. This video provides an excellent framework for this.
Watch the section that introduces the Metrics Pyramid. This will clarify the distinction between general metrics, Key Performance Indicators (KPIs), and Objectives and Key Results (OKRs). Then, watch the segment on how companies pick KPIs, which highlights that good KPIs are both influenceable and reflective of true performance.
With that framework in mind, let's formalize how it applies to an ML project. The ultimate goal is to ensure that optimizing your model evaluation metrics leads to an improvement in your business success metrics.
Framing an ML problem | Machine Learning
Now, let's tie this back specifically to machine learning projects.
Read the section Define the success metrics in the Google article. Note the clear examples for the weather and video apps and the important distinction between a model being "good enough" versus actually meeting the business success criteria.
A final, more advanced consideration from an architectural standpoint is handling conflicting objectives. For example, a newsfeed may want to maximize user engagement but also minimize the spread of misinformation. These goals can be at odds. In such cases, you might design a system with multiple models or a custom objective function that creates a weighted trade-off between the two. We will revisit such advanced designs later in the course.
Conclusion
In this lesson, we established the critical first step in any successful machine learning project: problem framing. You learned how to deconstruct a business goal and translate it into a well-defined predictive modeling task.
Here are the key takeaways:
- Start with Why: Always begin by understanding the business objective before defining the technical solution.
- Translate, Don't Just Solve: Frame the business problem in terms of machine learning by identifying the inputs (data), the desired output (e.g., a classification or a regression value), and the objective function.
- Mind the Gap with Proxies: When the ideal outcome isn't directly measurable, carefully select a proxy label and be aware of its potential pitfalls.
- Distinguish Success from Performance: A project's success is measured by its impact on business KPIs, not just by the model's technical evaluation metrics.
Now that we have a framework for defining the problem, our next step is to get our hands dirty with the data. In the next lesson, we will cover Exploratory Data Analysis (EDA), the process of investigating your dataset to understand its structure, find patterns, and uncover anomalies that will inform your feature engineering and modeling strategies.