Decomposing ML Projects: From Goals to Sub-tasks

Hello! In our previous lesson, you took a significant step by mapping the meal quality violations you've defined to the specific data modalities—images, text, and metadata—that can provide evidence for them. This gave you a strategic overview of where to find the signals for your project.

Today, we will move from strategy to a concrete technical plan. The goal is to decompose the overall project goal into a set of distinct Machine Learning (ML) sub-tasks. This is one of the most crucial steps in any applied ML project. We are essentially breaking down the big, complex problem of "monitoring meal quality" into smaller, manageable, and solvable pieces. For each piece, we will define exactly what goes in, what the model should do, and what comes out. This process creates a clear blueprint for the models you will eventually build.

From Policy Problem to ML Formulation

Before diving into code, experienced practitioners first translate a real-world problem into a formal ML problem. This ensures that the model you build actually helps solve the original problem. A paper from researchers at Carnegie Mellon University who specialize in "ML for Social Good" provides an excellent framework for this.

[PDF] Lessons Learned from Designing, Developing, and Deploying ...

This paper outlines practical lessons from real-world projects in public health, education, and human services—a context very similar to yours. We will use their structured approach to problem formulation.

First, briefly re-read the section "Start with project goals and actions, not ML" to reinforce the problem-first mindset. Then, focus on the section titled "Modeling problem formulation". Read the description of its three key elements: the modeling cohort, the target/label, and the system output. Finally, examine Figure 5 at the end of the paper. Notice how it explicitly maps a policy problem (preventing homelessness) to a precise ML formulation by answering specific questions. This is the same thought process we will apply.

The paper gives us a clear three-part structure for defining each ML sub-task:

Modeling Cohort: The "who" or "what" we are analyzing. In your project, the fundamental unit is a single meal instance. This instance might be represented by a photo, a text complaint, or both, along with associated metadata (date, school, location).
Modeling Target/Label: The outcome we want to predict for each unit in the cohort. This is the specific violation, such as is_contaminated or insufficient_quantity.
ML System Output: The final product of the model that an end-user or downstream system consumes. This could be a list of the 100 most severe violation cases this week, a severity score for every meal instance, or an automated alert.

With this framework in mind, let's start decomposing your project.

Defining Sub-Tasks, One Modality at a Time

We will now go through each of your primary modalities (image and text) and define the specific ML tasks required.

Image-Based Sub-Tasks

Let's start with a violation you identified as being primarily detectable in images: Insufficient Quantity. How do we frame this as an ML task? Should the model predict the exact weight of the food in grams, or should it classify the meal into categories?

This is a critical design decision. The Google Developers guide to ML problem framing provides excellent guidance on this.

Framing an ML problem | Machine Learning

This guide explains how to translate a desired outcome into a specific model output. We'll focus on its discussion of classification versus regression.

First, read the section on identifying the model's output, paying close attention to the different types of classification and regression in Table 1. Next, read the sections explaining Classification and Regression. The most important part is the video caching example. It brilliantly illustrates why predicting a decision directly (classification) is often better than predicting a number that is later used to make a decision (regression), especially when you have fixed thresholds.

The key insight from the Google guide is the principle to "Predict the decision." Your goal isn't to know the exact weight of a meal; your goal is to decide if the portion is sufficient or insufficient. Trying to build a regression model to predict weight is complex and prone to error (due to camera angles, food density, etc.). More importantly, it doesn't directly answer the question. A classification model is simpler, more robust, and directly solves the problem.

So, we can define our first image sub-task:

Sub-Task 1: Portion Size Classification
- ML Task Type: Binary Classification
- Input: A preprocessed image of a meal (e.g., resized to 224x224 pixels).
- Output: A probability score and a corresponding class label from [sufficient, insufficient].
- Model Goal: To predict whether the portion size shown in an image meets the defined sufficiency standard.

We can apply the same logic to other image-based violations:

Sub-Task 2: Visible Contamination Detection
- ML Task Type: Binary Classification
- Input: Meal image.
- Output: Class label from [contaminated, not_contaminated].
Sub-Task 3: Food Item Identification
- ML Task Type: Object Detection
- Input: Meal image.
- Output: Bounding boxes and class labels for each detected food item (e.g., dal, rice, chapati, vegetable). This directly uses the YOLO model from your technical plan.

Text-Based Sub-Tasks

Now let's consider the text modality. A complaint might mention multiple issues. For example, "The dal was watery and they didn't even serve the promised banana." This single text signals both a quality issue (watery dal) and a menu violation (missing banana). Therefore, we need a model that can assign multiple labels to a single piece of text.

Sub-Task 4: Complaint Violation Classification
- ML Task Type: Multiclass, Multi-label Classification
- Input: A tokenized text snippet from a complaint (prepared for a model like BERT).
- Output: A set of one or more labels from a predefined list of text-detectable violations, such as [spoilage, menu_violation, staff_behavior, sickness_report].
- Model Goal: To identify all violation types mentioned within a given complaint text.

Visualizing the Blueprint: High-Level System Design

We've now broken the problem down into independent sub-tasks. The next step is to visualize how these building blocks fit together to form your complete multimodal system. In ML system design, we often use block diagrams for this.

The following video provides a practical walkthrough of designing a system for a related problem: harmful content detection. It shows how to sketch a high-level architecture.

Harmful Content Detection / Content Moderation | ML System Design Problem Breakdown

This video on ML system design for content moderation demonstrates how to create a high-level diagram to connect different system components.

Watch the section on High-Level Design, from the beginning of the section. The key idea is not to create a perfect, detailed diagram, but to sketch out how the "pieces fit together and then how your system ultimately acts with the world."

The diagrams below illustrate this concept. The first shows a generic multimodal architecture, and the second shows a specific example.

This is a general blueprint for a multimodal system. Each input modality is processed by its own "unimodal model." The outputs of these models are then combined in a "fusion model" to produce a final prediction.

This diagram shows a specific implementation for a visual question-answering task. An image is processed by a CNN (a unimodal image model), and text is processed by a BLSTM (a unimodal text model). Their outputs are then combined in an LSTM (a fusion model) to generate the answer.

Your project will follow this exact pattern. You will have:

An Image Branch (using a CNN like ResNet) to handle the image-based sub-tasks.
A Text Branch (using a Transformer like BERT) to handle the text-based sub-tasks.
A Fusion Model that takes the outputs from the image and text branches (and your metadata) and makes a final, unified prediction about the presence and severity of a violation. This is the "late fusion" approach you plan to use.

Your Task: Create the Technical Blueprint

Your task is to formalize this decomposition. Complete the following table for the main violation types you defined in Module 2, Lesson 1. This table will serve as the definitive technical blueprint for your project's modeling phase. I've filled in the first two rows as examples.

Violation Category	Primary Modality	ML Sub-Task Name	ML Task Type	Model Input	Model Output
Insufficient Quantity	Image	Portion Size Classification	Binary Classification	Resized meal image	Label from `[sufficient, insufficient]`
Spoilage / Poor Quality	Text	Complaint Violation Classification	Multi-label Classification	Complaint text snippet	One or more labels, e.g., `spoilage`, `bad_taste`
Visible Contamination
Menu Violation
Adverse Health Event
(Add other violations)

Completing this table solidifies the link between the real-world problem and the machine learning solution.

Conclusion

In this lesson, you have translated your high-level project goal into a set of concrete, actionable machine learning sub-tasks. This is the final and most critical step in problem framing. You now have a clear technical blueprint that specifies what each model in your system needs to do.

Key Takeaways:

Decomposition is Key: Large, real-world problems are solved by breaking them into smaller, well-defined ML sub-tasks.
Predict the Decision: Frame your tasks to predict the decision you need to make (e.g., classification into sufficient/insufficient), which is often more practical than predicting a raw number (regression).
Define Inputs and Outputs: Every ML task must have a clearly specified input (the data it consumes) and output (the prediction it makes).
Sub-tasks are Building Blocks: These individual tasks for images, text, and other data types are the components that will be assembled into your final multimodal system.

You now have a robust plan. In our next and final lesson of this module, we will address the realities of this plan: what happens when our data doesn't have a clear signal for a violation, and what are the fundamental assumptions and limitations of framing this complex social problem as a set of ML tasks?

Recommended resources