Recommended resources
The freely available resources this lesson will direct you to.
Welcome to your second lesson. In our previous session, we established that the unique characteristics of AI systems—like their dependency on data and probabilistic nature—create new governance challenges that traditional IT frameworks aren't equipped to handle.
Today, we will build on that foundation to address the learning outcome: distinguishing the governance challenges posed by different AI system types. Not all AI is the same, and from a governance perspective, the risks associated with a model that predicts customer churn are vastly different from those of a model that generates marketing copy. We'll focus on the two most prominent categories: predictive AI and generative AI. Understanding this distinction is crucial for correctly scoping risks and applying the right controls.
1. Defining the Landscape: Predictive vs. Generative AI
Before we can analyze the governance challenges, we need a clear understanding of what separates these two types of AI. Predictive AI, the more established technology in the enterprise, is about forecasting outcomes. Generative AI, the technology behind recent headlines, is about creating something new.
The following video provides an excellent high-level overview of this distinction. The speaker, Eric Siegel, contrasts the two by focusing on their core purpose and potential for automation.
Generative AI vs. Predictive AI | Eric Siegel
This video, "Generative AI vs. Predictive AI," clearly lays out the fundamental differences between the two.
Please watch the first two segments: Generative AI: Pay attention to the discussion of its communication capabilities and the need for human oversight due to its tendency to be incorrect ("hallucinate"). Predictive AI: Note how it's described as a tool for improving large-scale operations through autonomous, data-driven predictions.
To summarize and reinforce these concepts, the following tables offer a direct comparison of the benefits and limitations of each AI type.
The infographic below provides an even more detailed, side-by-side comparison across ten different attributes, including their objectives, data requirements, and specific ethical concerns.
2. Governance Challenges of Predictive AI
Your experience in IT governance provides a strong foundation for managing predictive AI. The risks here are often an amplification of issues you've likely encountered with data-driven systems, but the scale and opacity of AI models make them more acute.
The core governance challenges for predictive AI include:
- Bias and Fairness: If historical data reflects societal biases (e.g., in loan approvals or hiring), a predictive model trained on that data will learn and perpetuate those biases, potentially leading to discriminatory outcomes.
- Explainability: For high-stakes decisions (e.g., a mortgage application being denied), regulations like the Equal Credit Opportunity Act in the US require an explanation. If your predictive model is a "black box," providing a clear, justifiable reason can be extremely difficult.
- Data Quality and Provenance: The principle of "garbage in, gospel out" is paramount. A model's predictions are only as reliable as the data it was trained on. Governance must ensure data is accurate, relevant, and representative of the deployment environment.
- Performance Drift: The world changes, and a model trained on past data may lose its predictive accuracy over time. For example, a model predicting consumer purchasing behavior pre-pandemic would likely perform poorly today. This requires continuous monitoring and a plan for retraining.
3. The New Frontier: Governance Challenges of Generative AI
Generative AI introduces novel categories of risk that go beyond the structured, predictable outputs of its predictive counterpart. Because it creates new content, the potential for misuse, error, and unintended consequences is vast and less bounded.
Recognizing this, the US National Institute of Standards and Technology (NIST) published a special companion document to its AI Risk Management Framework (RMF) focusing specifically on generative AI. The video below explains the relationship between different types of AI and highlights the unique risks that prompted this separate guidance.
NIST AI Risk Management Framework & Generative AI | Lunchtime BABLing 36
The speakers in this video, "NIST AI Risk Management Framework & Generative AI," discuss why generative AI requires its own risk considerations.
Please watch the following segments: AI definitions: This section provides a clear "concentric circles" model to understand how generative AI is a specific subset of machine learning and AI. Generative AI risks: Pay close attention to the discussion of information integrity (misinformation) and human-AI configuration (automation bias) as key risks exacerbated by generative AI.
The video highlighted a couple of the major risks. The official NIST publication provides a comprehensive list of 12 risks that are either unique to or exacerbated by generative AI. This list is an essential tool for any governance professional.
Artificial Intelligence Risk Management Framework
Please read the section titled "2. Overview of Risks Unique to or Exacerbated by GAI", found on pages 3-6 of the PDF.
As you read through the list of 12 risks, focus on understanding the definitions for risks like Confabulation, Information Integrity, Intellectual Property, and Value Chain and Component Integration. These represent some of the most pressing new challenges for organizational governance.
From this list and our broader understanding, we can summarize the top governance challenges for generative AI:
- Confabulation (Hallucinations): Models can produce confident, plausible-sounding, but factually incorrect information. If this output is used in decision-making without verification, it can lead to significant errors.
- Information Integrity & Disinformation: The ease of creating "deepfakes" or generating large volumes of convincing but false text poses a risk to both your organization's reputation and societal trust.
- Intellectual Property (IP) and Data Privacy: A model might reproduce copyrighted text or images from its training data. Even more common is the risk of employees pasting sensitive or proprietary company data into a public generative AI tool, resulting in a data leak. This practice is often referred to as "Shadow AI."
- Harmful or Unsafe Content: Without strong guardrails and ongoing monitoring, models can be manipulated ("jailbroken") into generating toxic language, malicious code, or dangerous instructions.
- Value Chain and Lack of Transparency: Many generative AI applications are built using third-party foundation models (e.g., from OpenAI, Google, or Anthropic). This creates a dependency where you may have little to no visibility into the model's training data or inherent risks, making your own risk management more difficult.
4. Comparing the Governance Focus
Given these distinct challenges, the focus of your governance efforts will necessarily differ depending on the type of AI system you are evaluating.
| Governance Domain | Predictive AI Focus | Generative AI Focus |
|---|---|---|
| Input Data | Mitigating historical bias; ensuring data is representative. | Vetting for copyrighted/toxic material; managing massive, unstructured datasets. |
| Output / Content | Validating prediction accuracy against ground truth. | Monitoring for factual correctness (hallucinations), harmfulness, and IP infringement. |
| Transparency | Explaining why a specific decision was made (explainability). | Disclosing that content is AI-generated and citing sources where possible (provenance). |
| Human Oversight | Reviewing and overriding high-stakes predictions. | Verifying outputs; establishing strict Acceptable Use Policies to prevent data leakage. |
| Security | Protecting the model and data from traditional cyber threats. | Preventing "prompt injection" attacks; securing against misuse for creating malware or phishing campaigns. |
As a governance professional, you won't treat all AI systems the same. You'll need to categorize them, understand their specific risk profile, and tailor your controls accordingly.
Conclusion
In this lesson, we've broken down the monolithic concept of "AI" into two distinct categories—predictive and generative—and explored their unique governance challenges.
Here are the key takeaways:
- Predictive AI makes forecasts based on historical data. Its governance challenges, like bias and explainability, are often an amplification of traditional data governance issues.
- Generative AI creates new content. It introduces novel governance challenges related to factual accuracy (confabulation), misinformation, intellectual property, and complex supply chain dependencies.
- Your governance approach must be tailored. Governing a predictive credit-scoring model requires a focus on fairness and auditable decisions. Governing an internal chatbot requires a focus on acceptable use policies, data leakage prevention, and managing hallucinations.
- Frameworks like the NIST AI RMF are adapting to these differences, even creating specific "profiles" to provide guidance on new technologies like generative AI.
In our next lesson, we will take a closer look at several of the key AI-specific risks we've introduced—model bias, opacity, and data drift—and define them more formally within the context of the end-to-end AI lifecycle. This will equip you with a more precise vocabulary for identifying and discussing risk with technical and business stakeholders.