Recommended resources
The freely available resources this lesson will direct you to.
Hello! Welcome to the first lesson of your course on AI governance.
Given your extensive background in IT governance, you're already familiar with the core principles of managing risk, ensuring compliance, and establishing robust processes. This course is designed to build directly on that foundation. Instead of starting from scratch, we will focus on the crucial differences that artificial intelligence introduces.
In this lesson, we will tackle the first learning outcome: identifying the governance gaps created by the unique characteristics of AI/ML systems. We'll explore why your established IT governance toolkit needs to be adapted and expanded for the world of AI. We'll pinpoint exactly what makes AI systems different from traditional software and how these differences create new challenges for governance, risk, and compliance.
1. From IT Governance to AI Governance: A Fundamental Shift
In traditional IT, governance functions like privacy, cybersecurity, and legal compliance often operate in well-defined silos. Each has its own frameworks (like GDPR for privacy or NIST CSF for security), priorities, and workflows. While this model has been effective, AI systems disrupt these neat boundaries.
The following article from ISACA—an organization you're likely familiar with from your IT governance work—provides an excellent overview of why this shift is happening. It explains how the interconnected risks of AI demand a more integrated approach, breaking down the traditional silos you've worked with.
Collaboration and the New Triad of AI Governance - ISACA
Please read the introduction and the section titled "The Fragmented Legacy of Governance."
As you read, focus on the author's argument for why the siloed approach to governance (separating privacy, security, and legal) is insufficient for managing AI. Think about how this contrasts with your own experience in IT governance.
The key takeaway is that AI systems create a complex, intersectional risk profile where a data privacy issue can simultaneously be a security vulnerability and a legal liability. A fragmented governance structure creates blind spots, making it impossible to see the full picture. This is the foundational problem that AI governance aims to solve.
2. The Unique Characteristics of AI Systems
To understand the governance gaps, we first need to pinpoint what makes AI technically different from traditional software. Traditional software is typically deterministic: given the same input, it will always produce the same output, following explicit, hard-coded rules. AI systems operate very differently.
The following video from IBM Technology provides a clear, high-level introduction to the core components of an AI system and the unique risks that arise from its fundamental nature.
The Importance of AI Governance
This video, "The Importance of AI Governance," clearly explains how AI's reliance on data and complex algorithms introduces new categories of risk.
Please watch the section from AI system risks. Pay close attention to the discussion of: How human bias in data gets reflected in AI outputs. The risk of privacy or copyright infringement. The challenge of "black box" models and the lack of transparency.
As the video explained, AI systems are not just complex code; they are dynamic systems shaped by data. To give you a more detailed comparison that directly contrasts with your IT background, the US National Institute ofStandards and Technology (NIST) provides an excellent summary in its AI Risk Management Framework (RMF).
Appendix B of this document is particularly useful, as it was written to help professionals like you bridge the gap between traditional software risks and AI risks.
Artificial Intelligence Risk Management Framework (AI RMF 1.0)
Please read Appendix B: How AI Risks Differ from Traditional Software Risks. You can find this on page 37 of the PDF.
This appendix provides a concise list of AI-specific risks that are either new or significantly increased compared to traditional software. As you read, reflect on which of these would be most challenging to manage using a standard IT risk register or control framework.
To synthesize these points, we can group the unique characteristics of AI into four main categories that challenge traditional governance:
-
Data Dependency and Dynamics: AI models are not just programmed; they are trained on vast datasets. Their performance is fundamentally tied to the quality, representativeness, and context of that data. Unlike traditional systems where data is simply processed, in AI, data shapes the system's logic itself. This leads to risks like inherited bias and performance degradation as real-world data changes over time (data drift).
-
Probabilistic and Opaque Nature: AI systems provide probabilistic, not deterministic, outputs. A model might be 95% confident in a prediction, but there's always a degree of uncertainty. Furthermore, the internal workings of complex models (like deep neural networks) can be inscrutable or a "black box," making it difficult to explain why a specific decision was made.
-
Scale and Emergent Properties: Modern AI models, especially large language models (LLMs), operate at a scale with billions or trillions of parameters. This complexity can lead to emergent properties—unforeseen capabilities or behaviors that were not explicitly programmed or anticipated during training.
-
Socio-technical Impact: AI systems are increasingly used to make or influence critical decisions about people's lives (e.g., in hiring, lending, and healthcare). This means their impact is socio-technical—it arises from the interaction between the technology and its social context, making ethical considerations and human oversight paramount.
3. From Characteristics to Governance Gaps
These unique characteristics directly create gaps in traditional IT governance frameworks. A control that works perfectly for a predictable, rule-based software application will often fail when applied to a dynamic, probabilistic AI system.
The table below maps each AI characteristic to the governance gap it creates:
| AI Characteristic | Resulting Challenge | Traditional IT Governance Gap |
|---|---|---|
| Probabilistic Nature | Outputs are not deterministic; performance is statistical. | Validation & Testing: Traditional pass/fail testing is insufficient. You can't just check if the output is "correct." You need new methods to measure statistical performance, robustness, and reliability over time. |
| Data Dependency | Model performance is tied to training data quality ("garbage in, gospel out"). | Data Governance: The focus shifts from just securing data to ensuring its quality, representativeness, and freedom from harmful bias. Data management becomes a continuous part of system validation, not just an input. |
| Opacity ("Black Box") | It can be difficult or impossible to explain a specific output. | Auditability & Accountability: You can't always audit the decision logic line-by-line. This creates a need for new "explainability" and "interpretability" techniques to ensure transparency and hold someone accountable. |
| Model/Data Drift | The system's performance degrades as real-world data changes. | Change & Configuration Management: "Set and forget" deployment is dangerous. AI systems require continuous monitoring of performance and input data, with triggers for retraining or recalibration that go far beyond typical software patching cycles. |
| Emergent Properties | The system develops unforeseen capabilities or failure modes. | Risk Management: Risk assessment cannot be a one-time, pre-deployment activity. It must be an ongoing process throughout the AI lifecycle to identify and mitigate emergent risks that were not present at launch. |
To effectively address these gaps, the field of AI governance focuses on building trustworthy AI. The NIST AI RMF outlines several characteristics of trustworthy systems. Thinking about these characteristics provides a positive goal for governance—a gap exists wherever one of these is missing.
This final video clip introduces these essential characteristics.
Mastering AI Risk: NIST’s Risk Management Framework Explained
In "Mastering AI Risk," the speaker outlines the seven characteristics that NIST defines as essential for trustworthy AI.
Please watch the section from trustworthy AI. This provides a clear framework for what "good" AI governance should achieve.
When an AI system lacks fairness, explainability, or transparency, it signifies a critical governance failure. Your role in AI governance will be to implement the frameworks, processes, and controls needed to build and maintain these very characteristics.
Conclusion
In this introductory lesson, we've established the fundamental premise of this course: AI is not just another type of software, and governing it requires a new way of thinking.
Here are the key takeaways:
- AI disrupts traditional governance silos because its risks are interconnected across privacy, security, and legal domains.
- The unique characteristics of AI—its data dependency, probabilistic nature, opacity, and potential for drift—create gaps in established IT governance controls.
- These gaps challenge traditional approaches to testing, data management, auditability, and change management.
- The goal of AI governance is to close these gaps and ensure systems are trustworthy, meaning they are valid, reliable, fair, transparent, and accountable, among other characteristics.
In our next lesson, we will delve deeper by exploring how these challenges differ across various types of AI systems. We will distinguish the governance needs of predictive AI (like a credit scoring model) from those of generative AI (like a chatbot or image creator). This will add another layer of nuance to your understanding of the AI risk landscape.