Prohibited AI Practices Under the EU AI Act

The Landscape of Prohibited AI

The EU has identified a set of AI practices that it considers a clear threat to fundamental rights, such as human dignity, privacy, and non-discrimination. These are not merely regulated; they are forbidden.

To start, take a look at these two visual summaries. They provide a concise overview of the eight main categories of prohibited practices.

As you can see, the prohibitions are not abstract. They target specific applications that raise profound societal concerns. To understand the legal detail, your main task for this lesson will be to study the relevant chapter from a comprehensive guide on the AI Act.

Now, let's synthesize and expand upon the key points from your reading, organizing the prohibitions into thematic groups. We'll also draw on expert commentary to clarify some of the more complex nuances.

Group 1: Practices Targeting Human Autonomy and Decision-Making

These prohibitions focus on AI systems designed to manipulate human behavior in harmful ways, striking at the core of individual autonomy and informed choice.

1. Subliminal, Manipulative, or Deceptive Techniques (Article 5(1)(a))

• What it is: A ban on AI that uses subliminal or purposefully manipulative/deceptive techniques to materially distort a person's behavior, causing them to make a decision they wouldn't have otherwise taken, leading to significant harm.
• Rationale: Protects cognitive autonomy and prevents exploitation. This is the Act's response to the fear of AI-powered "mind control."
• Key Nuances: All conditions must be met. A simple "dark pattern" in a user interface might not be prohibited unless it causes significant harm. The harm can be physical, psychological, or financial. Your experience with chatbot implementation is relevant here; a "chatbot gone rogue" that manipulates a vulnerable user into harmful behavior would be a prime example. Hallucinations in LLMs are not necessarily considered "deceptive" if the user has been adequately warned about the system's limitations.

2. Exploitation of Vulnerabilities (Article 5(1)(b))

• What it is: Prohibits AI systems that exploit the vulnerabilities of a specific group (due to age, disability, or social/economic situation) to materially distort their behavior in a way that causes significant harm.
• Rationale: This is a targeted protection for vulnerable populations who may be less able to resist manipulation.
• Example: An AI-powered mobile game aimed at children that uses addictive reinforcement schedules to encourage compulsive spending would likely be prohibited. However, an application that is merely inaccessible to people with disabilities would not trigger this specific prohibition.

Group 2: Practices Involving Social Control and Profiling

These prohibitions address dystopian scenarios of mass surveillance and algorithmic judgment by the state or private entities.

3. Social Scoring (Article 5(1)(c))

• What it is: Bans AI systems that evaluate or classify individuals based on their social behavior or personal characteristics, where the resulting "social score" leads to detrimental treatment in an unrelated context or is unjustified and disproportionate.
• Rationale: Directly counters systems like China's Social Credit System, which is seen as fundamentally incompatible with EU values of freedom and privacy.
• Key Nuances: This is highly context-dependent. A bank using a credit score (based on financial history) to decide on a loan is generally permissible. However, if that same bank used your social media activity or online shopping habits to set your mortgage rate, it would likely be considered prohibited social scoring.

4. Risk Assessment for Criminal Offenses (Article 5(1)(d))

• What it is: Forbids AI that assesses the likelihood of a person committing a crime based solely on profiling or personality traits. This is often called "predictive policing."
• Rationale: Upholds the presumption of innocence and human dignity. It prevents a "Minority Report" scenario where individuals are penalized for crimes they have not committed.
• Exception: This ban does not apply to AI systems used to support a human assessment that is based on objective, verifiable facts directly linked to a criminal activity. The AI can be a tool to assist, but not the sole judge. This prohibition can also apply to private actors, for instance, a bank using a prohibited profiling model for anti-money laundering (AML) screening.

Group 3: Practices Involving Biometric Data

This is a particularly sensitive area, and the Act draws several hard lines to prevent the misuse of our most personal data. Your technical interest in how AI works will be useful in appreciating the distinctions made here.

5. Creating Facial Recognition Databases (Article 5(1)(e))

• What it is: A ban on creating or expanding facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage.
• Rationale: A direct response to the privacy-invasive practices of companies like Clearview AI, which were seen as enabling mass surveillance on an unprecedented scale.
• Key Nuances: The key word is "untargeted." Targeted collection (e.g., collecting images of a specific suspect for an investigation) is not banned by this provision.

6. Emotion Recognition in Workplaces and Schools (Article 5(1)(f))

• What it is: Prohibits using AI to infer emotions of people in the context of work or education.
• Rationale: Protects individuals from intrusive emotional surveillance in environments with a significant power imbalance.
• Technical Distinction: Expert analysis has clarified that this ban applies specifically to emotion recognition based on biometric data (e.g., analyzing facial expressions, voice tonality, or even keystroke patterns). It does not prohibit systems that perform sentiment analysis on written text (e.g., analyzing employee feedback surveys). This is a crucial distinction.
• Exception: A narrow exception exists for medical or safety reasons, such as monitoring a pilot for fatigue.

7. Biometric Categorization on Sensitive Attributes (Article 5(1)(g))

• What it is: Bans using biometric data to deduce or infer sensitive characteristics like race, political opinions, religious beliefs, or sexual orientation.
• Rationale: Aims to prevent automated discrimination based on protected characteristics.
• Example: An AI system that analyzes headshots to infer a person's race or political leanings would be prohibited. This does not cover AI used by law enforcement to filter a database based on known characteristics (e.g., searching for a suspect with "brown hair").

8. Real-Time Remote Biometric Identification (RBI) by Law Enforcement (Article 5(1)(h))

• What it is: This is a general ban on the use of "real-time" biometric systems (like live facial recognition) in public spaces for law enforcement purposes.
• Rationale: This is considered the most privacy-intrusive form of surveillance, creating a potential chilling effect on freedoms of assembly and expression.
• Strictly Limited Exceptions: This is one of the most heavily debated parts of the Act. The ban is not absolute. Member states can allow its use in a few, exhaustively listed, and strictly necessary situations, such as:
  • Targeted search for specific victims (e.g., a missing child).
  • Prevention of an imminent terrorist attack.
  • Localization or identification of a suspect for a serious crime (punishable by at least 4 years in prison).
    Each use requires prior authorization from a judicial or independent administrative authority and is subject to strict safeguards.

For a final layer of nuance, it can be helpful to see how legal experts interpret the official guidelines published by the European Commission.

Conclusion

In this lesson, we have moved beyond a high-level overview and dissected the eight specific categories of AI practices that are strictly prohibited under the EU AI Act. These bans represent the EU's ethical and legal red lines, designed to protect fundamental rights from what are perceived as the most dangerous applications of artificial intelligence.

Key Takeaways:

• The EU AI Act establishes eight categories of prohibited practices under Article 5, carrying the highest penalties for non-compliance.
• These prohibitions target AI systems that manipulate human behavior, exploit vulnerabilities, enable social scoring, perform predictive policing, and misuse biometric data for mass surveillance or sensitive inferences.
• The legal definitions are highly specific. For a practice to be banned, it must often meet several cumulative conditions related to intent, context, and the significance of harm.
• Crucial technical and legal distinctions exist, such as the difference between biometric-based emotion recognition (banned in workplaces) and text-based sentiment analysis (not banned by Article 5).
• Even where exceptions exist (e.g., law enforcement's use of real-time RBI), they are narrowly defined and subject to strict authorization and oversight.

Preview of the Next Lesson:

Having established what is completely off-limits, we will now move one step down the risk pyramid. In our next lesson, we will focus on the "high-risk" category. These are AI systems that are not banned but are considered dangerous enough to warrant a comprehensive set of mandatory compliance obligations before they can be deployed. We will explore these requirements, including conformity assessments, risk management, and human oversight.