Introduction
In the previous module, we delved into the use of AI within the justice system, focusing on the profound risks to fairness, due process, and judicial independence. That discussion centered on the relationship between the state and the individual. We now pivot from public law to the private sphere, examining how AI impacts the relationship between businesses and consumers. Your experience in implementing digital transformation tools like chatbots and CRM systems has given you a firsthand view of how technology mediates this relationship.
This lesson directly addresses our learning outcome: to apply the concepts of 'defect' and 'deficiency' under the Consumer Protection Act, 2019 to AI products and services. We will dissect India's primary consumer protection law to understand its core liability principles and then grapple with the central challenge of fitting autonomous, intangible AI systems into its traditional legal categories. This will equip you to analyze who is legally responsible when an AI system causes harm to a consumer.
1. The Consumer Protection Act, 2019: A Modern Framework
The Consumer Protection Act (CPA) of 2019 was enacted to replace the older 1986 Act, specifically to address the realities of modern commerce, including e-commerce and complex new products. It provides a mechanism for consumers to seek redress against manufacturers and service providers for harm.
To get a foundational overview, let's watch a few key segments from a discussion on Sansad TV about the bill before it became law.
Laws in the making - The Consumer Protection Bill, 2019
This discussion, 'Laws in the making - The Consumer Protection Bill, 2019,' provides an excellent summary of the Act's purpose and key innovations. We will focus on the sections that define its core scope and introduce the crucial concept of product liability.
Please watch the following clips: Introduction to the Bill (00:00 - 01:13): Understand the overall purpose of replacing the 1986 Act to address new market challenges. Compensation for Harm (01:13 - 01:54): Note the broad definition of harm for which a consumer can seek compensation, including property damage, personal injury, and even emotional harm. Product Liability Actions (08:41 - 09:21): This is a critical section. Pay close attention to the introduction of a specific 'product liability' action, which imposes liability directly on the manufacturer for defects.
As the video highlights, the CPA, 2019 created a robust framework for consumer rights. At its heart are two concepts that determine liability:
- Defect: Defined in Section 2(10) of the Act as "any fault, imperfection or shortcoming in the quality, quantity, potency, purity or standard which is required to be maintained by or under any law... or as is claimed by the trader". This applies to products.
- Deficiency: Defined in Section 2(11) as "any fault, imperfection, shortcoming or inadequacy in the quality, nature and manner of performance which is required to be maintained... or has been undertaken to be performed by a person". This applies to services.
The 2019 Act significantly introduced a dedicated chapter on Product Liability (Chapter VI), creating a clearer, more direct path for consumers to hold manufacturers accountable for defects.
2. The Core Dilemma: Is AI a 'Product' or a 'Service'?
This seemingly simple question is the most significant hurdle in applying the CPA to AI-related harm. The answer determines the entire liability framework.
- If an AI system is a product with a defect, the manufacturer can be held liable under the strict liability provisions of the Act.
- If it's a service with a deficiency, the case is typically judged based on negligence, which requires proving the service provider failed to exercise a reasonable standard of care.
The following article offers a detailed legal analysis of this classification problem.
Addressing Product and Service Liability Concerns in ...
The article 'Addressing Product and Service Liability Concerns in Artificial Intelligence' from the Law School Policy Review directly tackles this foundational question.
Please read the following two sub-sections: 'Is AI a product under the Indian Law' (Part of Section I.A): Focus on the arguments for and against classifying AI as a product. Note the CPA's physical definition of a product ('gaseous, liquid, or solid') and the challenges this poses for intangible software. Pay attention to the international approaches mentioned. 'Argument against AI as service' (Section I.A.ii): Understand the four key problems with applying a negligence standard (the typical test for services) to AI. This is crucial for grasping why the 'product' classification is often preferred from a consumer protection standpoint.
As the article explains, the classification is ambiguous:
- AI as a Product: The CPA's definition is ill-suited for software. However, when AI is embedded in a tangible object (a smart thermostat, an autonomous vehicle, a medical diagnostic machine), the argument is stronger. The physical object is clearly a product, and the embedded AI is an integral component.
- AI as a Service: This seems to fit for cloud-based AI, like a SaaS legal research tool or a translation API. However, the article compellingly argues that this puts an unfair burden on the consumer. To prove negligence, a consumer would need to establish the "standard of care" for an AI system, which is incredibly difficult for even experts to define, let alone a layperson. It would require the user to constantly second-guess the AI, defeating its purpose.
This legal distinction is critical. Classifying AI as a product shifts the legal burden. It focuses on the quality of the item itself, moving closer to a strict liability standard where the manufacturer is responsible for a defect, regardless of whether they were negligent. Classifying it as a service focuses on the user's conduct and the provider's failure to be careful, a much higher bar for the consumer to clear.
3. Defining AI 'Defects' and 'Deficiencies'
Once we have a working classification, how do we identify the actual 'defect' or 'deficiency' in an AI system?
AI Defects as Design Defects
AI systems rarely suffer from "manufacturing defects" in the traditional sense, where one unit is faulty due to a production error. Instead, when an AI system is flawed, the flaw is usually inherent in its very design and thus affects every instance of that AI.
Addressing Product and Service Liability Concerns in ...
The same LSPR article provides a clear analysis of why AI flaws are best understood as design defects.
Please read the section 'AI defects as design defects' (Section I.C). Focus on the distinction between manufacturing and design defects. Note the discussion of the US 'risk-utility test' as a method for evaluating design defects. This connects to how one might balance an AI's benefits against its potential for harm.
As you read, an AI design defect can stem from:
- Biased training data: The algorithm learns and perpetuates harmful stereotypes (a concept we covered in Module 3).
- Flawed model architecture: The underlying neural network or algorithm is not suited for the task.
- Poorly defined objective function: The goal the AI is programmed to optimize leads to unsafe or undesirable outcomes.
AI Deficiency as Performance Failure
If we treat an AI system as a service, a deficiency would be an "inadequacy in the quality, nature and manner of performance." This is easier to identify in practice. Examples include:
- A medical chatbot giving incorrect dosage information.
- An algorithmic trading platform executing trades contrary to the user's instructions.
- A generative AI "hallucinating" and providing false information that a user relies on to their detriment.
The challenge, as noted before, remains proving that this failure was due to the provider's negligence, not just the inherent probabilistic nature of the AI.
4. Application through Case Studies
Let's apply these concepts to concrete scenarios. The following report uses hypotheticals based on real-world incidents to explore liability.
Crafting a Liability Regime for AI Systems in India
The report 'Crafting a Liability Regime for AI Systems in India' by the Esya Centre and Cyber Saathi Foundation provides excellent, practical case studies. We will examine two of them.
Please read the analysis of the following two case studies: The AI Diagnostic Tool (beginning on page 8, under the heading that starts 'We find and conclude as follows on this case study...'): This case deals with a medical AI. Focus on how it applies consumer protection law and identifies the 'defect' (unrepresentative datasets). Note the conflict it points out between the doctor's liability and the manufacturer's liability. The Autonomous Vehicle Sensor (beginning on page 9, under the heading that starts 'Case Study 4 involves a minor collision...'): This case deals with an AI as a component. Pay attention to the 'component part' doctrine and how Section 87(2) of the CPA might absolve a component manufacturer if they provided adequate warnings to the final product manufacturer (e.g., the car company).
These case studies illustrate the practical legal knots:
-
The AI Diagnostic Tool: This is a classic case of a design defect. The product is defective because it was trained on non-representative data. Under the CPA, the patient (consumer) could potentially file a product liability claim against the AI developer (manufacturer). However, the case is complicated by the doctor's role. Did the doctor, as a service provider, commit an act of negligence by over-relying on the tool? The CPA must untangle the liability of the product manufacturer and the service provider.
-
The Autonomous Vehicle Sensor: This scenario highlights the complexity of modern supply chains, something you've likely encountered in your consulting work. The AI sensor is a 'product' but also a component. The CPA's product liability rules specifically address this, creating a potential defense for the sensor maker if they warned the car company about the component's limitations. This shifts the ultimate responsibility to the final product manufacturer who integrated the component into the car.
Test your understanding!
A healthcare clinic chain (similar to the ones you've worked with) deploys a new AI-powered chatbot to answer patient queries about medication. The chatbot is developed by a third-party tech company. A patient asks the chatbot about the correct timing for a specific antibiotic. The chatbot, due to a flaw in its NLP model, misinterprets the query and advises the patient to take the medication with dairy, which significantly reduces its effectiveness. The patient's infection worsens, requiring hospitalization.
Analyze this scenario using the concepts from this lesson:
- Would you classify the chatbot as a 'product' or a 'service'? Justify your choice.
- Based on your classification, is the harm caused by a 'defect' or a 'deficiency'? Describe it.
- Who are the potential parties that could be held liable under the CPA, 2019, and on what grounds?
Show answer
-
Classification: This is ambiguous, but a strong argument can be made for classifying the chatbot as a product. While it provides a 'service' (answering questions), the harm stems from a flaw inherent in the chatbot software itself, which was mass-produced/licensed by the tech company. Following the reasoning in the readings, treating it as a product with a design defect allows for a more straightforward application of strict product liability, which is more consumer-friendly than proving negligence in a service.
-
Defect/Deficiency: If classified as a product, the harm is caused by a design defect. The defect is the flawed NLP model that leads to misinterpretation. It's not a one-time glitch (manufacturing defect) but an inherent shortcoming in the AI's design that would affect any user asking a similar question. If classified as a service, it would be a deficiency in the service provided by the clinic, characterized by the "inadequacy in the quality" of the information provided.
-
Liable Parties:
- The Tech Company (Developer): As the product manufacturer, they could be held liable in a product liability action under the CPA. The chatbot contained a design defect that made it unsafe for its intended use.
- The Healthcare Clinic: As the product seller or service provider, the clinic could also be held liable. They offered the defective product/deficient service to the patient. If the chatbot is seen as a service, the clinic would be liable for deficiency in service. Even if it's a product, the clinic chose to deploy it and could be seen as having a duty of care to ensure its safety. They could potentially have a cross-claim against the tech company. The patient would likely sue both parties.
Conclusion
Today, we've established how India's Consumer Protection Act, 2019, provides the primary legal avenue for addressing harm caused by AI in a commercial context. We've seen that while the Act is modern, applying its traditional categories to AI is a significant legal challenge.
Key Takeaways:
- The CPA, 2019 creates a framework for consumer redress based on defects in products and deficiencies in services.
- A central ambiguity is whether AI should be treated as a product (invoking stricter liability standards) or a service (requiring proof of negligence). The classification has profound implications for the consumer.
- AI failures are best understood as design defects—flaws in algorithms or training data—rather than manufacturing defects.
- In real-world scenarios involving multiple actors (developers, deployers, users), the CPA provides rules for allocating liability, but the process is complex, as seen in the medical AI and component part examples.
Preview of the Next Lesson:
We have focused on situations where AI systems malfunction or are flawed. But what happens when an AI functions exactly as designed, but its purpose is to manipulate consumers? In our next lesson, we will analyze 'unfair trade practices' in the context of algorithmic manipulation of consumer choice, exploring how AI can be used to exploit cognitive biases and create new challenges for consumer protection and competition law.