Recommended resources
The freely available resources this lesson will direct you to.
- Fraud Detection with AI: Ensemble of AI Models Improve Precision & Speedwww.youtube.com
- How to Build Human-in-the-Loop for AI Agents (Practical Guide)www.youtube.com
- 7 AI Tools That Automatically Escalate High-Risk Customer Cases in ...www.usefini.com
- The rise of agentic AI: transforming fraud risk management - EYwww.ey.com
Introduction
In our previous lesson, we designed the context-enrichment strategies that allow a fraud detection agent to build a nuanced, evidence-based understanding of a transaction. We established that by leveraging semantic, relational, and document-based context, the agent moves beyond simple rule-based flagging to produce a sophisticated risk assessment.
This lesson addresses the critical next step: what does the agent do with that assessment? Our objective is to specify escalation logic and automated alert mechanisms for suspicious activity. This is where the agent's analytical capabilities are translated into concrete, decisive action.
We will architect the decision-making framework that governs the agent's response, moving from the analysis phase to the action phase of the agentic cycle. You will learn to define a spectrum of possible actions, design the logic that triggers them, and specify the technical mechanisms for alerting human operators in a way that is both robust and actionable.
1. The Action Spectrum: Beyond Binary Decisions
Traditional fraud systems often operate on a binary basis: a transaction is either approved or flagged for review. An agentic system, with its richer contextual understanding, can deploy a more graduated and intelligent set of responses. The goal is not just to catch fraud but to do so with minimal friction for legitimate customers and optimal use of human analysts' time.
The EY article, "The rise of agentic AI," outlines a "value chain of defence" that provides a useful model for these actions. An agent can:
- Detect a suspicious event.
- Manage the alert by resolving it or escalating.
- Contact the customer for verification.
- Act on the case (approve, block, or escalate).
This leads to a spectrum of actions an agent can take, ordered by increasing severity and required certainty:
- Auto-Approve: For transactions with a very low risk score, the agent approves them without intervention. This is the default "happy path."
- Request Step-Up Authentication: For low-to-medium risk cases where uncertainty exists. Instead of blocking a potentially legitimate transaction, the agent can trigger an automated request for additional verification (e.g., a push notification, an SMS code, or a biometric prompt). This balances security with user experience.
- Auto-Block/Decline: For transactions with a very high confidence of being fraudulent (e.g., matching a known pattern, originating from a sanctioned entity). The agent acts decisively to prevent loss.
- Escalate for Human Review: For ambiguous, high-value, or novel cases that the agent cannot confidently resolve. This is the critical handoff point, and making it effective is the focus of this lesson.
2. Designing Escalation Logic: The Decision Engine
The "escalation logic" is the set of rules that determines which action the agent takes. A robust decision engine doesn't rely on a single risk score. Instead, it synthesizes multiple inputs to make a judgment.
These inputs include:
- Quantitative Risk Score: The probability of fraud (e.g., a score from 0.0 to 1.0) generated by the underlying analytical models.
- Qualitative Flags: Specific, explainable reasons for suspicion identified during context enrichment (e.g., "device ID linked to 3 other flagged accounts," "transaction description contains keywords from a money laundering typology").
- Business Context: The financial value of the transaction, the lifetime value of the customer, and the product type.
- Confidence Level: The model's own certainty about its risk score.
A powerful technique is to use a multi-stage process. Low-confidence or borderline cases aren't immediately sent to humans. Instead, they can be escalated to a more powerful, albeit more expensive, analytical process, such as a large language model (LLM).
The following video from IBM Technology explains this concept of an AI-driven escalation path.
Fraud Detection with AI: Ensemble of AI Models Improve Precision & Speed
This video, 'Fraud Detection with AI,' demonstrates an ensemble approach where a predictive model handles clear-cut cases and escalates ambiguous ones to a more powerful encoder LLM for deeper analysis. This illustrates a form of automated, tiered escalation before any human involvement.
Watch the segment from 05:41 to 07:55. Focus on how the system uses the initial model's 'confidence level' as a trigger. Low-confidence transactions are not immediately flagged for humans but are first 'escalated to an encoder model LLM for further analysis' using both structured and unstructured data.
This tiered approach allows you to create a sophisticated decision matrix. Here is a simplified example of what this logic could look like in practice:
| Risk Score | Confidence | Transaction Value | Qualitative Flags | Action |
|---|---|---|---|---|
| < 0.3 | High | Any | None | Auto-Approve |
| 0.3 - 0.7 | Low | < $5,000 | "Unusual Location" | Request Step-Up Auth |
| 0.3 - 0.7 | Low | > $5,000 | "Unusual Location" | Escalate to Tier-1 Analyst |
| > 0.9 | High | Any | "Known Fraud Pattern" | Auto-Block & Escalate |
| 0.7 - 0.9 | High | Any | "Novel Network Activity" | Escalate to Fraud Strategy Team |
This matrix is not static. As the EY article LINK suggests, an advanced agent can perform "control optimization and strategy tuning," continuously analyzing the effectiveness of this logic and suggesting updates to thresholds and rules.
3. Architecting Automated Alert Mechanisms
Once the logic dictates an escalation, the system must execute it through an alert mechanism. The architectural pattern for this depends on the user context: is a user actively waiting for a response, or is this a background process?
There are two primary patterns for implementing this, often referred to as "human-in-the-loop" systems.

The following video provides an exceptionally clear, practical guide to building the backend architecture for both real-time and asynchronous escalation patterns.
How to Build Human-in-the-Loop for AI Agents (Practical Guide)
The video 'How to Build Human-in-the-Loop for AI Agents' is a practical guide to the engineering behind escalation. It is essential viewing for understanding how to build robust alert systems.
Please watch two key sections. First, from 12:33 to 19:17, which details the architecture for a real-time (SSC streaming) escalation suitable for a user at a checkout. Second, from 19:17 to 22:21, which explains the asynchronous pattern for backend processes using notifications (e.g., Slack, email). Focus on the core principles of state management.
As the video emphasizes, the key architectural principles for any robust escalation system are:
- Deferred Execution: When approval or review is needed, the system must save the state of the process instead of executing the final action.
- State Serialization: Persist the entire agent context—including the transaction data, the reasoning trace, and the pending action—into a database. This ensures the process can be resumed later.
- Stateless Resume: The workflow should be able to resume by loading the state from the database, not from memory. This makes the system resilient to timeouts, network issues, or delays in human response.
By adhering to these principles, you can design a system that reliably hands off cases to human analysts without losing context or state, regardless of whether the review takes seconds or days.
4. Specifying the Content of an Alert
An alert that simply says "Suspicious Transaction: ID #XYZ" is ineffective. It forces the human analyst to start their investigation from scratch. The goal of an agentic escalation is to deliver a pre-analyzed case file, not just a flag.
To design an effective alert, we can draw from best practices in AI escalation tools.
7 AI Tools That Automatically Escalate High-Risk Customer Cases in ...
The article '7 AI Tools That Automatically Escalate High-Risk Customer Cases' provides an excellent checklist of what constitutes an intelligent and useful escalation. It moves beyond simple ticketing to focus on decision accuracy and context enrichment.
Please read the section 'Key Features to Look for in AI Escalation Software'. Focus on the descriptions for 'Explainable Decision Logic', 'Risk Scoring with Context', and 'Audit Trail Completeness'.
Based on these principles, a high-quality alert delivered to an analyst's dashboard (or via API to a case management system) should contain:
- Case Summary: A concise overview of the transaction details (Amount, Recipient, Timestamp, etc.).
- Risk Assessment: The final risk score and the agent's confidence level.
- Explainable Decision Logic (The "Why"): A clear, human-readable trace of the reasoning. For example:
Risk Score: 0.85 (High)Reasoning: Matched behavioral pattern of 'Rapid Small Transfers followed by Large Withdrawal'.Supporting Evidence: Recipient account shares IP address with 3 previously blocked accounts. See attached graph.
- Enriched Context: Direct links to the supporting evidence the agent used, such as similar past transactions, a visualization of the relational graph, and relevant snippets from KYC documents.
- Audit Trail: A timestamped log of the agent's analysis steps.
- Suggested Next Actions: The agent can even propose a course of action for the human, such as "Recommended Action: Initiate voice verification with customer" or "Recommended Action: Add entity to watchlist."
This turns the human from an investigator into a validator, drastically improving their efficiency and the accuracy of the final decision.
Conclusion
In this lesson, we have specified the logic and mechanisms that enable an agent to act decisively on its analytical findings. We've established that effective fraud management is not a binary decision but a nuanced spectrum of actions, governed by a sophisticated decision engine.
Key Takeaways:
- Action Spectrum: An agent's response to a suspicious transaction should range from auto-approval and step-up authentication to auto-blocking and intelligent escalation.
- Escalation Logic: The decision to escalate should be based on a matrix of inputs, including quantitative scores, qualitative flags, business context, and model confidence. Tiered escalation to more powerful AI models can precede human review.
- Robust Architecture: Alert mechanisms must be built on sound engineering principles, including deferred execution and stateless resume, to handle both real-time and asynchronous workflows reliably.
- Actionable Alerts: An escalation is not just a flag; it is an enriched case file containing an explainable reasoning trace, supporting evidence, and an audit trail to empower human analysts.
Preview of the Next Lesson:
Having designed the agent's actions and the audit trail for fraud detection, we will now shift our focus to another critical application in financial services: compliance. In the next lesson, "Design a data pipeline to ingest and prepare regulatory policy documents for agent-based retrieval," we will lay the foundation for an agent that can navigate the complex world of regulatory requirements, starting with building its essential knowledge base.