API Gateway: Patterns, Responsibilities, and Trade-offs

1. From Monolith to Microservices: The Rise of the API Gateway

As we've discussed, moving from a monolithic architecture to microservices introduces complexity. Instead of one large application, you now have dozens or even hundreds of small, independent services. This raises a critical question: how should external clients (like a mobile app or a web frontend) interact with this distributed system?

Having clients call each microservice directly would be chaotic. The client would need to know the address of every service, handle authentication for each one, and orchestrate complex calls. This tightly couples the client to the backend architecture, making any changes to the backend a nightmare.

The API Gateway pattern solves this problem by providing a single, unified entry point for all client requests. Let's start with a video that clearly explains this evolution and the gateway's primary role.

As the video explains, the API Gateway acts as a reverse proxy that sits between your clients and your backend microservices. It abstracts the complexity of the internal system, allowing clients to interact with a simple, unified API.

2. Core Responsibilities of an API Gateway

While its primary job is request routing, a modern API Gateway handles many other critical functions, often called "cross-cutting concerns," because they apply across multiple services. Centralizing these in the gateway simplifies the microservices themselves, allowing them to focus purely on business logic.

Let's dig into these responsibilities. The following reading provides a detailed breakdown of the request lifecycle through a gateway.

To summarize and consolidate those points, here are the key responsibilities you should be able to discuss in an interview:

• Request Routing: The fundamental task. The gateway inspects the incoming request (e.g., the URL path, HTTP method) and forwards it to the appropriate downstream service based on a routing map.
• Authentication & Authorization: The gateway can act as the first line of defense, authenticating the user (e.g., by validating a JWT token) and performing coarse-grained authorization checks (e.g., "Is this user a paying customer allowed to access this feature?").
• Rate Limiting & Throttling: Protects backend services from being overwhelmed by too many requests from a single client, preventing abuse and ensuring stability.
• SSL/TLS Termination: The gateway can handle the decryption of incoming HTTPS requests and send unencrypted traffic to the internal services, offloading this CPU-intensive task from the services themselves.
• Caching: For frequently accessed, non-personalized data, the gateway can cache responses, reducing latency for clients and load on backend systems.
• Request/Response Transformation: It can transform requests and responses on the fly. A common example is translating between external client-facing REST APIs and internal gRPC-based services.
• Logging, Monitoring, & Tracing: As all traffic passes through the gateway, it's the perfect place to centralize logging and collect metrics on API usage, error rates, and latency. It's also the ideal entry point for distributed tracing.

The following diagram provides a great visual summary of how these different functions fit together within an API Gateway's architecture.

3. API Gateway vs. Load Balancer vs. Reverse Proxy

A common point of confusion, and a frequent interview question, is the difference between these three components. While they have overlapping functionalities, their primary purposes are distinct.

• A Reverse Proxy is a server that sits in front of web servers and forwards client requests to those servers. It's a generic term.
• A Load Balancer is a specific type of reverse proxy that specializes in distributing traffic across multiple instances of the same service to ensure no single instance is overloaded.
• An API Gateway is a more advanced, application-aware (Layer 7) reverse proxy designed specifically for managing APIs in a microservices context. It includes load balancing capabilities but adds the rich set of features we just discussed (auth, rate limiting, etc.).

Think of it this way: all API Gateways are reverse proxies, but not all reverse proxies are API Gateways.

The following video uses a great analogy and walks through scenarios of increasing scale to clarify when you would use each component.

4. Design Patterns & Trade-offs

For a senior role, simply listing the gateway's functions isn't enough. You need to discuss architectural patterns and their trade-offs. The most significant decision is whether to use a single gateway or multiple gateways.

This next reading explores this choice and introduces a critical pattern: the Backend for Frontend (BFF).

Let's summarize the key patterns:

Single API Gateway

A single, centralized gateway handles all incoming requests for all clients.

• Pros: Simpler to deploy, manage, and monitor initially. A single place for all routing and global policies.
• Cons: Can become a monolithic bottleneck. A single point of failure. All teams must coordinate changes, which can slow down development. It may become bloated with logic for many different client types.

Multiple API Gateways / Backend for Frontend (BFF)

You deploy a separate gateway for each "frontend" or client type (e.g., one for the web app, one for the mobile app, one for public third-party developers).

• Pros: Each gateway is tailored to the specific needs of its client, providing optimized payloads and protocols. Teams can manage their own gateways independently, promoting autonomy and faster development cycles. Isolates failures; a problem with the mobile gateway doesn't affect the web gateway.
• Cons: More components to manage, leading to higher operational overhead. Potential for code duplication across gateways (though this can be managed with shared libraries).

The image below clearly illustrates the difference between these two patterns.

For any large-scale system with diverse clients, the BFF pattern is generally the superior approach as it aligns better with the principles of microservices and team autonomy.

5. Interview Strategy: How to Talk About API Gateways

In a 45-minute system design interview, you can't afford to get bogged down in the details of a standard component. The API Gateway is an expected part of a microservices design.

Here is the key advice, echoed across multiple expert resources:

1. Acknowledge and Move On: When starting your high-level design, draw a box for the API Gateway. State its purpose clearly: "I'll place an API Gateway here to act as the single entry point for all clients. It will handle request routing to the appropriate microservices, as well as cross-cutting concerns like authentication, rate limiting, and logging."
2. Don't Over-Design It: The interviewer knows what an API Gateway is. Unless they specifically ask you to deep-dive into it, don't waste time explaining its internal workings or debating specific open-source vs. managed solutions. The core of the problem lies elsewhere.
3. Be Ready for a Deep Dive: If the interviewer does probe, be prepared to discuss the trade-offs we've covered, especially the Single Gateway vs. BFF pattern. This demonstrates your seniority and architectural thinking.

This final clip reinforces this crucial interview strategy.

Conclusion

You've now learned how the API Gateway serves as the cornerstone of client interaction in a microservices architecture. It simplifies complexity for clients while centralizing critical cross-cutting concerns for developers.

Key Takeaways:

• Purpose: An API Gateway is a managed entry point for clients, abstracting the complexity of the backend microservices.
• Responsibilities: Its core job is routing, but it also handles authentication, rate limiting, caching, transformation, and observability.
• Patterns & Trade-offs: The main design choice is between a single gateway (simpler, but can become a bottleneck) and multiple gateways using the BFF pattern (more complex, but better for scalability, client optimization, and team autonomy).
• Interview Strategy: Acknowledge the API Gateway's role, state its responsibilities, and then move on to the more unique aspects of the system you are designing.

Next Lesson Preview:

An API Gateway needs to route requests to healthy service instances. But how does it, or the service registry it communicates with, know which instances are healthy? In our next lesson, we will dive into the patterns and mechanisms for this by learning how to design health checking mechanisms for service reliability and discovery.