Recommended resources
The freely available resources this lesson will direct you to.
Introduction
In our last lesson, we established that horizontal scaling—adding more servers—is the standard strategy for building resilient, large-scale applications. This approach, however, introduces a critical question: with multiple servers available, how do we intelligently direct incoming user traffic to them? Simply sending all requests to a single server defeats the purpose of scaling, while random distribution might not be efficient.
This lesson directly answers that question by introducing the load balancer, a fundamental component in any distributed system. You will learn to describe the load balancer's role as the "traffic director" of your application. We will explore its core functions, compare the different types you'll encounter, and dissect the common algorithms it uses to distribute requests. For you, as a developer aiming for a senior role, mastering this component is non-negotiable, as it's the linchpin that makes scalable architecture possible.
1. What is a Load Balancer?
At its heart, a load balancer is a device or software that sits in front of your servers and routes client requests across all servers capable of fulfilling those requests. Its goal is to ensure that no single server is overworked, which in turn leads to higher availability, better performance, and greater reliability for your application.
To get a quick, dynamic overview of what a load balancer is and the key benefits it provides, please watch the first part of this video from ByteByteGo.
What is a LOAD BALANCER really about?
This short video provides a concise and well-animated introduction to the concept of load balancing and its primary advantages.
Watch from the beginning until the end of the benefits section. Focus on how it enables workload distribution, dynamic scaling, and high availability.
As the video highlights, a load balancer is not just about spreading traffic; it's a foundational piece for building robust systems that can grow and handle failure gracefully.
2. The Three Core Functions of a Load Balancer
While traffic distribution is its most famous job, a production-grade load balancer is responsible for three distinct functions that work together to create a reliable system. We can think of these as: distributing traffic, monitoring server health, and ensuring its own availability.
The following reading from AlgoMaster's guide on designing a load balancer provides an excellent, structured breakdown of these responsibilities.
Design Load Balancer | System Design Interview
This article breaks down the design of a load balancer into its fundamental requirements, which is a great way to understand its role from first principles.
Please read the following three sections: Start at the High-Level Design section and read the introductory paragraphs that outline the three fundamental jobs: the three fundamental things. Next, jump to Requirement 2 and read the section on Health Monitoring. Pay close attention to how it uses health checks to automatically route around failed servers. Finally, read the section on High Availability. Focus on understanding why the load balancer itself can be a single point of failure and how the Active-Passive and Active-Active patterns solve this.
Let's recap and solidify those concepts:
- Traffic Distribution: This is the primary role. The load balancer acts as a single entry point for all incoming requests and forwards them to one of the backend servers.
- Health Monitoring: A load balancer constantly runs "health checks" on the backend servers. If a server fails a health check (e.g., it stops responding or returns an error), the load balancer will temporarily stop sending traffic to it until it becomes healthy again. This automatic removal and re-introduction of servers is key to building self-healing systems.
- High Availability: If the load balancer itself fails, the entire application goes down. To prevent this, load balancers are typically set up in a high-availability cluster. The most common pattern is Active-Active, where multiple load balancers share the traffic, providing both redundancy and increased capacity.
3. Types of Load Balancers: Layer 4 vs. Layer 7
Load balancers are often categorized by which layer of the OSI network model they operate on. The two most common types you'll work with are Layer 4 and Layer 7.
As the "Types of Load Balancers" section in the image illustrates:
- A Layer 4 (L4) load balancer works at the transport layer. It makes routing decisions based on information like the source/destination IP address and port. It's very fast because it doesn't need to inspect the content of the traffic. It simply forwards network packets.
- A Layer 7 (L7) load balancer works at the application layer. It can inspect the content of the request, such as HTTP headers, the URL path, or cookies. This allows for much more intelligent routing decisions.
For your work with React frontends and Spring Boot backends, an L7 load balancer is particularly powerful. It could, for example, route requests for /api/v1/... to your Java microservices while routing requests for /static/... to a different set of servers optimized for serving static assets.
To understand the trade-offs in detail, the following reading is highly recommended.
Design Load Balancer | System Design Interview
This section from the AlgoMaster guide provides a clear comparison of Layer 4 and Layer 7 load balancing.
Please read the entire section 6.4 Layer 4 vs Layer 7 Load Balancing. Focus on the table that compares their capabilities and the "When to Use Each" guide at the end.
The key takeaway is a classic system design trade-off: Speed vs. Intelligence. Layer 4 is faster but "dumber," while Layer 7 is "smarter" but has higher overhead because it does more work.
4. How Does a Load Balancer Distribute Traffic?
The "brain" of the load balancer is its configured algorithm, which is the rule it follows to decide which backend server gets the next request. Choosing the right algorithm is crucial for ensuring traffic is spread evenly and efficiently.
Let's explore the most common algorithms, many of which are shown in the "Load Balancing Algorithms" section of the infographic above.
Design Load Balancer | System Design Interview
This section provides a deep dive into the most common load balancing algorithms, explaining how they work and their pros and cons.
In section 6.1 Load Balancing Algorithms, please read the descriptions for the following four common approaches: Round Robin Weighted Round Robin Least Connections IP Hash As you read, focus on the "How It Works" and "When it falls short" parts for each one.
Here is a summary of the key algorithms and their use cases:
| Algorithm | How It Works | Best For | Main Drawback |
|---|---|---|---|
| Round Robin | Sequentially sends requests to servers in a list, one by one. | Identical servers with uniform, stateless requests. | Doesn't account for server load or health. A slow server still gets its share of traffic. |
| Weighted Round Robin | Like Round Robin, but servers with a higher "weight" (more capacity) get proportionally more requests. | Servers with different capacities (e.g., different instance sizes). | Still doesn't adapt to real-time server load. A powerful but busy server keeps getting traffic. |
| Least Connections | Sends the next request to the server with the fewest active connections. | Environments where request processing time varies. This is a common and effective default. | Requires tracking connection state, which adds slight complexity. |
| IP Hash | Uses a hash of the client's IP address to determine which server receives the request. | Achieving "session persistence" or "stickiness" without cookies. Ensures a user always lands on the same server. | Can cause uneven load if many users are behind a single IP (like a corporate NAT). |
For modern, stateless applications like the microservices you build with Spring Boot, the Least Connections algorithm is often a great choice because it adapts dynamically to the real-time load on your servers.
5. Load Balancers in a System Architecture
Finally, it's important to understand where load balancers fit within a larger architecture. They aren't just used at the front door of your application.
.webp)
This diagram shows two types of placements:
- External Load Balancer: This is the internet-facing load balancer. It takes traffic from users on the internet and distributes it to your front-end web servers.
- Internal Load Balancer: These load balancers manage traffic between services inside your own network. For example, an internal load balancer might sit between your web front-end servers and your backend application servers, or between your application servers and your database services.
This pattern is essential for microservices architectures. It allows you to scale each layer of your application independently. For example, you could scale your Spring Boot API services horizontally without needing to scale your database layer, with internal load balancers managing the traffic flow between them.
Conclusion
In this lesson, you've moved from the concept of horizontal scaling to understanding the critical component that makes it work: the load balancer. You are now able to describe its role not just as a traffic distributor, but as a comprehensive manager of availability and reliability.
Key Takeaways:
- A load balancer's primary role is to distribute incoming traffic across a pool of servers, preventing any single server from being overwhelmed.
- It performs three core functions: traffic distribution, health monitoring of backend servers, and ensuring its own high availability.
- Load balancers come in different types, most commonly Layer 4 (fast, but limited to network-level data) and Layer 7 (smarter, application-aware, but with higher overhead).
- The choice of algorithm (e.g., Round Robin, Least Connections) determines how traffic is spread and should be chosen based on application needs.
- Load balancers are used both externally (facing the internet) and internally (between services) to enable independent scaling of different application tiers.
You have now completed the "System Design Foundations" module, covering caching, scaling, and load balancing. These are the fundamental building blocks of any scalable system. In our next module, "Architecting for Business Impact," you will learn to combine these components and apply first-principles thinking to design entire systems that solve complex business problems.