System Design Fundamentals: What is a Load Balancer ?
Introduction to Load Balancers, How they work and Benefits of Load Balancers
Load Balancers is one of the fundamental topics for system design interviews. Load Balancer is a building block and is employed in all the real-world distributed systems.
Understanding load balancer and it’s working will help you understand the quality attributes of a system such as availability, scalability, and reliability in detail.
In this article, we will explore what a load balancer is and why it’s needed. We will understand the working of a load balancer with an analogy. Additionally, we will take a look at the different advantages of a load balancer.
What is a Load Balancer ?
Load Balancer is a component in distributed systems that is responsible for distributing the client requests to the different backend servers. The primary goal of a Load Balancer is to improve the reliability, availability, scalability and security of the system. It achieves this by balancing the load and preventing a single server from being overwhelmed.
Why do we need a Load Balancer ?
With the internet proliferation, the number of internet users has grown exponentially. To provide a seamless customer experience and stay competitive, companies are focussing on scalability and high availability.
To handle growing number of user requests, websites use lots of machines. Without a load balancer, a single system could get bombarded with large number of requests. As a result, the server could either fail, go down, or user response could get delayed.
A load balancer solves this problem by distributing the load evenly. Additionally, it also performs health checks and removes the servers from the pool if they are unreachable or unresponsive.
How do Load Balancers work ?
Analogy
Before delving into the functionality of a load balancer, let's consider the analogy of a supermarket with multiple counters. What could happen if customers randomly chose one queue among many?
The person at the chosen counter could become overwhelmed with work.
Other counters might remain underutilized.
Customers might experience longer waiting times in the queue.
Such a process could lead to frustration among customers due to the time-consuming nature.
This is why a coordinator is essential to assign customers to the appropriate counter. The coordinator monitors the queue lengths and allocates customers accordingly.
The coordinator ensures efficient utilization of all counters. Additionally, customers spend less time waiting, significantly enhancing their overall experience.
Load Balancer working
In the aforementioned analogy, substitute the supermarket counters with backend servers and the coordinator with a load balancer. Here is a step-by-step process that unfolds when clients submit requests to a website :-
Load Balancer receives the request from the client.
Employing the configured load-balancing algorithm, it identifies the server responsible for handling the request.
Subsequently, the load balancer forwards the request to the designated server and awaits the response.
Upon receiving the response, the load balancer processes it and sends the final response back to the client.
The process is illustrated in the below diagram :-
There are different types of algorithms that a Load Balancer can use to choose the backend server. Few of the common algorithms are round robin, least connections, weighted round robin, etc. We will discuss the algorithms and further dive deep into them in the next article.
Types of Load Balancers
Hardware Load Balancer - These are physical appliances that need to be installed in the data centres. These Load Balancers are costly and require time to set up. They also don’t offer the flexibility provided by software load balancers.
Software Load Balancer - These load balancers are software application and can be run on any machine. Few of the well-known software load balancers are NGINX, HAProxy, AWS ELB (Elastic Load Balancer), Azure Load Balancer
Cloud Load Balancer - These load balancers are software based & responsible for distributing the customer workload across different cloud providers. Many companies run multi-cloud setup. In such cases, it becomes necessary to optimize the costs and resources. Cloud Load Balancers help in distributing the traffic across the different cloud environments.
Advantages of Load Balancers
High performance - A load balancer distributes the work load evenly among the servers. This improves the efficiency of each server and reduces the latency to process each request. It thereby improves the response time and the user experience.
High scalability - A load balancer can help the system to add/remove servers from the pool. Auto-scaling systems in the cloud add or remove server instances based on the metrics (requests/sec) emitted by the load balancer.
Increased availability - A load balancer detects which server instance isn’t responding by performing health checks. It then redirects the requests to the healthy server instances. This improves the availability of the system.
Improved reliability - A load balancer improves reliability by ensuring that the system continues to function even in case of failures.
Increased security - At times bad actors can overwhelm the system with large number of requests. A famous example is DDOS (Distributed Denial of Service) attack done by malicious actors. Load balancers act as firewall and prevent DDOS attacks by rate limiting, anomaly detection and blacklisting/whitelisting IPs.
Conclusion
In short, load balancers are like traffic managers for a website. They ensure everything works fine when many people access the website. Load balancers are like organizers in supermarket who make sure every counter(server) gets a fair share of customers (requests).
Load balancers distribute the load evenly and prevent a single server from getting overwhelmed. They prevent the chances of a website crashing and disrupting the user experience.
In a nutshell, load balancers help websites perform well, scale easily and stay reliable. We will explore how the different load-balancing algorithms work in detail in upcoming articles. Also, we will dive deep into working of few open source load balancers such as NGINX and HAProxy.
For those of you, who are curious to understand the internals of load balancer, you can go through the working of NGINX. The article is detailed and covers all the different modules of NGINX.
I hope you guys enjoyed reading my article. Thanks for reading the article!
Before you go:
References
Here are few other references you can go through :-