Load Balancers Decoded

What are Load Balancers?

In today's world, websites are buzzing with thousands of users from all corners of the globe. Imagine you're trying to access information—text, images, videos—from your favorite site; you want it fast and within seconds right?

Well, companies uses many servers to make that happen, but here's where load balancing steps in as the superhero. Picture load balancing as the ultimate traffic cop of the internet and its job is to distribute all the incoming online traffic seamlessly among a group of servers, ensuring your requests find the quickest route to their destination.

Load balancers for web application servers are normally placed between your backend servers and your firewall. Using a load balancer, which distributes client requests to available servers, lets you handle high traffic conditions with zero downtime.

What is Load Balancing and how that works?

When a website is bombarded by thousands of users simultaneously, handling all those requests can strain certain applications, sometimes even causing the entire system to crash.

For which we enter load balancing for distributing workloads and computing resources efficiently across multiple servers, whether they're on-premise or in the cloud. For high-traffic websites aiming for seamless operation and speed, load balancing is a must. It doesn't just enhance performance; it also beefs up your system's security, shielding it from potential failures.

So, how does it work? Load balancing isn't just about website traffic; it considers memory capacity, network, and CPU load on a server. The main goal is to make sure each part of your network carries a fair share of the load. It juggles data based on server workload, preventing any one system from getting overloaded.

Think of it like processors having a conversation about job arrival rates and CPU processing rates, using various algorithms to keep things balanced.

Benefits ?

In the blink of an eye, your website needs to make an impact. Managing high volumes of user traffic ensures a positive experience for your thousands (or millions) of visitors. Load balancers, employed by giants like Amazon or eBay, play a crucial role in distributing workloads efficiently.

Load Balancer Handles Sudden Bursts of Traffic by automatically dividing website loads among different servers, preventing downtime and keeping your site running without delays.

Reduce Downtime Load balancers act as guardians against three downtime culprits: server overload (think DDoS attacks or too much traffic), maintenance, and failures. They redirect traffic to functional servers, ensuring your site stays online.

Security and Compliance Load balancers add an extra layer of security, filtering out malicious traffic like DDoS attacks. They also simplify PCI compliance, a must for websites processing credit cards, by redirecting traffic to a secure public cloud provider.

Recommended tools for load balancing

NGINX: NGINX is a popular load balancing tool that can be used to selectively distribute traffic to back-end servers. Supports load balancing across HTTP, TCP, and UDP protocols.

Apache HTTP Server: This is a widely used load balancing tool. It provides support for a variety of operating systems and supports various load balancing algorithms such as round-robin and IP hashing.

Microsoft Azure Load Balancer: It is a cloud-based load balancing tool that can be used to distribute traffic across multiple virtual machines. It supports autoscaling and integrates well with other Microsoft Azure services.

Amazon Elastic Load Balancer: One of Amazon's most widely used tools, it provides scalable, elastic load balancing solutions for AWS cloud-based web applications. It supports HTTP, HTTPS, and SSL protocols in multiple zones and provides advanced security features such as SSL termination and access controls.

Load Balancing with Nginx

It's time to get your load balancer up and running. First things first , deploy an EC2 instance if you haven't already. Currently, nginx packages are available on the latest versions of CentOS, Debian and Ubuntu. So you can pick up whichever of these you prefer.

After you have set up the server the way you like, install the latest stable nginx. Use one of the following methods.

# Debian and Ubuntu
sudo apt-get update
# Then install the Nginx Open Source edition
sudo apt-get install nginx

Test that the server replies to HTTP requests. Open the load balancer server’s public IP address in your web browser. When you see the default welcoming page for nginx, this means that the installation is successful.

Configuring nginx for load balancing

Once you've got NGINX installed and given it a little test run, it's time to make it your load balancer.

1. Configure NGINX: Think of this like giving your traffic cop (NGINX) a set of instructions. Tell NGINX what types of connections to keep an ear out for and where to send them. It's like planning the routes on a map.

2. Create a Configuration File: Open up your preferred text editor and, create a new configuration file. This file will be your guidebook for NGINX, filled with all the details on how to handle the traffic.

sudo nano /etc/nginx/conf.d/load-balancer.conf

In the load-balancer.conf you’ll need to define the following two segments, upstream and server, see the examples below.

# Define which servers to include in the load balancing scheme. 
# It's best to use the servers' private IPs for better performance and security.
# You can find the private IPs at your UpCloud control panel Network section.
http {
   upstream backend {
      server 10.1.0.101; 
      server 10.1.0.102;
      server 10.1.0.103;
   }

   # This server accepts all traffic to port 80 and passes it to the upstream. 
   # Notice that the upstream name and the proxy_pass need to match.

   server {
      listen 80; 

      location / {
          proxy_pass http://backend;
      }
   }
}

Final Touches on Your Configuration: Save and Declutter!

1. Save Your Changes: Once you've worked your magic in the configuration file, hit that save button, and gracefully exit the editor. Your instructions are now safely stored.

2. Disable the Default Setup: Remember the default configuration you checked earlier? It's time to give it a little break. The process varies depending on your operating system.

   • For Debian and Ubuntu Users: Think of this step like tidying up your server space. Navigate to the sites-enabled folder and remove the default configuration by removing its symbolic link.

       sudo rm /etc/nginx/sites-enabled/default
     

Then use the following to restart nginx.

sudo systemctl restart nginx

Check that nginx starts successfully. If the restart fails, take a look at the /etc/nginx/conf.d/load-balancer.conf you just created to make sure there are no mistypes or missing semicolons.

When you enter the load balancer’s public IP address in your web browser, you should pass to one of your back-end servers.

Navigating Load Balancing Methods

NGINX Open Source supports four load‑balancing methods, and NGINX Plus adds two more methods.

• least_conn: A request is sent to the server with the least number of active connections, again with server weights taken into consideration.

upstream backend {
    least_conn;
    server 10.1.0.101;
    server 10.1.0.102 weight=5;
}

• ip_hash: The server to which a request is sent is determined from the client IP address. In this case, either the first three octets of the IPv4 address or the whole IPv6 address are used to calculate the hash value. The method guarantees that requests from the same address get to the same server unless it is not available. To use this method, add the ip_hash -parameter to your upstream segment like in the example underneath.

upstream backend {
   ip_hash;
   server 10.1.0.101; 
   server 10.1.0.102;
   server 10.1.0.103;
}

If one of the servers needs to be temporarily removed from the load‑balancing rotation, it can be marked with the down parameter to preserve the current hashing of client IP addresses. Requests that were to be processed by this server are automatically sent to the next server in the group.

• hash: The server to which a request is sent is determined from a user‑defined key which can be a text string, variable, or a combination. For example, the key may be a paired source IP address and port, or a URI as in this example:

upstream backend {
    hash $request_uri consistent;
    server backend1.example.com;
    server backend2.example.com;
}

Add the optional consistent parameter, and now you've got consistent-hash load balancing. What does that mean? Your requests are evenly spread across all servers, based on your custom hashed key value. Even if you add or remove a server, only a few keys get shuffled around. This keeps things smooth, especially for cache servers or apps that collect state info.

• least_time: For each request, NGINX Plus selects the server with the lowest average latency and the lowest number of active connections, where the lowest average latency is calculated based on which of the following parameters to the least_time directive is included:

upstream backend {
    least_time header;
    server backend1.example.com;
    server backend2.example.com;
}

• header – Time to receive the first byte from the server

• last_byte – Time to receive the full response from the server

• last_byte inflight – Time to receive the full response from the server, taking into account incomplete requests

Conclusion

If you wish to improve your web application performance , a load balancer is definitely something to consider. Nginx is powerful yet relatively simple to set up to load balance a web server and there is much more to the topic that you check out on the official documentation of NGINX.

Thanks for reading!

You can connect with me on LinkedIn or Twitter, if you're interested.