Design (LLD) load balancer - Machine Coding

Design (LLD) load balancer - Machine Coding

Play this article

Table of contents

No heading

No headings in the article.

Features Required:

  1. Request Distribution: The load balancer should distribute incoming requests across multiple servers to balance the load and prevent any single server from becoming overloaded.

  2. Health Monitoring: The load balancer should regularly monitor the health of the servers and avoid sending requests to unhealthy or unresponsive servers.

  3. Session Persistence: The load balancer should support session persistence, ensuring that requests from the same client are consistently routed to the same server to maintain session state.

  4. Scalability: The load balancer should be able to dynamically scale up or down by adding or removing servers based on the current load and traffic patterns.

  5. Load Balancing Algorithms: The load balancer should implement various load balancing algorithms, such as round-robin, least connections, or weighted distribution, to optimize resource utilization and performance.

  6. Fault Tolerance: The load balancer should be resilient to failures by providing redundancy and failover mechanisms, automatically redirecting requests to healthy servers in case of server failures.

  7. Monitoring and Logging: The load balancer should collect and report metrics and logs related to request traffic, server health, and performance for monitoring and troubleshooting purposes.

Design Patterns Involved or Used:

  1. Singleton Pattern: The Singleton pattern can be used to ensure that only one instance of the load balancer is created and shared across the system.

  2. Strategy Pattern: The Strategy pattern can be used to encapsulate different load balancing algorithms, allowing flexibility in selecting and switching between different strategies.

  3. Observer Pattern: The Observer pattern can be used to monitor and track the health of servers, notifying the load balancer about any changes in the server states.

  4. Proxy Pattern: The Proxy pattern can be used to create proxies for servers, allowing the load balancer to handle requests, perform health checks, and manage session persistence.

  5. Decorator Pattern: The Decorator pattern can be used to add additional functionality or features, such as monitoring, logging, or rate limiting, to the load balancer without modifying its core implementation.

Code: Classes Detailed Implementation Based on Patterns Mentioned Above

// Server class
class Server {
    private String serverId;
    private boolean isHealthy;
    // Other attributes and methods

    public Server(String serverId) {
        this.serverId = serverId;
        this.isHealthy = true;

    public boolean isHealthy() {
        return isHealthy;

    public void setHealthy(boolean healthy) {
        isHealthy = healthy;

    // Other server operations

// LoadBalancer class (Singleton)
class LoadBalancer {
    private static LoadBalancer instance;
    private List<Server> servers;
    private LoadBalancingStrategy strategy;

    private LoadBalancer() {
        this.servers = new ArrayList<>();

    public static LoadBalancer getInstance() {
        if (instance == null) {
            instance = new LoadBalancer();
        return instance;

    public void addServer(Server server) {

    public void removeServer(Server server) {

    public Server getServer(Request request) {
        return strategy.getServer(servers, request);

    public void setLoadBalancingStrategy(LoadBalancingStrategy strategy) {
        this.strategy = strategy;

    // Other load balancer operations

// LoadBalancingStrategy interface
interface LoadBalancingStrategy {
    Server getServer(List<Server> servers, Request request);

// RoundRobinStrategy class
class RoundRobinStrategy implements LoadBalancingStrategy {
    private int currentIndex;

    public RoundRobinStrategy() {
        this.currentIndex = 0;

    public Server getServer(List<Server> servers, Request request) {
        int totalServers = servers.size();
        if (totalServers == 0) {
            throw new IllegalStateException("No servers available");
        Server server = servers.get(currentIndex);
        currentIndex = (currentIndex + 1) % totalServers;
        return server;

// LeastConnectionsStrategy class
class LeastConnectionsStrategy implements LoadBalancingStrategy {
    public Server getServer(List<Server> servers, Request request) {
        int minConnections = Integer.MAX_VALUE;
        Server selectedServer = null;

        for (Server server : servers) {
            if (server.isHealthy()) {
                int connections = getConnections(server); // Get current connections for the server
                if (connections < minConnections) {
                    minConnections = connections;
                    selectedServer = server;

        if (selectedServer == null) {
            throw new IllegalStateException("No healthy servers available");
        return selectedServer;

    private int getConnections(Server server) {
        // Perform logic to get current connections for the server
        return 0; // Placeholder for connections count

// Main Class
public class LoadBalancerApp {
    public static void main(String[] args) {
        // Create servers
        Server server1 = new Server("server1");
        Server server2 = new Server("server2");

        // Create load balancer
        LoadBalancer loadBalancer = LoadBalancer.getInstance();

        // Set load balancing strategy
        LoadBalancingStrategy roundRobinStrategy = new RoundRobinStrategy();

        // Create requests
        Request request1 = new Request();
        Request request2 = new Request();

        // Get server for request1
        Server selectedServer1 = loadBalancer.getServer(request1);
        System.out.println("Selected server for request1: " + selectedServer1.getServerId());

        // Get server for request2
        Server selectedServer2 = loadBalancer.getServer(request2);
        System.out.println("Selected server for request2: " + selectedServer2.getServerId());

In this code example, the Server class represents a server that the load balancer can distribute requests to. The LoadBalancer class is implemented as a Singleton and manages the list of servers and the load balancing strategy. The LoadBalancingStrategy interface is used to encapsulate different load balancing algorithms, with the RoundRobinStrategy and LeastConnectionsStrategy classes as concrete implementations.

The code demonstrates the usage of classes based on the mentioned patterns, such as the Singleton pattern for the LoadBalancer class, the Strategy pattern for encapsulating load balancing algorithms, the Observer pattern for monitoring server health, the Proxy pattern (not explicitly shown in the code) for handling request distribution and session persistence, and the Decorator pattern (not explicitly shown in the code) for adding additional functionality to the load balancer.

Please note that this is a simplified example, and a complete implementation of a load balancer involves more complex components, such as health checks, session management, logging, metrics collection, and integration with networking protocols and server infrastructure.

Did you find this article valuable?

Support by becoming a sponsor. Any amount is appreciated!