Load Balancing Algorithms

Load balancers distribute incoming network traffic across multiple servers to ensure high availability, reliability, and performance. Here are the six most common load balancing algorithms.

1. Round Robin

Description: Distributes requests sequentially across all available servers in a circular order.

How it works:

• Request 1 → Service A
• Request 2 → Service B
• Request 3 → Service C
• Request 4 → Service A (cycles back)

Pros:

• Simple to implement
• Even distribution when servers have equal capacity
• No server state tracking needed

Cons:

• Doesn’t account for server load or capacity differences
• May overload slower servers

Best for: Homogeneous server environments with similar workloads.

public class RoundRobinLoadBalancer
{
    private readonly List<string> _servers;
    private int _currentIndex = -1;

    public RoundRobinLoadBalancer(List<string> servers)
    {
        _servers = servers;
    }

    public string GetNextServer()
    {
        _currentIndex = (_currentIndex + 1) % _servers.Count;
        return _servers[_currentIndex];
    }
}

2. Sticky Round Robin (Session Affinity)

Description: Routes requests from the same client to the same server for session persistence.

How it works:

• User A always routes to Service A
• User B always routes to Service B
• Maintains session state on the same server

Pros:

• Session data stays on one server
• No need for distributed session storage
• Simpler application architecture

Cons:

• Uneven load if some users are more active
• Server failure affects all sticky users
• Horizontal scaling complications

Best for: Stateful applications, shopping carts, user sessions.

public class StickyRoundRobinLoadBalancer
{
    private readonly List<string> _servers;
    private readonly Dictionary<string, string> _clientServerMap = new();
    private int _currentIndex = -1;

    public string GetServerForClient(string clientId)
    {
        if (_clientServerMap.TryGetValue(clientId, out var server))
            return server;

        _currentIndex = (_currentIndex + 1) % _servers.Count;
        server = _servers[_currentIndex];
        _clientServerMap[clientId] = server;
        return server;
    }
}

3. Weighted Round Robin

Description: Assigns weights to servers based on capacity, routing more traffic to higher-capacity servers.

How it works:

• Service A (weight 3) gets 3x more requests
• Service B (weight 2) gets 2x requests
• Service C (weight 1) gets 1x requests

Pros:

• Accounts for different server capacities
• Better resource utilization
• Flexible configuration

Cons:

• Requires manual weight configuration
• Weights may become outdated

Best for: Heterogeneous environments with different server specs.

public class WeightedRoundRobinLoadBalancer
{
    private readonly List<(string Server, int Weight)> _servers;
    private int _currentIndex = 0;
    private int _currentWeight = 0;

    public WeightedRoundRobinLoadBalancer(List<(string, int)> servers)
    {
        _servers = servers;
    }

    public string GetNextServer()
    {
        while (true)
        {
            _currentIndex = (_currentIndex + 1) % _servers.Count;
            if (_currentIndex == 0)
            {
                _currentWeight--;
                if (_currentWeight <= 0)
                    _currentWeight = _servers.Max(s => s.Weight);
            }

            if (_servers[_currentIndex].Weight >= _currentWeight)
                return _servers[_currentIndex].Server;
        }
    }
}

4. IP/URL Hash

Description: Uses a hash of the client IP or request URL to determine the target server.

How it works:

• Hash(Client IP) % ServerCount = Server Index
• Same IP always goes to same server
• URL hash routes similar requests together

Pros:

• Predictable routing
• Good for caching (same content on same server)
• No state tracking needed

Cons:

• Uneven distribution with limited IPs
• Server changes affect all clients

Best for: CDN caching, API request routing.

public class IpHashLoadBalancer
{
    private readonly List<string> _servers;

    public IpHashLoadBalancer(List<string> servers)
    {
        _servers = servers;
    }

    public string GetServerForIp(string clientIp)
    {
        int hash = clientIp.GetHashCode();
        int index = Math.Abs(hash) % _servers.Count;
        return _servers[index];
    }

    public string GetServerForUrl(string url)
    {
        int hash = url.GetHashCode();
        int index = Math.Abs(hash) % _servers.Count;
        return _servers[index];
    }
}

5. Least Connections

Description: Routes requests to the server with the fewest active connections.

How it works:

• Track active connections per server
• New request → server with lowest count
• Dynamically adapts to load

Pros:

• Adapts to varying request complexity
• Better for long-lived connections
• Handles slow requests better

Cons:

• Requires connection state tracking
• More complex implementation
• Slight overhead for tracking

Best for: WebSocket connections, database connections, varying request durations.

public class LeastConnectionsLoadBalancer
{
    private readonly Dictionary<string, int> _serverConnections;

    public LeastConnectionsLoadBalancer(List<string> servers)
    {
        _serverConnections = servers.ToDictionary(s => s, _ => 0);
    }

    public string GetNextServer()
    {
        var server = _serverConnections
            .OrderBy(kvp => kvp.Value)
            .First().Key;

        _serverConnections[server]++;
        return server;
    }

    public void ReleaseConnection(string server)
    {
        if (_serverConnections.ContainsKey(server))
            _serverConnections[server]--;
    }
}

6. Least Time (Least Response Time)

Description: Routes to the server with the fastest response time and fewest active connections.

How it works:

• Monitors response times per server
• Combines latency + connection count
• Routes to optimal server

Pros:

• Best user experience
• Accounts for server performance
• Adapts to network conditions

Cons:

• Most complex to implement
• Requires continuous monitoring
• Health check overhead

Best for: Performance-critical applications, global deployments.

public class LeastTimeLoadBalancer
{
    private readonly Dictionary<string, ServerMetrics> _metrics;

    public string GetNextServer()
    {
        return _metrics
            .OrderBy(m => m.Value.AverageResponseTime + (m.Value.ActiveConnections * 10))
            .First().Key;
    }

    public void RecordResponse(string server, TimeSpan responseTime)
    {
        _metrics[server].RecordResponse(responseTime);
    }

    private class ServerMetrics
    {
        private readonly Queue<double> _responseTimes = new();
        public int ActiveConnections { get; set; }
        public double AverageResponseTime =>
            _responseTimes.Count > 0 ? _responseTimes.Average() : 0;

        public void RecordResponse(TimeSpan time)
        {
            _responseTimes.Enqueue(time.TotalMilliseconds);
            if (_responseTimes.Count > 100)
                _responseTimes.Dequeue();
        }
    }
}

Algorithm Comparison

Implementation in NGINX

# Round Robin (default)
upstream backend {
    server backend1.example.com;
    server backend2.example.com;
}

# Weighted
upstream backend {
    server backend1.example.com weight=3;
    server backend2.example.com weight=1;
}

# IP Hash
upstream backend {
    ip_hash;
    server backend1.example.com;
    server backend2.example.com;
}

# Least Connections
upstream backend {
    least_conn;
    server backend1.example.com;
    server backend2.example.com;
}

Sources

• C#/Teorie/load balancing algorithms.jpeg
• Credit: Rocky Bhatia (@learnwithrockybhatia)