When scaling applications from hundreds of users to millions of concurrent requests, monolithic single-server architectures fail. System design is the art of organizing software components, databases, load balancers, and caches to achieve high availability, fault tolerance, and low latency.
1. Vertical vs Horizontal Scaling
Scalability measures an application's ability to handle growing workload seamlessly.
Vertical Scaling (Scale Up)
Adding more CPU cores, RAM, or SSD storage to an existing single server instance.
- Pros: Simple setup, no code changes required.
- Cons: Hard hardware limits, single point of failure (SPOF), expensive server costs.
Horizontal Scaling (Scale Out)
Adding more server instances behind a load balancer to distribute request traffic.
- Pros: Unlimited scalability, high fault tolerance, cost-effective commodity servers.
- Cons: Requires stateless app design & complex load balancing.
2. Load Balancers & Traffic Algorithms
A Load Balancer sits between client browsers and backend app servers, distributing incoming traffic across healthy instances.
| Algorithm | How It Works | Best Use Case |
|---|---|---|
| Round Robin | Distributes requests sequentially to each server in order. | Homogeneous server clusters with equal capacity. |
| Least Connections | Routes traffic to the server with the fewest active connections. | Long-lived HTTP requests or WebSocket connections. |
| IP Hash | Hashes the client's IP address to map them to a specific server. | Sticky sessions requiring stateful user persistence. |
3. Caching Strategies (Redis / Memcached)
Caching stores frequently requested data in high-speed RAM to eliminate redundant database queries.
async function getUserProfile(userId) {
// 1. Check Redis cache first
const cachedUser = await redis.get(`user:${userId}`);
if (cachedUser) return JSON.parse(cachedUser);
// 2. Cache miss: fetch from SQL database
const user = await db.query('SELECT * FROM users WHERE id = ?', [userId]);
// 3. Store result in Redis with 1-hour expiration (TTL)
await redis.setex(`user:${userId}`, 3600, JSON.stringify(user));
return user;
}
4. CAP Theorem in Distributed Databases
The CAP Theorem states that a distributed data system can guarantee at most two of the following three properties simultaneously:
- Consistency (C): Every read receives the most recent write or an error.
- Availability (A): Every non-failing node returns a non-error response without guaranteeing latest write.
- Partition Tolerance (P): The system continues operating despite network message drops between nodes.
Prepare for System Design Interviews with Telugu IT Tutorials
Learn full-stack system architecture, microservices, and database scaling in our live interactive training courses. View Courses →