"No design is correct — only appropriate for the requirements and scale you have right now. System design isn't memorizing lots of technologies; it's knowing what to trade for what."

This is an intro to System Design — a topic that feels vague but really revolves around a handful of building blocks that repeat over and over. This article walks through seven of them: scaling, load balancing, caching, CDNs, databases (SQL/NoSQL, replication, sharding), the CAP theorem, and message queues. Understand these and you can assemble most real-world systems.

Every design decision is a trade-off between opposing quantities: latency vs throughput, consistency vs availability, cost vs complexity. Add a cache and you're faster but must handle stale data; shard a database and you scale, but cross-shard queries get hard.

So the number-one principle: start from the requirements, not from the technology. Don't jump to "use Kafka, use Redis" — first ask what the system actually needs.

Before drawing a single box, separate two kinds of requirements:

• Functional: what the system does — post an article, send a message, shorten a URL...
• Non-functional: how well it runs — how many users, target latency, availability (99.9%?), read/write ratio.

Then do a back-of-envelope estimate so you know what scale you're designing for:

# Estimate for a social network with 100 million users
DAU         = 100_000_000 * 0.20        # 20% active daily = 20 million
req_per_day = DAU * 20                  # ~20 requests/user/day = 400 million
QPS         = req_per_day / 86_400      # ~4,600 requests/second (average)
peak_QPS    = QPS * 3                   # peak hours ~3x = ~14,000 QPS

Part A · Scale & Speed

When a system is overloaded, there are two broad directions:

• Vertical (scale up): add more CPU, RAM, disk to the same machine. Simple, no code changes — but there's a hard ceiling and that machine is a single point of failure.
• Horizontal (scale out): add more machines running in parallel. Scales almost infinitely and tolerates failure — but needs load balancing and services must be stateless.

A load balancer sits in front of a server pool and distributes requests across them so no single server is overwhelmed. It also runs health checks: a dead server is pulled out of rotation automatically → it removes the single point of failure.

Common distribution algorithms: round-robin (in turn), least connections (favor the least-busy server), IP hash (a given user always lands on the same server). People distinguish L4 (balancing on TCP/IP, fast) from L7 (on HTTP content, smarter — route by path, cookie, etc.).

A cache stores frequently used data in fast memory (usually RAM — Redis, Memcached) to avoid re-reading a slow database or recomputing. It's the cheapest, most effective way to cut latency and offload the database.

The most common pattern is cache-aside (lazy loading):

def get_user(user_id):
    data = cache.get(user_id)          # 1. try the cache first
    if data is not None:
        return data                    # cache hit → return immediately
    data = db.query(user_id)           # 2. miss → read the DB
    cache.set(user_id, data, ttl=300)  # 3. write back to cache (expires in 5 min)
    return data

When the cache fills up, you must evict with a policy: LRU (drop least-recently-used), LFU (drop least-frequently-used), or TTL (expire by time). You can cache at many layers: browser, CDN, application tier, and inside the DB.

A CDN (Content Delivery Network) is a network of servers spread around the world that hold copies of static content (images, JS, CSS, video) at the "edge" near users. Someone in Vietnam downloads an image from a server in Vietnam instead of crossing half the planet to the US → latency drops sharply while the origin server is offloaded.

Two styles: pull (the CDN fetches content from the origin the first time it's requested, then caches it) and push (you proactively push content to the CDN). Most sites use pull for convenience.

Part B · Data

SQL (relational): data lives in tables with a fixed schema, supports ACID transactions, joins, and strong consistency. Great for structured data that must be exactly right (finance, orders). E.g. PostgreSQL, MySQL.

NoSQL: flexible schema, easy horizontal scaling, usually only eventual consistency (BASE) in return. Several kinds: key-value (Redis), document (MongoDB), wide-column (Cassandra), graph (Neo4j). A fit for large, loosely structured, write-heavy data.

Replication keeps multiple copies of data across machines. Two wins: if one machine dies another survives (availability), and you can spread read load across the copies.

The common model is leader–follower (primary–replica): every write goes to the leader and propagates to followers; reads can come from any follower. Because propagation takes time, a follower may trail the leader slightly — called replication lag — leading to reads that see slightly stale data (eventual consistency).

When data (or write volume) exceeds one machine, you shard: split the data into pieces placed on multiple machines by a shard key. For example, split users by hash(user_id) % N, or by a range of name letters.

It's the only way to scale writes and storage beyond a single machine — but it comes at a cost: cross-shard queries get hard, an uneven key creates hotspots, and resharding is painful. Consistent hashing helps add/remove machines while moving as little data as possible.

The CAP theorem says: under a network Partition — groups of machines temporarily lose contact — a distributed system can keep only one of two: Consistency or Availability, not both.

• CP (choose consistency): during a partition, refuse to serve where it can't be sure it's correct — better an error than wrong data. Fits banking, payments.
• AP (choose availability): keep serving, accepting data that may be temporarily stale/divergent and converges later. Fits social feeds, like counters.

Part C · Communication

A message queue — Kafka, RabbitMQ, SQS — sits between a producer and a consumer. Instead of doing heavy work inside the request (sending email, processing an image, generating a report), the app just pushes a message onto the queue and responds to the user immediately; background workers pick it up and process it later.

Four big benefits: decoupling (producer and consumer don't need to know each other), load buffering (traffic spikes pile into the queue instead of crashing the DB), retries (on failure, leave it in the queue to try again), and scaling (add workers when the queue grows).

Common Sections

Put the pieces together and a large-scale web system usually looks like this — follow a request from the browser all the way to the DB:

   User
     │
     ▼
   [ DNS ] ──▶ [ CDN ]  ◀── images, JS, CSS, video (static)
     │
     ▼  (dynamic content)
   [ Load Balancer ]   health checks · round-robin
     │
   ┌─┼───────────┐
   ▼ ▼           ▼
 [App1][App2]...[AppN]   STATELESS servers, scale out
   └─┬───────────┘
     │  hot reads          heavy work, run in background
     ▼                     │
   [ Cache: Redis ]        ▼
     │              [ Message Queue ] ──▶ [ Workers ]
     ▼
   [ Database ]  Primary (writes) + Replicas (reads), sharded

The flow: user → DNS points to the nearest edge → static content served straight from the CDN; dynamic content goes through the load balancer to a stateless server; the server tries the cache first, and on a miss reads the DB (replicas for reads, primary for writes); heavy work is pushed onto the queue for workers to process in the background.

💀 Pitfall #1 — Premature optimization / over-engineering. Microservices, Kafka, multi-region before you even have users. Start simple, scale when genuinely needed.

💀 Pitfall #2 — Single point of failure (SPOF). One load balancer, one DB, one cache — if it dies the whole system goes down. Always have redundancy for critical components.

💀 Pitfall #3 — Forgetting the data layer. The DB is usually the first bottleneck. Adding web servers is easy; scaling the DB (replicas, shards) is the hard part.

💀 Pitfall #4 — Stateful servers. Storing sessions on the box breaks horizontal scaling (users must always return to the same server). Push state out (Redis/DB).

💀 Pitfall #5 — No caching strategy. Careless caching gives unpredictable stale data; or a cache stampede crashes the DB when keys expire together.

💀 Pitfall #6 — Ignoring monitoring. No logs, no metrics, no alerts → you don't know whether the system is healthy or sick until users start shouting.

💀 Pitfall #7 — Designing for the wrong scale. Skip estimation and you either waste resources or fall apart under load. Always do the back-of-envelope first.

A few latency numbers worth knowing (orders of magnitude):

   RAM read              ~100 nanoseconds
   SSD random read       ~100 microseconds   (~1,000x slower than RAM)
   Round-trip in a DC    ~0.5 milliseconds
   HDD disk read         ~10 milliseconds    (~100,000x slower than RAM)
   Cross-continent network ~150 milliseconds

Four questions for any System Design problem: (1) What scale (users, QPS, storage)? (2) Read-heavy or write-heavy? (3) Strong consistency needed, or is eventual OK? (4) Where's the bottleneck and how do you scale it?

1. Start from requirements & numbers, not technology. Estimate QPS/storage before drawing boxes.
2. Scale up until you can't, then scale out with a load balancer + stateless servers.
3. Reads outnumber writes → cache + read replicas. The biggest, cheapest performance lever.
4. Beyond one machine → shard; need durability → replicate. And remember CAP: under a partition, pick C or A.
5. If it needn't be synchronous → push it to a queue. Fast requests, smooth handling of load spikes.

System design isn't magic — it's a few foundational blocks assembled deliberately, always paired with the question "what am I trading away?" Master the seven blocks in this article, ask the four framing questions each time you design, and you can assemble most real-world systems. 🏗️