What Is System Design

A different kind of problem

Coding problems have a correct answer. System design problems do not. You are asked to build something like "a URL shortener" or "a news feed", and the interviewer is watching how you reason, not whether you arrive at a specific diagram.

Every decision you make buys something and costs something. Adding a cache makes reads faster but introduces staleness. Adding replicas improves availability but complicates consistency. The skill is naming the tradeoff out loud and justifying your choice against the requirements.

The vocabulary

Four terms come up in every discussion.

Scalability — the system keeps working as load grows. Vertical scaling means a bigger machine, which is simple but has a hard ceiling and a single point of failure. Horizontal scaling means more machines, which scales further but requires your system to handle distributed state.

Availability — the fraction of time the system responds. Usually quoted in nines:

Each extra nine costs substantially more than the last.

Latency and throughput — latency is how long one request takes; throughput is how many requests you handle per second. They are related but distinct, and you can improve one while hurting the other. Batching raises throughput and raises latency.

Consistency — whether all users see the same data at the same time. Strong consistency guarantees they do. Eventual consistency allows a brief window where they do not, in exchange for better availability and performance.

The CAP theorem

In a distributed system you can guarantee at most two of:

• Consistency — every read sees the most recent write
• Availability — every request receives a response
• Partition tolerance — the system works despite network failures between nodes

Network partitions are a fact of life, not a choice, so P is mandatory. The real decision is what to do during a partition: refuse requests to stay consistent (CP), or keep serving possibly-stale data to stay available (AP).

Banking systems lean CP — showing the wrong balance is unacceptable. Social feeds lean AP — a slightly stale like count harms nobody.

CAP only describes failure. PACELC describes normal operation too.

CAP's weakness is that it only tells you what happens during a network partition — a relatively rare event. Most of the time, the network is fine, and yet systems still trade off consistency against latency every single day. PACELC extends the idea to cover this:

If Partitioned, choose between Availability and Consistency (this is just CAP). Else (normal operation), choose between Latency and Consistency.

This second half is the part CAP leaves out. Even with no partition at all, a system that wants strong consistency must wait for writes to reach every replica (or a quorum of them) before confirming — which costs latency. A system willing to accept eventual consistency can respond the instant the nearest replica has the data.

This is why "is it AP or CP" is an incomplete question for most real systems — PACELC is the fuller version interviewers who know the theorem well will actually expect, because it explains behavior on the 99.99% of days nothing is partitioned at all.

Consistency is a spectrum, not a switch

"Strong" and "eventual" are the two ends people quote, but production systems often pick a specific point between them:

• Strong consistency — every read reflects the latest completed write, no matter which replica answers. Expensive, but simplest to reason about.
• Eventual consistency — replicas converge to the same value eventually, with no guarantee about how long "eventually" takes. Cheapest, hardest to reason about.
• Read-your-writes consistency — a specific user always sees their own writes immediately, even if other users might briefly see stale data. Solves the most common complaint about eventual consistency (a user updating their profile and being confused when it doesn't look updated) without paying for full strong consistency.
• Causal consistency — writes that are causally related (a comment that replies to a post) are seen by everyone in the same order, but unrelated writes can be seen in different orders on different replicas.

Naming the specific consistency level you need — not just "strong" or "eventual" — is what separates a design that sounds thoughtful from one that sounds memorized.

Numbers worth memorising

Rough orders of magnitude let you sanity-check a design on the spot.

The headline: memory is roughly 100,000 times faster than disk, and network calls across regions dominate everything. This is the entire justification for caching and for keeping data near users.

Useful capacity estimates:

• 1 million requests/day is roughly 12 requests/second
• 1 billion requests/day is roughly 12,000 requests/second
• A commodity server handles perhaps 1,000–10,000 simple requests/second

How to answer a design question

Follow a consistent structure so you never freeze.

1. Clarify requirements (5 minutes). Never start drawing immediately. Ask what the system must do, who uses it, and what scale is expected. Separate functional requirements (shorten a URL, redirect) from non-functional ones (low latency, high availability).

2. Estimate scale. Convert the user count into requests per second and storage per year. This decides whether you need one database or a sharded cluster, and it justifies everything that follows.

3. Define the API. A few endpoint signatures pin down what the system actually does.

4. Sketch the high-level design. Client, load balancer, application servers, cache, database. Get the boxes and arrows down before adding detail.

5. Design the data model. Choose SQL or NoSQL and say why. Identify the access patterns.

6. Go deep on one or two areas. The interviewer will steer you. Common depth areas are sharding strategy, cache invalidation, and handling hot keys.

7. Identify bottlenecks. Name the weakest point and how you would address it. Volunteering this is a strong signal.

Worked estimate

Say you are designing a URL shortener with 100 million new URLs per month.

• Writes: 100M ÷ (30 × 86400) ≈ 40 writes/second
• Reads: assume a 100:1 read-to-write ratio → 4,000 reads/second
• Storage: 500 bytes per record × 100M × 12 months ≈ 600 GB/year

Those three numbers immediately tell you that reads dominate, so caching is the highest-value optimisation, and that storage is modest enough for a single well-provisioned database with replicas for years.

What interviewers reward

• Asking clarifying questions before designing
• Stating assumptions explicitly
• Naming tradeoffs rather than presenting one option as obviously correct
• Starting simple and adding complexity only when a requirement demands it
• Admitting uncertainty instead of bluffing

What to avoid

• Jumping to microservices, Kafka and Kubernetes before knowing the scale
• Designing for a billion users when the requirement is a thousand
• Silence — think out loud, since the reasoning is what is being assessed
• Treating any single technology as the answer to everything

Summary

System design rewards structured reasoning about tradeoffs. Clarify requirements, estimate scale with rough numbers, sketch a simple design, then deepen where asked. There is no correct answer — only choices you can defend against the requirements you were given.