System Design 101: From One Server to Millions of Users

System design is the process of deciding how the major pieces of an application work together.

You are not only choosing technologies. You are deciding:

• where data lives,
• how requests move through the system,
• what happens when something fails,
• how the application handles more traffic,
• which parts require strong consistency.

Start Simple

A new application can often begin with one backend and one relational database.

This is not bad architecture.

For an early product, simple architecture is usually easier to build, debug and operate.

Problems start when traffic increases.

The application server may become CPU-bound, the database may receive too many repeated queries, and the single machine can become a single point of failure.

A Typical Evolution

1. Add a load balancer.
2. Run multiple application instances.
3. Keep application servers stateless.
4. Add database indexes.
5. Cache frequently requested data.
6. Move slow work into background jobs.
7. Add database replicas if reads become heavy.
8. Add metrics, logs and tracing.

Start With Requirements

A payment ledger and a social-media feed have very different consistency requirements.

The Main Principle

Good system design is not about creating the largest diagram.

It is about building the simplest system that reliably satisfies the current requirements while leaving a reasonable path to scale later.