30 key systems design concepts

Introduction: System design can feel overwhelming — especially when vast topics like scalability, performance, reliability, and distributed architecture are thrown at you all at once. But once you master the foundational building blocks, designing robust and scalable systems transforms from stressful to doable. In this post, we’ll explore 30 essential system design concepts with explanation

1. Client–Server Architecture

What it is: The foundational model: thin clients (browsers, mobiles, IoT) make requests; servers process logic and return responses. Servers can be single or a pool of machines.
Why it matters: It separates concerns — UIs live in clients and heavy computation/state in servers — enabling centralized control, security, and shared business logic.
Trade-offs & details: Clients must handle availability issues and degraded connectivity. This model fits web apps, but for extreme scale you need caching, CDNs, and stateless servers (see Diagram 1). Session state should be externalized (cache or DB) to allow horizontal scaling.

2. IP Address

What it is: The numeric address that uniquely identifies a host on the internet (IPv4/IPv6).
Why it matters: DNS maps human domains to these addresses; networking routes traffic using IPs. Servers often live behind ephemeral IPs in cloud — load balancers and DNS abstracts that away.
Practical note: Use private/internal IPs for intra-VPC communication and public IPs only at the edge. Use IPv6 if you expect many clients and want simplified addressing.

3. DNS (Domain Name System)

What it is: Distributed system that maps domain names (example.com) to IP addresses.
Why it matters: DNS is the user-facing gatekeeper — it determines which IP (or set of IPs) clients connect to. TTL and record types (A, CNAME, ALIAS) control caching and failover behavior.
Operational tips: Use low TTLs for failover windows but beware extra DNS queries. Combine DNS-based routing with a CDN to direct users to nearest edge nodes (Diagram 1).

4. Proxy / Reverse Proxy

What it is: A proxy forwards client requests; a reverse proxy sits in front of servers and forwards requests to backend apps. Common examples: Nginx, HAProxy, Cloud Load Balancers.
Why it matters: Reverse proxies provide TLS termination, request routing, caching, compression, WAF and can offload expensive work from app servers. They also act as a single point for observability.
Caveats: They can become bottlenecks — scale them, use multiple layers (edge CDN + reverse proxy + internal LB). Diagram 1 places the reverse proxy / load balancer between clients/CDN and application servers.

5. Latency

What it is: Time between sending a request and receiving a response (often measured as RTT or end-to-end latency).
Why it matters: Latency affects user experience more than raw throughput. Small decreases (100s of ms) can significantly increase engagement or conversion.
Mitigation: Use CDNs, edge caching, reduce number of network hops, optimize database queries, enable connection reuse (HTTP keepalive), reduce TLS roundtrips (session resumption). Diagram 1 highlights CDN and edge caching as latency reducers.

6. HTTP / HTTPS

What it is: HTTP is the application protocol for request/response; HTTPS is HTTP over TLS (secure).
Details: HTTPS adds encryption and an initial handshake that increases latency slightly; use HTTP/2 or HTTP/3 to reduce overhead via multiplexing and header compression. Use proper cache-control headers for CDN caching.
Security: Always HTTPS for production; terminate TLS at CDN or load balancer to centralize cert management.

7. APIs (Application Programming Interfaces)

What it is: Contracted endpoints that expose functionality/data (REST, GraphQL, gRPC, etc.).
Design principles: Clear versioning, backwards compatibility, and documentation are crucial. Good APIs encapsulate the server internals so clients don't rely on implementation details.
Practical: Design APIs with idempotency and meaningful error codes; add observability (metrics, tracing).

8. REST API

What it is: Resource-driven APIs using HTTP verbs (GET/POST/PUT/DELETE) and stateless operations.
Why it works: Simplicity and cache-friendliness (GET becomes easily cacheable).
Limitations: Multiple round-trips for complex data shapes; overfetching/underfetching problems — GraphQL can solve those but introduces other trade-offs.

9. GraphQL

What it is: Flexible query language allowing clients to request exactly the shape they need from a single endpoint.
Benefits: Reduces over/under-fetch; single request for nested data.
Gotchas: Harder to cache (responses vary by query), can cause N+1 backend fetches unless you batch or use dataloader patterns. Good for client-driven UIs and mobile clients with limited bandwidth.

10. Databases (general)

What it is: Persistent storage for structured or unstructured data. Types include relational (RDBMS), wide-column, document, key-value, and columnar for analytics.
Design choice: Choose based on query patterns: transactional writes/joins → RDBMS; large-scale flexible schema → NoSQL; analytics → columnar stores. Diagram 2 shows DB roles (primary, replicas, shards).

11. SQL vs NoSQL

SQL: ACID transactions, strong schema, complex joins and relational integrity. Great for OLTP and financial systems.
NoSQL: Eventual consistency (often), flexible schema, simpler horizontal scaling. Good for high-velocity reads/writes, flexible documents, and caching heavy workloads.
Decision: Prefer SQL for transactional consistency; NoSQL when you need scale and schema flexibility — hybrid approaches are common.

12. Vertical Scaling (Scale Up)

What it is: Upgrade a single machine’s resources (CPU, RAM, disk).
Pros: Simple, no application changes.
Cons: Upper hardware limits and single point-of-failure. Cost per performance often increases non-linearly. Use vertical scaling initially, but plan to move horizontally.

13. Horizontal Scaling (Scale Out)

What it is: Add more instances/machines to handle load (stateless services are required to scale smoothly).
Why preferred: Better fault tolerance, easier to add capacity elastically. Requires stateless app design or externalized session/state (caches, databases). Diagram 1’s application server pool is horizontally scalable behind a load balancer.

14. Load Balancers

What it is: Distribute incoming traffic across multiple servers. Types: L4 (TCP) vs L7 (HTTP-aware).
Algorithms: Round-robin, least connections, IP-hash, weighted. Health checks remove unhealthy nodes automatically.
Design note: L7 balancers can do route-based rules, TLS offload, and path-based routing; they’re essential to horizontal scaling and high availability.

15. Database Indexing

What it is: Data structures (B-trees, hash indexes) that allow fast lookups on columns.
Trade-offs: Indexes speed reads but slow writes and consume space. Choose composite indexes when common query filters are combined. Use explain plans to validate indexes.
Practical tip: Index selectively on columns used in WHERE, JOIN and ORDER BY; avoid indexing highly volatile columns unnecessarily.

16. Replication

What it is: Copying data from a primary (writer) to replicas (readers). Can be synchronous or asynchronous.
Why use it: Scale read throughput and improve availability (read replicas can handle reporting).
Consistency: Asynchronous replication yields eventual consistency; synchronous replication ensures stronger consistency but increases write latency. Diagram 2 shows primary + read replicas.

17. Sharding (Horizontal Partitioning)

What it is: Splitting a single logical dataset into multiple physical databases (shards) by a shard key.
When to use: When a single database cannot handle the data volume or throughput. E.g., partition users by user_id modulo N.
Challenges: Resharding (changing shard count), cross-shard joins and transactions. A shard mapper or router is often required (shown in Diagram 2).

18. Vertical Partitioning (Column-based split)

What it is: Splitting a table into multiple tables by columns (e.g., hot columns vs rarely-read BLOBs).
Why it helps: Reduce I/O for hot queries, keep frequently accessed columns compact. Often used alongside sharding/replication for performance tuning.

19. Caching

What it is: Storing frequently accessed data in fast storage (in-memory caches like Redis or memcached, and CDNs for static assets).
Cache strategies: Cache-aside (application checks cache then DB), write-through, write-back. Invalidations: TTL, explicit purge, or event-based invalidation.
Pitfalls: Stale data, cache stampede (many misses at once), cache poisoning. Diagram 1 shows a cache between app servers and DB.

20. Denormalization

What it is: Intentionally duplicating data to optimize read performance (precomputed joins, embedded documents).
Benefits: Faster reads, fewer joins.
Costs: More complex writes (must update multiple places), possible data inconsistency if not carefully managed. Use when reads vastly outnumber writes or when response tail latency matters.

21. CAP Theorem

What it is: In the presence of network Partitions, a distributed system can provide either Consistency or Availability, but not both simultaneously.
Implications: Choose CP (strong consistency, may block) or AP (available but eventual consistency) depending on use-case. Banking favors CP; social feeds often choose AP for availability.

22. Blob Storage (Object Storage)

What it is: Systems optimized to store large binary objects (images, videos). Examples: S3, GCS, Azure Blob Storage.
Features: Infinite scaling, lifecycle policies, versioning, multipart upload. Often used with a CDN for delivery. Diagram 4 includes blob storage as the canonical spot for large files.

23. CDN (Content Delivery Network)

What it is: Globally distributed edge caches that serve static (and in some cases dynamic) assets from servers close to users.
Why critical: Reduces latency and origin load. Use proper caching headers, and implement cache invalidation/purging flows for deployments. Diagram 1 and Diagram 4 show CDNs at the edge.

24. WebSockets

What it is: A persistent, bidirectional TCP-based channel for real-time updates (over an initial HTTP/HTTPS handshake).
Use-cases: Chat apps, live dashboards, multiplayer games.
Scaling: Requires either sticky sessions to specific servers or a pub/sub hub (Redis/Kafka) to fan out messages. Diagram 4 shows WebSocket Server interacting with clients.

25. Webhooks

What it is: HTTP callbacks that notify your system when events happen in external systems (push model).
Best practices: Secure with HMAC signatures, provide retries with exponential backoff, accept idempotent requests. Consumers must validate inputs and respond quickly (offload long work to background workers).

26. Microservices

What it is: Architecture that decomposes a monolith into small, independently deployable services owning their own data and domain logic.
Benefits: Independent scaling, smaller codebases, polyglot tech stacks.
Hard parts: Distributed transactions, service discovery, tracing, and eventual consistency. Diagram 3 depicts multiple services each with own DB plus an API Gateway.

27. Message Queues (Async Messaging)

What it is: Durable or transient queues (RabbitMQ, Kafka) that decouple producers and consumers so work can be processed asynchronously.
Patterns: Work queues, pub/sub, event sourcing.
Advantages: Smoothing spikes, retry and backoff, better resilience. Watch ordering guarantees and consumer offset management (Diagram 3 places the message queue as the asynchronous backbone).

28. Rate Limiting

What it is: Mechanisms to limit request rate per key (IP, API key, user) — token bucket and leaky bucket are common algorithms.
Purpose: Protect services from abuse, prevent cascading failures, and provide fair usage.
Implementation: At API Gateway or edge proxy; differentiate soft throttling vs hard blocking and return clear responses (429).

29. API Gateways

What it is: A single entry point for clients that handles cross-cutting concerns: authentication, routing, rate limiting, monitoring, and aggregation.
Role: Simplifies client logic and adapts to backend microservices.
Trade-offs: Gateway becomes a critical piece — must be highly available and horizontally scalable (see Diagram 3).

30. Idempotency

What it is: Guarantee that repeating the same operation (same idempotency key) yields the same result as doing it once.
Why required: Network retries, user double-clicks, or mobile retries must not cause double-charges or duplicated records.
Pattern: Generate idempotency keys for client-initiated actions, store recent keys with result, and respect TTLs for memory cleanup (especially important for payment endpoints — ties into API Gateway and message queue flows).