Designing a URL Shortener: From MVP to 10M Requests/Day

A URL shortener looks deceptively simple.

You give it https://example.com/products/some-really-long-product-url?id=12345 and it gives you https://sho.rt/aB7xK2. When someone visits the short URL, the service redirects them to the original.

That is the MVP.

But what happens when the system needs to handle 10 million requests every day? Suddenly the questions multiply fast.

How do we generate unique short codes without collisions? Should we use hashing or Base62? What database stores the mappings? How do we keep redirects fast? What happens when millions of users hit the same URL? How do we prevent abuse? How do we collect analytics without slowing down redirects? When does Redis actually help?

This article starts with a simple first version and evolves it into a system capable of handling 10M+ requests per day — adding each layer only when there is a concrete reason to add it.

What Are We Building#

Our URL shortener has two primary operations.

Create a short URL:

POST /api/urls
Content-Type: application/json

{ "url": "https://example.com/products/12345" }

Response:
{ "shortUrl": "https://sho.rt/aB7xK2" }

Redirect:

GET /aB7xK2

302 Found
Location: https://example.com/products/12345

That is the core system. Everything else is an optimization or an additional feature.

Functional Requirements#

For the first version, the scope is clear.

Must have:

• Accept a long URL and generate a unique short URL
• Redirect short URLs to the original with minimal latency
• Persist URL mappings durably

Nice to have:

• Custom aliases (https://sho.rt/my-product)
• Expiration (expiresAt: 2027-01-01)
• Click analytics
• Abuse protection

Non-Functional Requirements#

At scale, these become more important than the API contract itself.

The system needs low redirect latency, high availability, zero short-code collisions, horizontal scalability, durable storage, and protection against abusive traffic.

The most important characteristic is that this is a read-heavy system. A user may create one short URL and that URL might be visited thousands or millions of times afterward.

The typical ratio looks something like:

1 URL creation
      ↓
100+ redirect requests

This read/write asymmetry strongly influences every architectural decision.

Start With the MVP#

The simplest architecture is a single Node.js API backed by PostgreSQL:

Client → Node.js API → PostgreSQL

The application handles both POST /api/urls and GET /:shortCode.

For a small application, this is more than enough. There is no need to start with Kafka, Redis Cluster, Kubernetes, microservices, CDN, or database sharding unless the requirements actually justify them.

Database Schema#

A straightforward PostgreSQL table:

CREATE TABLE urls (
    id         BIGSERIAL PRIMARY KEY,
    short_code VARCHAR(10)  UNIQUE NOT NULL,
    original_url TEXT        NOT NULL,
    created_at TIMESTAMP    NOT NULL DEFAULT NOW(),
    expires_at TIMESTAMP    NULL
);

The critical field is short_code. It needs a unique constraint — that gives a database-level guarantee against collisions.

How to Generate the Short Code#

This is one of the most important design decisions in the entire system.

Option 1 — Hash the URL. Take SHA256 of the long URL and truncate it to a few characters. The problem is that two different URLs can produce the same truncated hash. As the number of URLs grows, collision probability becomes a real concern. Do not blindly truncate a hash and assume it is unique.

Option 2 — Generate an ID and encode it. This is the better approach. Generate a unique numeric ID from the database sequence (1, 2, 3...) and encode that number using Base62.

Base62 uses the characters 0-9, A-Z, and a-z. That is 62 characters total. The mapping works like this:

1       → 1
62      → 10
1000    → G8
123456  → w7e

The reason Base62 is attractive is that it packs a large number into a compact, URL-safe string. Seven characters gives 62^7 which is approximately 3.5 trillion possible combinations. That is more than enough for any practical URL shortener.

Base62 Implementation#

const ALPHABET =
  "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
 
function encodeBase62(num: number): string {
  if (num === 0) return ALPHABET[0];
  let result = "";
  while (num > 0) {
    result = ALPHABET[num % 62] + result;
    num = Math.floor(num / 62);
  }
  return result;
}

The flow becomes: database sequence produces an ID, the application encodes it to Base62, and that becomes the short code. No collision detection logic needed.

The MVP Create Flow#

Client
  ↓
POST /api/urls
  ↓
API Server
  ↓
INSERT into PostgreSQL (auto-increment ID)
  ↓
Base62 encode the ID
  ↓
Return short URL

The Redirect Flow#

When someone visits GET /3D7:

async function redirect(shortCode: string) {
  const url = await db.urls.findUnique({
    where: { shortCode }
  });
 
  if (!url) return 404;
 
  return redirectTo(url.originalUrl);
}

For an MVP this works perfectly. But there is a problem that appears under load.

The Database Becomes the Hot Path#

At 10,000,000 requests per day, average traffic is around 116 requests per second. That does not sound scary in isolation. But averages hide spikes — traffic can jump to 1,000+ RPS during peak periods, and most of those requests are redirects.

Without caching, every single redirect hits PostgreSQL. The database becomes the bottleneck quickly. There is also an important observation: the same short URLs are requested repeatedly. A popular link shared on social media might receive thousands of hits per minute, all asking for the same row.

That is where caching changes everything.

Add Redis Caching#

The architecture gains a new layer:

Client
  ↓
Load Balancer
  ↓
API Servers
  ↓
Redis (cache-aside)
  ↓  (cache miss only)
PostgreSQL

The redirect flow becomes:

async function redirect(shortCode: string) {
  const cacheKey = `url:${shortCode}`;
 
  const cachedUrl = await redis.get(cacheKey);
  if (cachedUrl) {
    return redirectTo(cachedUrl);
  }
 
  const record = await db.urls.findUnique({ where: { shortCode } });
 
  if (!record) return notFound();
 
  await redis.set(cacheKey, record.originalUrl, "EX", 3600);
 
  return redirectTo(record.originalUrl);
}

The first request hits PostgreSQL and populates the cache. Every subsequent request for that URL is served from Redis in a few milliseconds. The database is protected from repetitive reads on the same data.

What Happens With a Popular Link#

Imagine a widely shared link receives one million clicks within an hour. Without caching, that is one million database queries. With caching:

1st request: Redis MISS → PostgreSQL → Redis SET
Remaining requests: Redis HIT → redirect

This is why caching matters so much for read-heavy systems. The math is not subtle.

The Cache Stampede Problem#

There is a subtle issue. If a cache entry expires at exactly the wrong moment and thousands of requests arrive simultaneously, they all see a cache miss and all hit the database at the same time. This is called a cache stampede or thundering herd.

Practical solutions include short distributed locks using Redis SET NX, probabilistic early refresh, or simply setting longer TTLs for URLs that are clearly popular. For a 10M/day system, a Redis lock lasting 200-300ms is usually sufficient.

Scaling the Application Layer#

Once Redis is in place, the next bottleneck shifts to the application servers themselves. The fix is straightforward: run multiple stateless API servers behind a load balancer.

Load Balancer
  ↓
API-1   API-2   API-3
  ↓
Redis
  ↓
PostgreSQL

Because the API servers hold no local state, any request can go to any server. Doubling traffic means adding more servers — no application-level coordination required.

Database Read Replicas#

As traffic increases further, PostgreSQL reads can be separated from writes. URL creation goes to the primary database. Redirect lookups can go to replicas.

However, since Redis should absorb most hot reads, adding replicas should be a measured decision. Adding a replica because it sounds like the right thing to do, without first confirming that PostgreSQL is actually the bottleneck, is premature.

Populating the Cache on Write#

One improvement to the create flow: populate Redis immediately after writing to PostgreSQL.

POST /api/urls
  ↓
Generate ID → Base62
  ↓
INSERT into PostgreSQL
  ↓
SET in Redis
  ↓
Return short URL

This means the first redirect never needs to hit the database at all.

Custom Aliases#

Some users want https://sho.rt/my-product instead of https://sho.rt/aB7xK2.

The implementation is conceptually simple:

async function createWithAlias(url: string, alias: string) {
  // Validate format and check reserved words
  if (!/^[a-zA-Z0-9_-]{3,50}$/.test(alias)) throw new Error('Invalid alias');
 
  try {
    await db.query(
      'INSERT INTO urls (short_code, original_url) VALUES ($1, $2)',
      [alias, url]
    );
    return alias;
  } catch (err: any) {
    if (err.code === '23505') throw new Error('Alias already taken');
    throw err;
  }
}

The database unique constraint remains the final authority. Application-level checks improve the user experience, but the constraint prevents race conditions.

URL Expiration#

Add an expires_at column and check it at redirect time:

GET /abc123
  ↓
Is expires_at < NOW()?
  ↓
Yes → 410 Gone
No  → redirect

Do not rely solely on Redis TTL for expiration correctness. The database is the source of truth. The cache may expire earlier or later, but the application enforces the business rule.

301 vs 302: An Important Decision#

301 Permanent Redirect — browsers and intermediate caches may aggressively cache the redirect. Great for performance, but subsequent requests may never reach your service, which makes analytics difficult or impossible.

302 Temporary Redirect — requests are more likely to reach your service each time. For a URL shortener where analytics matter, 302 is the safer default. Use 301 only when the destination is genuinely permanent and analytics are not a concern.

Analytics Should Not Block Redirects#

Suppose every click needs to record IP address, user agent, country, timestamp, and referrer. Doing this synchronously blocks the redirect:

Request → Lookup URL → Write analytics DB → Redirect

If analytics takes 100ms, your redirect now takes at least 100ms longer. That is unacceptable.

The correct approach:

Request → Lookup URL → Publish click event → Redirect immediately

The analytics pipeline processes the event asynchronously. The redirect path stays fast.

Add a Message Queue for Analytics#

At larger scale, a durable queue like Kafka or a managed equivalent decouples the redirect path from analytics completely:

API Servers
  ↓            ↓
Redis       Kafka
              ↓
       Analytics Workers
              ↓
       Analytics Database

The redirect path stays: lookup Redis, return 302. Kafka handles the rest asynchronously.

Rate Limiting and Abuse Prevention#

URL shorteners are attractive to attackers because short URLs hide destinations. Without rate limiting, a single client can generate millions of short URLs or flood the redirect endpoint.

Redis is a natural fit for distributed rate limiting:

async function checkRateLimit(ip: string): Promise<boolean> {
  const key = `ratelimit:${ip}`;
  const count = await redis.incr(key);
  if (count === 1) await redis.expire(key, 60);
  return count <= 100; // 100 requests per minute per IP
}

URL validation should also reject javascript:, file:, data:, and other unexpected schemes before any URL is accepted.

Capacity Planning#

At 10,000,000 requests per day:

Average RPS = 10,000,000 / 86,400 ≈ 116

With a 10x peak multiplier = ~1,160 RPS

That is a very manageable workload for a horizontally scaled application with Redis caching. The important takeaway: 10M requests per day does not automatically require a complicated distributed architecture. The system becomes difficult when traffic is highly bursty, URLs are extremely hot, availability requirements are very high, or analytics volume is large.

Storage Estimation#

At 1,000,000 new URLs per day at roughly 500 bytes per record:

Daily storage = 500 MB Annual storage = ~180 GB

Real storage will be higher due to indexes, replicas, backups, and overhead. But this is well within the capabilities of a standard managed PostgreSQL instance. There is no need for sharding at this scale.

Do We Need Sharding#

Probably not at 10M requests per day.

Start with PostgreSQL plus Redis plus horizontal API scaling. Only consider sharding when the database becomes a genuine bottleneck due to dataset size, write throughput, storage limitations, or replication constraints.

A common mistake in system design discussions is jumping from 10M/day directly to 50 database shards without demonstrating why they are needed. The architecture should earn its complexity.

Handling Failures Gracefully#

Redis goes down. The service should not stop. Fall back to PostgreSQL directly. Redirects are slower but correct.

async function redirect(shortCode: string) {
  try {
    const cached = await redis.get(`url:${shortCode}`);
    if (cached) return redirectTo(cached);
  } catch {
    // Redis unavailable — continue to database
  }
 
  const record = await db.query(
    'SELECT original_url FROM urls WHERE short_code = $1',
    [shortCode]
  );
  return record.rows[0] ? redirectTo(record.rows[0].original_url) : notFound();
}

PostgreSQL goes down. Existing cached URLs can still work. Creating new URLs will fail. This is acceptable — redirects are the critical path, not creation.

One API server goes down. The load balancer removes it. Other instances continue serving traffic without interruption.

Analytics pipeline goes down. Redirects continue working. Events can be buffered or retried. Analytics is never part of the critical redirect path.

CDN for Extremely Hot Links#

At very large scale, Redis does not have to be the first cache layer. Adding a CDN in front moves extremely popular traffic closer to users:

User → CDN → (cache hit) → redirect
            → (cache miss) → Load Balancer → API → Redis → PostgreSQL

This is especially useful when the same short links receive enormous volumes of global traffic. The critical path for a cache hit becomes: client, CDN edge, redirect. No backend involved at all.

The Evolution From MVP to 10M Requests Per Day#

Stage 1 — MVP

A single Node.js server talking directly to PostgreSQL. Good for thousands of requests per day. Fast to build and operate.

Stage 2 — Production Ready

Add a load balancer, multiple API servers, authentication, rate limiting, monitoring, and backups. Good practice regardless of scale.

Stage 3 — 10M Requests Per Day

Add Redis caching, read replicas on PostgreSQL, horizontal API scaling, and asynchronous analytics via a queue. The redirect path becomes: Redis, then 302. Database is protected.

Stage 4 — Much Larger

CDN in front, distributed ID generation if multiple regions need to write independently, database sharding only when data size or write throughput demands it, multi-region deployment for latency-sensitive global traffic.

The Most Important Design Decisions#

If you had to explain this system in a five-minute interview, these are the decisions that matter.

Use Base62 encoding of a database-generated ID. It produces compact, URL-safe identifiers with zero collision risk.

Do not hash the URL and truncate. Hash collisions are a correctness problem, not just a performance concern. Generate a unique ID first, then encode it.

PostgreSQL is enough to start. Do not introduce NoSQL or sharding without a concrete requirement backed by measurements.

Redis handles hot redirects. The redirect path should hit Redis and return a 302 — the database should almost never be involved for popular URLs.

Keep analytics off the critical path. Never let an analytics write block a redirect. Fire an event, return the redirect immediately, process the event asynchronously.

Keep API servers stateless. Stateless servers make horizontal scaling trivial.

Design for graceful degradation. Redis going down should degrade performance, not take the service offline.

Final Architecture#

For a system handling around 10M requests per day:

Clients
  ↓
CDN
  ↓
Load Balancer
  ↓
API-1   API-2   API-3
  ↓
Redis
  ↓ (cache miss)
PostgreSQL (primary + read replica)
  ↓ (analytics events)
Kafka
  ↓
Analytics Workers
  ↓
Analytics Database

The critical request path stays minimal:

Client → CDN or API → Redis → 302 Redirect

The database handles durability. Kafka and analytics stay entirely off the redirect path.

Conclusion#

A URL shortener is a great example of how a simple feature becomes a distributed systems design problem at scale.

The real lesson is not "use Redis, Kafka, PostgreSQL, and Base62." The real lesson is that the architecture should evolve based on actual measured bottlenecks.

Start simple: API talking directly to PostgreSQL. When reads become the bottleneck, add Redis. When application traffic grows, add load balancing and more API servers. When analytics becomes expensive, add a queue. When traffic becomes globally large, add a CDN.

At 10 million requests per day, the winning architecture is not the most complicated one. It is the simplest architecture that keeps the redirect path fast, protects the database from repetitive reads, and allows every layer to scale independently.

Every component added beyond that should have a concrete reason — a measured bottleneck, a business requirement, or a reliability constraint that justifies the added complexity.

Building a product with link management, referral tracking, or a custom URL shortener? Get in touch — I work on full-stack Node.js and Next.js products from early-stage through production scale.