Scaling a SaaS Beyond One Server

In Deploying a Full Stack SaaS, I got the app running in production on a single instance: Docker images, a CI/CD pipeline, one NestJS container and one Postgres database behind it. That setup will carry a real SaaS further than most people expect. It also has a hard ceiling. The day you hit it, "just add another server" turns out to be a much bigger sentence than it sounds.

This is Module 4 of the Full Stack SaaS Masterclass, and this article is about what actually changes when you go from one instance to several. It's less about picking an autoscaling number and more about the assumptions your app quietly made when it only ever ran on one box.

What follows is the application-level changes that have to happen before more servers help you at all, rather than a walkthrough of configuring Kubernetes. Get those wrong and adding capacity just spreads the same bugs across more machines.

Why one server works until it very suddenly doesn't

A single instance is simple because everything lives in one place: in-memory session storage, an in-process rate limiter, a WebSocket connection map, a scheduled job that runs "once" because there's only one process to run it. None of that is wrong on day one. It's the correct amount of complexity for the traffic you have.

The problem is that these shortcuts are invisible until you add a second instance. Then a user logs in, the load balancer routes their next request to a different container, and their session doesn't exist there. A cron job that ran "once" now runs twice, because both instances think they're the only one. A rate limiter that tracked requests per process now lets through double the traffic, because each instance has its own counter.

None of this shows up in code review. It shows up in production, as a support ticket that says "I got logged out for no reason" or "I got charged twice." So before scaling out, the real task is finding every place your app assumed there was exactly one of it.

Make the process stateless before you make it plural

The single most important property for horizontal scaling is that any instance can handle any request, with no memory of what happened before. If that's true, a load balancer can route however it wants and you can add or remove instances without anyone noticing.

Sessions are the classic offender. If you're using JWTs for auth (as covered earlier in this series), you're already stateless on that front, since the token carries everything the server needs to verify the user. If you're using server-side sessions, they need to live in Redis, not in process memory, so every instance reads the same session store.

Health checks matter just as much here, because a load balancer that can't tell a broken instance from a healthy one will happily send traffic to a dying container.

// health.controller.ts
import { Controller, Get } from '@nestjs/common';
import {
  HealthCheck,
  HealthCheckService,
  TypeOrmHealthIndicator,
  MemoryHealthIndicator,
} from '@nestjs/terminus';

@Controller('health')
export class HealthController {
  constructor(
    private health: HealthCheckService,
    private db: TypeOrmHealthIndicator,
    private memory: MemoryHealthIndicator,
  ) {}

  @Get('live')
  live() {
    // Liveness: is the process even running? No dependency checks.
    return { status: 'ok' };
  }

  @Get('ready')
  @HealthCheck()
  ready() {
    // Readiness: can this instance actually serve traffic right now?
    return this.health.check([
      () => this.db.pingCheck('database', { timeout: 1500 }),
      () => this.memory.checkHeap('memory_heap', 500 * 1024 * 1024),
    ]);
  }
}

The distinction matters once you're behind a load balancer or an orchestrator. Liveness answers "should this container be restarted?" Readiness answers "should traffic be sent here right now?" A container can be alive but not ready, for instance while it's warming up a database connection pool during startup, or draining connections during a graceful shutdown.

Graceful shutdown is the other half of this. When an instance is about to be terminated (during a deploy, a scale-down, or a spot instance reclaim), it needs to stop accepting new work, finish in-flight requests, and close its database and Redis connections cleanly.

// main.ts
async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.enableShutdownHooks();

  const server = await app.listen(process.env.PORT ?? 3000);

  process.on('SIGTERM', async () => {
    server.close(); // stop accepting new connections
    await app.close(); // run onModuleDestroy hooks: DB, Redis, queues
    process.exit(0);
  });

  return app;
}

Skip this and every deploy or scale-down event drops requests mid-flight. It's a common source of intermittent 502s that never show up in local testing, because locally you never kill the process while it's handling a request.

The database becomes the real bottleneck

Once the app tier is stateless and horizontally scaled, the next constraint shows up almost immediately: Postgres. Every app instance opens its own connection pool, and Postgres connections are not cheap; each one holds a backend process and a meaningful chunk of memory. Five instances with a pool of 20 each is 100 connections before you've served a single extra user, well past what Postgres defaults expect from a busy fleet.

This is where connection pooling at the infrastructure level earns its keep. PgBouncer sits between your app instances and Postgres, multiplexing many app-level connections onto a much smaller number of real database connections.

# pgbouncer.ini (relevant section)
[databases]
saas_prod = host=postgres-primary port=5432 dbname=saas_prod

[pgbouncer]
pool_mode = transaction
max_client_conn = 1000
default_pool_size = 25

transaction pooling mode is the one that actually helps at scale: a client (your app) borrows a real connection only for the duration of a transaction, then gives it back. It does mean you lose session-level features like advisory locks held across statements or SET commands that are meant to persist, so it's worth auditing anything relying on session state before flipping this on.

The other lever is read replicas. Reporting queries, dashboards, and analytics reads don't need to hit the primary, and separating them protects write throughput for the paths that actually matter, like checkout or signup.

// database.module.ts
export const databaseProviders = [
  {
    provide: 'PRIMARY_CONNECTION',
    useFactory: () =>
      new DataSource({
        type: 'postgres',
        host: process.env.DB_PRIMARY_HOST,
        // writes and anything read-after-write sensitive
      }).initialize(),
  },
  {
    provide: 'REPLICA_CONNECTION',
    useFactory: () =>
      new DataSource({
        type: 'postgres',
        host: process.env.DB_REPLICA_HOST,
        // dashboards, analytics, exports
      }).initialize(),
  },
];

The honest tradeoff is replication lag. A user who just updated their profile and immediately reloads the page can hit a replica that hasn't caught up yet and see stale data. The usual fix is routing anything read-after-write (the response to a mutation, or a page loaded right after a form submit) to the primary, and reserving replicas for reads that can tolerate a little staleness. Most reporting and list views fall into that category comfortably.

Move shared state into Redis, deliberately

Once you have more than one instance, anything that needs to be shared across the fleet has to live somewhere all instances can see it. Redis is the natural home for this because it's fast enough to sit in the request path and simple enough to reason about.

Rate limiting is the clearest example. An in-memory rate limiter on each instance only limits requests to that instance, so a user hitting five instances effectively gets five times the limit.

// redis-rate-limiter.ts
import { Injectable } from '@nestjs/common';
import Redis from 'ioredis';

@Injectable()
export class RedisRateLimiter {
  constructor(private readonly redis: Redis) {}

  async isAllowed(key: string, limit: number, windowSeconds: number): Promise<boolean> {
    const count = await this.redis.incr(key);
    if (count === 1) {
      await this.redis.expire(key, windowSeconds);
    }
    return count <= limit;
  }
}

The same pattern applies to WebSockets. If your app supports live features (notifications, presence, collaborative editing), a socket connected to instance A can't broadcast directly to a client connected to instance B. Socket.IO's Redis adapter solves this by publishing events through Redis pub/sub, so any instance can reach any connected client regardless of which instance it's on.

// socket-adapter.ts
import { IoAdapter } from '@nestjs/platform-socket.io';
import { createAdapter } from '@socket.io/redis-adapter';
import { createClient } from 'redis';

export class RedisIoAdapter extends IoAdapter {
  private adapterConstructor: ReturnType<typeof createAdapter>;

  async connectToRedis(): Promise<void> {
    const pubClient = createClient({ url: process.env.REDIS_URL });
    const subClient = pubClient.duplicate();
    await Promise.all([pubClient.connect(), subClient.connect()]);
    this.adapterConstructor = createAdapter(pubClient, subClient);
  }

  createIOServer(port: number, options?: any) {
    const server = super.createIOServer(port, options);
    server.adapter(this.adapterConstructor);
    return server;
  }
}

The same reasoning extends to scheduled jobs and locks. A cron-style job that assumes "there's only one process" will fire once per instance once you scale out. Distributed locks (Redis's SET NX with an expiry, or a library that wraps that pattern) keep only one instance actually executing the job while the others no-op.

Scaling the fleet without scaling the chaos

With the app stateless and shared state centralized, horizontal scaling becomes mostly an infrastructure decision. On AWS, that usually means an Application Load Balancer in front of an ECS or Fargate service, with a target-tracking scaling policy based on CPU or request count per target.

The tradeoff worth naming: autoscaling adds capacity reactively, after a metric crosses a threshold, and new containers take time to start and pass their readiness check. If traffic spikes faster than a new instance can come online, you'll see elevated latency during the gap. Capacity planning still matters here: autoscaling only handles the variance around a baseline you've already sized reasonably.

A few pitfalls show up reliably here. Deploys that don't drain connections drop requests exactly when you're proving the new version works. Health checks that only check "is the process up" rather than "can it reach the database" let broken instances stay in rotation. And a thundering herd on cache expiry, where many instances simultaneously miss a Redis key and hammer Postgres at once, can turn a scaling event into an outage. Staggered TTLs or a simple lock around cache repopulation avoids that specific failure mode.