All postsTutorials

Setting up a multi-region Kubernetes cluster

Real config, real failover testing, real cost numbers — running a cluster across three of our regions.

PAPriya AnandDeveloper AdvocateMar 30, 202614 min read

Most multi-region Kubernetes guides stop at the YAML. This one goes through the YAML, the failover test, and the actual cost at the end of the month. We'll deploy a stateless web tier across three of our regions — IAD, FRA, NRT — with traffic steering and a regional Postgres replica.

Topology

Three control planes, three data planes, one global service that routes via our anycast load balancer. Postgres is primary in IAD, with read replicas in FRA and NRT. Writes always go to IAD; reads can serve from anywhere.

The cluster manifest

yaml
apiVersion: rsh.cloud/v1
kind: KubernetesCluster
metadata:
  name: web-global
spec:
  regions:
    - name: iad
      nodes: 3
      size: vps-4
    - name: fra
      nodes: 3
      size: vps-4
    - name: nrt
      nodes: 2
      size: vps-4
  loadBalancer:
    type: anycast
    healthCheck:
      path: /healthz
      intervalSeconds: 5

The failover test

We tear down the IAD region while load is running. Within 18 seconds, anycast withdraws the IAD prefix, traffic shifts to FRA and NRT, and Postgres writes start failing because the primary is gone. We promote the FRA replica. Writes recover at the 90-second mark.

Postgres failover is the slowest part of any geographically-distributed setup. Plan for at least 60 seconds of write unavailability during a real region loss.

What it costs

Three regions, eight nodes, anycast LB, three Postgres instances, cross-region replication: about $480/month. Compare to running this on one of the big three clouds with the same redundancy and you're looking at roughly $1,400. The savings come mostly from not paying egress between regions.

PA
Written byPriya AnandDeveloper Advocate

Writes the tutorials she wishes existed when she was learning. Maintains three production WordPress sites in her spare time.

READY WHEN YOU ARE

Get the next post in your inbox.

Monthly engineering digest. No spam, no marketing — just what we're shipping and what we learned.

Subscribe View RSS