Never Down: Implementing Auto Fallback in Caddy

I host lots of private (and a few public) projects behind Caddy. I do like its simplicity and obvious happy path. Something I did struggle with at the start of my journey with Caddy, and something I routinely forget again, is how to set up a fallback page. This is a good idea if there’s a single-instanced service that might be taken offline for a few moments due to scheduled or unscheduled maintenance. The solution is to create a second, static upstream.

Let’s take a look at the Caddyfile for Findsight:

:9999 {
	respond "We'll be right back (usually within 3 minutes)!"
}

www.findsight.ai {
	redir https://findsight.ai{uri}
}

findsight.ai {
	handle {
		reverse_proxy localhost:8080 :9999 {
			lb_policy first
			lb_try_duration 1s
			fail_duration 30s
			transport http {
				dial_timeout 500ms
			}
		}
	}
}

Here’s what’s happening:

In normal operation, Caddy reverse proxies requests to localhost:8080 where the main Findsight app listens. The entry point is a domain (findsight.ai), and Caddy will automatically handle certificate provisioning and TLS termination. Additionally, there’s a request rewrite for the www subdomain.

If localhost:8080 would be the only upstream defined and the server goes down (e.g. for database migration) Cloudflare, which is the CDN for Findsight, would display a generic “upstream dead” error and that’s not a good look. It leaves the user guessing: why is it down? When will it be back?

We need to define a fallback. If Caddy considers an entry for reverse_proxy to be dead, it moves on to the next one. That behavior is defined by the lb (load balancing) policy first. Dead upstreams remain dead for 30s. This duration can be tuned to taste, I chose it mainly to avoid requests hitting the server during start-up after a crash, when the port is reserved but requests won’t be processed yet. Now requests hit the internal :9999 upstream, which serves a static response directly from Caddy. Here one could alternatively display a full HTML page.