As part of setting up my new home server, I decided to learn Caddy. Caddy is a replacement for nginx/apache. With its plugin system, zero-config SSL support, and modern architecture, it’s a good bit more powerful.

It’s been a while since I’ve used something that just felt right: well-designed, great abstractions, good defaults, etc. I’m really impressed with what they’ve built. It’s a beautiful piece of software with well-written documentation.

It’s pretty easy to use, but I ran into some sharp edges and wanted to share my experience and my example Caddyfile.

Caddy Features

• It’s designed for local development and production usage. It has some nice features for local development like locally signed SSL certs that are automatically added to your certificate store.
• There’s a nice extension architecture that makes it easy to create custom docker containers with the extensions you want bundled in.
• If you are running Caddy in a docker container, you want to link/copy your Caddyfile to /etc/caddy/Caddyfile
• Disabling https auto_https off will not automatically support http on servers. You must prefix any server blocks with http otherwise you’ll get a 502 gateway error. This took me a bit to figure out.
• You can dump the config by calling http :2019/config/. You’ll need to either expose this port to the host, or hop into the container (exec sh).
• If you want to forward to a non-https proxy, you have to prefix your host specification with http . Ex: reverse_proxy http://danswer_web_server:3000
• If bind is not specified, all available network interfaces are bound to.
• By default, access requests are not logged. You need to specify log within a server block in order to enable logging.
• Logs get sent to stdout by default.
• handle_path automatically drops the static path of the match
• You can reference ENV variables in your Caddyfile.
• It supports both "snippets" (blocks that can be reused) and variables (using @). This is a really significant improvement from Apache + Nginx—the lack of this feature was one of my biggest gripes with them.
• This site has some great snippets

Beyond this, Candy also allows you to cater to your needs more precisely. Want to set up a catch-all reverse proxy? You can do that.

:80 {
  log
  reverse_proxy http://dokku:9010
}

Do you have a local TLD that you don’t want SSL enabled for? You can specify a global option to disable ssl (there’s a bunch more you can do too):

{
  auto_https off
  debug
}

Development

In my case, I had Caddy running inside of a docker container. I was editing the caddy file through a VS Code SSH remote session. Here’s the snippet I used to automatically reload changes to the caddy file when it was updated.

echo Caddyfile | entr docker compose exec -it caddy sh -c 'caddy reload --config /etc/caddy/Caddyfile'    

However, just because the file is mounted via bind-mount, it doesn’t mean it will always be updated. I learned that:

Bind mounts are based on inodes, and when the file is deleted and recreated, the bind-mount is broken. These changes aren’t propagated to the bind-mount until a container restart, so it picks the new inode.

This happened to me multiple times, most likely because of a git rebase. Be aware that a bind-mount file link can easily break!

Alpine Linux

Alpine linux is annoying to work with since it’s so bare bones. Many of the containers I was trying to weave together were on alpine, so I found it helpful to install some tools to help inspect the network:

apk add bind-tools # for dig and friends
apk add httpie

The right thing to do is to set the default user for a docker image to nonroot. If you run into this, you’ll encounter this error:

/app $ apk add httpie
ERROR: Unable to lock database: Permission denied
ERROR: Failed to open apk database: Permission denied

You can specify the root user within an exec call (this was new to me!):

docker compose exec -u 0 -it container sh

Example Caddyfile

Here’s an example Caddy config:

• hello.lan for testing various config options
• Support for ArchiveBox, Windmill, and Danswer.
• Catch-all reverse proxy to dokku
• Disables https redirect since everything is off a local-only TLS config.

{
  auto_https disable_redirects
  debug
}

hello.lan {
    respond / "Hello World" 200
    tls internal
    log
}

http://windmill.lan {
        reverse_proxy /ws/* http://windmill_lsp:3001
        reverse_proxy /api/srch/* http://windmill_indexer:8001
        reverse_proxy /* http://windmill_server:8000
}

http://danswer.lan {
    reverse_proxy http://danswer_web_server:3000

    # this is handled by the /api* handler below
    rewrite /openapi.json /api/openapi.json

    handle_path /api* {
      reverse_proxy http://danswer_api_server:8080
    }

    # Set max body size to 5G
    request_body {
        max_size 5G
    }
}

http://archivebox.lan {
  reverse_proxy http://archivebox:8000
}

:80 {
  log
  reverse_proxy http://dokku:9010
}

Wildcard DNS With DNSMasq on Pihole

I wanted to specify a wildcard DNS configuration so that all unmapped domains would point to my new home server. You can’t do this through Pihole and instead need to use dnsmasq for this:

# dnsmasq.conf
address=/.lan/100.75.228.62
address=/.lan/192.168.7.34

You can define multiple IPs for a given wildcard DNS as well. For me, this is helpful since I want the local statically defined IP to be used first, with the tailscale IP used only if this is down (note that last is first in DNSMasq config). If the DNS server responds with two IPs, order is not guaranteed, but most systems respect it.

I then have a make command to copy this (and a custom.list) config into my pihole container:

pihole-custom-dns:
    docker compose cp custom.list pihole:/etc/pihole/custom.list
# wildcard DNS can only be done through lower-level dnsmasq
    docker compose cp dnsmasq.conf pihole:/etc/dnsmasq.d/05-custom-wildcards.conf
    docker compose restart pihole

Conclusion

Overall, Caddy is a great software product. Throughout learning it, I have been very impressed with its overall function, documentation, and unique features. The learning process was smooth, and the implementation was beneficial.