You can have multiple configs in a single Caddyfile and reload when you make changes, and it'll just route them as you wish, e.g.

domain1.com -> service on port 1234

domain2.com -> service on port 5678

domain3.com -> serving a file directory.

And then you still access domain1.com, domain2.com, domain3.com on port 80/443