I use Caddy V2 (running in Docker/Podman). Configuration can be even simpler than the below. It automatically sorts out the SSL certs from Let's Encrypt for you. If you use Cloudflare DNS challenge like I do, you can get SSL without the server having to be exposed to the internet.
I used docker to get nextcloud and nginx conf to reverse proxy to it. It works well and is not difficult to set up by following their guide on github. It works pretty much out of the box.
I would keep octopi off the Internet (local network only). There's too much risk that if somebody did get access they could heat your hot-end up to 300C and just leave it there or something.. Setup a vpn if you want remote access to it.
Nginx is pretty easy to set up. Look up "nginx virtual hosts". You might want to use certbot/acme if you don't have SSL certificates for your domain names. You need either a wildcard certificate (*.example.com), a certificate with SAN (Subject Alternative Name) containing the second subdomain, or two certificates (one for each subdomain).
Note that subdomains can be found more easily than path based websites, if you allow connections from the whole WAN.
I use both nginx and haproxy, and nginx is much easier to configure. That being said, haproxy has more features, like working as a load balancer with traffic shaping/shifting. But it sounds like you don’t need those features.