I don't have much of a homeserver, its more of an experiment - but I have Fedora IoT as hypervisor OS running a Open Media Vault guest and another Fedora IoT VM for container services.
I'm a big fan of Fedora's Ostree setup, and have used Silverblue on the desktop for a while now, so IoT makes a lot of sense for me.
Ubuntu LTS server. I chose it because it's got good support both in community and enterprise support. Also it's pretty simple to use and almost forgot it's got zfs built in.
I think it really depends on what you intend to do with it... Many answers here will mention what they use but not why.
In my case I want to have various services installed in docker containers, and I have the skills to manage Linux in console. A very simple solution for me was to use a rock-solid, established Linux distro on the host (Debian stable) with Docker sourced from its official apt repo. It's clean, it's simple, it's reliable, it's easy to reinstall if it explodes.
Why containers (as opposed to directly on the host)? I've done both over several years and I've come to consider the container approach cleaner. (I mention this because I've seen people wondering why even bother with containers.) It's a nice sweet spot in-between dumping everything on the host and a fully reproducible environment like nixOS or Ansible. I get the ability to reproduce a service perfectly thanks to docker compose; I get to separate persistent data very cleanly thanks to container:host mapping of dirs and files; I get to do flexible networking solutions because containers can be seen as individual "machines" and I can juggle their interfaces and ports around freely; I get some extra security from the container isolation; it's less complicated than using VMs etc.
TrueNAS scale.
Why: my main concern is backup and data protection, and TrueNAS offers just that. On top of that it's flexible enough to build a media suite on top of it, and it's easy to manage.
I could have also gone unRAID, but since trueNAS is free and offer a bit better protection imo (at the cost of flexibility), I picked that
I went from freenas to unraid and couldn't be happier.
Unraid has a ton of really amazing features, it's super easy to use, the docker support is great (freenas didn't have docker support when I left), the parity drives are magic, and just being able to slap random disks of any size in your NAS is great.
I've had a few issues with freenas, but I've never had a single issue with unraid. That shit just works.
Edit:
I have a live stream porn downloader, that'll watch when people come online and start capturing the stream. I don't want this to be part of my system and putting strain on it, so with unraid I'm able to put disks in my system and use a plugin called unassigned devices, which allows me to add them to the system, but have them be separate from my main array.
That's why I just love unraid. The flexibility is great.
It's fantastic, though I should add that my Proxmox and Unraid are separate hardware, as with my Raspberry-Pi DNS Servers and Proxmox backup machine.
After started with Unraid a little over a year ago, this motivated me to move to a real hypervisor Unraid can virtualize, dockerize, etc, but has far too many drawbacks to justify the hardware commitments, though it was an excellent starting point.
Add to this, a (custom) DNS and SSL termination, and we well on our way to a proper home lab setup.
It's worth mentioning that while there are people virtualizing Unraid on Proxmox, that the implications of inter-service dependencies were just not worth the hassles against diversifying backend services.
Next up; UPS, pfSense (router), and forwarding DNS servers on a 10GbE backplane :D
Arch, because I've always had a better experience with it than Ubuntu, be it server or desktop. I also daily drive it on my desktops.
It's so much easier to setup. Only with Docker and MergerFS it's a command and easily updatable, instead of the PPA setups or bash installs you have to do on Ubuntu. The wiki is still the best.
And it's way easier to maintain when there's less stuff.
I used unraid last year, excellent experience learning how to use docker + vm in a user-friendly interface.
Now I use debian (installed via debootstrap) headless (docker only).
Currently running 1 server. Ubuntu, with docker because I'm not doing a ton, but I don't want to have to take down my pi-hole if I need to reboot jellyfin. etc.
I tried Unraid - left due to annoyance of losing server function when I had to take drive arrays down and only scrubbing on demand
Went to True Nas scale for the promise of ZFS and liked it, but had issues with the interface and getting GPU passthrough to containers was impossible.
Ended up ditching my large server case (still have it - anyone need a large 12 drive 3U case with a big threadripper and bunch of ram - hit me up) and switched to a NUC running ubuntu with portainer managing my dockers on it and all my data stored on a Synology NAS.
I've been on Unraid for years now with no complaints, much better then all the other NAS platforms. The docker app catalog is great. I also run a number of VMs on it including GPU pass through.
VMware on one, runs Ubuntu (and then docker) and various appliances. Xpenology on the other. Xpenology also runs Docker for more OOTB containerized apps.
I use Fedora.
Why not Promox or unraid or any of those? Because I didn't see a personal benefit to it over Linux+libvirt+qemu (I'm sure there is a benefit, I just don't care enough to find out)
Why not Arch/Debian/Ubuntu/etc? Because I partially use it as a way to learn or practice things from work and I want to stick as close to RHEL as possible.
Thanks, very interesting read. I've been to ESXi for 15 years more or less (first box was an atom miniitx vanilla board) but you really made me interested in PVE. As if I needed another project...
NixOS. Only been running it on that server for a couple of weeks now, but so far I'm happy with it.
I like being able to manage almost every aspect of the system from a single declarative configuration file.
I don't do any fancy NAS and/or RAID stuff though. Just the OS living on an nvme, and a logical volume running across two spinning disks. If I need direct file access I use scp, but the storage is more for jellyfin and a syncthing node.
I use OpenMediaVault, with a Docker plugin and a few containers for Plex and Transmission. Although, I don't actually remember if OpenMediaVault is the OS itself, or running on top of it, which is a testament to stability, I suppose.
I have only ever used UnRaid, so I can't speak to the differences between server options. But I love UnRaid!
I am not very experienced in using text-based command terminals, so having a GUI is very helpful for me. In the past, I've tried to really understand the Linux terminal, making reference sheets for all the commands and writing down file paths, but in the end, I just spend way too much time trying to remember/ figure out how to do something that would only take clicking an icon. (I'm not here to argue the merits of terminal vs GUI. I understand the power of the terminal and still occasionally use it when necessary).
I originally started using UnRaid because of the Linus Tech Tips video, where they made 2 gaming PCs in one computer. I really wanted to do that too for my partner and I to game together. I mainly wanted to do that for the cool factor, but ultimately, it was cheaper to buy all the parts for one PC and just get a second GPU and an UnRaid license than it was to buy everything for 2 PCs.
UnRaid's built-in Docker and VM support is amazing! The Community Applications plugin has also made Docker Containers a breeze! People make pre-configured apps that only take minimal setup on my side to get running (mostly setting the file path for save folders). I've never made much luck with Docker itself on other OSs, but now I run Plex, Home Assistant, Blender, Cura Slicer, photo backup, minecraft server, etc. All only took a few clicks and setting file paths, and then they are up and running!
The ability to quickly spin up VMs in UnRaid and choose how many resources you want to give it (CPU cores, RAM, passing through USB or any PCIe device) has been amazing! It is really making it easier for me to learn Linux since I can easily access Windows or Linux as VMs and easily give GPU access to either if I need to. I guess it's like having all the benefits of bare metal installs and VMs.
UnRaid also has a terminal for more advanced users. Really, it seems anything is possible with some of the stuff I've read about people doing. And I never imagined I could build two gaming PCs into one!