What's the deal with Docker?

IMHO with docker and containerization in general you are trading drive space for consistency and relative simplicity.

a hypothetical:
You set up your mumble server and it requires the leftpad 3.7 package to run. you install it and everything is fine.
Now you install your ftp server but it needs leftpad 5.5. what do you do? hope the function that mumble uses in 3.7 still exists in 5.5? run each app in its own venv?

Docker and containerization resolve this by running each app in its own mini virtual machine. A container running mumble and leftpad 3.7 can coexist on host that also has a container running a ftp server with leftpad 5.5.

Here is a good video on what hole docker and containerization looks to fill
https://www.youtube.com/watch?v=Nm1tfmZDqo8

There have been some great answers on this so far, but I want to highlight my favourite part of Docker: the disposability.

When you have a running Docker container, you can hop in, fuck about with files, break stuff as you try to figure something out, and then kill the container and all of the mess you've created is gone. Now tweak your config and spin up a fresh one exactly the way you need it.

You've been running a service for 6 months and there's a new upgrade. Delete your instance and just start up the new one. Worried that there might be some cruft left over from before? Don't be! Every new instance is a clean slate. Regular, reproducible deployments are the norm now.

As a developer it's even better: the thing you develop locally is identical to the thing that's built, tested, and deployed in CI.

I <3 Docker!

It's virtual machines but faster, more configurable with a considerably larger set of automation, and it consumes less computer resources than a traditional VM. Additionally, in software development it helps solve a problem summarized as "works on my machine." A lot of traditional server creation and management relied on systems that need to be set up perfectly identical every deployment to prevent dumb defects based on whose machine was used to write it on. With Docker, it's stupid easy to copy the automated configuration from "my machine" to "your machine." Now everyone, including the production systems, are running from "my machine." That's kind of a big deal, even if it could be done in other ways naturally on Linux operating systems. They don't have the ease of use or the same shareability.

What you're doing is perfectly expected. That's a great way of getting around using Docker. You aren't forced into using it. It's just easier for most people

This blog post explains it well:

https://cosmicbyt.es/posts/demistifying-containers-part-1/

Essentially, containers are means of creating environments in which you can run software, and those environments are:

• isolated, which makes it a very controlled environment. Much harder to run into errors
• reproducible: we have tools that reproduce the same container from an image file
• easy to distribute: just have the container image.
• little to no compromises on performance (at least on Linux)

It is essentially a way for you to run a program without having to worry how to set up the environment, why it didn't work as expected, what dependencies you're missing, etc.

I feel that a lot of people here are missing the point. Docker is popular for selfhosted services for a few main reasons:

1. It is one package that can be used on any distribution (or even OS with a Linux VM).
2. The package contains all dependencies required to run the software so it is pretty reliable.
3. It provides some basic sandboxing against non-malicious services. Basically the service can't scribble all over your filesystem. It can only write to specific directories that you have given it access to (via volumes) other than by exploiting security vulnerabilities.
4. The volume system also makes it very obvious what data is important and needs to be backed up or similar, you have a short list.

Docker also has lots of downsides. I would generally say that if your distribution packages software I would prefer the distribution's package over the docker image. A good distribution package will also solve all of these problems. The main issue you will see with distribution packages is a longer delay before new versions are made available.

What Docker completely dominates was previous cross-distribution packaging options which typically took one of the previous strategies.

1. Self-contained compiled tarball. Run the program inside as your user. It probably puts its data in the extracted directory, maybe. How do you upgrade? Extract and copy a data directory? Self-update? Code is mutable and mixed with data, gross.
2. Install script. Probably runs as root. Makes who-knows what changes to your system. Where is the data, is the service running? Will it auto-start on boot. Hope that install script supports your distro.
3. Source tarball. Figure out the dependencies. Hope they don't conflict with the versions your distro has. Set up users and setup scripts yourself. Hope the build doesn't take too long.

For your use case, consider it to be a packaging format (like AppImage, Flatpak, Deb, RPM, etc.) that includes all the dependencies (including services, not just libraries) for the app in question.

Should I change this?

If it's not broken don't fix it.

Use Podman (my preferred - the SystemD approach is awesome), containerd, or Incus. Docker is a graveyard of half-finished pet projects that have no reason for existing. Podman has a Docker-compatible socket, so 100% of Docker tooling will work with it.

Try to run something that requires php7 and something else that requires php8 on the same web server; or python 2 and python 3.

You actually can, but it's not pretty.

(The thing about a declarative setup isn't much of a difference, you can do it for any popular Linux distro.)

I have a reason I don't think is covered. A few programs I have come across that I want to try recommend docker and some only provide instructions for docker. They can spend less time trying to help you with dependencies and installations knowing they've included everything you need in the docker file. I don't have a background in Linux or programming so unless they tell you exactly how to install something, I can struggle. Their installation page is then just the docker compose file with a note on the environment variables you can change.

When I asked this question

So there are many reasons, and this is something I nowadays almost always do. But keep in mind that some of us have used Docker for our applications at work for over half a decade now. Some of these points might be relevant to you, others might seem or be unimportant.

• The first and most important thing you gain is a declarative way to describe the environment (OS, dependencies, environment variables, configuration).

• Then there is the packaging format. Containers are a way to package an application with its dependencies, and distribute it easily through the docker hub (or other registries). Redeploying is a matter of running a script and specifying the image and the tag (never use latest) of the image. You will never ask yourself again "What did I need to do to install this again? Run some random install.sh script off a github URL?".

• Networking with docker is a bit hit and miss, but the big thing about it is that you can have whatever software running on any port inside the container, and expose it on another port on the host. Eg two apps run on port :8080 natively, and one of them will fail to start due to the port being taken. You can keep them running on their preferred ports, but expose one on 18080 and another on 19080 instead.

• You keep your host simple and empty of installed software and packages. Less of a problem with apps that come packaged as native executables, but there are languages out there which will require you to install a runtime to be able to start the app. Think .NET, Java but there is also Python out there which requires you to install it on the host and have the versions be compatible (there are virtual environments for that but im going into too much detail already).

I am also new to self hosting, check my bio and post history for a giggle at how new I am, but I have taken advantage of all these points. I do use "latest" though, looking forward to seeing how that burns me later on.

But to add one more:- my system is robust, in that I can really break my containers (and I do), and to recover is a couple clicks in Portainer. Then I can try again, no harm done.

One of the the main reasons why docker and kubernetes take off is they standardized the deployment process. Say, you have 20 services running on your servers. It's much easier to maintain those 20 services as a set of yaml files that follow certain standard than 20 config files each with different format. If you only have a couple of services, the advantage is probably not apparent. But as you add more and more services, you'll start to appreciate it.

The thing that confused me when first learning about docker was, that everybody compares it to a virtual machine. It's not. Containers dont virtualize anything. They take a (single) process from the host OS and separate that into its own environment. All system calls, memory access, file writes etc are still handled by the same os (same kernel). However the process is separated both on the file system and process level. It can't see other processes outside of the container and it also doesn't see the real filesystem. It sees a filesystem provided by the container. This also means it sees different file and user permissions. When you run a alpine Linux docker container on an Ubuntu system, the container only containes the (few) files for alpine but no Linux kernel no desktop environment. A process inside that container only sees the alpine files and not the Ubuntu files. It also means all containers see a filesystem independent of each other and can use libraries and dependencies of different versions (they are only files after all).

For administration it makes running complex services easy. You define how to setup that service (what base Linux distro to use, what packages to install, what commands to run, and how to start the process). You can then be save to assume the setup of that service did not interfere with the setup of any other service. "Service 1 needs a certain system wide config changed? Service 2 needs that config in the default state? And both need a different version of the same library?" In containers you can have all at the same time because they each see a different version of the same config and library.

And all this is provided by the kernel itself. All docker does is provide an "easy" way to create and manage containers but could could do all of that using chroot, runc and a few other.

As a note, containers usually don't come with systemd as they don't need an init system. You would run the service directly inside the container and then use systemd outside the container to make sure the container is started/restarted, or just docker as it can already do that.

I found a great article demystifying containers recently

Install Portainer, it helps you get used to managing docker images and containers before going full command line.

One benefit that might be overlooked here is that as long as you don’t use any Docker Volumes (and instead bind mount a local directory) and you’re using Docker Compose, you can migrate a whole service, tech stack and everything, to a new machine super easily. I just did this with a Minecraft server that outgrew the machine it was on. Just tar the whole directory, copy it to the new host, untar, and docker compose up -d.

I started self-hosting a bit prior to when Docker took off, and getting multiple services running was much harder. Service A wants a certain version of PHP installed with certain plugins while Service B wants a different version. You'd follow a tutorial for installing Service C and desperately hope that it wouldn't somehow break Service A or B. You installed Service D for a bit despite all the installation pain and now want to uninstall it - I hope you tracked exactly what config changes you made throughout the system so you can undo it.

Docker fixed all of this by making each service independent through containers which made self-hosting 10x easier. I'd also add that I love how easy it is to transfer my setup to a new server - I keep all of my container volumes in a specific directory and my docker-compose files in another and that's all I need to backup / transfer. Without Docker you'd have to specifically handle each & every configuration file and database location, and if you later upgrade to a newer version of the OS or a different distro you'd have to handle possible conflicts between your versions and what the distro expects.

A lot of people here really do be describing docker like flatpak

I've used Docker a fair bit over the years because it's a simple line of code I can copy/paste to get a simple web server running.

I ran Home Assistant Supervised in Docker for many years. It was a few lines of code and then I basically had Home Assistant OS running on my Pi without it taking over the whole Pi, meaning I could run other things on it too.

That ended when HA just died one day and I had no clue how to get it running again. I spent a day trying, then just installed HA OS on the Pi instead.

Anyway I now have a Dell Optiplex and Proxmox and I've gone back to Docker. Why? Well I discovered that I could make a Linux VM and install Docker on it, then add the Docker code to install a Portainer client to it, then make that into a template.

Meaning I can clone that template and type the IP address into Portainer and now I have full access to that Docker instance from my original Portainer container. That means I can bang a Docker Compose file into the "Stack" and press go, then tinker with the thing I wanna tinker with. If I get it working it can stay, if I don't then I just delete the VM and I've lost nothing.

Portainer has made Docker way more accessible for me. I love a webui

If you're already using systemd, do not switch to Docker. Use Podman instead. Docker runs all your services under the Docker service. Podman can both run the same containers as systemctl services.

How’s the Pi 3? I was considering the idea of getting one to avoid the crazy prices for newer models

Dockers documentation is actually pretty good, I’d recommend taking a look at it because it’s written really well and can be used as a decent primer on learning to read documentation.

I would recommend learning docker / containerization. For your use case you likely won’t see a big benefit HOWEVER it is a good technology to know.

As far as the “why” you’d use it there are too many to list but for your use case the why I’d argue is “just so you know how to do it” and you’ll come up with your own why along the way.

Simplest why beyond “it’s a good technology to know” is that updating an app is as simple as pulling a new container and relaunching it.

Docker is amazing but not needed. You can compare it to a simpler VM. You can take a docker and run it on any machine. You have an environment that is separate from your host and you and the container can only access it via defined points (volumes and ports).

Imagine you need to run a 2nd Mumble Server. I never set on up but its often that a 2nd instance is not that easy. With docker its easy. The only difference is that you need to use different ports, when you have only one network access or you use a reverse proxy. You can create a 2nd instance to test stuff, without interrupting your productive system. Its a security benefit, because its isolated to some degree and you can remove one easily.

I started using it with MSSQL Server, because I hated how invasive it is on a windows machine, especially I just needed it temporarily to do stuff with it. I'm not a microsoft admin and I know that Servers from Microsoft are a different level. Docker allowed me to start and stop it and remove it very easily. After that I started using it for a lot of and brought my NAS on the next level.

Also one thing worth mentioning are Linux Containerx (LXC). They are in Proxmox but I have less knowledge. It feels more like a full VM than docker but uses less resources. This is the reason why containers in general are more popular. They are less resource hungry than a full VM but have some benefits than running everything on one machine. LXC feels more like a full system, than docker. With docker you rarely get into the system. You may execute some commands, like a create user command or a one time job but don't access it via a shell from the inside (its possible). LXC on the other hand, you use the shell.

Docker of one version of software that uses Linux containers to encapsulate software and that software's dependencies, while limiting that software's access to the underlying OS. It's chroot, but for more of the system. It can make running software that has a lot of moving parts and dependencies easier. It can also improve your security running that software.

For how-tos, watch one of the 875,936 YouTube tutorials, or read one of the 3 million text tutorials. Or ask ChatGPT, if you really need hand-holding.

Vs. LXD?

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

9 acronyms in this thread; the most compressed thread commented on today has 15 acronyms.

[Thread #592 for this sub, first seen 11th Mar 2024, 17:25] [FAQ] [Full list] [Contact] [Source code]

Recent video that explains Docker very well: https://www.youtube.com/watch?v=rIrNIzy6U_g

The thing with Docker is that people don't want to learn how to use Linux and are buying into an overhyped solution that makes their life easier without understanding the long term consequences. Most of the pro-Docker arguments go around security and that's mostly BS because 1) systemd can provide as much isolation a docker containers and 2) there are other container solutions that are at least as safe as Docker and nobody cares about them.

Companies such as Microsoft and GitHub are all about re-creating and reconfiguring the way people develop software so everyone will be hostage of their platforms. We see this in everything now Docker/DockerHub/Kubernetes and GitHub actions were the first sign of this cancer. We now have a generation that doesn’t understand the basic of their tech stack, about networking, about DNS, about how to deploy a simple thing into a server that doesn’t use some Docker BS or isn’t a 3rd party cloud xyz deploy-from-github service.

Before anyone comments that Docker isn’t totally proprietary and there’s Podman consider the following: It doesn’t really matter if there are truly open-source and open ecosystems of containerization technologies. In the end people/companies will pick the proprietary / closed option just because “it’s easier to use” or some other specific thing that will be good on the short term and very bad on the long term.

Docker may make development and deployment very easy and lowered the bar for newcomers have the dark side of being designed to reconfigure and envelope the way development gets done so someone can profit from it. That is sad and above all set dangerous precedents and creates generations of engineers and developers that don’t have truly open tools like we did. There's LOT of money into transitioning everyone to the "deploy-from-github-to-cloud-x-with-hooks" model so those companies will keep pushing for it.

Note that technologies such as Docker keep commoditizing development - it’s a negative feedback loop that never ends. Yes I say commoditizing development because if you look at it those techs only make it easier for the entry level developer and companies instead of hiring developers for their knowledge and ability to develop they’re just hiring “cheap monkeys” that are able to configure those technologies and cloud platforms to deliver something. At the end of the they the business of those cloud companies is transforming developer knowledge into products/services that companies can buy with a click.

Docker makes sense if you are deploying thousands of machines in the cloud. I don't think it makes as much sense if you have your own hardware.

Some services do have 1-line installers with docker, so those might be useful. But they usually have 1-line non-docker installers too.

Avoid Docker as much as possible. It gives you a quick way to run apps, but in a way that you can't see what is going to be installed.

It is also double resource heater.

And it has many frequent vulnerabilities.