For me, docker is just a superpower that let's me build larger, more complex applications. That's why it's worth learning. It raises the ceiling of what I'm able to create. Creating a site with multiple services that integrate with each other is complicated. You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about--the complexity you want to architect and program. You'd be too busy programming solutions to the problems that docker already solves.
> You could do it from scratch, but it would take so long that you would likely never be able to manage the actual complexity you care about
Except plenty of people have and continue to manage? Do we think that _most_ people are using Docker now?
This is how we do it:
We use baremetal servers. We use systemd, including the systemd sandboxing capabilities. We have deploy scripts (almost all bash, and yes, they are largely idempotent). A host could be deployed with any number of: cockroachdb, openresty, monetdb, rabbitmq or postgresql. They're all running vector (log ingestion). And they can be running N apps. Each host also get public keys for encrypting on-site, credentials for uploading backups to Google Cloud, systemd-timers for tasks like running lynis and rkhunter. Everything is connected via wireguard (over a physical private network).
Our apps (mostly go and elixir) are built on a self-hosted gitlab and we deploy like normal. Each project can pick what it wants, e.g. auto deploy dev branch to dev, auto deploy master to stage, manually-triggered to prod.
We run separate tool servers for our internal stuff, gitlab, elastic search, kibana, prometheus, vault and grafana.
We have some clustering/mesh networking which we "discover" with an internal DNS. But everything is pretty static/fixed. We can survive downtime because either something is HA and stateless (e.g. we keep N monetdb "reporting" servers in sync from the OLTP->OLAP process), or stateful and part of a cluster (e.g., cockroachdb).
Here's the kicker: we have 0 devops. We don't spend a huge deal of time on this. We used to have devops and K8 and everything was worse. It was _much_ more expensive, everything took _a lot_ longer to get done, it all ran slower, it was much buggier, and it was less secure (because we aren't sharing a host).
I feel like we could build almost anything this way. I don't see what complexity we'd run into that Docker would help with. I think we'd run into management issues...applying patches to hundreds of servers, reviewing hardening and access logs, etc. But that's infrastructure complexity, not application complexity.
We use docker in apps that require a complex setup for end-to-end testing.
What you have sounds great, and sounds about the same as running containers with orchestration, just via SystemD cgroups instead of container cgroups, and bash and humans instead of a real orchestrator. You lose a lot of features ( idempotent and self-contained ( so zero risk of dependency clashing), cluster-wide auto-healing, load distribution and autoscaling, centralised management) and ease of use of existing solutions ( want to run a CockroachDB cluster? There's probably a k8s operator for that; want automatic DNS? Consul from Nomad, k8s already do that), but you understand it. There could be very serious downsides around maintenance of your bash scripts, and onboarding new people.
Personally, I don't want to think about all this complexity. I just build simple web applications. I want a box in my basement that has everything production has - the whole thing, all the web services running all the time.
It ought to be possible in my mind. After all, there is really only one user: me. I just want to be able to automatically get the latest code that reservations team or order management system team or what have you deploys to production.
Why do I need to connect to VPN just to work on a stupid web application? None of the code we write has any special sauce in it and we can have fake data...
I love docker as a home user as well. Sometimes when I'm bored I'll build an IM bot. Docker makes hosting it super simple. I just start from the ruby docker image, copy my script in and then dump it on gitlab which will build and host the container which lets me pull it from my server.
I can then copy a simple systemd config to start it on boot and restart it if it fails. This is all much simpler than managing multiple versions of ruby installed locally. Not to mention the security benefits of keeping it in a container. Perhaps this is all so easy and convenient for me because I learned docker for pro use and now its just mentally free for casual use.
Yeah I have seen this but for some reason it just wasn't working for me. I think that maybe the docker service just wasn't being started at boot but once I set it up with systemd it all just works now.
