Depending on how much of the Kube API you need, Podman is that. It can generate containers and pods from Kubernetes manifests [0]. Kind of works like docker compose but with Kubernetes manifests.

This even works with systemd units, similar to how it's outlined in the article.

Podman also supports most (all?) of the Docker api, thus docker compose, works, but also, you can connect to remote sockets through ssh etc to do things.

[0] https://docs.podman.io/en/latest/markdown/podman-kube-play.1...

[1] https://docs.podman.io/en/latest/markdown/podman-systemd.uni...

I hate sounding like an Oracle shill, but Oracle Cloud's Free Tier is hands-down the most generous. It can support running quite a bit, including a small k8s cluster[1]. Their k8s backplane service is also free.

They'll give you 4 x ARM64 cores and 24GB of ram for free. You can split this into 1-4 nodes, depending on what you want.

[1] https://www.oracle.com/cloud/free/

I have a couple dedicated servers I fully manage with ansible. It's docker compose on steroids. Use traefik and labeling to handle reverse proxy and tls certs in a generic way, with authelia as simple auth provider. There's a lot of example projects on github.

A weekend of setup and you have a pretty easy to manage system.

But you've already said yourself that the cost of using K8s is too high. In one sense, you're solving those solutions more efficiently, it just depends on the axis you use to measure things.

That's more than what I'm paying for far fewer resources than Hetzner. I'm paying about $8 a month for 4 vCPUs and 8GB of RAM: https://www.hetzner.com/cloud

Note that the really affordable ARM servers are German only, so if you're in the US you'll have to deal with higher latency to save that money, but I think it's worth it.

Which I guess makes it more than good enough for hobby stuff - I'm playing with a multi-node cluster in my homelab and it's also working fine.

In terms of the cloud, I think Digital Ocean costs about $12 / month for their control plane + a small instance.

For single server setups, it uses k3s, which takes up ~200MB of memory on your host machine. Its not ideal, but the pain of trying to wrangle docker deployments, and the cheapness of hetzner made it worth it.

I don't use ingresses or loadbalancers because those cost extra, and either have the services exposed through tailscale (with tailscale operator) for stuff I only use myself, or through cloudflare argo tunnels for stuff I want internet accessible

(Once a project graduates and becomes more serious, I migrate the container off this cluster and into a proper container runner)

It looks like Nomad has a driver to run software via isolated fork/exec, as well, in addition to Docker containers.

Let it not be idempotent. Let it crash sometimes.

We lived without kubs for years and the web was ok. Your users will survive.

Out of curiosity, what is so bad about this for smaller projects?

The marginal cost of an additional project on the cluster is essentially $0

Recently I switched my entire setup (few Pi's, NAS and VM's) to NixOS. With Colmena[0] I can manage/update all hosts from one directory with a single command.

Kubernetes was a lot of fun, especially the declarative nature of it. But for small setups, where you are still managing the plumbing (OS, networking, firewall, hardening, etc) yourself, you still need some configuration management. Might as well put the rest of your stuff in there also.

[0] https://colmena.cli.rs/unstable/

    6 vCore (ARM64)
    8 GB RAM
    256 GB NVMe

The more I look into it, the more I think of k8s as a way to "move to micro services" without actually moving to micro services. Loosely coupled micro services shouldn't need that level of coordination if they're truly loosely coupled.

To put this in perspective, that’s less compute than a phone released in 2023, 12 years ago, Samsung Galaxy S4. To find this level of performance in a computer, we have to go to

The main issue is that Kubernetes has created good API and primitives for managing cloud stuff, and managing a single server is still kinda crap despite decades of effort.

I had K3S on my server, but replaced with docker + Traefik + Portainer - it’s not great, but less idle CPU use and fewer moving parts

Or maybe look into Kamal?

Or use Digital Ocean app service. Got integration, cheap, just run a container. But get your postgres from a cheaper VC funded shop :)

It can manage multiple machine with just ssh access and docker install.

https://github.com/virtual-kubelet/virtual-kubelet

https://minikube.sigs.k8s.io/

Another way to look at this is the Kubernetes created solutions to problems that were already solved at a lower scale level. Crontabs, http proxies, etc… were already solved at the individual server level. If you’re used to running large coordinated clusters, then yes — it can seem like you’re reinventing the wheel.

From my perspective, it got a lot of hate in its first few years (decade?), not because the project itself was bad -- on the contrary, it succeeded in spite of having loads of other issues, because it was so superior. The problem was the maintainer's attitude of wantonly breaking things that used to work just fine, without offering any suitable fixes.

I have an old comment somewhere with a big list. If you never felt the pain of systemd, it's either because you came late to the party, or because your needs always happened to overlap with the core maintainer's needs.

    #define _XOPEN_SOURCE 700
    #include <signal.h>
    #include <unistd.h>
    int main() {
        sigset_t set;
        int status;
        if (getpid() != 1) return 1;
        sigfillset(&set);
        sigprocmask(SIG_BLOCK, &set, 0);
        if (fork()) for (;;) wait(&status);
        sigprocmask(SIG_UNBLOCK, &set, 0);
        setsid();
        setpgid(0, 0);
        return execve("/etc/rc", (char *[]){ "rc", 0 }, (char *[]){ 0 });
    }

You can spawn systemd from there, and in case anything goes wrong with it, you won't get an instant kernel panic.

Sure. It worked for _50 years_ just fine but obviously it is very wrong and should be replaced with - of course - systemd.

never have your filesystem mounted at the right time, because their automount rules are convoluted and sometimes just plain don't work despite being 1:1 according to the documentation.

What isn't great, and where the hate comes from, is that it makes the life of a distribution or upstream super easy, at the expense of adding a (slowly growing) complexity at the lowest levels of your system that--depending your perspective--does not follow the "unix way": journalctl, timedatectl, dependencies on/replacing dbus, etc. etc. It's also somehow been conflated with Poettering (he can be grating in his correctness), as well as the other projects Poettering works on (Avahi, Pulse Audio).

If all you want to do is coordinate some processes and ensure they run in the right order with automatic activation, etc. it's certainly capable and, I'd argue, the right level of tool as compared to something like k8s or docker.

My only bugbear with it is that there's no equivalent to the old timeout default you could set (note that doas explicitly said they won't implement this too). The workaround is to run it in `sudo -i` fashion and not put a command afterwards which is reasonable enough even though it worked hard against my muscle memory + copypaste commands when switching over.

> Systemd gets a lot of hate

I'd argue it doesn't and is simply another victim of loud internet minority syndrome.

It's just a generic name at this point, basically all associated with init and service units and none of the other stuff.

https://man.archlinux.org/man/run0.1.en

impossible to have a clear picture of what's up with home dir, where is now located, how to have access to it or whether it will suddenly disappear. Obviously, plain /home worked for like five decades and therefore absolutely has to be replaced.

Learning curve is not the annoying part. It is kind of expected and fine.

systemd is annoying is parts that are so well described over the internet, that it makes it zero sense to repeat it. I am just venting and that comes from the experience.

never boot into the network reliably, because under systemd you have no control over the sequence.

BTW, I think that's one of the main pros and one of the strongest features of systemd, but it is also what makes it unreliable and boot unreproducible if you live outside of the very default Ubuntu instance and such.

To me podman/systems/quadlet could just as well be an implementation detail of how a k8s node runs a container (the.. CRI I suppose, in the lingo?) - it's not replacing the orchestration/scheduling abstraction over nodes that k8s provides. The 'here are my machines capable of running podman-systemd files, here is the spec I want to run, go'.

I believe the podman-compose project is still actively maintened and could be a nice alternative for docker-compose. But the podman's interface with systemd is so enjoyable.

> Of course, as my luck would have it, Podman integration with systemd appears to be deprecated already and they're now talking about defining containers in "Quadlet" files, whatever those are. I guess that will be something to learn some other time.

    docker context create --docker 'host=ssh://user@remote-host' remote-host

    docker -c remote-host compose -f compose.yaml up -d --pull always
           ^^^^^^^^^^^^^^         ^^^^^^^^^^^^^^^

Also, another pro tip: set up your ~/.ssh/config so that you don't need the user@ part in any ssh invocations. It's quite practical when working in a team, you can just copy-paste commands between docs and each other.

    Host *.example.com
        User myusername