Okay I'm a Nix enthusiast but you'll have to trust me when I say that I'm not criticizing them for moving away from Nix; it isn't that strong of an emotional attachment. However, I'm not really sure I understand some of these complaints and they really could use more explanation. For example:
> The biggest problem with Nix is its commit-based package versioning. Only the latest major version of each package is available, with versions tied to specific commits in the nixpkgs repo.
While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well.
> no way of splitting up the Nix dependencies into separate layers
That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even.
> We also changed the codebase from Rust to Go because of the Buildkit libraries.
This part is not related to Nix, but I find it interesting anyways. Obviously most people don't transition programming languages on a whim, it's generally something you do when you're already planning on building from scratch anyways. To me it almost sounds like different people worked on Railpacks vs Nixpacks.
(I've definitely seen what happens when people not familiar with Nix wind up having to deal with unfinished Nix solutions within an organization. It is not pretty, as most people are unwilling to try to figure out Nix. I don't generally use Nix at work out of fear of causing this situation.)
> While Nixpkgs is an amazing resource, Nix != Nixpkgs.
If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix.
> That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even.
This isn’t “most users”, this is a large company building a product on top of Nix. I’m pretty sure most orgs using Nix at a minimum have a custom overlay.
If you identify these things as an issue, any competent engineer should find a variety of solutions with search and/or LLM assistance within an hour, since they’re not super obscure requirements.
I’m not saying Railway didn’t do this and realize that these common solutions weren’t viable for them, but it’s odd to not mention anything they tried to get around it.
Nixpkgs isn't Nix and in production you rarely just use Nixpkgs verbatim. It's trivial to overlay whatever versions you want (including forks), and I'd say it's expected for any company in production to manage their package set.
We are talking about a company full of professionals. If they need something obvious, simple, and default to manage their build - the core business function that turns their text into deployable artifacts - maybe there is a skill culture issue.
In the documentation[0] they're right next to each other, and `buildImage`[1] (builds a single layer) specifically calls out that you probably want to use `buildLayeredImage` or `streamLayeredImage`[2] (both produce a separate layer per dependency) instead.
Neither should cause the final image to include build dependencies, that sounds like they were doing something silly like running `nix-build` from inside a Dockerfile and just taking that as their final image. Which.. yes, would include build cruft. Oh,[3] I guess that was exactly what they were doing after all. And mixing in Debian packages... for reasons, I guess.
> If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix.
This is not what parent commenter is getting at. Nix itself is a deterministic build tool. There is also a package manager built on top which uses a large collection of nix files to describe each package - this is nixpkgs.
They use the same primitives, but the same way you don't just yolo your node/rust etc build versions to whatever your OS comes with and use a lock file, you also want to have more control over the exact versions and thus may use something other than what nixpkgs packages. Especially that it makes it easy to override any property of your dependencies, unlike any other tool out there.
Yes. There are two options IIRC, minimum layers and maximum layers (one per dep by default unless that makes too many, which is handled automatically) depending on what you want, and it’s a Boolean flag. If you need more control it’s more complicated but this one really is a strange criticism unless they’re using non-standard wrappers for the usual nix way to do this.
> If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix.
This feels rather dismissive. They wrote a bespoke solution, not a weekend toy. Surely you'd agree that they have more than just surface-level knowledge of Nix, to be able to distinguish between Nix and Nixpkgs? They're already doing non-trivial things by merging multiple commits of Nixpkgs in order to get different versions of different tools!
> Is this obvious, simple, and default behaviour?
Well, Nix doesn't do much of anything "by default", it's a pretty generic tool. But insofar as it matters, Yes, pretty much. `dockerTools.buildLayeredImage` will in fact automatically build a layered image, and it is the most "obvious" way (IMO) to build a docker image. There is also `dockerTools.buildImage` but there's no particular reason to use it unless you specifically want a flattened image. (The documentation available is clear enough about this. In fact, in practice, much of the time you'd probably actually want `dockerTools.streamLayeredImage` instead, which is also documented well enough, but that's beyond the point here.)
But that's not my point. As far as I know, Nixpacks don't even use this functionality, I'm pretty sure they wrote their own OCI image building tools. And in that sense, it is not obvious why they can't split the Nix store and the article doesn't explain it.
My point wasn't to be dismissive about the difficulties of Nix, it's that the blog post doesn't really do a good job of explaining things. It makes it sound like these are normal problems in Nix, but they are not; even the official Nixpkgs documentation often points to third party solutions for when you're working outside of Nixpkgs, since most of the Nixpkgs tools is geared for Nixpkgs and NixOS usage. As an example, take a look at this section of the Rust documentation in Nixpkgs:
So even if you're relatively new to Nix, as long as you are reading the documentation you will indeed definitely be aware of the fact that there is more to the Nix ecosystem than just Nixpkgs. It may not be surface-level Nix knowledge, but it's certainly close.
> While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well.
It’s so exhausting that every single time a basic use issue comes up with nix, the response is “but there’s a way to work around it (that you need tribal knowledge for and will require writing dozens to hundreds of lines of code to fix in a language that doesn’t work like any of the mainstreams with bad error messages and poorly documented standard libraries)”.
People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works.
If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead.
Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating.
Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.
You can use nixpkgs as a single versioned input, and you can also give certain things their own distinct input to be tracked separately in flake.lock, but I wouldn't describe either as a "workaround" for the other.
It is a bit of a headache to have to pick which mode you want for each dependency, but I'm not sure that's a headache that can be dispensed with via UX improvements.
> why even bother using nix instead of mainstream tools with good documentation?
nix lets you have a single source of truth for all of it. It's not nix vs apt or nix vs pip, but rather nix vs (pip & (apt|brew)).
So far as I know, the only other tool that scratches that itch is bazel, otherwise you're kind of stuck with multiple overlapping packaging strategies that don't communicate, which is a recipe for "works on my machine".
> It’s so exhausting that every single time a basic use issue comes up with nix, the response is “but there’s a way to work around it (that you need tribal knowledge for and will require writing dozens to hundreds of lines of code to fix in a language that doesn’t work like any of the mainstreams with bad error messages and poorly documented standard libraries)”.
> People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works.
Well, first of all, this article isn't even about the UX of using Nix directly, it is about Nixpacks, a tool built on top of Nix. If Nix already solved the problems Nixpacks were trying to solve, they would've had no reason to write it, so I don't really see how this could be relevant.
> If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead.
Would it matter if there was good documentation if you were not willing to read it anyway? Both of the issues I talked about actually are covered in the official Nixpkgs documentation and have been for years. For example, here is some of the documentation for layered docker images:
Nixpkgs and NixOS are maybe not perfectly documented, but on the other hand, they're absolutely enormous projects. The documentation that does exist is already staggering. And of course it is! Seriously, try finding a singular project that has as big of a scope as Nixpkgs...
Of course, what people seem to read from what I said is that Nix is actually perfect and there are no problems, but what I'm really saying is they didn't go into very much detail on how they tried to solve the issues they ran into. The two possibilities are that they didn't try very much, or that they did and they omitted it. The latter is certainly as plausible, but it leads to this confusing problem where their complaints don't really make much sense on their own. What I am talking about is not obscure bespoke tribal knowledge. It's pretty close to the first stuff you would learn if you read docs or tutorials.
> Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating.
> Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.
This really seems like it's veering far away from the discussion about Nixpacks and into personal grievances about Nix, but honestly there's thousands of build systems and package managers that don't work the way Nix does, I do not really see why Nix should compromise its ideals. for the sake of UX. But please don't get me wrong: I agree on the point that Nix has a worse UX than it could; I think the language is full of unneeded papercuts and confusing things and meanwhile the time it has taken to stabilize flakes and the new Nix command has really been a drag and introduced a lot of confusion. On the other hand, though, I'm not even sure it's worth wasting too many tears over this: as hard as it is to get started in Nix and as many things as there are that could be improved with Nix and Nixpkgs itself, really the chief pain I feel when dealing with anything involving Nix is not Nix itself but the world it exists in.
Nix has existed for over 20 years and in those years the world of OS design and package managers only really started to move towards immutability and purity relatively recently. Most software still likes to have tons of mutable, impure shared state and trying to encapsulate this into Nix can be very painful. Nix has grown many limbs and improved in many ways to try to deal with this, but it is still far and away one of the biggest sources of confusion that I have with Nix.
It is not a given that things have to work this way, but it is a natural consequence of the fact that Nix is trying to do something that is very much at odds with the way software has worked for a very long time. The impedance mismatch is massive and unavoidable, but I don't think the answer is that Nix should bend to deal with this. If someone finds what they feel is a better sweet spot between what Nix is today and the world outside of Nix, they should feel free to pursue that, but I'm involved with Nix because I think it has the right ideals just way too early.
Having dealt with systems that try to be hermetic like Bazel, you can certainly get some meaningful benefits from sitting at a midway point, but what we're chasing are the benefits you get much closer to the end-game of hermetic systems, when hermeticity is enforced strongly across the entire system. This is about more than just being a convenient tool for developers and much more into the future of how systems are designed. And yeah, sure, when you try to build on top of this in today's world, it can be awkward, nobody is denying that, but a lot of this awkwardness is, unfortunately, a feature, and while clever Nix solutions may eventually exist for some of those problems (a lot of clever work is being done, with concepts like dynamic derivations) I really feel strongly that you shouldn't hold your breath.
And sure, by all means, switch to OCI tooling if you feel like it works better. It may very well actually be better for some use cases! But you literally can not replace most of what Nix is used for and can do with OCI tooling, it's ultimately a very small subset of what Nix is capable of overall.
I think the part that's easy to miss is that their users are devs who will want to specify their own dependencies and versions for arbitrary packages.
The way nix works with the way nixpkgs is structured, pinning a version of any package means pinning a commit of the entire nixpkgs tree. Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. It is also a leaky abstraction, so they will need to expose that to their users, who now may run into situations where they need to align various states of the nixpkgs repo when they just wanted to "yarn add new-fancy-nodejs-package-with-linked–native-deps".
Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.
> Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits.
> Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.
Nixpkgs isn't all-or-nothing. You're right that Nixpkgs itself rarely packages more than one version of something, but the standard approach for "language package managers" is that you use a tool like crate2nix[0] which automatically generates pinned derivations for all of your dependencies.[1] For system dependencies which aren't covered by your language package manager.. you're basically in the same position as for something like Debian: you can either pull it from Nixpkgs (and give up control beyond "which Nixpkgs am I on?") or you can write/fork your own package. Or you can pull specific packages from specific Nixpkgs checkouts and splice them into your "main" Nixpkgs version as an overlay (though this is definitely getting into the Weird Territory(tm)).
> The way nix works with the way nixpkgs is structured, pinning a version of any package means pinning a commit of the entire nixpkgs tree.
You can do that for a single package. For example, the latest version of linux-firmware is currently broken for my wifi cards so I pinned linux-firmware to commit dda3dcd3f of nixpkgs, whereas the rest of my software comes from the latest version of nixpkgs. There are multiple ways to accomplish this, but the way I did it was:
I've been told this when trying FreeBSD in regards to freebsd ports. pkg generally works fine for me, but one day I tried to go off the beaten path and compile vim with some custom USE flags (I forget what they are called in freebsd) in the ports section. It pulled down 20+ dependencies and each `make menuconfig` kept asking me "would you like any of these options" I selected a few that seemed reasonable, and lo-and-behold, package 16 out of 23 fails because "this-requires-that and that-needs-Fubar3.32.1 and Fubar3 is deprecated for Fubar4" and I just gave up. I get that the Core OS devs can't support all 10k+ packages, but they should also be very clear that if you actually try to use them (i.e. enable custom features, not just compile stock code) there's a high chance they won't work. Another option would be to yank them from the ports list if they don't compile and require at least some standard, independently-produced build to succeed before they appear in portsnap fetch.
I think this is pretty well stated. I'll add that while nixpkgs isn't nix, nixpkgs is kind of the good part. I use NixOS and for the first time in my life, I'm using the latest version of the Linux kernel on release day. That's pretty excellent. While I've come to tolerate Debian Stable in my old age, it is always like stepping a few years into the past ;)
The Nix language is something I could criticize for hours without getting bored, but it is what it is. It's old and they did the best they could and it's probably not worth changing. The Nix build system feels awfully primitive to me, often rebuilding stuff that doesn't need to be rebuilt for no good reason. (For example, my NixOS installer ISO has a ton of the build depend on the cmdline I pass to the kernel [just console=ttyS2,1500000n8], and so changing the speed of my serial port requires about 3 minutes of build time. It's goofy and makes me laugh, I'm not going to stop using Nix because of it... but it's also something that I wouldn't let happen in MY build.)
Nix for Docker images is, in my opinion, what it's the worst at. A long time ago, I was writing some software in Go and needed to add the pg_dump binary from Postgres to my container image. The infrastructure team suggested using Nix, which I did, but our images blew up from 50MB of our compressed go binary to 1.5GB of God Knows What. pg_dump is 464K. I ended up doing things my way, with Bazel and rules_debian to install apt packages, and the result (on top of distroless) was much cleaner and more compact. My opinion with some actual Nix experience is that a Nix system always ends up being 1.4GB. My installer ISO is 1.4GB. My freshly installed machine is 1.4GB. That's just how it is, for whatever reason.
Finally, the whole "I would like to build a large C++ project" situation is a well worn path. s/C++/Rust doesn't change anything material. There are build systems that exist to make the library situation more tolerable. They are all as complicated as Nix, but some work much better for this use case. Nix is trying to be a build system for building other people's software, supporting nixpkgs, and lands on the very generic side of things. Build systems that are designed for building your software tend to do better at that job. Personally, I'm happy with Bazel and probably wouldn't use anything else (except "go build" for go-only projects), but there are many, many, many other options. 99% of the time, you should use that instead of Nix (and write a flake so people can install the latest version of Your Thing with home-manager; or maybe I'm just the only person that uses their own software day to day and you don't actually need to do that...)
That's strange, I never had problems building really tiny docker (release) images with nix, in fact it felt easier than doing it with alpine. You just get exactly what you specify, no more.
(OTOH, when developing in nix, I always end up with a huge /nix/store and have no idea how to clean it without garbage collecting everything and having to wait all over)
The version selection part sounds weird. The versions in nixpkgs make sense when you're running/building the system. If you're providing runtimes/compilers as a platform, you really want something like what devenv does - provide the versions yourself. You really don't want to end up building an old system to provide an old nodejs - you're leaving security patches in dependencies behind. Devenv does it for example through https://github.com/cachix/nixpkgs-python "All Python versions, kept up-to-date on hourly basis using Nix."
> Railway injects a deployment ID environment variable into all builds.
They could've done it in the next layer after installation. Also, you can split packages into different layers. There's even automation for it if you need batches to keep the number of layers down.
> we don’t have any problem with Nix itself. But there is a problem with how we were using it.
A good example of 'use the right tool for the right job'. Nix is great for some use cases and awful for others. The problem is the Nix learning curve is so high that by the time you've grasped it enough to make a decision you feel you've invested too much time to back out now and pivot to something else so you try to shoehorn it to solve the original need.
I feel it like that as well, but in some ways Nix is a more normal programming paradigm than other OS’s. We’re just not used to thinking about an OS that way. Nix expressions have inputs (a package repo, lots of key-value pairs) and outputs (a Linux system). Idk perhaps in a couple of years it will be much more normal.
Ie it is very easy for an AI to create a to-spec shell.nix (some Python packages, some Linux packages, some env vars, some path entries etc), or configuration.nix because of this paradigm.
I do this a lot to include envs with repos that fully support the package. It would probably be more reproducible with flakes (a flake.nix is like a shell.nix but with version pinning… or something, I’m still climbing that learning hill).
I often find it takes about 10 minutes of my time to package your average python package into nix, or 1 minute if its pyproject.toml based (uv/poetry projects!.
There is a long tail though (cough weasyprint cough).
Looks like they are trying to force versions into where there are none. Just like trying to force a square cube into a round hole.
"Default versions" breaking things that depend on them? What is that? It is like using docker's ":latest" tag and being surprised each time that a new server falls on its face because the "default" image is actually a different version from the previous "default" image.
I don't understand any of the explanations in this blog post. Seems like people who have zero clue about what a "version" of a software is.
"no way of splitting up the Nix dependencies into separate layers" - Why? Of course you can split /nix/store into as many layers as you need. Do they even know how to use containers and how to use Nix in the first place?
With the clear incompetence of these people, no wonder that their proposed solution smells like a decomposed fish.
Classic NIH syndrome. There is going to be no surprise to see them meet the exact same problems they didn't solve with Nix to infest their new "solution".
I’m all for not using nix, especially where it doesn’t make sense. But rebuilding a working system from scratch, for reasons that aren’t actually a problem if they were to have spent even a couple hours looking at how people already solve those issues, seems fundamentally insane.
Like others have said here, nix2container and flakes seem like they would address every problem they have.
With regard to versioning, I have flakes written 3 years ago that still build with exactly the same versions and the same output as when it was first written.
Sure does sound like that want to go to market and raise off a platform :D
Edit: Literally just checked nixpacks’ github and it immediately jumped out to me that they are using rustPlatform in nixpkgs, not oxalica’s rust-overlay[0] which would have come up in any cursory search for the rust issues. And is one of the most useful and powerful overlays I’ve used.
Yea, i kinda hate Nix - but it's fundamentally changed how i view OSs and the minimum features i want from them.
Though I really want another language than Nixlang, but i've been stuck on Nix for years despite not liking it because the concrete builds are just so good. If Nix actually had a language i liked i'd probably go full immutable user config too and fully embrace nix. It's just so, so good.
I'd switch in a heartbeat if someone iterated on Nix for some of my complaints, though. But i'm not switching away from the concrete system builds.. it's just amazing.
I don't think any VC worth the time is going to sit around nitpicking how much Nix matters to their offering if they're making increasing amounts of money.
> What works better to raise VC (which for many is the goal)[?] A nix wrapper or a deployment platform
Well, if I wanted VC money for this, even if I was buliding a simple nix wrapper, I'd still tell the VCs that it was a deployment platform. (to be clear I have no opinion on TFA or any idea whether this is what they're doing)
For instance, if your images use bash, you can explicitly create a layer containing the bash closure. This layer can then be used across all your images and is only rebuild and repushed if this bash closure is modified.
> > pull in dependencies often results in massive image sizes with a single /nix/store layer
This is the case for the basic nixpkgs.dockerTools.buildImage function but this is not true with nix2container, nor with nixpkgs.dockerTools.streamLayeredImage. Instead of writing the layers in the Nix store, these tools build a script to actually push the image by using existing store paths (which are Nix runtime dependencies of this script). Regarding the nix2container implementation, it builds a JSON file describing the Nix store paths for all layers and uses Skopeo to push the image (to a Docker deamon, a registry, podman, ...), by consuming this JSON file.
Just wanted to say thanks for nix2container. I’ve been using it to do some deploys to AWS (ECR) and my iteration time between builds is down to single digit seconds.
The main problem here is wanting to hang on to the "bespoke version soup" attitude that language package managers encourage (and is totally unsustainable). The alternative Mise doesn't appear to have any ability to understand version constraints between packages and certainly doesn't run tests for each installed package to ensure it works correctly with the surrounding versions. So you're not getting remotely the same thing.
Bespoke version soup is unsustainable, but part of why people keep doing it is that it tends to work fine. It tends to work fine in part because OS-level libraries come from a different, much more conservative world, in which breaking backwards compatibility is something you try to avoid as much as possible.
So they can take a stable, well-managed OS as a base, use tools like mise and asdf to build a bespoke version soup of tools and language runtimes on top, then run an app on top of that. It will almost never break. When it does break, they fiddle with versions and small fixes until it works again, then move on. The fact that it broke is annoying, but unimportant. Anything that introduces friction, requires more learning, or requires more work is a waste of time.
Others would instead look for a solution to stop it from breaking ever again. This solution is allowed to introduce friction, require more learning, or require more work, because they consider the problem important. These people want Nix.
Most people are in the first group, so a company like Railway that wants to grow ends up with a solution that fits that group.
Package maintainers often think in terms of constraints like I need a 1.0.0 <= pkg1 < 2.0.0 and a 2.5.0 <= pkg2 < 3.0.0. This tends to make total sense in the micro context of a single package but always falls apart IMO in the macro context. The problem is:
- constraints are not always right (say pkg1==1.9.0 actually breaks things)
- constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it
- even if you can use a given version, your package may have a hidden dependency on one if pkg1’s dependencies, that is only apparent once you start changing pkg1’s version
Constraint solving is really difficult and while it’s a cool idea, I think Nixpkgs takes the right approach in mostly avoiding it. If you want a given version of a package, you are forced to take the whole package set with you. So while you can’t say take a version of pkg1 from 2015 and use it with a version of pkg2 from 2025, you can just take the whole 2015 Nixpkgs and get pkg1 & pkg2 from 2015.
It's the idea that every application can near-arbitrarily choose a bespoke-but-exact mix of versions of every underlying package and assume they all work together. This is same attitude that leads to seemingly every application on planet earth needing to individually duplicate the work of reacting to every single dependabot update for their thousands of underlying packages and deal with the fallout of conflicts when they arise.
Packages in nixpkgs follow the "managed distribution" model, where almost all package combinations can be expected to work together, remain reasonably stable (on the stable branch) for 6 months receiving security backports, then you do all your major upgrades when you jump to the next stable branch when it is released.
Nix generally speaking has a global "nixpkgs" version (I'm greatly over-simplifying here ofc) in which there is a single version of each package.
This is likely the source of their commit based versioning complaint/issue, i.e the commits in question are probably https://github.com/NixOS/nixpkgs versions if they aren't maintaining their own overlay of derivations.
This is in contrast to systems that allow all of the versions to move independently of each other.
i.e in the Nix world you don't just update one package, you move atomically to a new set of package versions. You can have full control over this by using your own derivations to customise the exact set of versions, in practice most folk using Nix aren't deep enough in it for that though.
Put out fewer versions of things. It is entirely possible to write a piece of software and only change the interface of it at rare intervals. The best solution I can think of though would be to allow one version of a package to provide multiple versions of its interface. Suppose you want to increment the minor version number of your code and this involves changing the signatures of a number of functions, you could design a programming language packaging system such that both versions are defined in the same package, sharing code when needs be.
You can actually have both if you do it right. It's trivial to build a rust package with Nix from a Cargo.lock file, for example. Nixpkgs is contrary to bespoke version soup, but Nix itself can be fine with it.
In my experience as a DevOps/SRE, I feel like every time someone tries to have a system to manage dependencies etc, its goes one of two ways (I'll use Python as an example):
OPTION 1
"We'll have one big shared mono repo"
Pros:
- it's all in one place
- it's "batteries included"
- everyone uses the same one (so things like vulnerability issues are easy to fix)
Cons:
- someone always wants a special version
- hard to do tiered rollouts so changes tend to be big bang
- "But what about if we want to build a small docker version?"
OPTION 2
"Everyone gets their own conda/venv!"
Pros:
- Everyone gets exactly what they want
- Don't use packages they don't need
- Easy to upgrade in phases/tiers etc
Cons:
- "Wait, we have HOW MANY different conda environments??"
- Libraries from different groups may not be tested with the same Python libraries
- Vulnerability management is a nightmare b/c you don't even know were all of the different conda envs are.
The above is why I'm always skeptical of "This new way will fix it all!".
In short, "there are no solutions, only tradeoffs" gets more and more true the later I go in my career.
As someone with only a little experience with Nix, the points here don’t really seem right?
> This approach isn’t clear or maintainable, especially for contributors unfamiliar with Nix’s version management.
> For languages like Node and Python, we ended up only supporting their latest major version.
What is not maintainable about this? That they need to make a list of available versions? So, can this not be automated?
Furthermore, why is Railway defining how a user uses Nix?
Surely one of the points of Nix is that you can take a bare machine and have it configured with exactly what versions of packages you want? Why would Railway need to get in the way of the user and limit their versions anyway?
Or did I misunderstand and they don’t even expose Nix to the user? If so, the original question still stands: can’t they automate that list of package versions?
The version limits come from the fact that the Nix cache doesn't maintain older versions. So, if you use an older version, you will have to compile from sources. It sounds like they didn't want to take it upon themselves and provide a cache with older versions, even though it doesn't sound like much effort.
Honestly, the reasons given don't feel very solid. Maybe the person who introduced Nix left and the ones remaining didn't like it very much (the language itself is not very nice, the docs weren't great either in the past).
Still, I'm not familiar enough with the stack they chose, but does it provide a level of determinism close to Nix? If not, it might come to bite them or make their life harder later on.
Nix cache (cache.nixos.org) does in fact maintain older versions[0]. In fact, they maintain so much older stuff (binaries and associated source), that they are used in both research[1][2] and are having issues with gigantic cache size[3][4].
And yes, their reasoning implies NIH and just unfamiliarity combined with unwillingness to really understand Nix.
Nix cache does provide old versions. What they seem to want is old versions built with new versions of dependencies. That's also possible, but you will have to build things.
> The way Nixpacks uses Nix to pull in dependencies often results in massive image sizes with a single /nix/store layer ... all Nix and related packages and libraries needed for both the build and runtime are here.
This statement is kinda like “I’m giving up on automobiles because I can’t make them go forward”. This is one of the things Nix can do most reliably. It automates the detection of which runtime dependencies are actually referenced in the resulting binary, using string matching on /nix/store hashes. If they couldn’t make it do that, they’re doing something pretty weird or gravely wrong. I wouldn’t even know where to start to try to stop Nix from solving this automatically!
I wouldn’t read too much into their experience with it. The stuff about versioning is a very normal problem everyone has, would have been more interesting if they attempted to solve it.
To be fair to the authors, this IS a problem, albeit one they phrased poorly, especially with building docker images via nix. The store winds up containing way more than you need (eg all of postgres, not just psql), and it can be quite difficult to patch individual packages. Derivations are also not well-pruned in my experience, leading to very bloated docker images relative to using a staged Dockerfile.
Image size isn’t something we’ve focused a lot on, so I haven’t spent a ton of time on it, but searching for “nix docker image size” shows it to be a pretty commonly encountered thing.
you can not have a dockerfile in your project at all, push your code to them, and they’d build an image for it with nixpacks. you’d see nix stuff in your build logs, but it’s behind the scenes for the most part.
Readit News
> The biggest problem with Nix is its commit-based package versioning. Only the latest major version of each package is available, with versions tied to specific commits in the nixpkgs repo.
While Nixpkgs is an amazing resource, Nix != Nixpkgs. Nixpkgs is highly unideal for cases where you want to be able to pull arbitrary versions of toolchains, but it is not the only way to go. For example, there is amazingly good Nix tooling for pulling an arbitrary version of Rust. Other Nix-based developer tools have shown how you can do this well.
> no way of splitting up the Nix dependencies into separate layers
That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even.
> We also changed the codebase from Rust to Go because of the Buildkit libraries.
This part is not related to Nix, but I find it interesting anyways. Obviously most people don't transition programming languages on a whim, it's generally something you do when you're already planning on building from scratch anyways. To me it almost sounds like different people worked on Railpacks vs Nixpacks.
(I've definitely seen what happens when people not familiar with Nix wind up having to deal with unfinished Nix solutions within an organization. It is not pretty, as most people are unwilling to try to figure out Nix. I don't generally use Nix at work out of fear of causing this situation.)
> While Nixpkgs is an amazing resource, Nix != Nixpkgs.
If Nixpkgs is the default and alternatives require additional research and effort then for most users it _is_ Nix.
> That doesn't make any sense. You can literally just split them into separate layers in whatever arbitrary fashion you'd like. The built-in Nixpkgs docker tooling has some support for this even.
Is this obvious, simple, and default behaviour?
If you identify these things as an issue, any competent engineer should find a variety of solutions with search and/or LLM assistance within an hour, since they’re not super obscure requirements.
I’m not saying Railway didn’t do this and realize that these common solutions weren’t viable for them, but it’s odd to not mention anything they tried to get around it.
We are talking about a company full of professionals. If they need something obvious, simple, and default to manage their build - the core business function that turns their text into deployable artifacts - maybe there is a skill culture issue.
The industry is full of ineptitude though.
In the documentation[0] they're right next to each other, and `buildImage`[1] (builds a single layer) specifically calls out that you probably want to use `buildLayeredImage` or `streamLayeredImage`[2] (both produce a separate layer per dependency) instead.
Neither should cause the final image to include build dependencies, that sounds like they were doing something silly like running `nix-build` from inside a Dockerfile and just taking that as their final image. Which.. yes, would include build cruft. Oh,[3] I guess that was exactly what they were doing after all. And mixing in Debian packages... for reasons, I guess.
[0]: https://nixos.org/manual/nixpkgs/stable/#sec-pkgs-dockerTool...
[1]: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerToo...
[2]: https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerToo...
[3]: https://github.com/railwayapp/nixpacks/blob/205b33b515282cdf...
This is not what parent commenter is getting at. Nix itself is a deterministic build tool. There is also a package manager built on top which uses a large collection of nix files to describe each package - this is nixpkgs.
They use the same primitives, but the same way you don't just yolo your node/rust etc build versions to whatever your OS comes with and use a lock file, you also want to have more control over the exact versions and thus may use something other than what nixpkgs packages. Especially that it makes it easy to override any property of your dependencies, unlike any other tool out there.
Yes. There are two options IIRC, minimum layers and maximum layers (one per dep by default unless that makes too many, which is handled automatically) depending on what you want, and it’s a Boolean flag. If you need more control it’s more complicated but this one really is a strange criticism unless they’re using non-standard wrappers for the usual nix way to do this.
This feels rather dismissive. They wrote a bespoke solution, not a weekend toy. Surely you'd agree that they have more than just surface-level knowledge of Nix, to be able to distinguish between Nix and Nixpkgs? They're already doing non-trivial things by merging multiple commits of Nixpkgs in order to get different versions of different tools!
> Is this obvious, simple, and default behaviour?
Well, Nix doesn't do much of anything "by default", it's a pretty generic tool. But insofar as it matters, Yes, pretty much. `dockerTools.buildLayeredImage` will in fact automatically build a layered image, and it is the most "obvious" way (IMO) to build a docker image. There is also `dockerTools.buildImage` but there's no particular reason to use it unless you specifically want a flattened image. (The documentation available is clear enough about this. In fact, in practice, much of the time you'd probably actually want `dockerTools.streamLayeredImage` instead, which is also documented well enough, but that's beyond the point here.)
But that's not my point. As far as I know, Nixpacks don't even use this functionality, I'm pretty sure they wrote their own OCI image building tools. And in that sense, it is not obvious why they can't split the Nix store and the article doesn't explain it.
My point wasn't to be dismissive about the difficulties of Nix, it's that the blog post doesn't really do a good job of explaining things. It makes it sound like these are normal problems in Nix, but they are not; even the official Nixpkgs documentation often points to third party solutions for when you're working outside of Nixpkgs, since most of the Nixpkgs tools is geared for Nixpkgs and NixOS usage. As an example, take a look at this section of the Rust documentation in Nixpkgs:
https://github.com/NixOS/nixpkgs/blob/master/doc/languages-f...
So even if you're relatively new to Nix, as long as you are reading the documentation you will indeed definitely be aware of the fact that there is more to the Nix ecosystem than just Nixpkgs. It may not be surface-level Nix knowledge, but it's certainly close.
It’s so exhausting that every single time a basic use issue comes up with nix, the response is “but there’s a way to work around it (that you need tribal knowledge for and will require writing dozens to hundreds of lines of code to fix in a language that doesn’t work like any of the mainstreams with bad error messages and poorly documented standard libraries)”.
People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works.
If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead.
Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating.
Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.
It is a bit of a headache to have to pick which mode you want for each dependency, but I'm not sure that's a headache that can be dispensed with via UX improvements.
> why even bother using nix instead of mainstream tools with good documentation?
nix lets you have a single source of truth for all of it. It's not nix vs apt or nix vs pip, but rather nix vs (pip & (apt|brew)).
So far as I know, the only other tool that scratches that itch is bazel, otherwise you're kind of stuck with multiple overlapping packaging strategies that don't communicate, which is a recipe for "works on my machine".
> People’s problems with nix are not that it isn’t turing-complete, it’s that it often creates more problems than it solves by refusing to provide a simple first-class API that interoperates with idiomatic projects in that ecosystem that just works.
Well, first of all, this article isn't even about the UX of using Nix directly, it is about Nixpacks, a tool built on top of Nix. If Nix already solved the problems Nixpacks were trying to solve, they would've had no reason to write it, so I don't really see how this could be relevant.
> If every project you try to use nix for devolves into centering around trying to fix issues with nix that you have to write your own modules for, why even bother using nix instead of mainstream tools with good documentation? Exactly what happened in this case. In most cases people are probably just deciding to use docker instead.
Would it matter if there was good documentation if you were not willing to read it anyway? Both of the issues I talked about actually are covered in the official Nixpkgs documentation and have been for years. For example, here is some of the documentation for layered docker images:
https://nixos.org/manual/nixpkgs/stable/#ssec-pkgs-dockerToo...
Nixpkgs and NixOS are maybe not perfectly documented, but on the other hand, they're absolutely enormous projects. The documentation that does exist is already staggering. And of course it is! Seriously, try finding a singular project that has as big of a scope as Nixpkgs...
Of course, what people seem to read from what I said is that Nix is actually perfect and there are no problems, but what I'm really saying is they didn't go into very much detail on how they tried to solve the issues they ran into. The two possibilities are that they didn't try very much, or that they did and they omitted it. The latter is certainly as plausible, but it leads to this confusing problem where their complaints don't really make much sense on their own. What I am talking about is not obscure bespoke tribal knowledge. It's pretty close to the first stuff you would learn if you read docs or tutorials.
> Nix’s refusal to address practical developer experience issues for a developer-facing product on a non-geologic timescale in favor of ideological pure flakes is rather frustrating.
> Yes, people are contributing their own time, but it’s so damn frustrating to see so much technical effort going into something that’s rendered practically unusable because of bad UX.
This really seems like it's veering far away from the discussion about Nixpacks and into personal grievances about Nix, but honestly there's thousands of build systems and package managers that don't work the way Nix does, I do not really see why Nix should compromise its ideals. for the sake of UX. But please don't get me wrong: I agree on the point that Nix has a worse UX than it could; I think the language is full of unneeded papercuts and confusing things and meanwhile the time it has taken to stabilize flakes and the new Nix command has really been a drag and introduced a lot of confusion. On the other hand, though, I'm not even sure it's worth wasting too many tears over this: as hard as it is to get started in Nix and as many things as there are that could be improved with Nix and Nixpkgs itself, really the chief pain I feel when dealing with anything involving Nix is not Nix itself but the world it exists in.
Nix has existed for over 20 years and in those years the world of OS design and package managers only really started to move towards immutability and purity relatively recently. Most software still likes to have tons of mutable, impure shared state and trying to encapsulate this into Nix can be very painful. Nix has grown many limbs and improved in many ways to try to deal with this, but it is still far and away one of the biggest sources of confusion that I have with Nix.
It is not a given that things have to work this way, but it is a natural consequence of the fact that Nix is trying to do something that is very much at odds with the way software has worked for a very long time. The impedance mismatch is massive and unavoidable, but I don't think the answer is that Nix should bend to deal with this. If someone finds what they feel is a better sweet spot between what Nix is today and the world outside of Nix, they should feel free to pursue that, but I'm involved with Nix because I think it has the right ideals just way too early.
Having dealt with systems that try to be hermetic like Bazel, you can certainly get some meaningful benefits from sitting at a midway point, but what we're chasing are the benefits you get much closer to the end-game of hermetic systems, when hermeticity is enforced strongly across the entire system. This is about more than just being a convenient tool for developers and much more into the future of how systems are designed. And yeah, sure, when you try to build on top of this in today's world, it can be awkward, nobody is denying that, but a lot of this awkwardness is, unfortunately, a feature, and while clever Nix solutions may eventually exist for some of those problems (a lot of clever work is being done, with concepts like dynamic derivations) I really feel strongly that you shouldn't hold your breath.
And sure, by all means, switch to OCI tooling if you feel like it works better. It may very well actually be better for some use cases! But you literally can not replace most of what Nix is used for and can do with OCI tooling, it's ultimately a very small subset of what Nix is capable of overall.
The way nix works with the way nixpkgs is structured, pinning a version of any package means pinning a commit of the entire nixpkgs tree. Since package builds of node/python/ruby packages depend on stuff outside of the package dir in the tree, you need that mapping between versions and commits. It is also a leaky abstraction, so they will need to expose that to their users, who now may run into situations where they need to align various states of the nixpkgs repo when they just wanted to "yarn add new-fancy-nodejs-package-with-linked–native-deps".
Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.
> Using nix without nixpkgs may be fine for more scoped use but seems hard to justify for a platform like Railway.
Nixpkgs isn't all-or-nothing. You're right that Nixpkgs itself rarely packages more than one version of something, but the standard approach for "language package managers" is that you use a tool like crate2nix[0] which automatically generates pinned derivations for all of your dependencies.[1] For system dependencies which aren't covered by your language package manager.. you're basically in the same position as for something like Debian: you can either pull it from Nixpkgs (and give up control beyond "which Nixpkgs am I on?") or you can write/fork your own package. Or you can pull specific packages from specific Nixpkgs checkouts and splice them into your "main" Nixpkgs version as an overlay (though this is definitely getting into the Weird Territory(tm)).
[0]: https://github.com/nix-community/crate2nix
[1]: For example: https://github.com/stackabletech/secret-operator/blob/30f0eb...
You can do that for a single package. For example, the latest version of linux-firmware is currently broken for my wifi cards so I pinned linux-firmware to commit dda3dcd3f of nixpkgs, whereas the rest of my software comes from the latest version of nixpkgs. There are multiple ways to accomplish this, but the way I did it was:
1. Add an input of a specific version of nixpkgs (which I called `nixpkgs-dda3dcd3f`): https://code.fizz.buzz/talexander/machine_setup/src/commit/e...
2. Add an "overlay" which replaces linux-firmware in the regular nixpkgs with the copy from nixpkgs-dda3dcd3f: https://code.fizz.buzz/talexander/machine_setup/src/commit/e...
I've been told this when trying FreeBSD in regards to freebsd ports. pkg generally works fine for me, but one day I tried to go off the beaten path and compile vim with some custom USE flags (I forget what they are called in freebsd) in the ports section. It pulled down 20+ dependencies and each `make menuconfig` kept asking me "would you like any of these options" I selected a few that seemed reasonable, and lo-and-behold, package 16 out of 23 fails because "this-requires-that and that-needs-Fubar3.32.1 and Fubar3 is deprecated for Fubar4" and I just gave up. I get that the Core OS devs can't support all 10k+ packages, but they should also be very clear that if you actually try to use them (i.e. enable custom features, not just compile stock code) there's a high chance they won't work. Another option would be to yank them from the ports list if they don't compile and require at least some standard, independently-produced build to succeed before they appear in portsnap fetch.
The Nix language is something I could criticize for hours without getting bored, but it is what it is. It's old and they did the best they could and it's probably not worth changing. The Nix build system feels awfully primitive to me, often rebuilding stuff that doesn't need to be rebuilt for no good reason. (For example, my NixOS installer ISO has a ton of the build depend on the cmdline I pass to the kernel [just console=ttyS2,1500000n8], and so changing the speed of my serial port requires about 3 minutes of build time. It's goofy and makes me laugh, I'm not going to stop using Nix because of it... but it's also something that I wouldn't let happen in MY build.)
Nix for Docker images is, in my opinion, what it's the worst at. A long time ago, I was writing some software in Go and needed to add the pg_dump binary from Postgres to my container image. The infrastructure team suggested using Nix, which I did, but our images blew up from 50MB of our compressed go binary to 1.5GB of God Knows What. pg_dump is 464K. I ended up doing things my way, with Bazel and rules_debian to install apt packages, and the result (on top of distroless) was much cleaner and more compact. My opinion with some actual Nix experience is that a Nix system always ends up being 1.4GB. My installer ISO is 1.4GB. My freshly installed machine is 1.4GB. That's just how it is, for whatever reason.
Finally, the whole "I would like to build a large C++ project" situation is a well worn path. s/C++/Rust doesn't change anything material. There are build systems that exist to make the library situation more tolerable. They are all as complicated as Nix, but some work much better for this use case. Nix is trying to be a build system for building other people's software, supporting nixpkgs, and lands on the very generic side of things. Build systems that are designed for building your software tend to do better at that job. Personally, I'm happy with Bazel and probably wouldn't use anything else (except "go build" for go-only projects), but there are many, many, many other options. 99% of the time, you should use that instead of Nix (and write a flake so people can install the latest version of Your Thing with home-manager; or maybe I'm just the only person that uses their own software day to day and you don't actually need to do that...)
That's strange, I never had problems building really tiny docker (release) images with nix, in fact it felt easier than doing it with alpine. You just get exactly what you specify, no more.
(OTOH, when developing in nix, I always end up with a huge /nix/store and have no idea how to clean it without garbage collecting everything and having to wait all over)
> Railway injects a deployment ID environment variable into all builds.
They could've done it in the next layer after installation. Also, you can split packages into different layers. There's even automation for it if you need batches to keep the number of layers down.
A good example of 'use the right tool for the right job'. Nix is great for some use cases and awful for others. The problem is the Nix learning curve is so high that by the time you've grasped it enough to make a decision you feel you've invested too much time to back out now and pivot to something else so you try to shoehorn it to solve the original need.
Ie it is very easy for an AI to create a to-spec shell.nix (some Python packages, some Linux packages, some env vars, some path entries etc), or configuration.nix because of this paradigm.
I do this a lot to include envs with repos that fully support the package. It would probably be more reproducible with flakes (a flake.nix is like a shell.nix but with version pinning… or something, I’m still climbing that learning hill).
There is a long tail though (cough weasyprint cough).
"Default versions" breaking things that depend on them? What is that? It is like using docker's ":latest" tag and being surprised each time that a new server falls on its face because the "default" image is actually a different version from the previous "default" image.
I don't understand any of the explanations in this blog post. Seems like people who have zero clue about what a "version" of a software is.
"no way of splitting up the Nix dependencies into separate layers" - Why? Of course you can split /nix/store into as many layers as you need. Do they even know how to use containers and how to use Nix in the first place?
With the clear incompetence of these people, no wonder that their proposed solution smells like a decomposed fish.
Classic NIH syndrome. There is going to be no surprise to see them meet the exact same problems they didn't solve with Nix to infest their new "solution".
Like others have said here, nix2container and flakes seem like they would address every problem they have.
With regard to versioning, I have flakes written 3 years ago that still build with exactly the same versions and the same output as when it was first written.
Sure does sound like that want to go to market and raise off a platform :D
Edit: Literally just checked nixpacks’ github and it immediately jumped out to me that they are using rustPlatform in nixpkgs, not oxalica’s rust-overlay[0] which would have come up in any cursory search for the rust issues. And is one of the most useful and powerful overlays I’ve used.
[0] https://github.com/oxalica/rust-overlay
Though I really want another language than Nixlang, but i've been stuck on Nix for years despite not liking it because the concrete builds are just so good. If Nix actually had a language i liked i'd probably go full immutable user config too and fully embrace nix. It's just so, so good.
I'd switch in a heartbeat if someone iterated on Nix for some of my complaints, though. But i'm not switching away from the concrete system builds.. it's just amazing.
A nix wrapper or a deployment platform
I don't think any VC worth the time is going to sit around nitpicking how much Nix matters to their offering if they're making increasing amounts of money.
Well, if I wanted VC money for this, even if I was buliding a simple nix wrapper, I'd still tell the VCs that it was a deployment platform. (to be clear I have no opinion on TFA or any idea whether this is what they're doing)
nix2container [1] is actually able to do that: you can explicitly build layers containing a subset of the dependencies required by your image. An example is provided in this section: https://github.com/nlewo/nix2container?tab=readme-ov-file#is...
For instance, if your images use bash, you can explicitly create a layer containing the bash closure. This layer can then be used across all your images and is only rebuild and repushed if this bash closure is modified.
> > pull in dependencies often results in massive image sizes with a single /nix/store layer
This is the case for the basic nixpkgs.dockerTools.buildImage function but this is not true with nix2container, nor with nixpkgs.dockerTools.streamLayeredImage. Instead of writing the layers in the Nix store, these tools build a script to actually push the image by using existing store paths (which are Nix runtime dependencies of this script). Regarding the nix2container implementation, it builds a JSON file describing the Nix store paths for all layers and uses Skopeo to push the image (to a Docker deamon, a registry, podman, ...), by consuming this JSON file.
(disclaimer: i'm the nix2container author)
[1] https://github.com/nlewo/nix2container
So they can take a stable, well-managed OS as a base, use tools like mise and asdf to build a bespoke version soup of tools and language runtimes on top, then run an app on top of that. It will almost never break. When it does break, they fiddle with versions and small fixes until it works again, then move on. The fact that it broke is annoying, but unimportant. Anything that introduces friction, requires more learning, or requires more work is a waste of time.
Others would instead look for a solution to stop it from breaking ever again. This solution is allowed to introduce friction, require more learning, or require more work, because they consider the problem important. These people want Nix.
Most people are in the first group, so a company like Railway that wants to grow ends up with a solution that fits that group.
Care to elaborate what that means and what the alternative is?
Package maintainers often think in terms of constraints like I need a 1.0.0 <= pkg1 < 2.0.0 and a 2.5.0 <= pkg2 < 3.0.0. This tends to make total sense in the micro context of a single package but always falls apart IMO in the macro context. The problem is:
- constraints are not always right (say pkg1==1.9.0 actually breaks things)
- constraints of each dependency combined ends up giving very little degrees of freedom in constraint solving, so that you can’t in fact just take any pkg1 and use it
- even if you can use a given version, your package may have a hidden dependency on one if pkg1’s dependencies, that is only apparent once you start changing pkg1’s version
Constraint solving is really difficult and while it’s a cool idea, I think Nixpkgs takes the right approach in mostly avoiding it. If you want a given version of a package, you are forced to take the whole package set with you. So while you can’t say take a version of pkg1 from 2015 and use it with a version of pkg2 from 2025, you can just take the whole 2015 Nixpkgs and get pkg1 & pkg2 from 2015.
Packages in nixpkgs follow the "managed distribution" model, where almost all package combinations can be expected to work together, remain reasonably stable (on the stable branch) for 6 months receiving security backports, then you do all your major upgrades when you jump to the next stable branch when it is released.
This is likely the source of their commit based versioning complaint/issue, i.e the commits in question are probably https://github.com/NixOS/nixpkgs versions if they aren't maintaining their own overlay of derivations.
This is in contrast to systems that allow all of the versions to move independently of each other.
i.e in the Nix world you don't just update one package, you move atomically to a new set of package versions. You can have full control over this by using your own derivations to customise the exact set of versions, in practice most folk using Nix aren't deep enough in it for that though.
OPTION 1
"We'll have one big shared mono repo"
Pros:
- it's all in one place
- it's "batteries included"
- everyone uses the same one (so things like vulnerability issues are easy to fix)
Cons:
- someone always wants a special version
- hard to do tiered rollouts so changes tend to be big bang
- "But what about if we want to build a small docker version?"
OPTION 2
"Everyone gets their own conda/venv!"
Pros:
- Everyone gets exactly what they want
- Don't use packages they don't need
- Easy to upgrade in phases/tiers etc
Cons:
- "Wait, we have HOW MANY different conda environments??"
- Libraries from different groups may not be tested with the same Python libraries
- Vulnerability management is a nightmare b/c you don't even know were all of the different conda envs are.
The above is why I'm always skeptical of "This new way will fix it all!".
In short, "there are no solutions, only tradeoffs" gets more and more true the later I go in my career.
> This approach isn’t clear or maintainable, especially for contributors unfamiliar with Nix’s version management.
> For languages like Node and Python, we ended up only supporting their latest major version.
What is not maintainable about this? That they need to make a list of available versions? So, can this not be automated?
Furthermore, why is Railway defining how a user uses Nix?
Surely one of the points of Nix is that you can take a bare machine and have it configured with exactly what versions of packages you want? Why would Railway need to get in the way of the user and limit their versions anyway?
Or did I misunderstand and they don’t even expose Nix to the user? If so, the original question still stands: can’t they automate that list of package versions?
Honestly, the reasons given don't feel very solid. Maybe the person who introduced Nix left and the ones remaining didn't like it very much (the language itself is not very nice, the docs weren't great either in the past).
Still, I'm not familiar enough with the stack they chose, but does it provide a level of determinism close to Nix? If not, it might come to bite them or make their life harder later on.
And yes, their reasoning implies NIH and just unfamiliarity combined with unwillingness to really understand Nix.
[0]: https://discourse.nixos.org/t/how-long-is-binary-cache-kept-...
[1]: https://hal.science/hal-04913007
[2]: https://luj.fr/blog/is-nixos-truly-reproducible.html
[3]: https://discourse.nixos.org/t/nixos-foundations-financial-su...
[4]: https://discourse.nixos.org/t/the-nixos-foundations-call-to-...
> The way Nixpacks uses Nix to pull in dependencies often results in massive image sizes with a single /nix/store layer ... all Nix and related packages and libraries needed for both the build and runtime are here.
This statement is kinda like “I’m giving up on automobiles because I can’t make them go forward”. This is one of the things Nix can do most reliably. It automates the detection of which runtime dependencies are actually referenced in the resulting binary, using string matching on /nix/store hashes. If they couldn’t make it do that, they’re doing something pretty weird or gravely wrong. I wouldn’t even know where to start to try to stop Nix from solving this automatically!
I wouldn’t read too much into their experience with it. The stuff about versioning is a very normal problem everyone has, would have been more interesting if they attempted to solve it.
Image size isn’t something we’ve focused a lot on, so I haven’t spent a ton of time on it, but searching for “nix docker image size” shows it to be a pretty commonly encountered thing.
That's at least my understanding, yes.