Nix is hard in the way that programming is hard. Not everyone gets over the activation energy to be successful. The ones that do don't regret the effort. Nix is complex because the problems it solves are high complexity problems that other systems don't solve. Docker is not a substitute for Nix.
The solution isn't to use a weaker tool, because the weaker tool doesn't solve your problem. It's not uncommon to see a programmer use a spreadsheet, but you wouldn't expect to see a programmer use a spreadsheet where a database is needed. And you don't see people trying to use garbage-collected languages to write operating systems, even though they are easier to use than C. It's perhaps inefficient when a tool is too powerful for what you need, but it's a fatal flaw if the tool you use is too weak for what you need.
Nix let's you control your dependencies in a way no other tool even attempts. I can pin and patch any combination of dependencies, even conflicting ones in single environment, with reproducible builds— I'm in control of every detail. I would never consider a downgrade from that, but I'm open to upgrades if something even more capable came along.
Yes and no. Fundamentally, I agree with you. Nix is hard because it solves an inherently complex problem. But it's also hard because it has some usability flaws.
For example, I recently wrote a bunch of Nix and got an error that boiled to me forgetting to add a "name" to a derivation. That is absolutely my bad. However, Nix didn't tell me which file this derivation was in (it said 'unknown file' iirc), the stack trace also wasn't any help . And so I went hunting through the hundreds of lines of Nix I just added to find where I might be missing a "name".
Should I have tested my code incrementely? Absolutely, yes! But Nix should also be able to tell me which file I am missing the attribute in.
I say all of this as a huge Nix advocate and fan. It's wonderful, it solves so many problems I previously had and I intend to keep using it, but it's far from perfect.
Yeah in many ways, Nix is a research project that escaped the lab. It’s the first tool of its kind that also happened to become successful. You have to write in this dynamically typed Haskell with poor affordances for being a build system and fragmented ways of addressing pinning.
PNPM (a package manager for JavaScript/NPM ecosystem) does this with much better usability:
> let's you control your dependencies in a way no other tool even attempts. I can pin and patch any combination of dependencies, even conflicting ones in single environment
It does so with much lower user facing complexity than Nix. I like flakes because they’re trying to follow a similar model, but still quite hard to teach newcomers:
- The CLI should literally have the same verbs as PNPM, but needs a bunch of complicated flags to do the same actions.
- The nix expression language is intimidating and has kinda bad tooling out of the box. I would love to use Starlark/Python/Typescript instead. Even Guix’s scheme language is more appealing to me, although Guix is too niche and GNU focused to be practical.
- Nix package files evaluate down to configuration for the Nix package manager, but I haven’t ever seen a good explanation for the basic essentials underneath all the abstraction. Every guide I’ve learned from and all the package defs I’ve read seem to cargo cult many layers of mysterious config composing config. Without easy to learn essentials it’s difficult to grok the system as a whole.
I would love to adopt Nix for developer tooling for Notion’s engineers, but today it’s about infinity times easier to work around the limitations mentioned of Docker+Ubuntu+NPM than to work around the limitations of Nix.
I don't think you can ever get Nix as simple as PNPM, simply because native libraries are sometimes annoying, need to be configured at build time to a greater degree and because the problem space it attacks is so much larger than PNPM, which only deals with the JS/Node.js ecosystem.
However, I do think that there exist reasonable levels of abstraction that sacrifice some expressive power for simplicity and such systems could maybe expose a PNPM-like CLI. One example that comes to mind is devenv.nix [1]. While it doesn't yet have a CLI, its configuration file is YAML and relatively simple. I think there's more to be done in this space and I hope for tools that are easier to grasp in the future.
> Nix package files evaluate down to configuration for the Nix package manager, but I haven’t ever seen a good explanation for the basic essentials underneath all the abstraction. Every guide I’ve learned from and all the package defs I’ve read seem to cargo cult many layers of mysterious config composing config. Without easy to learn essentials it’s difficult to grok the system as a whole.
To me it sounds like the essential that you're referring to is the 'derivation' primitive, which is almost always hidden behind the mkDerivation abstraction from nixpkgs. This [2] blog post is an exploration of what exactly that means.
I'd also love for the documentation situation to be much better, in particular in terms of official, curated resources. But I'm not convinced that you actually need to know the difference between derivation and mkDerivation to make effective use of Nix, because in practice you would always use the latter. That said, mkDerivation and the whole of nixpkgs is essentially a huge DSL (I believe this is what you meant when you said 'config composing config') that you do need to know and is woefully underdocumented.
> I would love to adopt Nix for developer tooling for Notion’s engineers, but today it’s about infinity times easier to work around the limitations mentioned of Docker+Ubuntu+NPM than to work around the limitations of Nix.
One approach I have taken to is to specify the environment in Nix, but then generate Docker devcontainers from it, so most people don't come into contact with Nix if they don't want to.
I recognize this argument, but eventually it just becomes a wall of difficulty for people. I see the same problem with Gradle and Bazel. There's just something to be said about the work required to abstract fundamentally complex concepts into a simple and grokkable API that these systems just don't manage to get right.
People generally want to program to accomplish goals, and they want to minimize the amount of setup time and yak-shaving needed to start working on those goals.
Nix does solve a lot of complex problems and gives you a lot of control over your dependencies. But if the complexity to start using Nix for your project is too high, then you're going to opt for a less powerful solution that lets you get stuff done.
I say this as someone who really loves Nix and the power it provides -- the UX is a huge challenge for adoption. It's better to confront that fact if we want Nix to succeed.
Nix solves packaging your application and its dependencies better than Docker does!
</qoute>
What am I missing? They just said that Nix solves the last two and the first one better than docker does.
> Running your container in a secure, multi-tenant fashion is definitely one of the problems Docker solves
Docker does not solve anything related to a secure multi-tenant environment. Docker can provide isolation between mostly trusted parties. Anything running on the same kernel must not be considered securely isolated.
So since I'm not getting security anyways running services in Nix is simple and provides the isolation that I care about. Applications aren't going to accidentally break each other or cause dependency hell. I can use UNIX users if I want or even use simple containers like systemd-nspawn. But none of these are secure.
I agree that Docker provides nice UX, that is why it won. It made it easy to get something that works and can run fairly reliably across machines. It has flaws, especially reproducibility, but it works and is relatively easy to understand.
I sometimes wonder if the Nix stdenv does too much. It is optimized for running configure and make for you but ends up being a lot of complexity that most people don't need with different phases and hooks. If you just use `pkgs.runCommand` you actually get a very simple docker-like experience where you just run commands and copy your build result to the output directory. Plus there is no messing around with build images vs output images to get small results.
> I sometimes wonder if the Nix stdenv does too much.
Yeah, NixOS is a Linux-based OS, and historically most of the system stuff is C/C++. So `stdenv` is just that - an abstraction for "a standard Linux environment with a working C toolchain to build system packages". I also tend to just use `runCommand` when I can, but even when building things like Rust, Python or JS, quite often there's some little plugin/package that needs to be built with a C compiler, and then `stdenv` comes handy.
The main thing that made understanding the language much easier was the realization that you arent actually looking at imperative commands. Nix is just a big lazily-evaluated JSON blob. All of the language is just commands to manipulate the values of that JSON. Once you understand that, I think its more straightforward (except some funky syntax like `//`)
Agreed. For me it helped to think about how modules are composed--something about the order-invariance aspect of it made me understand why this was a good language choice. Suppose you were actually using JSON. You've got
X = {a:1, b:2}
and
Y = {b:3, c:4}
In imperative land, if you start with X and then apply Y, "b" will be 3. And if you start with Y and then apply X "b" will be 2. That's bad if you're a package manager, you don't want the order that packages are installed to change their function.
The imperative solution is to add some kind of metadata which indicates whether X and Y are compatible, and to consult some kind of solver which prevents you from even trying to do the thing that would create the order-dependence nightmare.
But because of how the Nix language works, you never even get to the point where you're clobbering things. Nor do you have to consult a version solver. The fact that you've tried to make "b" both 2 and 3 is a language error. It's as close as you can get to making config errors syntactically invalid and helps you sidestep a whole domain of trouble.
It also means that pretty much all problems are code problems, and that's a dimension that we know how to collaborate in. sorta.
My biggest frustration with Nix is the lack of typing. Writing anything more than the basics, I feel like I quickly run into an issue of not being able to reason easily about what structures I'm manipulating, and I haven't gotten very far by trying to lean on the editor I'm using either.
There's some prospect of Nickel (https://github.com/tweag/nickel) meeting this need, though haven't tried it and can't really vouch.
Typing might be really helpful for building your own layers up, but I don't know if it can meaningfully help with problems around consuming existing Nix ~APIs.
Something will evolve from that space which doesn't have the flaw. Whether that thing will be called Nix or not, I don't know. I think that means that it's not a fatal flaw, it's just a regular flaw.
I'm not convinced that the upsides of Nix don't necessarily include the downside.
Docker files are basically "copy these files in and run these commands with full network access" which is a recipe for both "easy" and "not reproducible"
Oh, I need python? "RUN apt-get install -y python" done. Which python version is installed? I don't have to think about that.
If you want to ensure the inputs are the same, then you need to impose structure. That structure is a large part of what is confusing with Nix.
I hate the Nix language as much as the next-guy, but I don't think that if it were Javascript instead, everyone would be like "oh this is so easy!"
It's no more difficult than packer. The challenge is architectural, because a lot of times (and I am guilty of this as a developer myself) the infra architecture comes later or after the fact.
A lot of what makes declarative stuff nice is it comes with an implicit opinionated standard way of doing stuff. Nix tries to be way more general to the point of a whole programming language.
Where everything is closer to true configuration not declarative programming. Apt and PyPi packages in, snap package out, install it like it was an exe or put it in an app store, give it some permissions, done.
They don't introduce any new concepts and there are few intermediate extra steps, it's all more of less pretty common ideas found in any tech workflow.
I totally agree. I love and hate Nix every day that I use it. I hope that someday they'll make it more user friendly, but I suspect it'll be someone else with a new approach.
I struggle to see how another tool will take the spot. I think the power of Nix is that the underpinnings are so pure, academic and well thought out that you can rethink package management in a much cleaner and sane way. Unfortunately that results in the UI not being the priority, though I think the opposite approach results in a conventional package manager with their many flaws and trade offs.
I generally think the UI first approach is a better way of building tools, but there are special cases where it doesn’t work due to their being so many edge cases that you need pure underpinnings to make it actually work (e.g. version control).
Docker is easy to use because it leverages something that most programmers already know - how to run commands in the shell.
Writing a Dockerfile is mostly just figuring out which commands you need to execute, and writing them down.
Writing a Nix package is mostly just scratching your head at incomplete documentation, and hanging out at Nix IRC channel hoping someone will help you.
I don't think so, not from what I'm reading. Not fully. Just comparing with Docker is a sign of limited understanding. Nix is not really a Docker competitor. You can run Nix stuff in a Docker, and it can build you a docker container image as well. Orthogonal. It's just so happen that basic applications of Nix overlap with what Docker is often used for, so comparing the two might seems natural. But Nix is more and more fundamental: Nix is a shared language for software composition. Nix creates a Linux OS for me, prepares custom ISO with my fav. stuff built-in, prepares my home dir, maintains my servers, configures services I use, makes me a dev environment I can share inside my project, future-proofs ad-hoc scripts I wrote, builds me docker containers, builds rpm&debs, makes me cross-compiling toolchains, and so much more.
Anyway, Nix does have an usability problem, but it's not fatal.
It's not worse than usability of e.g. git, or even usability of Docker.
Yeah. Docker. People now consider Docker a bread and butter of SWE, but there are still plenty of people that don't know how to use it, or need lots of help using it, and not much more than 5 years ago you'd have to drag your team and explain to everyone why it's beneficial to use it. And when things go south it takes quite a bit of understanding of the under-the-hood machinery to figure things out.
And there are plenty of devs who know like 5 commands to handle git + github UI and when anything goes wrong need help. And people complain about UX of git every week on HN.
Usability is 90% familiarity. Once you have enough people who know X well, they help people who don't and live goes on.
Once you build a good mental model of Nix, it's actually very simple, elegant, natural and very usable. The error messages and other-UX-stuff sometimes suck, but it's surface level and fixable (thought requires lots of dev work).
At this point I can't even imagine working without Nix, and giving up on things it allows me to do. It's really like a super-power. And it's been a great accelerator for teams and project I've introduced it to. I could maybe see something even better just replacing it, but once you really get Nix, there's not going back.
It's not exactly a 1:1 comparison, but NixOS' tight and extremely simple integration with systemd and podman has made NixOS native containers a pleasure to work with. OP seems to maybe not be aware of them?
Readit News
The solution isn't to use a weaker tool, because the weaker tool doesn't solve your problem. It's not uncommon to see a programmer use a spreadsheet, but you wouldn't expect to see a programmer use a spreadsheet where a database is needed. And you don't see people trying to use garbage-collected languages to write operating systems, even though they are easier to use than C. It's perhaps inefficient when a tool is too powerful for what you need, but it's a fatal flaw if the tool you use is too weak for what you need.
Nix let's you control your dependencies in a way no other tool even attempts. I can pin and patch any combination of dependencies, even conflicting ones in single environment, with reproducible builds— I'm in control of every detail. I would never consider a downgrade from that, but I'm open to upgrades if something even more capable came along.
For example, I recently wrote a bunch of Nix and got an error that boiled to me forgetting to add a "name" to a derivation. That is absolutely my bad. However, Nix didn't tell me which file this derivation was in (it said 'unknown file' iirc), the stack trace also wasn't any help . And so I went hunting through the hundreds of lines of Nix I just added to find where I might be missing a "name".
Should I have tested my code incrementely? Absolutely, yes! But Nix should also be able to tell me which file I am missing the attribute in.
I say all of this as a huge Nix advocate and fan. It's wonderful, it solves so many problems I previously had and I intend to keep using it, but it's far from perfect.
https://nickel-lang.org
As far as more loosely related technology, there's also https://cuelang.org, which is... something different.
Powerful tools, but they’ll rip your arm off.
Learning nix, slowly, really enjoying it though.
> let's you control your dependencies in a way no other tool even attempts. I can pin and patch any combination of dependencies, even conflicting ones in single environment
It does so with much lower user facing complexity than Nix. I like flakes because they’re trying to follow a similar model, but still quite hard to teach newcomers:
- The CLI should literally have the same verbs as PNPM, but needs a bunch of complicated flags to do the same actions.
- The nix expression language is intimidating and has kinda bad tooling out of the box. I would love to use Starlark/Python/Typescript instead. Even Guix’s scheme language is more appealing to me, although Guix is too niche and GNU focused to be practical.
- Nix package files evaluate down to configuration for the Nix package manager, but I haven’t ever seen a good explanation for the basic essentials underneath all the abstraction. Every guide I’ve learned from and all the package defs I’ve read seem to cargo cult many layers of mysterious config composing config. Without easy to learn essentials it’s difficult to grok the system as a whole.
I would love to adopt Nix for developer tooling for Notion’s engineers, but today it’s about infinity times easier to work around the limitations mentioned of Docker+Ubuntu+NPM than to work around the limitations of Nix.
(Disclaimer: I work on garn)
[0] https://github.com/garnix-io/garn
However, I do think that there exist reasonable levels of abstraction that sacrifice some expressive power for simplicity and such systems could maybe expose a PNPM-like CLI. One example that comes to mind is devenv.nix [1]. While it doesn't yet have a CLI, its configuration file is YAML and relatively simple. I think there's more to be done in this space and I hope for tools that are easier to grasp in the future.
> Nix package files evaluate down to configuration for the Nix package manager, but I haven’t ever seen a good explanation for the basic essentials underneath all the abstraction. Every guide I’ve learned from and all the package defs I’ve read seem to cargo cult many layers of mysterious config composing config. Without easy to learn essentials it’s difficult to grok the system as a whole.
To me it sounds like the essential that you're referring to is the 'derivation' primitive, which is almost always hidden behind the mkDerivation abstraction from nixpkgs. This [2] blog post is an exploration of what exactly that means.
I'd also love for the documentation situation to be much better, in particular in terms of official, curated resources. But I'm not convinced that you actually need to know the difference between derivation and mkDerivation to make effective use of Nix, because in practice you would always use the latter. That said, mkDerivation and the whole of nixpkgs is essentially a huge DSL (I believe this is what you meant when you said 'config composing config') that you do need to know and is woefully underdocumented.
> I would love to adopt Nix for developer tooling for Notion’s engineers, but today it’s about infinity times easier to work around the limitations mentioned of Docker+Ubuntu+NPM than to work around the limitations of Nix.
One approach I have taken to is to specify the environment in Nix, but then generate Docker devcontainers from it, so most people don't come into contact with Nix if they don't want to.
[1] https://devenv.sh
[2] https://ianthehenry.com/posts/how-to-learn-nix/derivations/
Deleted Comment
Nix does solve a lot of complex problems and gives you a lot of control over your dependencies. But if the complexity to start using Nix for your project is too high, then you're going to opt for a less powerful solution that lets you get stuff done.
I say this as someone who really loves Nix and the power it provides -- the UX is a huge challenge for adoption. It's better to confront that fact if we want Nix to succeed.
On Ubuntu you can make something work, then ship it in a box and it works anywhere, with some sandboxing. What more would I want?
Reproducible builds are nice, but I don't currently do any significant compiled language work aside from embedded.
1. Docker Build
2. Docker Run
3. Docker Hub
Nix solves the last two.
Nix solves packaging your application and its dependencies better than Docker does!
</qoute>
What am I missing? They just said that Nix solves the last two and the first one better than docker does.
> Running your container in a secure, multi-tenant fashion is definitely one of the problems Docker solves
Docker does not solve anything related to a secure multi-tenant environment. Docker can provide isolation between mostly trusted parties. Anything running on the same kernel must not be considered securely isolated.
So since I'm not getting security anyways running services in Nix is simple and provides the isolation that I care about. Applications aren't going to accidentally break each other or cause dependency hell. I can use UNIX users if I want or even use simple containers like systemd-nspawn. But none of these are secure.
I agree that Docker provides nice UX, that is why it won. It made it easy to get something that works and can run fairly reliably across machines. It has flaws, especially reproducibility, but it works and is relatively easy to understand.
I sometimes wonder if the Nix stdenv does too much. It is optimized for running configure and make for you but ends up being a lot of complexity that most people don't need with different phases and hooks. If you just use `pkgs.runCommand` you actually get a very simple docker-like experience where you just run commands and copy your build result to the output directory. Plus there is no messing around with build images vs output images to get small results.
Yeah, NixOS is a Linux-based OS, and historically most of the system stuff is C/C++. So `stdenv` is just that - an abstraction for "a standard Linux environment with a working C toolchain to build system packages". I also tend to just use `runCommand` when I can, but even when building things like Rust, Python or JS, quite often there's some little plugin/package that needs to be built with a C compiler, and then `stdenv` comes handy.
X = {a:1, b:2}
and
Y = {b:3, c:4}
In imperative land, if you start with X and then apply Y, "b" will be 3. And if you start with Y and then apply X "b" will be 2. That's bad if you're a package manager, you don't want the order that packages are installed to change their function.
The imperative solution is to add some kind of metadata which indicates whether X and Y are compatible, and to consult some kind of solver which prevents you from even trying to do the thing that would create the order-dependence nightmare.
But because of how the Nix language works, you never even get to the point where you're clobbering things. Nor do you have to consult a version solver. The fact that you've tried to make "b" both 2 and 3 is a language error. It's as close as you can get to making config errors syntactically invalid and helps you sidestep a whole domain of trouble.
It also means that pretty much all problems are code problems, and that's a dimension that we know how to collaborate in. sorta.
https://dhall-lang.org/
Typing might be really helpful for building your own layers up, but I don't know if it can meaningfully help with problems around consuming existing Nix ~APIs.
Docker files are basically "copy these files in and run these commands with full network access" which is a recipe for both "easy" and "not reproducible"
Oh, I need python? "RUN apt-get install -y python" done. Which python version is installed? I don't have to think about that.
If you want to ensure the inputs are the same, then you need to impose structure. That structure is a large part of what is confusing with Nix.
I hate the Nix language as much as the next-guy, but I don't think that if it were Javascript instead, everyone would be like "oh this is so easy!"
Compare that with Snap packages: https://snapcraft.io/docs/python-apps
Where everything is closer to true configuration not declarative programming. Apt and PyPi packages in, snap package out, install it like it was an exe or put it in an app store, give it some permissions, done.
They don't introduce any new concepts and there are few intermediate extra steps, it's all more of less pretty common ideas found in any tech workflow.
I generally think the UI first approach is a better way of building tools, but there are special cases where it doesn’t work due to their being so many edge cases that you need pure underpinnings to make it actually work (e.g. version control).
Writing a Dockerfile is mostly just figuring out which commands you need to execute, and writing them down.
Writing a Nix package is mostly just scratching your head at incomplete documentation, and hanging out at Nix IRC channel hoping someone will help you.
I don't think so, not from what I'm reading. Not fully. Just comparing with Docker is a sign of limited understanding. Nix is not really a Docker competitor. You can run Nix stuff in a Docker, and it can build you a docker container image as well. Orthogonal. It's just so happen that basic applications of Nix overlap with what Docker is often used for, so comparing the two might seems natural. But Nix is more and more fundamental: Nix is a shared language for software composition. Nix creates a Linux OS for me, prepares custom ISO with my fav. stuff built-in, prepares my home dir, maintains my servers, configures services I use, makes me a dev environment I can share inside my project, future-proofs ad-hoc scripts I wrote, builds me docker containers, builds rpm&debs, makes me cross-compiling toolchains, and so much more.
Anyway, Nix does have an usability problem, but it's not fatal.
It's not worse than usability of e.g. git, or even usability of Docker.
Yeah. Docker. People now consider Docker a bread and butter of SWE, but there are still plenty of people that don't know how to use it, or need lots of help using it, and not much more than 5 years ago you'd have to drag your team and explain to everyone why it's beneficial to use it. And when things go south it takes quite a bit of understanding of the under-the-hood machinery to figure things out.
And there are plenty of devs who know like 5 commands to handle git + github UI and when anything goes wrong need help. And people complain about UX of git every week on HN.
Usability is 90% familiarity. Once you have enough people who know X well, they help people who don't and live goes on.
Once you build a good mental model of Nix, it's actually very simple, elegant, natural and very usable. The error messages and other-UX-stuff sometimes suck, but it's surface level and fixable (thought requires lots of dev work).
At this point I can't even imagine working without Nix, and giving up on things it allows me to do. It's really like a super-power. And it's been a great accelerator for teams and project I've introduced it to. I could maybe see something even better just replacing it, but once you really get Nix, there's not going back.