What's all the hype with Nix?

Learn
#8

Nix is a time sink and a constantly moving target of ecosystem trends (now it’s haskell.nix). The documentation is extensive, but also incredibly inaccessible. You won’t make anything work without reading through countless blog posts, anonymous gists and asking on reddit, github or IRC, just to wait for a nix guru 1 month later to have discovered the impossible: a bug. And one you need a couple of years experience to spot.

If you’re up for all that, go ahead. For me, the benefits absolutely do not outweigh the cost.

10 Likes
#9

Note that one could say the same to say, managing effects in Haskell.

Language support is still being explored and there are different takes. I do agree we have to do a better job at describing the differences.

I’m working on tutorials for all kinds of ways to get started with Nix at https://nix.dev/, but these things take time and effort.

19 Likes
#10

Further discussion about this topic on reddit.

2 Likes
#11

It took me a long time to warm up to Nix, but after six flawless major OS upgrades on my home and work laptops, which are always configured exactly the same way I might add, I’m completely convinced that Nix’s innovations represent the future.

So yes, it’s like Haskell for me: already good enough for production use, maybe never completely mainstream, but the epicenter of innovation for its domain.

10 Likes
#12

a bit like git, there is an elegant and clean core hidden underneath a, well, imperfect UI.

Ah, this is reassuring to hear. I used to hate git, but as soon as I (accidentally) came to understand its (simple but inexplicably hidden) model (via git log --graph --decorate --all --oneline) I immediately started loving it and now I use it with great pleasure.

If Nix also has an elegant and clean core then I am much more likely to dig into to. I would welcome the thoughts of others.

2 Likes
#13

It does! The two pieces are

  • There is a pure lazy untyped functional programming language. So, almost Haskell. The syntax is boring in some places, neat in others, and idiosyncratic in yet other corners. Like with any language.
  • Some values are special, namely “derivations”:
    • These values (I can’t say special type…) denote build instructions. Evaluating a nix expression to such a value is called instantiation.
    • These values can be realized. This executes the build instructions, and (if they don’t fail), produce an output path. These builds are hermetic, i.e. only stuff specified in the derivation can affect the output. Output paths are put in /nix/store, and are addressed by essentially the hash of the build instructions (not the output content).
    • Nix understands how derivations depends on storage paths, and how store paths depends on each other, so it builds what’s needed, can copy stuff including all referenced paths from caches.

That’s what I’d consider the elegant and clean core. All the rest (the sometimes obscure CLI, with old-style and new-style commands; nixpkgs with it’s concept of overlays based on recursive knots; stuff like haskell.nix) become manageable with that in mind.

9 Likes
#14

I love nix and use NixOS, but I must also say that nix has bad documentation, bad errors, no typechecking, really lacking debugging capabilities (to my knowledge) and generally a disorganized community. Still it is the best build system to date. So I think the solution is to replace the nix language with something (dependently) typed. I think nix will evolve a lot though.
My experience using it for haskell was not very uplifting. Rust was better. I think it needs a sort of nix protocol which makes it easier to integrate with excisting build tools like cargo or cabal.

5 Likes
#15

but I must also say that nix has bad documentation

There’s been quite some effort going into documentation lately, from RFC converting docbook to markdown to a bunch of tutorials being written.

We still have a lot of work to be done for sure and getting feedback what part of documentation is bad helps us prioritize what’s next.

bad errors

In Nix 2.4 (to be released soon) error reporting has been revamped thanks to all the donors and you can still donate and report errors to be improved.

no typechecking

That one probably deserves its own thread, but in short, most of the time evaluation error is still at build time, rather than at run time. So it’s not as bad as general purpose dynamic languages that fail at runtime. There is however research how we might have static types.

really lacking debugging capabilities (to my knowledge)

That’s something I’d love if we had a budget for! There’s a prototype that needs a lot of work. I’m hopeful as Nix gets more popular that eventually we’ll get resources to fix this.

generally a disorganized community.

Watch the news in the following weeks/months :slight_smile:

My experience using it for haskell was not very uplifting. Rust was better. I think it needs a sort of nix protocol which makes it easier to integrate with excisting build tools like cargo or cabal.

Did you try haskell.nix tutorial?

9 Likes
#16

Does haskell.nix offer a lot of benefits over the default haskell-nix infrastructure? I’ve seen the name fly by a couple of times, but it seems that it isn’t as often used.

#17

Yes. Official docs have some insight into it, although the main emphasis of that document should be that haskell.nix can map cabal plans or stack resolvers exactly into the same versions in Nix.

Default infrastructure ships with a flat list of packages, usually resolved from the latest LTS.

2 Likes
#18

Last time I looked into using nix for some new project, I stopped at this point. It seemed like someone made a big workaround for some failing in nix that I didn’t understand. That kind of thing is immediately off-putting, since now my development setup would depend on both nix being maintained and understood, and some third party’s workaround being maintained and understood. Whose docs do I read when something goes wrong, who do I file issues with? I’m probably making it into a bigger problem than it is, but I’ve experienced enough of these situations that it makes me wary.

OTOH, I do use nix-based IHP because they’ve done all the work for me and I didn’t have to think at all, just run two commands. They’re also very good at making Upgrading documentation. For IHP, I mostly don’t even notice nix being there. But that doesn’t help my non-IHP projects.

5 Likes
#19

I’ve had similar concerns, and ended up avoiding both Nix and haskell.nix for a while, and like yourself, I found the approach from IHP quite good, and almost seamless. I was also quite impressed with their deployment integration, all nix based, to their own “cloud provider”. That was for me the catalyst to dig further into it. I stumbled upon nixkell, and that was my starting point.

I must say, Nix is unnecessarily confusing, a lot of self-inflicted pain, especially by naming 3 things that are almost completely unrelated, Nix. Searching for solutions can be quite overwhelming, and easy to end up down the wrong rabbit hole. But once you get it working, it works really well. I think nix-shell, despite its ergonomics, is a game-changer for team work. The way Nix integrates with Docker can also be quite powerful, even if the documentation is sparse.

I ended up with the following setup (shameless plug), and it’s been working for me quite well. Starting new projects is easy, and ensuring that I can rebuild it and work on it regardless of machine setup (as long as it has Nix package manager installed), or that a coworker or someone else interested in a particular repository can do the same is very good.

That being said, I’m far from an expert on the subject, so take it with a grain of salt. :slight_smile:

3 Likes
#20

Personally, I’ve had a difficult time getting into nix itself, but I am finding NixOS a real pleasure. Even as a beginner, in daily, practical, production use, NixOS is no more effort to maintain than Ubuntu. However, NixOS is more consistent and easier to ‘figure out’, so it gets easier as you go, whereas Ubuntu/etc are inconsistent and moving targets, requiring a consistent (or growing) amount of effort to understand and maintain. For a solid experience, you’ll sometimes need to contribute to nixpkgs, but that’s easy and a great way to learn more about nix and get that experience. If you are interested in nix, and not interested in figuring it all out ASAP, I would recommend starting with NixOS as a way to “ease” into nix.

That all said, despite being a bit mysterious and similar to haskell in mentorship requirements, nix still seems (IMHO) to be best-in-class when it comes to a modern packaging solution that needs to encompass all aspects of building and managing project dependencies. If your project is not simple in that regard (dependencies), using nix to build your project is probably worth the expense.

9 Likes
#21

Let’s say you want to update some tool to a newer version in you project. You need to make sure everyone updates their version of the tool on their machine as well. With nix you just update the tool version in shell.nix and when your colleagues pull it down they will now also use the new version. Same goes for if you have multiple projects, it you want to try something out, just try it out for that project, you don’t need to worry about whether or not it’s compatible with the tools in the other project. I’ve seen it reduce the friction with experimenting and testing new things.

6 Likes
#22

I agree!

Pinning project dependencies is a great practice, regardless of the size of your team, and regardless of how you do it. Nix is a good way, and you can do it in other ways as well. Floating dependency versions will eventually bite you if you don’t pin those.

2 Likes
#23

Many good answers here, but there is another perspective too I would like to add:

For many of us, the usual situation where computers are full of random…stuff, opaque binaries that might be compiled open source but it’s not clear how, is an annoyingly messy situation. It feels like not really being in control of one’s machine, and that all these dark dusty corners make everything more complex that need be because it’s impossible to tell what is actually needed vs what is just there.

Nix and it’s ecosystem together is a solution to that problem by allowing when to have a “complete blueprint” for all the software on their machine.

Nevermind how Nix works, or that it was inspired by functional programing. I think this appeals to the same sort people who also get into Haskell in the first place. If Haskell inspires the dream that all the junk could be rewritten, Nix inspires the dream that we can at least clime on top of the current garbage heap to get a better vantage point!

8 Likes
#24

Not if you want bugfixes and security patches. That’s why PVP and semver were invented.

5 Likes
#25

Can I chime-in with a “to the core” explanation.

Linux ecosystem & IT as a whole - is a huge “bazaar” development process. It creates a structural mess as a biproduct. There are ~20 package managers & ~+100 central distribution software repositories.

The biggest repository that is famous for being usable & have almost all of the software packages - is Arch Linux sidekick AUR (Arch User Repository). AUR has 1 major design flaw - every package needs(encourages) a separate repository just for that package configuration, & so only one particular maintainer of that repository & package (the strong “bus factor” there - maintainer is not responding - nobody can do anything about the package), all package configurations are a separate configuration. It is a bit of a hassle to contribute to AUR - because every time the new repository must be cloned & the patch is on the mercy of 1 particular person in the world giving attention to the patch.

In the DevOps/SRE space - specialists need to learn/know several hundred tools & ~20 domain-specific languages to put all that together. Docker, Ansible (great language but not centralized), Hashicorp (HCL - great language), Kubernetes (its overengineering complexity of configuration & its repetitiveness & formatting - is what drove me away from DevOps, apart from learning & understanding the main classical DevOps tools language - Go) - it is just a beginning of what DevOps/SRE is about - a 50% of a job there is putting things together & keeping updates to those parts in sync with one another, it is a job largely about “bridging the gap” in structural disorganization.

Main DevOps/SRE invention is keeping infrastructure as a code.

& here comes Nix.
Nix language is (because of its scientific properties of it) - is a powerful language, it to put simply - infinitely programmable. So “infrastructure as a code” can be achieved (from the beginning - downloading, configuring, installing a package; to the end: managing a 1 000 node cluster deployment of servers with 100k “containers”, on cloud provider (& its services) running an enterprise software services that is the main source of company money). Nix language “laziness” allows to put all configuration for the world of software into 1 repository & grow the codebase there infinitely. So the world of people can pile-on unto 1 repository & polish & polish it & maintain it. Changing 1 structural thing in Nixpkgs - affects not 1 package, as in AUR - Nixpkgs maintenance allows to do a sweeping functionality introduction, compatibility changes & refactoring changes - across the whole world of computer software at once. The only caveat of it - is that Nix language & its complexity & complexity of a tooling, working with it & consequences of a design approach.

Nix gives DevOps/SRE a completely new perspective. No DevOps/SRE ecosystems except NixOS (NixPkgs) options - have that level of being a 1-stop candyshop for infrastructure automation deployment:

  1. Kubernetes: NixOS Search - Loading...
  2. PostgreSQL: NixOS Search - Loading...
  3. Grafana: NixOS Search - Loading...
  4. Haskell: NixOS Search - Loading... & Hoogle server: NixOS Search - Loading...

& It is not only DevOps/SRE options, those options are suitable for any deployments, so they also provide just as good Desktop configuration as well.

Those Options get a (now standardized) crowd-maintenance from a worl-wide network of according narrow-specialized professionals, & further general Nix code maintenance is done by NixPkgs maintainer teem as a side-gig of NixPkgs updates & refactors.

Nix puts Open Souce idea of “contributing through scratching one owns itch” - into a proper position, if someone does something - that is contributed to & immediately but gradually used by everyone - so the more people - the more correction & quality. Because of paradigms & context - “quantity transforming into quality” is the most strong here.

I also see NixPkgs as a main testing ground for open source projects being done right. On the integration of a package - if there is any assuming infrastructure/configuration hardcode - it would be found & reported upstream to be fixed. In that way NixPkgs (& Hydra) is a CI for compatibility & portability of software.

Nix actually has no competitors in DevOps/SRE field, no comparable products (besides GNU GUIX that sadly does not has the scale required & enterprise views contradict freedom of software license frequently). Only Nix combines multiple abilities that other solutions adopt into old paradigms through containerization, deduplication, FS snapshotting, jailed package bundling - but all those solutions is a patch-work that is even more diverse & complex than the Nix ecosystem, moreover that approach can not rely as strongly as Nix on crowd-source management of those solutions as one whole.

In business terms, Nix is not “just” “visionary” - it also has an objective scientific & technical superiority and solution got to the level - that further it only can grow and disrupt the market of IT infrastructure management & deployment.

To give some proof that it does a proper science-technical leap is: Graphs - Repology

The complexity of Nix maintenance & tooling is another topic - “errors were made” there & Nix ecosystem bears consequences & tries to solve them.

And Nix is a technical needle, also because of those “errors were made”. Companies can pick or classical DevOps, or Nix DevOps - having two at the same time require the ability to attract & hold a too highly skilled specialists that are few and far between & demand a lot from them. Once a business deployed NixOS infrastructure - in simple terms - it requires or to be torn down, or new deployments to be NixOS also & so hire NixOS DevOps’es. It locks-in companies hard on the stack.

So Nix has all needed to disrupt, seize & hold the market for a long time.

5 Likes
#26

The two work together beautifully! Or they could. Pinning doesn’t prevent rolling upgrades, it just ensures everyone/every environment where code is evaluated is rolling in sync.

3 Likes
#27

Would disagree.

Haskell works with Nix, only because Haskell maintainers in Nixpkgs are constantly churning hard at work in time.
Nixpkgs forms a Haskell set from Stackage with Hackage on top of it, & because Cabal & Stack configurations are different from Nix code - so Nixpkgs maintainers need to work quite a lot to keep haskellPackages set.

Also Hackage/Cabal versioning way & Nixpkgs way are different. For example - Cabal would align all tree of dependencies to use 1 version of every dependency, while Nix would not, NixPkgs designed & allows to use one library version for one package & another package in the dependency tree uses a different library version (& Haskell APIs/Hackage/Cabal system does not account for those) - if there are differences in library behavior/code - they can produce deviations.

Nixpkgs standardizes on 1 version & use of older/never versions introduce … work.

From my view the most default way of boot Haskell package into Nix system (considering Nixpkgs being the main software infrastructure) is something like: GitHub - haskell-nix/haskell-with-nixpkgs: Drop-in files ready to integrate your project with Nix ecosystem and give Nixpkgs Haskell Lib abilities for your development and CI.

Until there is a full-blown Haskell community Nix flake.

But people have as many ways to bootstrap Haskell projects - as Turing complete possible. As many projects I’ve seen/maintained - as much Nix bootstrapping setups I saw in projects (as now I also have mine “default one”).

IOHK was hard at work & provided GitHub - input-output-hk/haskell.nix: Alternative Haskell Infrastructure for Nixpkgs, that is a byproduct of their internal struggles & bootstrapping of Haskell projects is just a byproduct of a different agenda of a project & it bootstraps & builds project in a non-conventional way, afaik, does not have caching/build reuse & it does not relate to Nixpkgs infrastructure at all (at the time I studied it).

Bootstrapping to build 1 project in the repository - may be a low effort experience.
While maintaining sizable Nix infrastructure is in itself a full time work.

Nix upstream has a pattern to making implicit changes & approaching from a point that official Nix tooling is the only tooling there, for example, nix in update 2.3.14 to 2.3.15 rolled-out an inNixShell breakage change - that broke all nix shell/default.nix code - without marking mentioning it during release, in docs or changelog, etc… Then in very short order the 2.4 was rolled-out, which additionally broke quite a few things, behaviours in language changed without upstream changing languageVersion & without number of those changes being documented.

Which is why a section of upstream maintainers signed a request to change the status quo on this breakage processes: Nix 2.4, and what’s next? - Development - NixOS Discourse.

Nix should not be simplified - it is quite a complex thing in itself, Haskell & Nix can go together effectively, but there are quite a lot of Nix hurdles & big infrastructure would require a lot of effort & dedicated people on the task. Nix can be simple in project bootstrapping, maintaining middle-size infrastructures probably would be easier through classical means, while maintaining a large infrastructure again classical approach becomes high in maintenance, and so Nix should be easier and long term saves a lot of effort, when there are several people on the task & they can work with upstreams.

1 Like