Why we use the Nix package manager
At Grafted-In we’re seeking to bring a new level of software quality to the online marketing arena. As part of this effort, we’ve adopted a relatively young but powerful way of managing servers, systems, and software: it’s called Nix. In this post we want to discuss what the Nix package manager is, how it compares to other solutions, and why we picked it.
What Is Nix?
At its core, Nix is a package manager. If you’re a developer, you know what that means. If not, then let me explain.
A package manager is sort of like an advanced “app store.” Just like an app store, package managers have a large registry of “apps,” except they’re called “packages.” Anyone can look at the packages in the store. Anyone can download them. Anyone can even add new ones, if they know how.
A package manager, though, is more advanced than an app store because it manages more than just apps. The “packages” in a package manager can be apps, tools for building apps, images, music, books, you name it. A package manager’s job is to keep track of all this stuff and help people use it. But a package manager’s most important job is keeping track of how packages depend on each other. For example, the “Google Chrome” package actually needs other packages in order for it to work. It might need a package for downloading webpages, a package for encrypting HTTPS traffic, a package for supporting various languages, etc. When a package manager installs Google Chrome it needs to make sure that the user also gets these necessary tools and resources too. If any of these things change, then the package manager is responsible for helping the user upgrade his packages and their resources so that nothing breaks.
Still, there are lots of package managers around. If you’re on macOS, you could use Homebrew or MacPorts. On Windows you could use Chocolatey. Or depending on which variant of Linux you might be using, there are countless options: Aptitude, Yum, and Pacman are just a few.
But the Nix package manager is not just another one among the bunch. It takes a unique theoretical stance toward how things should be done. In technical terms, Nix’s stance is called “purely functional.” To most, those words mean very little, but in essence this means that Nix tries very, very hard to not ever change anything. Every package you install is frozen in time along with all of its necessary resources (called dependencies). While most package managers will allow packages to change on your system, Nix religiously refuses to allow this possibility. When even the tiniest detail changes in a package or any of its dependencies, it gets treated as an entirely new package. The old one remains unchanged. While this approach does require most people to rethink how they use their package manager, we’ll see that it comes with numerous big-deal benefits.
Nix packages are defined using a well-designed, turing-complete language, also called “Nix.”
What the Nix Package Manager Offers
In addition to the traditional features of a package manager, Nix offers several benefits not found in most of its competition:
Declarative Configuration and Reproducibility
With Nix you define your system setup with a clean, declarative configuration. This means you can always know exactly what’s installed on your system and how those pieces interact with each other. In the context of NixOS, this means you can write a complete specification of your entire system. In addition to making system management easier, having a declarative specification also means you have a blueprint of your setup that you can use to reproduce it identically as many times as you want. If you keep your configuration in a version control system like Git or Mercurial then you can even keep track of your configuration as it changes over time. These are an invaluable assets particularly to teams which often need to share configuration among members. Instead of teaching everyone on the team how to configure their computer (which is a painful experience), you can give them a reliable blueprint that will reproduce the configuration for them.
Declarative configuration is also extremely valuable in the context of building servers for your various applications. Anyone who has managed servers can understand the frustration involved in trying to keep dozens (or even thousands) of servers in sync and working seamlessly together. NixOps is a tool that will take a system specification and create servers to meet it exactly. To make changes you simply modify the blueprint and tell NixOps to make it so (this is sometimes called “Desired State Configuration”).
Yet another huge benefit of a reproducible system is that you can very easily try out alternative systems, say while building something new or testing out a change. If you don’t like your change, you revert the blueprint and everything goes back to how it was. NixOps, for example, takes advantage of this and allows you to build test servers in a virtual machine so you can try out important changes without ever needing to touch your production systems.
On a more technical note, the Nix package manager is able to offer fully declarative and reproducible systems precisely because it enforces strict determinism and refuses to mutate anything. Changes to your configuration are treated as a brand new configuration and nothing is actually modified underneath except for a pointer to your “current-state”.
Because Nix packages are fully deterministic and immutable (i.e. they never change), upgrades on your system actually perform almost no true in-place mutation. This is in stark contrast to most package managers which actually modify your system, often in ways that cannot be traced or undone. Nix upgrades, however, can be trivially undone, which is called a rollback. For example, say you installed a new driver on your system which ended up making your Internet stop working. In most packaging systems, you’d have to scramble around trying to find a way to reinstall the old driver, and without Internet! What version of the driver were you using before? Where did you get it? Does it install the same way? These are all questions you’d be frantically asking in a traditional upgrade. But with Nix (particularly NixOS), you have a much better option: simply rollback. When the operating system boots, you simply tell it to use the previous version of your configuration. No files get downloaded. No installations happen. The operating system simply uses the setup you had before!
In addition, Nix also supports another unique feature called “atomic upgrades.” This is a bit more technical, but it basically means that installations will never fail in the middle and leave you with a partial installation. Because most packaging systems are actively modifying your system in untraceable ways, a failure during the installation (like a bug or a sudden power outage) can leave you with a system that is terribly broken and in some undecipherable state between two worlds. With Nix, this doesn’t happen.
While this is obviously a nice feature to have for any system, it’s a dream-come-true for DevOps teams in managing servers. The ability to safely upgrade and rollback a large, complex server setup is rare and powerful.
Because of its philosophy, Nix actually lends itself to a more secure system. Since packages are totally immutable, attackers can’t hijack packages. It also means that many users can use the same system without breaking each other or gaining access to other user accounts. In fact, a huge benefit of Nix is that it actually provides an extremely satisfying and elegant solution to the “dependency hell” problem, which is a great source of frustration in software development.
Why We Picked Nix
First off, let’s be fair and explain some of the reasons that tempted us not to use Nix:
- No Windows support. Nix doesn’t work on Windows. Theoretically it could, but practically that has not happened and likely never will.
- Smaller community. Largely because Nix is so different, it doesn’t attract the popular crowd. While popularity is a terrible metric for wisdom, it does come with some perks like polish, documentation, and integrations. The Nix community is very helpful and and kind but many features have less-than-awesome polish and documentation.
- Rough edges. Due to #2, Nix has some rougher edges. For example, it has particularly bad error messages and tools like NixOps could use some work to have more integrations and documentation.
However, we felt that the unique benefits of Nix outweighed these cons. At this point I’m confident that the benefits of Nix have doubly paid for the learning curve and adoption pains.
Ultimately we picked Nix for a few big reasons:
- Nix offers huge benefits to system management in general.
- Nix allows us to easily test our systems.
- Nix allows us to keep fully working setups in version control for our clients. Since we work on many different websites and applications, maintaining consistent development and production environments is extremely taxing and error-prone. Nix gives us the ability to encapsulate every last detail of a client’s project and not risk losing anything.
- Nix allows us to kill several birds with one stone: development environments, testing rigs, system management, server management, and deployments!
- The Nix community (particularly the IRC channel, #nixos) is very kind and welcoming and helped us get on our feet.
- Nix does support both Linux and macOS, which means we can use the same package system on both. If we find that developing on Windows is important, we will likely set up a Docker container or Vagrant box to support our development setup. In the future, perhaps the Windows Subsystem for Linux will mature enough for Nix to work. (Cygwin could also be an option.)
And yes, this site is running on a NixOS server deployed with NixOps. In fact, we built a template for WordPress sites using Nix.
Elliot Cameron is the VP of Technology at Grafted-In LLC.