My experience with NixOS is: if whatever I want to do is covered by the documentation, I just need to add the necessary enchantments (often verbatim) from the documentation and things _just work_. It certainly takes way less time to do compared to any distribution I've ever used. It's trivial to undo mistakes and I don't have to write down what I did - the code replaces my notes entirely.
However, if I can't find what I need in the documentation then it can be a problem. That's specially true for installing software. I'm not too familiar with the language to make entire new packages. Most things are using flakes and I haven't wrapped my head around them yet. Other things are not really compatible with its philosophy (like software packaged as Wine bottles).
Nix itself: same problem when the package doesn't exist. But when it does, it's wonderful. I'm even using it in OSX, instead of homebrew (with home-manager).
Hopefully this team will help smooth some of the rough edges.
A few of these problems are fixed with flakes - namely the dramatically improved composability. It becomes much simpler to understand how to use something when you are able to work on it in isolation. With the previous mono-repo nixpkgs assumption, you were basically required to not only learn nix-lang, but also nixpkgs and the nixpkgs workflow making it much harder to grasp the fundamentals.
With flakes, it is now significantly easier to create an standalone package or module for your specific needs since you import it just like nixpkgs. It also means that software doesn't have to be part of nixpkgs to be usable on the system - if a git repo has a flake it's just a matter of adding it as an input. Flakes represent such a massive improvement to the way that you interact with nix, I sincerely hope they become the default here soon. In the meantime, it is well worth your effort to learn them even if they aren't 'required' yet.
Unfortunately all of this is contingent on being able to grok the language in the first place which is where the documentation really falters. The nix pills talk about the basics of the language but it is very difficult to make anything useful without importing some libraries (i.e. nixpkgs) - and these libraries are not very well documented. I am aware of a documentation team being formed to help address this but it still remains the number one issue with nix, especially for newcomers. The unfortunate thing is that there are so many moving parts that it will take a considerable effort to explain the entire ecosystem without being overwhelming but still providing enough intuition for newcomers to get things done with Nix.
If anyone is interested, a recent SoN talk was given about efforts to improve the documentation. It touches on many of the things I bought up here: https://www.youtube.com/watch?v=WFRQvkfPoDI
I really hope to see a lot of packages move out of nixpkgs, only to be re-imported as flakes. There are currently 3.9k pull requests for nixpkgs; the current model really isn't working.
I don't know if packages should necessarily be moved out of nixpkgs since I doubt most people who don't use nix would want to maintain flake.nix in their project, nor would users want to have 30+ flakes in their inputs. However, I can see a good case for separating the package definitions from the nixos modules and the stdlib (mkDerivation, stdenv, etc). Maybe there could be a flake for each.
As for keeping up with PRs and other responsibilities, that seems to be one of the stated goals of the this new Nix Team. I guess we'll see how they intend to handle this soon.
Look at the ratio instead: 4k open vs 162k closed. Or 4k open vs 80k+ packages.
If we split things up, there will be less open PRs per project of course. But I doubt the ratio will be much better. It will actually increase the number of PRs as large scale refactors will be multiplied by the number of repositories.
I generally agree that the documentation is a problem. A lot of the code is commented, so if you can find a package in nixpkgs that is using the same language, you can usually do some copypaste, somehow find the function definition it's referencing, and then it's not so hard to make the changes to package your new package. But "somehow find the function definition it's referencing" is not always as easy as it sounds... I think it's the only way to learn though. The documentation just isn't up to the level it needs to be to be self-sufficient.
As for flakes, they're not that big a change. There's some entry point which you have to learn about - a special file (flake.nix) that defines inputs and outputs according to some predefined structure. But the output leafs are just normal nix code - it's just that there's some builtins that can't be used because they do side effects at runtime. But that's less important than it sounds, because if you're using nixpkgs (as a library/input or as a source for copypasting) then it's flake-safe.
This is generally true, but I have yet to successfully make a nix overlay for any haskell package not already in nixpkgs. I can't make heads-nor-tails of how the haskell packaging works, other than forking nixpkgs and rerunning the scripts that generate the packages from cabal.
I always wondered why there are no wine packages using nix. Sure, it is likely not legal, but I would absolutely love to have an MS office package that mandates a specific hashed file (that I can torrent and add to the nix store - this way I don’t even risk viruses), and the install itself is deterministically done (perhaps with some headless GUI clicking here and there).
Windows application installers generally are just binary blobs that can do as they please, so just pinning an installer will not imply that the resulting installation will be deterministic - a lot of installers by design change the global state of the system and download stuff during the installation process. If you want packaged on Windows, try choco.
For what it's worth most of the "packages" on chocolatey are just instructions for downloading those same installers and executing them in their respective silent modes. No one is repackaging those apps except for maybe some of the ones marked as "portable".
You are right in the first approximation, windows application installers can do whatever they want. However, many applications work already if you extract the files from the installer and put them into the appropriate directories, which you can make a deterministic process. There are tools for this, e.g. msiextract.
You could probably wrap Scoop packages into a Wine environment with Nix fairly easily, since all the packages in their main repos are well-behaved, portable, and install non-interactively.
Fantastic. This year I managed to get my development environment(s) with declarative, shared configurations almost completely managed by nix. This is the first success for me after trying to do this with multiple tools over the past decade.
I'm blown away by what Nix/NixOS has accomplished without having a formal team in place. Kudos to the maintainer(s), community and best wishes to the new Nix Team.
I'm glad this is happening. I was hoping that maybe the Nix/NixOS developers could have a dedicated documentation maintainer or two to make deeper learning about each system more accessible. Although I can make my way around NixOS, I feel that's only possible because I could look over the obscure parts of the configs of people more obsessed with Nix than me.
The old development style mentioned reminds me of that excessively mentioned essay about Lisp hackers keeping to themselves. I wonder if there's something intrinsic to certain developer-oriented projects like these that lead people to tinker with them on their own for long periods of time.
About 80% of Henrik Lissner's fame is Doom Emacs. The other 20% is his Nix config that people clone and then show up in the doom discord asking why their computer won't log in anymore.
Should I use flakes or not? Every single library I find is a Nix Flake, but insists that Flakes are experimental. Who makes the decision that they are not experimental? At what point does majority adoption of flakes overrule? Why are they still behind a feature flag in the CLI?
What is Nickel? Is this an officially endorsed project? It’s run by Tweag — do they run Nix? Most core contributors work for Tweag. What is the fate of the Nix language? Should I start investing in this new language?
Why is documentation still so bad? Why is there not a clear and official answer to documentation generation? Syntax linting? Testing? Who is in charge of trying to fix SEO for documentation?
Nix is technologically sound but is struggling organizationally from its explosive growth
> In this case, "it's experimental" should be read as "maybe the team will decide to make changes", rather than "it's not going to work half the time".
In that case, I can't in good conscience recommend we use such a feature at my workplace. We can't afford huge breaking changes, which is what that flag suggests. Which means my workplace is stuck with a bunch of `shell.nix` and `direnv` magic instead of clean flake-based envs.
It's not enough to gate features behind a flag, unless your only audience is enthusiasts. Versioning your features makes them accessible to a wider audience and encourages more risk-averse folks to adopt the features despite the possibility that they may undergo slight changes.
> Versioning your features makes them accessible to a wider audience and encourages more risk-averse folks to adopt the features despite the possibility that they may undergo slight changes.
just pin the nix package itself and that should ensure the flake API remains consistent for however many months/years you want?
That creates risk when we decide to upgrade, should bugs be fixed or new tools be implemented. Right now we are up to date and hope to keep it that way.
> That creates risk when we decide to upgrade, should bugs be fixed or new tools be implemented.
This is true of literally any dependency, though.
Semver is supposed to grant you confidence that things won't change very much, but it's not a technical guarantee.
In terms of "risk of adopting nix flakes", the lion's share of the risk is in "adopting nix" (because nix is difficult, hard to find people well versed in nix, etc.). The risks from adopting flakes on top of that are marginal.
> Which means my workplace is stuck with a bunch of `shell.nix` and `direnv` magic instead of clean flake-based envs.
if flakes win out, this other stuff might well be deprecated someday. timeline for that is probably lengthier than the timeline where flakes don’t win, but it’s still there. in either scenario you’re gonna have a month or more between one approach being deprecated and being dropped altogether (6 months if you’re on the stable channel). you’re implicitly betting for or against the success of flakes here, weighted by the cost of refactoring should you bet on the “losing” side plus maintenance until the winner emerges. you describe non-flake maintenance as being higher cost, so from the outside it just feels like a suspicious bet to make.
AFAIK you don't have to use flakes. Same as it's fine to only use shell.nix. No system is for eternity, nix builds are deterministically cached, often never reproduced, so I see no strong arguments against using direnv with shell.nix.
The Nix team doesn't allow any community proposals for making changes to Flakes because it is still 'experimental' and hasn't yet gone through an RFC. So it's 'experimental' in the sense that it might change, but the people who have suggested changes to the design are generally told that they won't discuss changes because it's not yet stabilized.
That kind of circular reasoning seems problematic.
Don't get me wrong, I like flakes, I use them for my NixOS setup, but the way that flakes development has been handled is very concerning.
> The Nix team doesn't allow any community proposals for making changes to Flakes because it is still 'experimental' and hasn't yet gone through an RFC.
You sound like you have something particular in mind.
> Well, that's the opposite problem of 'floundering'.
+1. The explosive growth has resulted in organic blossoming of several new ideas, approaches and templates. Not all of them are great and it takes time to sift through them to see if there are canonical approaches.
The creation of the team to streamline these ideas, bless some official paths is well timed.
>> Should I use flakes or not? Every single library I find is a Nix Flake, but insists that Flakes are experimental.
> In this case, "it's experimental" should be read as "maybe the team will decide to make changes", rather than "it's not going to work half the time".
This I think is part of the problem. People see the "experimental" label and don't understand that it's not "experimental" in the sense they're used to. Maybe labeling flakes as a "beta" feature and featuring the documentation more prominently would ease the confusion.
Agreed, they should just bite the bullet and commit to flakes as enabled and not "experimental". There are plenty of other things that break regularly with my nix deployment and it's never flakes.
The new Nix command syntax that was there up to 2.3 was always marked as experimental, but it worked fine for many years, I used it in many projects ... until it got deleted and replaced with flakes. I really hope they don't do that again. Flakes are de facto adopted by now, if they break it once more, it will fragment the ecosystem. Though there are rough edges where a breaking change would be beneficial in the long run.
> In this case, "it's experimental" should be read as "maybe the team will decide to make changes", rather than "it's not going to work half the time".
This is exactly the clarification that needs to happen. It hasn't yet, and we've been needing it. That's what I mean by "floundering". Nix isn't broken, but its development progress has been needing explicit direction and clarity.
That's why I'm excited to hear a team is being put together. This is a great opportunity to bring some decisiveness into Nix's development.
These questions highlight the overwhelm that users can feel as they're getting started with Nix. The Nix ecosystem is so replete with divergent possibilities that navigating it can absolutely play into paralysis by analysis, and lead users to hesitate to learn more because they don't know where to start or which competing tools in the ecosystem are more suitable or more likely to be 'the future'. That is an experience of floundering for sure.
But in terms of contributions, capabilities, and userbase, the Nix lately been subject to impressive and exciting growth that you mention. I think the best is yet to come, but it's already fair to say that Nix is flourishing.
The community and the codebases also do have growing pains, of course. But I don't think that Nix is floundering at all.
SEO for Nix is kinda hard unless they decide to change the name. I've resorted to searching github with `extension:nix`, the forum and the manual directly.
Not necessarily... Nix is compelling enough today that it could experience explosive growth even if core development was completely stuck. If that was the case, newcomers would still find enough value to stick around and spread the word, but existing pain points would go unaddressed.
I'm not close enough to Nix development to know whether this is the case or not in this instance, but the Discourse post makes a compelling argument about pull request statistics. Sibling commenters also point out that flakes have become a de facto standard despite being hidden behind an experimental flag, suggesting that the flag has lingered too long and without clear ownership or authority around stabilizing the feature. That indecision feels like floundering.
> Nix is compelling enough today that it could experience explosive growth even if core development was completely stuck. [...] I [do not] know whether this is the case
It felt like that to me for a really long time, in between the releases of Nix 2.3 and 2.4, when there were lots of features being added but there was no release schedule. The divide between flakes users and non-flakes users was also larger then, as the feature was gated behind not just an experimental flag, but pre-release versions of Nix.
Since then, Nix has gotten into a regular, fairly rapid release cycle and new functionality is released pretty often. There are various concerns about what that change does and doesn't fix for Nix, but for me at least there's no longer that feeling of stagnation or mystery about when Nix will see a stable update released.
> flakes have become a de facto standard despite being hidden behind an experimental flag, suggesting that the flag has lingered too long and without clear ownership or authority around stabilizing the feature. That indecision feels like floundering
Flakes have been conceived and pushed forward by the original author and chief maintainer of Nix. In the earliest stages of their development, consensus proved difficult to build and the community RFC process was very young. Suspending that RFC and adding the features to the master branch but marking them 'experimental' was an attempted compromise on the part of Eelco Dolstra, who had previously more or less taken a BDFL approach to maintaining Nix.
Since then, the flakes implementation has matured quite a bit. I think most users expect that it will evolve into an official feature soon and without many fundamental changes, although some community members still feel some bitterness about that initial process as well as skepticism about the scope and necessity of the whole suite of features that come with flakes.
At the same time, many RFCs have gone through the process since then. The organization of the projects and processes for getting ambitious changes approved are much more explicit now. In those ways, it's clear that leadership/governance in the Nix community is becoming more organized and more effective.
To ~rectify some problems caused by the really large gap between the release of 2.3 and the next minor/major, the release of Nix 2.4 in November 2021 marked a move to a 6-week release cadence. Since they're driven by a calendar now, some will inevitably be bigger/smaller.
despite the shortcomings and ambition of Nix, I've been amazed at how it's catching on. I see shell.nix files in tons of mainstream projects now, or Nix used to set up CI environments or package projects.
it gives me hope. I just wish the boilerplate were less arcane. Nix is a relatively elegant language, I'm not sure why nixpkgs is so ugly.
I'm trying to get Nix to compile software against host-system provided cuda libraries (and the libc those libraries are dependent on), for a Jetson ARM-based system. So I'm trying to create overrides for these basic libraries based on tar'ed system libraries. Has anyone tried to use Nix this way?
Yes! Although not for CUDA. I used opencl. nixgl helped here: https://github.com/guibou/nixGL. CUDA should be even simpler since you just add the CUDA .so paths to the RPATH. Allthough I might be missing something. Custom libc might be a whole can of worms though if it happens to be incompatible.
Nixpkgs is certainly the core of the ecosystem, but it's not at all the whole ecosystem.
Other important parts of the NixOS ecosystem include:
- NixOS, the main module system which adds many configuration management capabilities to Nix
- Nix-Darwin and home-manager, extremely popular module systems for using Nix for configuration management on other operating systems
- various deployment tools: NixOps, Hail, Colmena, deploy-rs, Terranix
- pre-flake version pinning tools like Niv
- flake libraries like flake-utils, flake-utils-plus, std, and flake-parts
- development environment utilities and libraries like devshell, Lorri, and integrations like those of direnv and shadowenv
- the Nix User Repository and various important overlays (e.g., the Emacs overlay, the Rust overlays)
- the many 2nix conversion tools, some of which are chiefly distributed outside of Nixpkgs and used to package things in private or company collections rather than in Nixpkgs
- arguably some proprietary commercial products and SaaS offerings, like Cachix, Hercules CI, and Flox
- the NixOS hardware profiles collection
- the Nix and NixOS manuals
- important unofficial documentation sources, like nix.dev and nixos.wiki, and even important blog series like the Nix Pills or Ian Henry's reading of the official documentation
- wrappers and developer applications based on Nix, like devboxes and nixpacks
- GUI tools for working with Nix code or configuring NixOS, graphical app stores, etc.
- editor integrations like rnix-lsp, nix-mode, and environment management plugins for various editors
- a couple of really cool tools for automatically creating portable/deployable shell scripts with Nix (binlore + resholve)
- tools for generating magic fat binaries based on container technologies, like nix-bundle
- partial reimplementations of various parts of the Nix stack for specialized use in other programs, or sometimes aspiring to compete with Nix (like tvix)
Some of that stuff is included in Nixpkgs, but that's almost incidental— the ecosystem is not just the packages qua packages, but also various specialized tools and community knowledge which has been accumulated in the form of configuration modules and documentation. Those projects are first-class members of the Nix ecosystem in their own right, not only as things that might be included as packages in Nixpkgs :)
(And of course leveraging Nix successfully doesn't mean one has to evaluate or learn all of these tools. Most users will organically discover and stick with just a handful of them according to the demands of their own use cases. But in total, there is a really wide landscape of Nix-based and Nix-related software out there.)
In this case, "Nix" refers to the Nix CLI and the Nix language, and not to closely related projects like NixOS, Nixpkgs, Home Manager, and many others that can be understood as "ecosystem" projects dependent on the Nix CLI/language. It's also worth noting that there's a vast constellation of third-party Nix tools that can also be understood as dependent on the core NixOS/nix project on GitHub.
That's really just a formalization and documentation of the way things are named now. I don't think it precludes renaming some things in the long term.
Nix is the package manager that works on a multitude of operating systems. Then there was the thought “what if we use this same underlying technology to make a Linux distribution” and “we can manage services in a declarative manner” which became NixOS.
However, if I can't find what I need in the documentation then it can be a problem. That's specially true for installing software. I'm not too familiar with the language to make entire new packages. Most things are using flakes and I haven't wrapped my head around them yet. Other things are not really compatible with its philosophy (like software packaged as Wine bottles).
Nix itself: same problem when the package doesn't exist. But when it does, it's wonderful. I'm even using it in OSX, instead of homebrew (with home-manager).
Hopefully this team will help smooth some of the rough edges.