Add better ways of handling/changing `nix` implementations

[getchoo]

Note

Because this is also inconsistent and confusing:

• When I say "Nix", I am referring to the repository hosted at NixOS/nix
• When I say nix, I am referring to the a generic nix binary implementation

The Problem

Over the last year(s), the community has grown a fair bit and with it, new implementations of nix itself have sprung up. This has introduced a problem we haven't really had before: many users in Nixpkgs now want to switch out their nix. So how can they do that?

Current Solutions

I've seen three main ways:

• Set nix.package in NixOS/home-manager
  • This (mostly) only changes the default nix in your $PATH
  • Some programs will respect this, some (most?) won't
  • It avoids rebuilds(!!)
• Overlay nix
  • This is the primary approach taken in lix-project/nixos-module
  • It's a bit of a force brute method, and can (and has) cause(d) issues with software that uses some of the newer APIs introduced by Nix that aren't present in Lix
  • Vastly improved by introducing the alias nixForLinking, which allowed packages to specifically depend on the Nix C API not present in Lix
  • Causes rebuilds(!!)
• Overriding packages
  • Sorta a middleground between both of the above?
  • Inconsistent, as some packages won't take in nix as an input
  • Nice for fine-grained control, or patching the effects of the above overlay

Issues With Them

IMO, much of the above boils down to a question of what exactly is nix?

By the pure Nixpkgs definition, it's the latest "stable" version of Nix. This implies that programs which depend on it currently have compatibility with at least v2.28 of Nix

nixpkgs/pkgs/top-level/all-packages.nix

Line 15136 in 86eba14

nix = nixVersions.stable;

Some packages - even in Nixpkgs itself - make different assumptions though. Namely:

• nix is a generic implementation of nix
• nix is something that's already present in the environment and doesn't need to be listed

These three definitions of what nix is and how it's meant to be used create a good amount of inconsistency and confusion. Packagers/developers for example now have to ask questions like if they need to test against other implementations for compatibility, if they need to pin to a specific implementation/version, or if they should even include a nix input at all. Likewise, users are faced with multiple, distinct ways of replacing their nix implementation that might not replace it everywhere reasonable, could directly break some packages, or isn't possible outside of manipulating $PATH

Better Solutions?

The main point of this issue is to encourage discussion on how to better solve the above, and make it easy enough for most users of Nixpkgs/NixOS to override the version or implementation of nix they're using.

An idea I had was to make a more generic nix alias (similar to nixForLinking) that implies the use of features compatible with Nix 2.18 - as these are mostly shared between Lix and Nix. Most packages could then probably use this nixGeneric (placeholder name 😉) in the same way they use nix now, hopefully after they've tested for compatibility. This would give nix implementation upstreams a nice override target, developers and packagers and nice way to express generic compatibility support, and avoid the pitfalls of not including a nix input at all

An alternative would of course be to just document nix itself as the "generic implementation", but I think that'd be slightly more breaking and put less emphasis on packagers actually testing to make sure they can use a generic implementation

Regardless, I would really love to find a more consistent and document-able solution to all this here :)

Related work

#384099
#391402
#396601
#396442
#433617
#436447

Upstream Nix overlay
Upstream Lix overlay

[getchoo]

CC @NixOS/nix-team @NixOS/lix-maintainers @lovesegfault @Artturin

[RaitoBezarius]

Do note that Lix evolved its approach and the option you mention is not accurate anymore, we have now a lixPackageSets scope which acts like the linuxPackages scope.

In a short future, I intend to add a lix top-level module akin to nix and add enable option (while leaving obviously the defaults of today).

The lix top-level module will behave like boot.kernelPackages and receive the scope and automatically wire up the right overlays from-inside nixpkgs.

This doesn't address your immediate question of performing fine-grained feature detections of Lix versions. I'm not sure what can be done about that. nixForLinking was attempted and CppNix brought forth things like nixComponents.

Perhaps an adapter can be made for Lix which can feed nixComponents from a monolithic Lix scope as long as the argument can be discovered statically (via callPackage or so).

But I don't think these abstractions are set in the stone and things seems to be consistently evolving as we understand how to do things better.

[lf-]

Part of the complication is that given the different stability policies of Lix and CppNix, it's highly plausible (and partially true today in the form of nix path-info --json) that the CLIs will diverge over time, so it's very possible that this winds up being less of a question people want to answer in the future, but it's unlikely they'll ever be completely incompatible.

[Mic92]

nixForLinking got replaced by nixComponents. The ABI today is already very different so there is no point in trying to find an abstraction as there is no project that is compatible with both Lix/Nix. Instead we we have different projects that explicitly link to one or the other.

[Mic92]

I think lixPackages is the better approach than trying to overlay packages adhoc and hope for the best. This already has caused bug reports in projects of mine in the past where the CLI was in fact not compatible.