Skip to content

meta.problems: Custom package problems [RFC127] #478539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
infinisil merged 6 commits into from
Feb 26, 2026
Merged

meta.problems: Custom package problems [RFC127] #478539

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7ae278d
meta-types.nix: Refactor
infinisil Jan 9, 2026
60ef776
meta-types.nix: Better error messages for attrsOf and listOf
infinisil Jan 9, 2026
27fcf5c
meta-types.nix: Introduce enum type
infinisil Jan 9, 2026
2032a6b
check-meta.nix: Remove unnecessery check-meta attributes
infinisil Feb 20, 2026
8fe8f0e
check-meta.nix: Internal refactor errormsg -> msg
infinisil Feb 20, 2026
27dd434
stdenv.mkDerivation: Initial RFC 127 implementation
infinisil Jan 9, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions ci/OWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,8 +58,10 @@
/pkgs/top-level/by-name-overlay.nix @infinisil @philiptaron
/pkgs/stdenv @philiptaron @NixOS/stdenv
/pkgs/stdenv/generic @Ericson2314 @NixOS/stdenv
/pkgs/stdenv/generic/check-meta.nix @Ericson2314 @adisbladis @NixOS/stdenv
/pkgs/stdenv/generic/meta-types.nix @adisbladis @NixOS/stdenv
/pkgs/stdenv/generic/problems.nix @infinisil
/pkgs/test/problems @infinisil
/pkgs/stdenv/generic/check-meta.nix @infinisil @Ericson2314 @adisbladis @NixOS/stdenv
/pkgs/stdenv/generic/meta-types.nix @infinisil @adisbladis @NixOS/stdenv
/pkgs/stdenv/cross @Ericson2314 @NixOS/stdenv
/pkgs/build-support @philiptaron
/pkgs/build-support/cc-wrapper @Ericson2314
Expand Down
3 changes: 3 additions & 0 deletions doc/redirects.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -629,6 +629,9 @@
"chap-stdenv": [
"index.html#chap-stdenv"
],
"sec-problems": [
"index.html#sec-problems"
],
"sec-using-llvm": [
"index.html#sec-using-llvm"
],
Expand Down
53 changes: 53 additions & 0 deletions doc/using/configuration.chapter.md
View file
Open in desktop

@infinisil infinisil Feb 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Originally posted by @ConnorBaker in #478539 (comment)

@ConnorBaker Would you like some help with that? If you give me some more details about what you need I could take a look!

I'd love some help!

Most of the CUDA packages come from binary archives provided by NVIDIA (we call these "redists" because we found them at a URL with "redist" in the path, the presence of a "redistrib_.json" manifest, and the LICENSE for some of them allowing redistribution). We have a generic buildRedist function we use to make packages from those binary archives, defined here: https://github.com/NixOS/nixpkgs/blob/9774bb59bd3bb978cf86b38afc70f9c6d5505983/pkgs/development/cuda-modules/buildRedist/default.nix.

Since availability of CUDA packages depends on platform, CUDA version, CUDA capabilities (think the type of GPU), and versions of dependencies, there's a lot of ways things can break (either when trying to patchelf the binary or at runtime). Making matters more complicated, NVIDIA does a fair amount of optimistic binary loading through dlopen calls at runtime, depending on the code path and the environment (e.g., CUDA version, CUDA capability, platform, etc.).

We try to fail early (during evaluation) when possible to avoid very expensive (time, space, and compute) builds that would fail or produce binaries things known to fail. To do that, we have two different attributes set through passthru by buildRedist: brokenAssertions and platformAssertions.

brokenAssertions is defined here:

nixpkgs/pkgs/development/cuda-modules/buildRedist/default.nix

Lines 388 to 425 in 9774bb5

# brokenAssertions :: [Attrs]
# Used by mkMetaBroken to set `meta.broken`.
# Example: Broken on a specific version of CUDA or when a dependency has a specific version.
# NOTE: Do not use this when a broken assertion means evaluation will fail! For example, if
# a package is missing and is required for the build -- that should go in platformAssertions,
# because attempts to access attributes on the package will cause evaluation errors.
brokenAssertions = [
{
message = "lib output precedes static output";
assertion =
let
libIndex = findFirstIndex (x: x == "lib") null finalAttrs.outputs;
staticIndex = findFirstIndex (x: x == "static") null finalAttrs.outputs;
in
libIndex == null || staticIndex == null || libIndex < staticIndex;
}
{
# NOTE: We cannot (easily) check that all expected outputs have a corresponding outputNameVar attribute in
# finalAttrs because of the presence of attributes which use the "output" prefix but are not outputNameVars
# (e.g., outputChecks and outputName).
message = "outputNameVarFallbacks is a super set of expectedOutputs";
assertion =
subtractLists (map mkOutputNameVar finalAttrs.passthru.expectedOutputs) (
attrNames finalAttrs.passthru.outputNameVarFallbacks
) == [ ];
}
{
message = "outputToPatterns is a super set of expectedOutputs";
assertion =
subtractLists finalAttrs.passthru.expectedOutputs (attrNames finalAttrs.passthru.outputToPatterns)
== [ ];
}
{
message = "propagatedBuildOutputs is a subset of outputs";
assertion = subtractLists finalAttrs.outputs finalAttrs.propagatedBuildOutputs == [ ];
}
]
++ brokenAssertions;

and platformAssertions is defined here:

nixpkgs/pkgs/development/cuda-modules/buildRedist/default.nix

Lines 427 to 446 in 9774bb5

# platformAssertions :: [Attrs]
# Used by mkMetaBadPlatforms to set `meta.badPlatforms`.
# Example: Broken on a specific system when some condition is met, like targeting Jetson or
# a required package missing.
# NOTE: Use this when a failed assertion means evaluation can fail!
platformAssertions =
let
isSupportedRedistSystem = _redistSystemIsSupported hostRedistSystem finalAttrs.passthru.supportedRedistSystems;
in
[
{
message = "src is null if and only if hostRedistSystem is unsupported";
assertion = (finalAttrs.src == null) == !isSupportedRedistSystem;
}
{
message = "hostRedistSystem (${hostRedistSystem}) is supported (${builtins.toJSON finalAttrs.passthru.supportedRedistSystems})";
assertion = isSupportedRedistSystem;
}
]
++ platformAssertions;

Then, in meta, buildRedist uses the final copies of those values to set meta.broken and meta.badPlatforms:

nixpkgs/pkgs/development/cuda-modules/buildRedist/default.nix

Lines 455 to 456 in 9774bb5

broken = _mkMetaBroken finalAttrs;
badPlatforms = _mkMetaBadPlatforms finalAttrs;

The helper functions produce the expected values, additionally printing out failed assertions through builtins.traceVerbose: https://github.com/NixOS/nixpkgs/blob/3f96296da66f5ecf3d8106c61281b823949a56c0/pkgs/development/cuda-modules/_cuda/lib/meta.nix.

As an example of how this is used, look at cuDNN, which uses platformAssertions to ensure it is only available for supported CUDA capabilities:

nixpkgs/pkgs/development/cuda-modules/packages/cudnn.nix

Lines 52 to 104 in 9774bb5

# NOTE:
# With cuDNN forward compatiblity, all non-natively supported compute capabilities JIT compile PTX kernels.
#
# While this is sub-optimal and we should warn the user and encourage them to use a newer version of cuDNN, we
# have no clean mechanism by which we can warn the user, or allow silencing such a warning if the use of an
# older cuDNN is intentional.
#
# As such, we only warn about capabilities which are no longer supported by cuDNN.
#
# NOTE:
#
# NVIDIA promises forward compatibility of cuDNN for major versions of CUDA. As an example, the cuDNN build for
# CUDA 12 is compatible with all, and will remain compatible with, all CUDA 12 releases. However, this does not
# extend to static linking with CUDA 11!
#
# We don't need to check the CUDA version to see if it falls within some supported range -- if a user decides
# to do static linking against some odd combination of CUDA 11 and cuDNN, that's on them.
#
platformAssertions =
let
# Create variables and use logical OR to allow short-circuiting.
cudnnAtLeast912 = cudnnAtLeast "9.12";
cudnnAtLeast88 = cudnnAtLeast912 || cudnnAtLeast "8.8";
cudnnAtLeast85 = cudnnAtLeast88 || cudnnAtLeast "8.5";
allCCNewerThan75 = lib.all (lib.flip lib.versionAtLeast "7.5") cudaCapabilities;
allCCNewerThan50 = allCCNewerThan75 || lib.all (lib.flip lib.versionAtLeast "5.0") cudaCapabilities;
allCCNewerThan35 = allCCNewerThan50 || lib.all (lib.flip lib.versionAtLeast "3.5") cudaCapabilities;
in
[
# https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-850/support-matrix/index.html#cudnn-cuda-hardware-versions
{
message =
"cuDNN releases since 8.5 (found ${finalAttrs.version})"
+ " support CUDA compute capabilities 3.5 and newer (found ${builtins.toJSON cudaCapabilities})";
assertion = cudnnAtLeast85 -> allCCNewerThan35;
}
# https://docs.nvidia.com/deeplearning/cudnn/archives/cudnn-880/support-matrix/index.html#cudnn-cuda-hardware-versions
{
message =
"cuDNN releases since 8.8 (found ${finalAttrs.version})"
+ " support CUDA compute capabilities 5.0 and newer (found ${builtins.toJSON cudaCapabilities})";
assertion = cudnnAtLeast88 -> allCCNewerThan50;
}
# https://docs.nvidia.com/deeplearning/cudnn/backend/v9.12.0/reference/support-matrix.html#gpu-cuda-toolkit-and-cuda-driver-requirements
{
message =
"cuDNN releases since 9.12 (found ${finalAttrs.version})"
+ " support CUDA compute capabilities 7.5 and newer (found ${builtins.toJSON cudaCapabilities})";
assertion = cudnnAtLeast912 -> allCCNewerThan75;
}
];

I'd love to see how (or if) the current implementation of the problems RFC would address such a use case.

As an added bonus, I'd love to see the definition for cudaPackages.backendStdenv simplified: https://github.com/NixOS/nixpkgs/blob/9774bb59bd3bb978cf86b38afc70f9c6d5505983/pkgs/development/cuda-modules/backendStdenv/default.nix.

Beyond ensuring a compatible version of GCC (patched to use glibc/libstdc++ from the default stdenv) is available for NVCC, it also performs a fair amount of logic in determining the default set of CUDA capabilities for some version of CUDA as well as whether explicitly requested CUDA capabilities are valid for the given CUDA version. It doesn't make sense to attach these through buildRedist since it affects all CUDA packages, and causing evaluation of backendStdenv to fail also allows us to fail evaluation in the presence of incorrect configurations (since a number of the utilities in cudaPackages.flags are derived from cudaPackages.backendStdenv).

Originally posted by @infinisil in #478539 (comment)

@ConnorBaker Thanks for the detailed explanation! I've created another draft PR based on the one here to implement a broken error kind: tweag#111

With that, it should be possible for you to do this for assertions:

meta.problems = optionalAttrs (! assertion) { 
  libBeforeStatic.message = "lib output precedes static output"
} // optionalAttrs (! anotherAssertion) {
  # ...
};

We could also consider adding a generic enable/condition/assertion field to problems to make this more ergonomic.

If an assertion is wrong, it causes evaluation to fail with the message displayed, along with info on how specific assertions can be switched to just warn or be entirely ignored.

I believe this would be appropriate for both the current brokenAssertions and platformAssertions, while not setting any meta.broken and meta.badPlatforms values. I plan to make a PR switching all of them for you to test, but early feedback is also appreciated :)

Originally posted by @infinisil in #478539 (comment)

@ConnorBaker Can you check out tweag#112?

Originally posted by @ConnorBaker in #478539 (comment)

I had a chance to look at tweag#111 and tweag#112 and I like what you've done!

We could also consider adding a generic enable/condition/assertion field to problems to make this more ergonomic.

I personally would like to see that, or something which aligns the attributes with the NixOS module system assertions/warnings.

In testing out tweag#112, I noticed it seems there's a double newline before "Package problems":

$ NIXPKGS_ALLOW_UNFREE=1 nix eval .#pkgsForCudaArch.sm_61.cudaPackages.cutlass -L --impure --json
error:
       … in the condition of the assert statement
         at /nix/store/yxy9bvxkvijkxpaz9qzap8gsjn4qf5rq-source/lib/customisation.nix:450:9:
          449|       outPath =
          450|         assert condition;
             |         ^
          451|         drv.outPath;

       … while evaluating the attribute 'handled'
         at /nix/store/yxy9bvxkvijkxpaz9qzap8gsjn4qf5rq-source/pkgs/stdenv/generic/check-meta.nix:677:11:
          676|           valid = if isNull problems.error then "warn" else "no";
          677|           handled = handle {
             |           ^
          678|             inherit attrs meta;

       (stack trace truncated; use '--show-trace' to show the full, detailed trace)

       error: Package ‘cuda12.8-cutlass-3.9.2’ in /nix/store/yxy9bvxkvijkxpaz9qzap8gsjn4qf5rq-source/pkgs/development/cuda-modules/packages/cutlass.nix:211 has the following problems that must be acknowledged: [ "capabilities" ], refusing to evaluate.


       Package problems:

       - capabilities (kind "broken"): Not all capabilities are >= 7.0 (["6.1"])

       You can use it anyway by ignoring its problems, using one of the
       following methods:

       a) For `nixos-rebuild` you can add "warn" or "ignore" entries to
         `nixpkgs.config.problems.handlers` inside configuration.nix,
         like this:

           {
             nixpkgs.config.problems.handlers = {
               cutlass.capabilities = "warn";
             };
           }

       b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
         "warn" or "ignore" to `problems.handlers` in
         ~/.config/nixpkgs/config.nix, like this:

           {
             problems.handlers = {
               cutlass.capabilities = "warn";
             };
           }

       See this page for more details: https://nixos.org/manual/nixpkgs/unstable#sec-problems

Is that intentional?

I'm glad to see these impact meta.available (which I use in some places to handle different configurations).

What do you imagine the use-case for the unsupported kind being and how would you say it differs from broken?

Originally posted by @infinisil in #478539 (comment)

I personally would like to see that, or something which aligns the attributes with the NixOS module system assertions/warnings.

Noted, but probably better in a follow-up PR to keep the scope of this one smaller.

Is that intentional?

Nope. Really the error format needs some tweaking in general imo, it's too big and contains duplicate info. How about this instead:

error: Package ‘a-0’ in /nix/store/eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee-default.nix:18 has problems:
- deprecated: Package is deprecated and replaced by b.
- maintainerless: This package has no declared maintainer, i.e. an empty `meta.maintainers` and `meta.teams` attribute.
- removal: Package will be removed.

See https://nixos.org/manual/nixpkgs/unstable#sec-problems for info. To allow evaluation regardless, use:
- Nixpkgs import: import nixpkgs { config = <below code>; }
- NixOS: nixpkgs.config = <below code>;
- nix-* commands: Put below code in ~/.config/nixpkgs/config.nix

  {
    problems.handlers = {
      a.deprecated = "warn"; # or "ignore"
      a.maintainerless = "warn"; # or "ignore"
      a.removal = "warn"; # or "ignore"
    };
  }

What do you imagine the use-case for the unsupported kind being and how would you say it differs from broken?

That's actually a very good point. These have very different notions:

  • Unsupported: Not supposed to work, fix upstream
  • Broken: Supposed to work according to upstream, fix the Nix side

We'd want to use both of these for CUDA

Originally posted by @ConnorBaker in #478539 (comment)

Ah, that makes sense. I worry that unsupported may not exactly align with how we'd want to use it for CUDA -- that is, to explain that the user has selected a configuration which will not or cannot work and that there is no upstream fix that would enable it. Does that make sense?

@infinisil infinisil Feb 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ConnorBaker Right, I see! Then I think it would be best to introduce a separate assertion problem kind for generic internal assertions that have been violated, not necessarily relating to broken or unsupported.

@ConnorBaker ConnorBaker Feb 16, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Part of the reason I've done broken or unsupported system previously is to indicate or communicate that such a configuration just isn't possible and that even more than "this will fail to build" the user might get evaluation errors (e.g., trying to do a lookup in an attribute set for a CUDA binary for an unsupported platform, like Darwin, will throw a missing attribute exception) -- kind of like a "stop digging! nothing you want is here!".

@infinisil infinisil Feb 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For those cases I actually think a simple foo.attrThatMightNotExist or (throw "This does not work!") would be best, because meta.problems will inherently allow users to ignore the problem and try anyways, which doesn't make a lot of sense if we know exactly where during evaluation it will fail.

So I think of meta.problems as a way for package maintainers to say that

I'm warning you that there's a problem, but if you wanna see for yourself or think it's not a problem for you, feel free to ignore this, in which case you're on your own and can't complain if you run into the problem you ignored

@Eveeifyeve Eveeifyeve Feb 17, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A better message could be a warning that says:

Warning!!!!!! The package <package> <problem> and may lead into these problems like: <list>, We strongly recommend <recommendation>. 

Please note: This issues listed above may not be covered in an issue.

So with a package being like nim1 and problem being it doesn't exist, recommend nim2 & problems may include security vulnerabilities, package building issue.
Would look like:

Warning!!!!!! The package nim1, doesn't exist and may lead into these problems like: security vulnerabilities & package building issues , We strongly recommend using nim or nim-unwrapped (v2). 

Please note: This issues listed above may not be covered in an issue/will be automatically closed.

Because I find the message that you mentioned @infinisil having a lot of idioms personally.

Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ By default, Nix will prevent installation if any of the following criteria are t

- The package has known security vulnerabilities but has not or can not be updated for some reason, and a list of issues has been entered into the package's `meta.knownVulnerabilities`.

- There are problems for packages which must be acknowledged, e.g. deprecation notices.

Each of these criteria can be altered in the Nixpkgs configuration.

:::{.note}
Expand Down Expand Up @@ -166,6 +168,57 @@ There are several ways to tweak how Nix handles a package which has been marked

Note that `permittedInsecurePackages` is only checked if `allowInsecurePredicate` is not specified.

## Packages with problems {#sec-problems}

A package may have several problems associated with it.
These can be either manually declared in `meta.problems`, or automatically generated from its other `meta` attributes.
Each problem has a name, a "kind", a message, and optionally a list of URLs.
Not all kinds can be manually specified in `meta.problems`, and some kinds can exist only up to once per package.
Currently, the following problem kinds are known (with more reserved to be added in the future):
Comment thread
infinisil marked this conversation as resolved.

- "removal": The package is planned to be removed some time in the future. Unique.
Comment thread
infinisil marked this conversation as resolved.
- "deprecated": The package relies on software which has reached its end of life.

@Eveeifyeve Eveeifyeve Feb 7, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two questions:

  1. How would deprecation of software process would look like?
  2. If the process is different, what would the migration process from throw look like?

@infinisil infinisil Feb 20, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As already replied in #478539 (comment), I'd rather not try to specify how this should be used exactly. For now I just want to make sure the mechanism works as the RFC specifies.

- "maintainerless": Automatically generated for packages with `meta.maintainers == []`. Unique, not manually specifiable.

Each problem has a handler that deals with it, which can be one of "error", "warn" or "ignore".

@Atemu Atemu Jan 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: I think I'd prefer if this was called "severity".

Declaring what it is rather than what should be done with it feels more right to me.

@Atemu Atemu Jan 10, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Handler" also sort of implies that what's declared is an action or a command; perhaps even a custom one.

"Severity" OTOH is just a property of the problem itself.

@infinisil infinisil Jan 10, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea. It's a derivation from the RFC, but I don't think we need to follow it to the dot.

@infinisil infinisil Jan 13, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although, this also implies that we should change config.problems.handlers, and I'm not sure I want to go that far.

"error" will disallow evaluating a package, while "warn" will simply print a message to the log.

The handler for problems can be specified using `config.problems.handlers.${packageName}.${problemName} = "${handler}";`.

There is also the possibility to specify some generic matchers, which can set a handler for more than a specific problem of a specific package.
This works through the `config.problems.matchers` option:

```nix
{
problems.matchers = [
# Fail to build any packages which are about to be removed anyway
{
kind = "removal";
handler = "error";
}

# Get warnings when using packages with no declared maintainers
{
kind = "maintainerless";
handler = "warn";
}

# You deeply care about this package and want to absolutely know when it has any problems
{
package = "hello";
handler = "error";
}
];
}
```

Matchers can match one or more of package name, problem name or problem kind.
If multiple conditions are present, all must be met to match.
If multiple matchers match a problem, then the highest severity handler will be chosen.
The current default value contains `{ kind = "removal"; handler = "warn"; }`, meaning that people will be notified about package removals in advance.

Package names for both `problems.handlers` and `problems.matchers` are taken from `lib.getName`, which looks at the `pname` first and falls back to extracting the "pname" part from the `name` attribute.

## Modify packages via `packageOverrides` {#sec-modify-via-packageOverrides}

You can define a function called `packageOverrides` in your local `~/.config/nixpkgs/config.nix` to override Nix packages. It must be a function that takes pkgs as an argument and returns a modified set of packages.
Expand Down
Loading
Loading