C API: Some functions are too strict

[roberth]

Describe the bug

Some functions, such as those for getting an attribute value are strict.
This is unexpected in a lazy language, and for the bindings to diverge from the language semantics is a recipe for subtle mistakes that make applications behave worse, probably unknowingly.

Example: forward an attribute in a primop implementation

1. write a primop that expects an argument like { meta, ... }:, and returns something like x // { inherit meta; }
2. get the argument
3. make sure it's an attrset
4. store the meta Value in a local variable
5. do whatever the primop is supposed to do
6. return an attrset which includes meta
7. observe that this code breaks

   let x = myprimop { inherit meta; foo = "bar"; }
        meta = x.foo.meta // { bar = true; };
   in x

   There's no reason for this not to work! The only reason why this fails is that nix_get_attr_byname forces the value.

I know that the team has argued for making the C API simpler by avoiding thunks, but I now believe this is contradictory.
By "avoiding thunks", we create an API that is less capable and will have to eventually be extended to included mirrored functions that do the exact same thing, except the redundant .force call.
Simplicity in a library is achieved by orthogonalizing its features, not by spreading responsibilities across the whole interface.

Also note that we are currently trading an obvious error with a simple and obvious solution for a subtle error that requires a thorough understanding of Nix laziness to understand that it can even be solved. Let's not do that.

Steps To Reproduce

See above for example.

Expected behavior

The C API is no more eager than the language itself.

nix-env --version output

Additional context

#8699

Priorities

Add 👍 to issues you find important.

[o-santi]

+1

It makes very hard to iterate through big package sets (like nixpgs) when every attribute is read strictly, no only because of infinite recursion (pkgs.pkgs) but also because it is much slower than the same thing on plain nix. If not possible to completely remove these force calls, maybe it should offer a non-forcing equivalent?