Jetpack 6 Support (WIP)

README.md

@@ -1,6 +1,6 @@
 # NixOS module for NVIDIA Jetson devices
 
-This repository packages components from NVIDIA's [Jetpack SDK](https://developer.nvidia.com/embedded/jetpack) for use with NixOS, including:
+This repository packages components from NVIDIA's [JetPack SDK](https://developer.nvidia.com/embedded/jetpack) for use with NixOS, including:
  * Platform firmware flashing scripts
  * A 5.10 Linux kernel from NVIDIA, which includes some open-source drivers like nvgpu
  * An [EDK2-based UEFI firmware](https://github.com/NVIDIA/edk2-nvidia)
@@ -11,19 +11,22 @@ This repository packages components from NVIDIA's [Jetpack SDK](https://develope
    - Graphics: Wayland, GBM, EGL, Vulkan
    - Power/fan control: nvpmodel, nvfancontrol
 
-This package is based on the Jetpack 5 release, and will only work with devices supported by Jetpack 5.1:
- * Jetson Orin AGX
- * Jetson Orin NX
- * Jetson Xavier AGX
- * Jetson Xavier NX
+This package supports JetPack 5 and 6. It works with NVIDIA's developer kits supported by these versions only:
 
-The Jetson Nano, TX2, and TX1 devices are _not_ supported, since support for them was dropped upstream in Jetpack 5.
-In the future, when the Orin Nano is released, it should be possible to make it work as well.
+|       Device       | JetPack 5 | JetPack 6 |
+| ------------------ | --------- | --------- |
+| Jetson Orin AGX    |     ✓     |     ✓     |
+| Jetson Orin NX     |     ✓     |     ✓     |
+| Jetson Orin Nano   |     ✓     |     ✓     |
+| Jetson Xavier AGX  |     ✓     |           |
+| Jetson Xavier NX   |     ✓     |           |
+
+The Jetson Nano, TX2, and TX1 devices are _not_ supported, since support for them was dropped upstream in JetPack 5.
 
 ## Getting started
 
 ### Flashing UEFI firmware
-This step may be optional if your device already has recent-enough firmware which includes UEFI. (Post-Jetpack 5. Only newly shipped Orin devices might have this)
+This step may be optional if your device already has recent-enough firmware which includes UEFI. (Post-JetPack 5. Only newly shipped Orin devices might have this)
 If you are unsure, I'd recommend flashing the firmware anyway.
 
 Plug in your Jetson device, press the "power" button" and ensure the power light turns on.
@@ -49,7 +52,7 @@ At this point, your device should have a working UEFI firmware accessible either
 
 Now, build and write the customized installer ISO to a USB drive:
 ```shell
-$ nix build github:anduril/jetpack-nixos#iso_minimal
+$ nix build github:anduril/jetpack-nixos#iso_minimal # iso_minimal_jp5 for Xavier (or otherwise wanting to use JetPack 5)
 $ sudo dd if=./result/iso/nixos-22.11pre-git-aarch64-linux.iso of=/dev/sdX bs=1M oflag=sync status=progress
 ```
 (Replace `/dev/sdX` with the correct path for your USB drive)
@@ -95,6 +98,20 @@ Concretely, that means that you cannot modify the EFI variables from Linux, so U
 You may need to enter the firmware menu and reorder it manually so NixOS will boot first.
 (See [this issue](https://forums.developer.nvidia.com/t/using-uefi-runtime-variables-on-xavier-agx/227970))
 
+### JetPack Versions
+The major JetPack version may be changed by setting `hardware.nvidia-jetpack.majorVersion`.
+It defaults to the latest version supported by the board (e.g. JetPack 6 for Orin and JetPack 5 for Xavier).
+The "generic" som defaults to JetPack 6.
+
+Note that the JetPack firmware of a given version is incompatible with the kernel and rootfs of
+the other. You cannot run the JetPack 5 firmware with the JetPack 6 kernel (and vice versa). Be
+certain when upgrading that you are able to update both simulatenously. 
+
+If not installing JetPack 6 firmware via flashing scripts, it is recommended to install the
+JetPack 6 bootloader configuration (without live switch), then install the capsule update,
+and finally reboot. The effect is that the device reboots, installs the JetPack 6 firmware
+via a capsule update, and when finished, boots into a NixOS configuration for JetPack 6.
+
 ### Graphical Output
 As of 2023-12-09, the status of graphical output on Jetsons is described below.
 If you have problems with configurations that are expected to work, try different ports (HDMI/DP/USB-C), different cables, and rebooting with the cables initially connected or disconnected.
@@ -127,7 +144,7 @@ Set `hardware.nvidia-jetpack.modesetting.enable = true;`
 Weston and sway have been tested working on Orin devices, but do not work on Xavier devices.
 
 ### Updating firmware from device
-Recent versions of Jetpack (>=5.1) support updating the device firmware from the device using the UEFI Capsule update mechanism.
+Recent versions of JetPack (>=5.1) support updating the device firmware from the device using the UEFI Capsule update mechanism.
 This can be done as a more convenient alternative to physically attaching to the device and re-running the flash script.
 These updates can be performed automatically after a `nixos-rebuild switch` if the `hardware.nvidia-jetpack.bootloader.autoUpdate` setting is set to true.
 Otherwise, the instructions to apply the update manually are below.

device-pkgs/flash-script.nix

@@ -46,6 +46,9 @@
   ${lib.optionalString (dtbsDir != null) "cp -r ${dtbsDir}/. kernel/dtb/"}
   ${lib.optionalString (uefi-firmware != null) ''
   cp ${uefi-firmware}/uefi_jetson.bin bootloader/uefi_jetson.bin
+  if [ -e "${uefi-firmware}/uefi_jetson_minimal.bin" ] ; then
+    cp ${uefi-firmware}/uefi_jetson_minimal.bin bootloader/uefi_jetson_minimal.bin
+  fi
 
   # For normal NixOS usage, we'd probably use systemd-boot or GRUB instead,
   # but lets replace the upstream L4TLauncher EFI payload anyway

flake.nix

@@ -34,7 +34,9 @@
           buildPlatform = "x86_64-linux";
           hostPlatform = "aarch64-linux";
         };
-
+      };
+      jetpack5_config = {
+        hardware.nvidia-jetpack.majorVersion = "5";
       };
     in
     {
@@ -45,6 +47,12 @@
         installer_minimal_cross = nixpkgs.lib.nixosSystem {
           modules = [ aarch64_cross_config installer_minimal_config ];
         };
+        installer_minimal_jp5 = nixpkgs.lib.nixosSystem {
+          modules = [ aarch64_config installer_minimal_config jetpack5_config ];
+        };
+        installer_minimal_cross_jp5 = nixpkgs.lib.nixosSystem {
+          modules = [ aarch64_cross_config installer_minimal_config jetpack5_config ];
+        };
       };
 
       nixosModules.default = import ./modules/default.nix;
@@ -56,7 +64,7 @@
           let
             supportedConfigurations = lib.listToAttrs (map
               (c: {
-                name = "${c.som}" + (lib.optionalString (c.super or false) "-super") + "-${c.carrierBoard}";
+                name = c.som + lib.optionalString (c.super or false) "-super" + "-${c.carrierBoard}" + lib.optionalString (c ? majorVersion) "-jp${c.majorVersion}";
                 value = c;
               }) [
               { som = "orin-agx"; carrierBoard = "devkit"; }
@@ -65,6 +73,12 @@
               { som = "orin-nano"; carrierBoard = "devkit"; }
               { som = "orin-nx"; carrierBoard = "devkit"; super = true; }
               { som = "orin-nano"; carrierBoard = "devkit"; super = true; }
+              { som = "orin-agx"; carrierBoard = "devkit"; majorVersion = "5"; }
+              { som = "orin-agx-industrial"; carrierBoard = "devkit"; majorVersion = "5"; }
+              { som = "orin-nx"; carrierBoard = "devkit"; majorVersion = "5"; }
+              { som = "orin-nano"; carrierBoard = "devkit"; majorVersion = "5"; }
+              { som = "orin-nx"; carrierBoard = "devkit"; super = true; majorVersion = "5"; }
+              { som = "orin-nano"; carrierBoard = "devkit"; super = true; majorVersion = "5"; }
               { som = "xavier-agx"; carrierBoard = "devkit"; }
               { som = "xavier-agx-industrial"; carrierBoard = "devkit"; } # TODO: Entirely untested
               { som = "xavier-nx"; carrierBoard = "devkit"; }
@@ -89,8 +103,8 @@
             uefiCapsuleUpdates = lib.mapAttrs' (n: c: lib.nameValuePair "uefi-capsule-update-${n}" c.system.build.uefiCapsuleUpdate) supportedNixOSConfigurations;
           in
           {
-            # TODO: Untested
             iso_minimal = self.nixosConfigurations.installer_minimal_cross.config.system.build.isoImage;
+            iso_minimal_jp5 = self.nixosConfigurations.installer_minimal_cross_jp5.config.system.build.isoImage;
 
             inherit (self.legacyPackages.x86_64-linux)
               board-automation python-jetson;
@@ -104,6 +118,7 @@
 
         aarch64-linux = {
           iso_minimal = self.nixosConfigurations.installer_minimal.config.system.build.isoImage;
+          iso_minimal_jp5 = self.nixosConfigurations.installer_minimal_jp5.config.system.build.isoImage;
         };
       };
 
@@ -140,7 +155,7 @@
               ];
             });
         in
-        pkgs.nvidia-jetpack // { inherit (pkgs) nvidia-jetpack5; }
+        pkgs.nvidia-jetpack // { inherit (pkgs) nvidia-jetpack5 nvidia-jetpack6; }
       );
     };
 }

mk-overlay.nix

@@ -5,6 +5,7 @@
 { jetpackMajorMinorPatchVersion
 , l4tMajorMinorPatchVersion
 , cudaMajorMinorPatchVersion
+, cudaDriverMajorMinorVersion
 , bspHash
 }:
 final: _:
@@ -27,6 +28,9 @@ let
 
   l4tMajorVersion = versions.major l4tMajorMinorPatchVersion;
 
+  l4tAtLeast = versionAtLeast l4tMajorMinorPatchVersion;
+  l4tOlder = versionOlder l4tMajorMinorPatchVersion;
+
   sourceInfo = import ./sourceinfo {
     inherit l4tMajorMinorPatchVersion;
     inherit (final) lib fetchurl fetchgit;
@@ -35,6 +39,7 @@ in
 makeScope final.newScope (self: {
   inherit (sourceInfo) debs gitRepos;
   inherit jetpackMajorMinorPatchVersion l4tMajorMinorPatchVersion cudaMajorMinorVersion;
+  inherit l4tAtLeast l4tOlder;
 
   callPackages = callPackagesWith (final // self);
 
@@ -85,7 +90,7 @@ makeScope final.newScope (self: {
     # Nvidia's recommended toolchain is gcc9:
     # https://nv-tegra.nvidia.com/r/gitweb?p=tegra/optee-src/nv-optee.git;a=blob;f=optee/atf_and_optee_README.txt;h=591edda3d4ec96997e054ebd21fc8326983d3464;hb=5ac2ab218ba9116f1df4a0bb5092b1f6d810e8f7#l33
     stdenv = final.gcc9Stdenv;
-    inherit (self) bspSrc gitRepos l4tMajorMinorPatchVersion uefi-firmware;
+    inherit (self) bspSrc gitRepos l4tMajorMinorPatchVersion l4tAtLeast uefi-firmware;
   }) buildTOS buildOpteeTaDevKit opteeClient;
   genEkb = self.callPackage ./pkgs/optee/gen-ekb.nix { };
 
@@ -130,11 +135,15 @@ makeScope final.newScope (self: {
     inherit (finalSamples) callPackage;
   });
 
-  tests = final.callPackages ./pkgs/tests { inherit l4tMajorMinorPatchVersion; };
+  tests = final.callPackages ./pkgs/tests { inherit l4tMajorMinorPatchVersion l4tAtLeast; };
 
-  kernelPackagesOverlay = final: _: {
-    nvidia-display-driver = final.callPackage ./pkgs/kernels/r${l4tMajorVersion}/display-driver.nix { inherit (self) gitRepos l4tMajorMinorPatchVersion; };
-  };
+  kernelPackagesOverlay = final: _:
+    if self.l4tAtLeast "36" then {
+      devicetree = self.callPackage ./pkgs/kernels/r${l4tMajorVersion}/devicetree.nix { };
+      nvidia-oot-modules = final.callPackage ./pkgs/kernels/r${l4tMajorVersion}/oot-modules.nix { inherit (self) bspSrc gitRepos l4tMajorMinorPatchVersion; };
+    } else {
+      nvidia-display-driver = final.callPackage ./pkgs/kernels/r${l4tMajorVersion}/display-driver.nix { inherit (self) gitRepos l4tMajorMinorPatchVersion; };
+    };
 
   kernel = self.callPackage ./pkgs/kernels/r${l4tMajorVersion} { kernelPatches = [ ]; };
   kernelPackages = (final.linuxPackagesFor self.kernel).extend self.kernelPackagesOverlay;
@@ -171,5 +180,6 @@ makeScope final.newScope (self: {
   # attribute set, we cannot use self.callPackages because we would end up with infinite recursion.
   # Instead, we must either use final.callPackages or packagesFromDirectoryRecursive.
   // final.callPackages ./pkgs/l4t {
-  inherit (self) debs l4tMajorMinorPatchVersion;
+  inherit (self) debs;
+  inherit l4tMajorMinorPatchVersion cudaDriverMajorMinorVersion l4tAtLeast l4tOlder;
 })