Skip to content

Commit 7d75557

Browse files
cottandcottand
authored
Improve examples/ subfolder with cross-compilation and more docs (#25)
- Adds examples for cross-compilation of EIFs in `./examples`, and gives more detail on what commands to run in order to build EIFs. - Fixes a bug where the examples lockfile would fail to find the parent flake if the flake is not already in the Nix store. - Updates architecture diagram to reflect changes from #24 - Removes quick start example as it is redundant with `./examples/` and means maintaining an extra flake in markdown
1 parent e0ef65c commit 7d75557

6 files changed

Lines changed: 125 additions & 60 deletions

File tree

‎README.md‎

Lines changed: 19 additions & 47 deletions
Original file line numberDiff line numberDiff line change
@@ -8,66 +8,38 @@ You can think of it as an alternative to `nitro-cli build-enclave` for building
88
- give users complete control over their enclave images, providing additional options like BYOK (Bring Your Own Kernel)
99
- easily build EIFs on systems other than Amazon Linux, including M1+ Macs (e.g, it's possible to build an x86_64 Linux EIF on an ARM Mac)
1010

11-
We recommend [this excellent blog post](https://blog.trailofbits.com/2024/02/16/a-few-notes-on-aws-nitro-enclaves-images-and-attestation) to learn more about the EIF Nitro image format in general.
11+
12+
> We wrote [a blog post](https://monzo.com/blog/securing-our-software-supply-chain-better-with-reproducible-builds-for)
13+
> about our motivation for building this tooling at Monzo. We recommend you read it if you use AWS Nitro Enclaves
14+
> and you are wondering why you might want to use it.
15+
> We also recommend [this other excellent blog post](https://blog.trailofbits.com/2024/02/16/a-few-notes-on-aws-nitro-enclaves-images-and-attestation) to learn more about the EIF Nitro image format in general.
1216
1317

1418
The tradeoffs between using this repo and AWS' `nitro-cli` are:
15-
| Feature | `nitro-cli build-enclave` | monzo/aws-nitro-util |
16-
|---------|-----------|----------------------|
17-
| EIF userspace input | Docker container | plain files, including nix packages and unpacked OCI images
18-
| EIF bootstrap input | pre-compiled kernel binary provided by AWS | use pre-compiled kernel by AWS or bring your own kernel (see [example](./examples/README.md))
19-
| dependencies | Docker, linuxkit fork, [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/) | Nix, [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/)
20-
| Source-reproducible | no, uses pre-compiled blobs provided by AWS | yes, can be built entirely from source
21-
| Bit-by-bit reproducible EIFs | no, EIFs are timestamped | yes, building the same EIF will result in the same SHA256
22-
| cross-architecture EIFs | yes, if you provide a container for the right architecture | yes, if you provide binaries for the right architecture
23-
| OS* | [Amazon Linux](https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-cli-install.html) unless you [compile `nitro-cli` from source](https://github.com/aws/aws-nitro-enclaves-cli/tree/main/docs) for other Linux. No MacOS. | any Linux or MacOS with a Nix installation
2419

20+
| Feature | `nitro-cli build-enclave` | monzo/aws-nitro-util |
21+
|------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------|
22+
| EIF userspace input | Docker container | plain files, including nix packages and unpacked OCI images |
23+
| EIF bootstrap input | pre-compiled kernel binary provided by AWS | use pre-compiled kernel by AWS or bring your own kernel (see [example](./examples/README.md)) |
24+
| dependencies | Docker, linuxkit fork, [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/) | Nix, [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/) |
25+
| Source-reproducible | no, uses pre-compiled blobs provided by AWS | yes, can be built entirely from source |
26+
| Bit-by-bit reproducible EIFs | no, EIFs are timestamped | yes, building the same EIF will result in the same SHA256 |
27+
| cross-architecture EIFs | yes, if you provide a container for the right architecture | yes, if you provide binaries for the right architecture |
28+
| OS* | [Amazon Linux](https://docs.aws.amazon.com/enclaves/latest/user/nitro-enclave-cli-install.html) unless you [compile `nitro-cli` from source](https://github.com/aws/aws-nitro-enclaves-cli/tree/main/docs) for other Linux. No MacOS. | any Linux or MacOS with a Nix installation |
2529

2630
(*): OS for building EIFs. Note that
2731
- to make EIFs on a Mac, you have to be able to cross-compile the userspace binaries from Darwin to Linux
2832
- even if you make an EIF on a Mac, it can still only run on Linux.
2933

3034
## Examples
3135

32-
Flake quick start, to build an enclave with nixpkgs' `hello` :
33-
```nix
34-
{
35-
inputs = {
36-
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
37-
nitro-util.url = "github:/monzo/aws-nitro-util";
38-
nitro-util.inputs.nixpkgs.follows = "nixpkgs";
39-
};
40-
outputs = { self, nixpkgs, flake-utils, nitro-util, ... }:
41-
let system = "x86_64-linux";
42-
nitro = nitro-util.lib.${system};
43-
eifArch = "x86_64";
44-
pkgs = nixpkgs.legacyPackages."${system}";
45-
in {
46-
packages.${system}.eif-hello-world = nitro.buildEif {
47-
name = "eif-hello-world";
48-
49-
# use AWS' nitro-cli binary blobs
50-
inherit (nitro.blobs.${eifArch}) kernel kernelConfig nsmKo;
51-
52-
arch = eifArch;
53-
54-
entrypoint = "/bin/hello";
55-
env = "";
56-
copyToRoot = pkgs.buildEnv {
57-
name = "image-root";
58-
paths = [ pkgs.hello ];
59-
pathsToLink = [ "/bin" ];
60-
};
61-
};
62-
};
63-
}
64-
```
36+
You can find examples in [`examples/`](./examples/README.md).
6537

66-
See more examples in [`examples/`](./examples/).
38+
Note that you need to install [Nix](https://nixos.org/) and [enable flakes](https://nixos.wiki/wiki/Flakes) to use this repo.
6739

6840
## Design
6941

70-
monzo/aws-nitro-util is made up of a small CLI that wraps [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/) (which allows building an EIF from a specific file structure) and of Nix utilities to reproducibly build the CLI and its inputs.
42+
monzo/aws-nitro-util compiles a CLI from [aws/aws-nitro-enclaves-image-format](https://github.com/aws/aws-nitro-enclaves-image-format/) (which allows building an EIF from a specific file structure) and of Nix utilities to reproducibly build AWS' tooling, the EIF, and its dependencies.
7143

7244
A typical EIF build would look like the following:
7345

@@ -96,7 +68,7 @@ graph TD
9668
yourRepo("your source code \n or OCI image")
9769
end
9870
initBin("init \n compiled init.c \n (or bring your own)")
99-
eifCli("📦 eif-cli \n in this repo")
71+
eifCli("📦 eif_build CLI \n")
10072
nsm("nsm.ko \n compiled Nitro \n kernel module \n (or bring your own)")
10173
10274
subgraph PCR1
@@ -125,7 +97,7 @@ graph TD
12597
nsm-->doSysInit
12698
doSysInit ==> sysInit
12799
128-
eifFormatRepo ---> |pulled as \n build-time Rust \n cargo dependency|eifCli
100+
eifFormatRepo ---> |compile \n from source|eifCli
129101
eifCli -->doEif
130102
kernel -->doEif
131103
sysInit ==>doEif

‎examples/README.md‎

Lines changed: 45 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,53 @@
11
# Usage examples
22

3-
Examples are structured as a single flake containing packages of potential EIFs.
3+
You need to install [Nix](https://nixos.org/) and [enable flakes](https://nixos.wiki/wiki/Flakes) to use this repo.
4+
Examples are structured as an additional Nix flake containing [derivations](https://zero-to-nix.com/concepts/derivations) (ie, build recipes, like Dockerfiles) for potential EIFs.
45

56
To see the overall plumbing to use the aws-nitro-util flake, see [flake.nix](./flake.nix).
67

78
To see examples for specific EIFs, see the individual package definitions:
89

910
- Booting an enclave with a shell script only: [`withShellScript.nix`](./withShellScript.nix)
10-
- Booting an enclave with your own, compiled-from-source kernel: [`bringYourOwnKernel.nix`](./bringYourOwnKernel.nix)
11+
- Booting an enclave with your own, compiled-from-source kernel: [`bringYourOwnKernel.nix`](./bringYourOwnKernel.nix)
12+
13+
## Building the examples
14+
15+
### To show what examples can be built
16+
17+
```bash
18+
nix flake show
19+
```
20+
21+
### To build `shellScriptEif` for your current architecture:
22+
```bash
23+
nix build '.#shellScriptEif'
24+
```
25+
Note this will produce an `aarch64-linux` EIF if you are running it in an ARM Mac.
26+
27+
28+
### To build for a different architecture via a remote builder
29+
Nix allows compiling 'natively' for other architectures by building in a different machine.
30+
31+
To do this you need to set up a [linux remote builder](https://nix.dev/manual/nix/2.18/advanced-topics/distributed-builds) first.
32+
This can be any machine you can SSH into, including a VM.
33+
34+
Then, for example, to compile EIFs natively for `x86_64-linux` on an ARM Mac:
35+
```bash
36+
nix build '.#packages.x86_64-linux-crossCompiledEif'
37+
```
38+
39+
Using remote builders makes builds simpler (because it is a linux x86 machine compiling linux x86 binaries) but requires setting
40+
up that additional machine and telling your local Nix installation about it.
41+
42+
### To build for a different architecture via cross-compilation
43+
44+
If you do not have remote builders, you can cross-compile. Keep in mind this requires all dependencies
45+
of your EIF to be cross-compiled too (which is tricky for bash scripts).
46+
47+
48+
To cross-compile an EIF from your local system
49+
to `x86_64-linux`:
50+
51+
```bash
52+
nix build '.#x86_64-linux-crossCompiledEif'
53+
```

‎examples/flake.lock‎

Lines changed: 9 additions & 6 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎examples/flake.nix‎

Lines changed: 21 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
inputs = {
33
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
44

5-
nitro-util.url = "path:../";
5+
nitro-util.url = "github:monzo/aws-nitro-util";
66
nitro-util.inputs.nixpkgs.follows = "nixpkgs";
77

88
flake-utils.url = "github:numtide/flake-utils";
@@ -14,7 +14,7 @@
1414
in
1515
{
1616
packages = {
17-
17+
# the EIFs below will be for your machine's architecture
1818
shellScriptEif = pkgs.callPackage ./withShellScript.nix {
1919
inherit nitro;
2020
};
@@ -28,6 +28,25 @@
2828
inherit nitro;
2929
};
3030

31+
# the EIFs below will be for the architecture in the package name,
32+
# even if you build from a different machine
33+
x86_64-linux-crossCompiledEif =
34+
let
35+
crossArch = "x86_64";
36+
crossPkgs = import nixpkgs { inherit system; crossSystem = "${crossArch}-linux"; };
37+
in
38+
crossPkgs.callPackage ./withCrossCompilation.nix {
39+
inherit crossArch nitro;
40+
};
41+
42+
aarch64-linux-crossCompiledEif =
43+
let
44+
crossArch = "aarch64";
45+
crossPkgs = import nixpkgs { inherit system; crossSystem = "${crossArch}-linux"; };
46+
in
47+
crossPkgs.callPackage ./withCrossCompilation.nix {
48+
inherit crossArch nitro;
49+
};
3150
};
3251
}));
3352
}

‎examples/withCrossCompilation.nix‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
{ buildEnv
2+
, hello
3+
, nitro # when you call this function pass `nitro-util.lib.${system}` here
4+
, crossArch
5+
}:
6+
nitro.buildEif {
7+
arch = crossArch;
8+
kernel = nitro.blobs.${crossArch}.kernel;
9+
kernelConfig = nitro.blobs.${crossArch}.kernelConfig;
10+
11+
name = "eif-hello-world";
12+
13+
nsmKo = nitro.blobs.${crossArch}.nsmKo;
14+
15+
copyToRoot = buildEnv {
16+
name = "image-root";
17+
# the image passed here must be a Nix derivation that can be cross-compiled
18+
# we did not use a shell script here because that is hard for GNU coreutils
19+
paths = [ hello ];
20+
pathsToLink = [ "/bin" ];
21+
};
22+
23+
entrypoint = ''
24+
/bin/hello
25+
'';
26+
27+
env = "";
28+
}

‎examples/withShellScript.nix‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -6,8 +6,8 @@
66
}:
77
let
88
myScript = writeShellScriptBin "hello" ''
9-
# note busybox can be used for building EIFs but only on Linux
10-
# so remove this line if you are building an EIF on MacOS
9+
# this will fail when compiling on MacOS
10+
# see cross-compilation examples for alternatives
1111
export PATH="$PATH:${busybox}/bin"
1212
1313
while true;
@@ -25,7 +25,7 @@ nitro.buildEif {
2525

2626
name = "eif-hello-world";
2727

28-
nsmKo = nitro.blobs.aarch64.nsmKo;
28+
nsmKo = nitro.blobs.${arch}.nsmKo;
2929

3030
copyToRoot = buildEnv {
3131
name = "image-root";

0 commit comments

Comments
 (0)