Skip to content

make-disk-image: replace cptofs with mkfs.ext4 -d#281

Open
chessai-ar wants to merge 100 commits into
awakesecurity:mainfrom
chessai-ar:chessai/stop-using-cptofs-in-make-disk-image
Open

make-disk-image: replace cptofs with mkfs.ext4 -d#281
chessai-ar wants to merge 100 commits into
awakesecurity:mainfrom
chessai-ar:chessai/stop-using-cptofs-in-make-disk-image

Conversation

@chessai-ar

Copy link
Copy Markdown
Collaborator

Use mkfs.ext4's -d flag to populate the filesystem directly from the staging directory during creation, instead of creating an empty filesystem and copying files into it via cptofs (LKL).

cptofs boots a Linux kernel in userspace (LKL) and copies files through it 4KB at a time with a hardcoded buffer. For large NixOS closures this takes potentially an hour or more. mkfs.ext4 -d writes blocks natively and completes in 1-3 minutes for the same images. [TODO: Put actual measurements here]

This is the same approach already used by make-ext4-fs.nix (PR 82718). The -d flag is supported alongside -E offset in e2fsprogs >= 1.43.

Also removes lkl from binPath since cptofs is no longer needed.

Things done

  • Built on platform:
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • Tested, as applicable:
  • Ran nixpkgs-review on this PR. See nixpkgs-review usage.
  • Tested basic functionality of all binary files, usually in ./result/bin/.
  • Nixpkgs Release Notes
    • Package update: when the change is major or breaking.
  • NixOS Release Notes
    • Module addition: when adding a new NixOS module.
    • Module update: when the change is significant.
  • Fits CONTRIBUTING.md, pkgs/README.md, maintainers/README.md and other READMEs.

Add a 👍 reaction to pull requests you find important.

cookiebaker444 and others added 30 commits January 6, 2026 11:51
To configure necessary timeouts and parameterize user/group.
To avoid the situation where a service unit file may have a [Unit] but
no valid [Service].

This warning is explicitly omitted for some services that are generated
by packages instead of by the systemd module.

Co-authored-by: Parnell Springmeyer <parnell@arista.com>
                Mel Zuser <mel.zuser@arista.com>
                Tomas Drtina <tdrtina-ext@arista.com>
Co-Authored-By: John Soo <jsoo1@users.noreply.github.com>
mailsend-go is a rewrite of mailsend in go.

mailsend is not receiving feature updates as of 2019-02-11.
Introduces clickhouse-jdbc, liquibase-clickhouse Java modules.
Uses liquibase-clickhouse local version to avoid use of obsolete clickhouse-jdbc
Make clickhouse users.xml configurable
Use `shaded` uberjar for liquibase-clickhouse to avoid managing transitive dependencies.
This change adds a derivation for the [`terraform-cloud-agent`][1]
executable and a NixOS module that defines a systemd service for
running that agent.

[1]: https://developer.hashicorp.com/terraform/cloud-docs/agents.

`stripRoot = false`

Specify the hash

Install tfc-agent binaries to `$out/bin`

Declare the `mainProgram` metadata for this package

... so that it can be used with `lib.getExe`.

Use `lib.getExe`

Suggested by @jsoo1.

Use freeformType for NixOS options that are turned into flags

Suggested by @jsoo1.

No need to guard on whether `cache-dir` and `data-dir` are null

Suggested by @jsoo1.

Ensure `/etc/terraform-cloud-agent/`

Suggested by @jsoo1.

Use the systemd-configured configuration directory

Use the standard systemd environment identifiers `%S` and `%C`

Suggested by @jsoo1.

Add `user` and `group` and drop privileges

Rider: add sandoxing arguments cribbed from the nginx module.

Just give the `flags` attrset as an argument

Sequence after `network-online.target` instead

... since that's what we really mean.

Add doc strings and examples
- preActivationHook is inserted after: nix-build before:
switch-to-configuration
- postActivationHook is inserted after: switch-to-configuration before:
reboot
Since the configuration assumes nscd.conf exists and nss is configured
pyrox0 and others added 27 commits March 18, 2026 21:10
Remove some unused arguments and use tag instead of rev in fetchers.
Some dependencies have advanced and now need to be pinned.

apache-airflow itself does not build, yet, because it needs pendulum
2.x, which requires python < 3.12, I believe.

Co-authored-by: Sergey Volkov <taranarmo@gmail.com>
Inactive since 2023, doesn't respond to maintainer pings.
Inactive since 2023/2024, does not react to maintainer pings.
Use replace-fail, tag and sort some deps and use
writableTmpDirAsHomeHook.
Once upon a time, these were important, but not anymore.
Co-authored-by: Wolfgang Walther <walther@technowledgy.de>
Not sure how these were supposed to be enabled, without exposing them on
the package interface itself.
- add nixfmt-tree call to format generated providers.nix
- fix update-providers script to include correct imports
Newer versions currently don't build with fastapi 0.121

Co-authored-by: Ethan Carter Edwards <ethan@ethancedwards.com>
Use mkfs.ext4's -d flag to populate the filesystem directly from the
staging directory during creation, instead of creating an empty
filesystem and copying files into it via cptofs (LKL).

cptofs boots a Linux kernel in userspace (LKL) and copies files through
it 4KB at a time with a hardcoded buffer. For large NixOS closures this
takes 20+ minutes. mkfs.ext4 -d writes blocks natively and completes
in 1-3 minutes for the same images.

This is the same approach already used by make-ext4-fs.nix (PR NixOS#82718).
The -d flag is supported alongside -E offset in e2fsprogs >= 1.43.

Also removes lkl from binPath since cptofs is no longer needed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.