Is your feature request related to a problem? Please describe.

Not exactly a problem, but I'm one of those people waiting for flakes to become stable as it's an experimental feature and I don't quite understand what it does.Reading a normal nix file is easier for me to understand and figure out what's going on, which seems kind of important before installing an OS.
Not sure how the split it in the nix community between flakes users and non-flakes users, but if it's 50/50 then about 50% of the nix community can't use nixos-anywhere.

Describe the solution you'd like

Not entirely sure if it fits but something like flake-compat for the project and instructions how to use non-flakes?

Describe alternatives you've considered

Waiting until flakes is stable.

Additional context

Is your feature request related to a problem? Please describe.

On my first use of nixos-anywhere, the remote machine did not boot anymore.

Describe the solution you'd like

As a user I'm not interested to deal with hardware specifics. I just want a working system. Any solution that requires zero interaction is preferred.

Describe alternatives you've considered

Alternative 1

Mention the necessity of a hardware-configuration in the docs, and describe how to generate one.

Alternative 2

Include all-hardware.nix from nixpkgs by default

Alternative 3

Generate a hardware-configuration for my system by default and include it automatically + notify user about it.

Additional context

If we generate a hardware-configuration.nix automatically, the question would be, how do we include it.
As currently users have to expose their system via nixosConfigurations, we could use ``:

nix-repl> nixosConfigurations.my-machine.extendModules {modules=[./hardware-configuration.nix];}

The problem with that approach is, that including the hardware config becomes an implementation detail of nixos-anywhere, and the setup will break again as soon as another tool will be used for future deployments.

Describe the bug

While the Gandi Cloud VPS offering supports NixOS their IaaS platform, which has some interesting features, doesn't.

In an attempt to use nixos-anywhere, I created a debian 11 instance with 2GB of RAM and prepared a flake to install. When running nixos-anywhere it failed with a kexec error.
I can post a log if necessary, but the error itself was: Could not get memory layout.

I looked around for instances of this error and found a bug report that suggests that kexec needs to be compiled with the xen libraries in order for it to work on xen guests. I believe I was targeting a xen guest.

I looked at the default kexec bundle used, which uses a statically linked kexec from nixpkgs, and indeed it isn't compiled with xen libraries. Since the xen libraries are part of the whole xen package, and it is now marked as insecure, this is about as far as I'm willing to go with this on my own.

To Reproduce

Steps to reproduce the behavior:

1. Create an instance of Gandi Cloud V5 server running debian 11, 2GB RAM
2. Use nixos-anywhere

Expected behavior

I expected the server to reboot to NixOS.

System information

Running from a macOS system, targeting debian 11, Xen guest.

Some machines may not support kexec for various reasons.
On those we can install nix along with the necessary tools.
I used this from the hetzner rescue system:

curl https://gist.githubusercontent.com/Mic92/53fedc01c006ab69efb8c9f347bd3d30/raw/a1eafa327b990cac15c26b2b62211ea638ecdb6d/nixosify.sh | bash -x

Only limitation is the missing zfs support since this usually requires an OS specific zfs package.

Using disko simple-efi.nix example:

installing the boot loader...
setting up /etc...
updating GRUB 2 menu...
Died at /nix/store/xlfyjbcgi8x6i4klw8r0rqx66hma7qzf-install-grub.pl line 586.
installation finished!
### Waiting for the machine to become reachable again ###
### Done! ###

Describe the bug

I'm trying to use nixos-anywhere to install NixOS onto a machine on my local network. The machine is a brand new System76 Thelio Mega running Ubuntu 22.04. When running nixos-anywhere, everything seems to be going fine, until...

+ echo machine will boot into nixos in 6s...
+ test -e /dev/kmsg
+ exec
Warning: Permanently added '10.144.139.249' (ED25519) to the list of known hosts.
Warning: Permanently added '10.144.139.249' (ED25519) to the list of known hosts.
ssh: connect to host 10.144.139.249 port 22: Connection timed out
ssh: connect to host 10.144.139.249 port 22: Connection timed out
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
ssh: connect to host 10.144.139.249 port 22: No route to host
^C

...I lost ssh connection to the target, and it is never regained.

Here is the full output.

The target box freezes, and I have to hard reboot. Upon rebooting, nothing seems to have changed.

To Reproduce

Steps to reproduce the behavior:

1. Have a machine on your local network with Ubuntu 22.04 installed (not sure if it matters if it's in the cloud or not...)
2. sudo apt install net-tools (for ifconfig)
3. sudo apt-get install openssh-server, sudo systemctl enable ssh --now
4. sudo nano /etc/ssh/sshd_config and add the lines Port 22 and PermitRootLogin yes
5. sudo systemctl start ssh
6. nix run github:numtide/nixos-anywhere -- --flake github:chessai/thelio-mega#thelio-mega root@<ip of local target>
7. Wait for it to call exec and then lose connection, then stay disconnected forever.

Expected behavior

Either a successful install, or a clear failure and exit.

System information

Source machine:

❯ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.25, NixOS, 23.05 (Stoat), 23.05.20230421.2362848`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - channels(root): `"nixos-22.05pre343321.78cd22c1b86"`
 - channels(chessai): `""`
 - nixpkgs: `/nix/store/22z4n4mxs2vz3l3lg41dz3mgnq1d4wxs-source`

Target machine:

chessai@system76-pc:~$ uname -a
6.2.6-76060206-generic #202303130630~1679424972~22.04~4a8cde1~dev-Ubuntu SMP PREEMPT_DY x86_64 x86_64 x86_64 GNU/Linux

chessai@system76-pc:~$ cat /boot/config-6.2.6-76060206-generic | grep KEXEC
CONFIG_KEXEC=y
CONFIG_KEXEC_FILE=y
CONFIG_ARCH_HAS_KEXEC_PURGATORY=y
CONFIG_KEXEC_SIG=y
# CONFIG_KEXEC_SIG_FORCE is not set
CONFIG_KEXEC_BZIMAGE_VERIFY_SIG=y
CONFIG_KEXEC_JUMP=y
CONFIG_KEXEC_CORE=y
CONFIG_HAVE_IMA_KEXEC=y
CONFIG_IMA_KEXEC=y

Additional context

I stopped X on the machine and re-ran, this time it hit "ssh: connect to host 10.144.139.249 port 22: Connection timed out" and exited immediately.

Describe the bug

When trying to build my x86_64-linux machine from my M1 Macbook, I get the following error:

Failed to find a machine for remote build!
derivation: bm173dm4why0gsh6h485v6q9dgz1r9gb-nv-codec-headers-11.1.5.2.drv
required (system, features): (i686-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([x86_64-linux], 1, [], [])

Previously I was able to build this system successfully if running locally. The x86_64-linux machine in question is a local bare metal PC in my office, running the latest NixOS installer (23.05).

I'm unsure why i686-linux is being targeted here.

To Reproduce

Steps to reproduce the behavior:

I ran this on my local LAN:

nix run github:dustinlyons/nixos-anywhere -- [email protected] --flake .#felix --build-on-remote -L

Expected behavior

I expect the system closure to be built. I'm unsure why ffmpeg headers are being targeted for i686-linux.

System information

Here's my code: https://github.com/dustinlyons/nixos-config

Additional context

Logs:

nix run github:numtide/nixos-anywhere -- [email protected] --flake .#felix --build-on-remote -L
### Uploading install SSH keys ###
/Users/dustin/.nix-profile/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/tmp/tmp.6LIk7zZn2o/nixos-anywhere.pub"
/Users/dustin/.nix-profile/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/Users/dustin/.nix-profile/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added '192.168.0.134' (ED25519) to the list of known hosts.
([email protected]) Password:

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh -o 'ConnectTimeout=10' -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no' '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

### Gathering machine facts ###
Warning: Permanently added '192.168.0.134' (ED25519) to the list of known hosts.
### Building disko script ###
warning: Git tree '/Users/dustin/.local/share/src/nixos-config' is dirty
warning: not writing modified lock file of flake 'git+file:///Users/dustin/.local/share/src/nixos-config':
• Removed input 'nixos'
### Formatting hard drive with disko ###
Warning: Permanently added '192.168.0.134' (ED25519) to the list of known hosts.
Warning: Permanently added '192.168.0.134' (ED25519) to the list of known hosts.
umount: /mnt/boot unmounted
umount: /mnt unmounted
++ realpath /dev/nvme0n1
+ disk=/dev/nvme0n1
+ lsblk --output-all --json
++ dirname /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate
+ jq -r --arg disk_to_clear /dev/nvme0n1 -f /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate.jq
+ set -fu
+ wipefs --all -f /dev/nvme0n1p1
/dev/nvme0n1p1: 8 bytes were erased at offset 0x00000036 (vfat): 46 41 54 31 36 20 20 20
/dev/nvme0n1p1: 1 byte was erased at offset 0x00000000 (vfat): eb
/dev/nvme0n1p1: 2 bytes were erased at offset 0x000001fe (vfat): 55 aa
+ wipefs --all -f /dev/nvme0n1p2
/dev/nvme0n1p2: 2 bytes were erased at offset 0x00000438 (ext4): 53 ef
++ zdb -l /dev/nvme0n1
++ sed -nr 's/ +name: '\''(.*)'\''/\1/p'
+ zpool=
+ [[ -n '' ]]
+ unset zpool
+ wipefs --all -f /dev/nvme0n1
/dev/nvme0n1: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 8 bytes were erased at offset 0x1d1c1115e00 (gpt): 45 46 49 20 50 41 52 54
/dev/nvme0n1: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
++ mktemp -d
+ disko_devices_dir=/tmp/tmp.9CsDqk3PLa
+ trap 'rm -rf "$disko_devices_dir"' EXIT
+ mkdir -p /tmp/tmp.9CsDqk3PLa
+ device=/dev/nvme0n1
+ imageSize=2G
+ name=nvme0n1
+ type=disk
+ device=/dev/nvme0n1
+ type=gpt
+ sgdisk --new=1:0:+100M --change-name=1:disk-nvme0n1-ESP --typecode=1:EF00 /dev/nvme0n1
Creating new GPT entries in memory.
The operation has completed successfully.
+ udevadm trigger --subsystem-match=block
+ udevadm settle
+ device=/dev/disk/by-partlabel/disk-nvme0n1-ESP
+ extraArgs=()
+ declare -a extraArgs
+ format=vfat
+ mountOptions=('defaults')
+ declare -a mountOptions
+ mountpoint=/boot
+ type=filesystem
+ mkfs.vfat /dev/disk/by-partlabel/disk-nvme0n1-ESP
mkfs.fat 4.2 (2021-01-31)
+ sgdisk --new=2:0:-0 --change-name=2:disk-nvme0n1-root --typecode=2:8300 /dev/nvme0n1
The operation has completed successfully.
Discarding device blocks: + udevadm trigger --subsystem-match=block
+ udevadm settle
+ device=/dev/disk/by-partlabel/disk-nvme0n1-root
+ extraArgs=()
+ declare -a extraArgs
+ format=ext4
+ mountOptions=('defaults')
+ declare -a mountOptions
+ mountpoint=/
+ type=filesystem
+ mkfs.ext4 /dev/disk/by-partlabel/disk-nvme0n1-root
mke2fs 1.47.0 (5-Feb-2023)
done
Creating filesystem with 488352785 4k blocks and 122093568 inodes
Filesystem UUID: e59daada-f9ac-4699-871e-14606c0afc86
Superblock backups stored on blocks:
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
	4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
	102400000, 214990848

Allocating group tables: done
Writing inode tables: done
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

+ set -efux
+ findmnt /dev/disk/by-partlabel/disk-nvme0n1-root /mnt/
+ mount /dev/disk/by-partlabel/disk-nvme0n1-root /mnt/ -t ext4 -o defaults -o X-mount.mkdir
+ findmnt /dev/disk/by-partlabel/disk-nvme0n1-ESP /mnt/boot
+ mount /dev/disk/by-partlabel/disk-nvme0n1-ESP /mnt/boot -t vfat -o defaults -o X-mount.mkdir
+ rm -rf /tmp/tmp.9CsDqk3PLa
### Building the system closure ###
warning: Git tree '/Users/dustin/.local/share/src/nixos-config' is dirty
warning: not writing modified lock file of flake 'git+file:///Users/dustin/.local/share/src/nixos-config':
• Removed input 'nixos'
trace: warning: optionsDocBook is deprecated since 23.11 and will be removed in 24.05
trace: warning: optionsDocBook is deprecated since 23.11 and will be removed in 24.05
trace: warning: optionsDocBook is deprecated since 23.11 and will be removed in 24.05
trace: warning: The option `fonts.fonts' defined in `/nix/store/xb426qj1fkj48rw3fwx6mqh6vj0488mk-source/nixos' has been renamed to `fonts.packages'.
Failed to find a machine for remote build!
derivation: bm173dm4why0gsh6h485v6q9dgz1r9gb-nv-codec-headers-11.1.5.2.drv
required (system, features): (i686-linux, [])
1 available machines:
(systems, maxjobs, supportedFeatures, mandatoryFeatures)
([x86_64-linux], 1, [], [])
error: a 'i686-linux' with features {} is required to build '/nix/store/bm173dm4why0gsh6h485v6q9dgz1r9gb-nv-codec-headers-11.1.5.2.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

When following the quickstart guide using the example configs (flake.nix and disk-config.nix) an error is thrown on step 6: (Optional) Test your nixos and disko configuration:

> nix run github:numtide/nixos-anywhere -- --flake .#hetzner-cloud --vm-test --show-trace
error:
       … while evaluating the attribute 'drvPath'
         at /nix/store/lj7rw28yzc4lvgfvncs5z05ykag6ir0g-source/lib/customisation.nix:222:7:

          221|     in commonAttrs // {
          222|       drvPath = assert condition; drv.drvPath;
             |       ^
          223|       outPath = assert condition; drv.outPath;

       … while calling the 'derivationStrict' builtin

         at /builtin/derivation.nix:9:12: (source not available)

       (stack trace truncated; use '--show-trace' to show the full trace)

       error: The option `disko.devices.disk."/dev/sda".content.partitions."[definition 1-entry 2]".content._pkgs' is
read-only, but it's set multiple times. Definition values:
       - In `/nix/store/r0rz80bnhl3j3ysq6bvxrksmhg1n29cc-source/lib/types/mdraid.nix': <function>
       - In `<unknown-file>': <function>

To Reproduce
Follow step 1-6 of the quickstart guide.

Expected behavior
A result from the vm test.

System information
nixos-anywhere: the version from https://github.com/numtide/nixos-anywhere as of now (Sat Aug 12 05:52:31 PM CEST 2023).

> nix-info -m
 - system: `"x86_64-linux"`
 - host os: `Linux 6.3.8, NixOS, 23.05 (Stoat), 23.05.1092.c7ff1b9b956`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.13.3`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

Describe the bug

targeting a Raspberry Pi OS system with nixos-anywhere will fail at the kexec stage with the error

+ /root/kexec/kexec/kexec --load /root/kexec/kexec/bzImage --initrd=/root/kexec/kexec/initrd --no-checks --command-line init=/nix/store/SNIPPED/init console=tty0 console=ttyAMA0,115200 console=ttyS0,115200 loglevel=4
Can't open (/proc/kcore).
Warning, can't get the VA_BITS from kcore
Can't open (/proc/kcore).

To Reproduce

Steps to reproduce the behavior:

1. Install Raspberry Pi Linux on a Raspberry Pi
2. Try to deploy any system config to it using nixos-anywhere

Expected behavior

The deployment should succeed.

System information

nixos-anywhere 5cdd6d6d2633498ff65dd6b15eaec8b9b8e7a3e2
raspberry pi os: 2023-05-03-raspios-bullseye-armhf-lite.img (available at https://downloads.raspberrypi.org/raspios_lite_armhf/images/raspios_lite_armhf-2023-05-03/2023-05-03-raspios-bullseye-armhf-lite.img.xz )

also tried after running rpi-update to grab latest unstable kernel.

Additional context
It seems that this platform may just be missing kcore entirely, and require a different approach.
I'm not too learned in these details, though, so could be wrong.

I've been looking for a way to pre-plan the installation of NixOS on two of my servers with as little downtime as possible, and just discovered nixos-anywhere which looks promising.

One thing that confuses me though, the instructions talk about creating a flake for the target installation, which largely looks like a regular NixOS configuration, but includes disko partitioning info. Which makes me assume that this flake is only to be used once, to get NixOS installed, and then never again for fear of re-partitioning the drive over and over.

Is this correct? Do I need to create a minimal configuration flake to use with nixos-anywhere that is separate from the actual flake configuration I intend to use on the target server? Or is the intent to use a final configuration during installation, and the partition info is somehow only used once?

Describe the bug

When the NixOS configuration we have kexec'd into is 23.05 it has nix 2.13.1, whilst nixos-anywhere is using another version of Nix (2.15.1). At the point of trying to copy the disko configuration from the local machine to the remote machine, it will hang, similar to MatthewCroughan/nixinate#43 (comment)

This is the command that hangs nix copy --extra-experimental-features 'nix-command flakes' --no-write-lock-file -L --substitute-on-destination --to ssh://[email protected] /nix/store/6717d5a5l98yak7apc06j6snd8cwxrab-disko

I have verified that changing to nixUnstable fixes this https://github.com/numtide/nixos-anywhere/blob/a75b9c5163a02d3fb02eb862f29624c7485f509d/src/default.nix#L5

To Reproduce

Steps to reproduce the behavior:

1. Use the current nixos-anywhere against a 23.05 installer

Expected behavior

That nix copy would complete successfully

Describe the bug

Example in README didn't work

To Reproduce

I copy-pasted the disko config which complains with

error: The option `disko.devices.disk.main.content.partitions."[definition 1-entry 1]".type' does not exist. Definition values:
       - In `/nix/store/3lrc9lybsdxdwqv9fadbf0xzw40qa7j5-source/flake.nix': "partition"

I think it might have to be this?

              part-type = "primary";

Expected behavior

example works

System information

nixos 22.11

Additional context

Is your feature request related to a problem? Please describe.

When provisioning a new system, we might want to provide metadata to configure things like the machine's hostname, authorized SSH keys, .... But not all targets have a metadata service (eg: a Hetzner machine doesn't provide the metadata).

Describe the solution you'd like

Cloudinit can source some of its metadata from a specially crafted disk partition. nixos-remote could be used to prepare that disk partition.

See https://cloudinit.readthedocs.io/en/latest/topics/datasources/configdrive.html

Describe alternatives you've considered

Additional context

Describe the bug

The deployment is quite slow for me, likely because the derivations are built using the nix remote build protocol which is very sensitive to network delay.

Is your feature request related to a problem? Please describe.

I wanted to use nixos-anywhere to setup NixOS on a work machine, which I can only reach by going through a seperate login server. For this host, I have an entry in my ~/.ssh/config along the lines of

Host to-be-provisioned
  User myusername
  HostName internal.address
  ProxyJump login.veryrealcompany.com

I was able to SSH into the host successfully and ran nixos-anywhere ... myusername@to-be-provisioned.
Initially, nixos-anywhere was able to connect as well and installed its SSH key.
However, at some point, it switched to trying to connect via internal.address which obviously did not work.

To mitigate this, I added a Host internal.address entry to my ~/.ssh/config and was able to get a step further.
Subsequently, ssh-keyscan failed because apparently it does not read ~/.ssh/config.

As a last resort, I tried to port-forward internal.address:22 to localhost:2222 and target that instead, but again ssh-keyscan failed because it tried to scan username@localhost rather than localhost.

Describe the solution you'd like

I'm sure each of these issues can (and maybe should be) fixed individually.
However, I do believe that my initial attempt should ideally just work.
To that end, nixos-anywhere should stick with the originally provided hostname for its connections or manually specify the appropriate ProxyJump. Further, if a ProxyJump is required, ssh-keyscan must be run on the bastion host.

In case there is a good reason to for nixos-anywhere to switch the hostname it uses for connections in some scenarios, it may be sufficient to enable this behavior through a flag.

Describe alternatives you've considered

Instead of automatically detecting the configured bastion host, it could also be specified via command line args.
While I find that slightly more cumbersome, an argument could be made that that solution is purer since the bastion host needs to be explicitly specified.
That said, the SSH config is currently respected so I believe implicit detection does not go against the spirit of the tool.

Additional context

In this setup, I have no possibility to run nix on the bastion host. I have instead performed the installation manually by running the same steps that nixos-anywhere would, only with the correct SSH options.

Description of the bug

While attempting to install NixOS on a bare metal x86_64 machine using nixos-anywhere, I encountered an issue. I followed the guide on installing on a system without an operating system. First, I downloaded the latest Gnome ISO image from https://nixos.org, then I booted it on the target machine, and finally, I executed the following command on another machine running NixOS:

nix run github:numtide/nixos-anywhere -- --flake .#my-nixos-config [email protected]

However, this resulted in an error related to signatures:

error: cannot add path '/nix/store/lkdb5zm8s6zpgxmpn5map96072v08g1r-disko' because it lacks a signature by a trusted key

Interestingly, when I used the root user account:

nix run github:numtide/nixos-anywhere -- --flake .#my-nixos-config [email protected]

everything worked as expected and the installation succeeded. Hence, it seems to be an issue with permissions on the target machine. The root user account appears to be able to add store paths without a trusted signature, while the nixos user account cannot.

Expected behavior

Following the steps in the how-to guide should not result in an error.

Additional information

Output when run with --debug option:

### Formatting the hard drive with disko ###
+ nix_copy --to ssh://[email protected] /nix/store/lkdb5zm8s6zpgxmpn5map96072v08g1r-disko
+ NIX_SSHOPTS='-o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -i /tmp/tmp.0DKmLmqnBD/nixos-anywhere '
+ nix copy --extra-experimental-features 'nix-command flakes' --no-write-lock-file -L --substitute-on-destination --to ssh://[email protected] /nix/store/lkdb5zm8s6zpgxmpn5map96072v08g1r-disko
error: cannot add the path '/nix/store/lkdb5zm8s6zpgxmpn5map96072v08g1r-disko' because it lacks a signature by a trusted key
error: unexpected end-of-file

Describe the bug

Hi, i just tried to nixos-anywhere into an installed Debian 5.10.162 on a Hetzner VM that was configured by someone else.
On Hetzner rescue systems this typically works.

What's obviously different here is that i don't have the root password, but a normal user account with sudo-rights.

The kexec part does not work. The machine does not come online after the kexec part, and the VM console shows this (typed it off the graphical screen):

[96822.464035] + nohup bash -c 'sleep 6 && \''/root/kexec/kexec/kexec/kexec'\'' -e '
[96822.517973] shell-init: error retrieving current directory: getcwd: cannot access parent directories: No such file or directory

Is it obvious to someone what happened here, by looking at this message?

Describe the bug

Using disko config to create btrfs filesystems with mountOptions = ["compress=zstd:1]. Disko will create the filesystems and I presume the mount units. I am unsure if it is disko or nixos-anywhere that then mounts the filesystems -- but whatever it is it does not use the mountOptions, so the system installed without compression enabled. After reboot the mountOptions take effect as expected.

To Reproduce

Steps to reproduce the behavior:

            content = {
              type = "btrfs";
              extraArgs = ["-f"]; # Override existing partition
              subvolumes = {
                # Subvolume name is different from mountpoint
                "/rootfs" = {
                  mountpoint = "/";
                };
                # Subvolume name is the same as the mountpoint
                "/home" = {
                  mountOptions = ["compress=zstd:1"];
                  mountpoint = "/home";
                };
                # Sub(sub)volume doesn't need a mountpoint as its parent is mounted
                "/home/sedlund" = {};
                # Parent is not mounted so the mountpoint must be set
                "/nix" = {
                  mountOptions = ["compress=zstd:1" "noatime"];
                  mountpoint = "/nix";
                };
              }; # subvolumes
            }; # content.btrfs
          }; # content.luks

Expected behavior

I would think mountOptions should be respected, or some other mechanism provided to allow the system to be installed with compression enabled - or being new to all this I am missing something. Any help appreciated.

System information

nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05";
disko.url = "github:nix-community/disko";
disko.inputs.nixpkgs.follows = "nixpkgs";

Additional context

Using nix-anywhere to try to install on OVH VPS with 4GB of RAM running Debian 11 non-EFI.

Install seems to work fine up 'til this point:

...

+ /root/kexec/kexec/kexec --load /root/kexec/kexec/bzImage --initrd=/root/kexec/kexec/initrd --command-line 'init=/nix/store/w8x4ggsygn5g8np2gwn8i3dvhfpss9bz-nixos-system-nixos-22.11pre-git/init console=tty0 console=ttyS0,115200 nohibernate loglevel=4'
machine will boot into nixos in in 6s...
+ echo 'machine will boot into nixos in in 6s...'
+ [[ -e /dev/kmsg ]]
+ exec

On server console I see Kernel panic:

Any ideas? Let me know what additional details might be helpful.

cc: @tfc

Describe the bug

I just tried nixos-anywhere (for the first time :D) and it failed because my target has no setsid (OpenWRT snapshot with ash a shell).

Here's a trace:

[email protected]'s password: 

Number of key(s) added: 1

Now try logging into the machine, with:   "ssh -o 'ConnectTimeout=10' -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no' '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

Warning: Permanently added '192.168.86.1' (ED25519) to the list of known hosts.
Warning: Permanently added '192.168.86.1' (ED25519) to the list of known hosts.
     _________
    /        /\      _    ___ ___  ___
   /  LE    /  \    | |  | __|   \| __|
  /    DE  /    \   | |__| _|| |) | _|
 /________/  LE  \  |____|___|___/|___|
 \        \   DE /
  \    LE  \    /  -------------------------------------------
   \  DE    \  /    OpenWrt SNAPSHOT, r5089-4f4d41021
    \________\/    -------------------------------------------

Warning: Permanently added '192.168.86.1' (ED25519) to the list of known hosts.
kexec/
kexec/bzImage
kexec/kexec
kexec/run
kexec/ip
kexec/initrd
Warning: Permanently added '192.168.86.1' (ED25519) to the list of known hosts.
     _________
    /        /\      _    ___ ___  ___
   /  LE    /  \    | |  | __|   \| __|
  /    DE  /    \   | |__| _|| |) | _|
 /________/  LE  \  |____|___|___/|___|
 \        \   DE /
  \    LE  \    /  -------------------------------------------
   \  DE    \  /    OpenWrt SNAPSHOT, r5089-4f4d41021
    \________\/    -------------------------------------------

-ash: setsid: not found

To Reproduce

Steps to reproduce the behavior:

1. use it on a target that does not have setsid

Expected behavior

It should work by trying to find alternatives to setsid or asking about installing it.

System information

OpenWrt SNAPSHOT, r5089-4f4d41021 (R86S stock OS)

I've been trying our different configurations for about half a day and none of them work. It's either that I get errors such as:

updating GRUB 2 menu...
installing the GRUB 2 boot loader on /dev/sda...
Installing for i386-pc platform.
/nix/store/kps2xxdg43x7wkjxh97wp6zpqx2gzdqy-grub-2.06/sbin/grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible.
/nix/store/kps2xxdg43x7wkjxh97wp6zpqx2gzdqy-grub-2.06/sbin/grub-install: warning: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
/nix/store/kps2xxdg43x7wkjxh97wp6zpqx2gzdqy-grub-2.06/sbin/grub-install: error: will not proceed with blocklists.
/nix/store/mpli8kczf4x9j9icfr5gjgl0a9y7rybx-install-grub.pl: installation of GRUB on /dev/sda failed: No such file or directory

Or that I got a machine that is incapable of boot.

Each time this fails I ask my VPS provider to reinstall their default OS distro, and then I sudo passwd and give root ssh access for nixos-anywhere.

Here's the configuration that produced the errors above:

let
    device = "/dev/sda";
in {
  disko.devices = {
    disk = {
      "${builtins.baseNameOf device}" = {
        inherit device;
        type = "disk";
        content = {
          type = "table";
          format = "gpt";
          partitions = [
            {
              name = "ESP";
              start = "1MiB";
              end = "100MiB";
              bootable = true;
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            }
            {
              name = "root";
              start = "100MiB";
              end = "-1G";
              part-type = "primary";
              content = {
                type = "filesystem";
                format = "ext4";
                mountpoint = "/";
              };
            }
            {
              name = "swap";
              start = "-1G";
              end = "100%";
              part-type = "primary";
              content = {
                type = "swap";
                randomEncryption = true;
              };
            }
          ];
        };
      };
    };
  };

  boot = {
    kernelPackages = pkgs.linuxPackages_latest;
    loader.grub = {
      enable = true;
      inherit device;
    };
  };
  # Other settings ...
}

Help will be appreciated!

Is your feature request related to a problem? Please describe.

Ssh configurations can be custom defined, when IdentitiesOnly enabled, ssh will no longer iterate over its ~/.ssh/ keys. Instead of requiring each user to configure their global ~/.ssh/config per install, i would propose an extra option, --ssh-option which would allow you to set arguments that are passed into each ssh call, similar to how nix has it with --option.

Describe the solution you'd like

This can help greatly by allowing a pathway to manually define an ssh key.

For example: --ssh-option '-i ~/.ssh/key'.

Describe alternatives you've considered

Manually writing/changing each machine's .ssh/config to make it work with nixos-anywhere. But this seems less than ideal and requires a lot of manual intervention unless ssh defaults are strictly followed by everyone.

Additional context

None

Is your feature request related to a problem? Please describe.

Allow looking up to top level for an attribute. Hit a failure when i attempted to use a top-level images.hetzner-cloud-kexec naming scheme. The configuration works correctly when moved to nixosConfigurations.

nix run github:numtide/nixos-anywhere -- [email protected] --flake .#images.hetzner-cloud-kexec
error: flake 'git+...' does not provide attribute 'packages.x86_64-linux.nixosConfigurations.images.hetzner-cloud-kexec.config.system.build.disko', 'legacyPackages.x86_64-linux.nixosConfigurations.images.hetzner-cloud-kexec.config.system.build.disko' or 'nixosConfigurations.images.hetzner-cloud-kexec.config.system.build.disko'

Describe the solution you'd like

Let nixos-anywhere walk all the way up to the top-level to look for an attribute to allow writing full lookups.

Describe alternatives you've considered

Using nixosConfigurations. Could work okay, but I personally do not like mixing install images and live systems. This could be solved by letting nixos-anywhere install full new systems at the kexec step. Unsure if there are any good reasons to seperate into a two stage deployment. (Install then a bigger configuration-install after reboot).

Additional context

None

we all more or less agree that the current name is not the nicest one. This issue is about finding a new one. Make a proposal and than we do thumb up / thump down to vote for it.

Describe the bug

The kexec call during a run fails with kexec_load failed: Permission denied. Quick googling suggests that

1. It probably has to do with secure boot.
2. It can actually be made viable if the kexec call receives the -s flag.

To Reproduce

Steps to reproduce the behavior:

1. Take any sample nixos-anywhere config and try to apply it to a secure-booted target.
2. What I see in my case of an Ubuntu 23.04 target is the following

❯ nix run github:numtide/nixos-anywhere -- --flake .#myhost root@<my-ip>
...
+ /root/kexec/kexec/kexec --load /root/kexec/kexec/bzImage --initrd=/root/kexec/kexec/initrd --no-checks --command-line init=/nix/store/gqm5m9b7ywkckj6azs1x5y2vak73a1v5-nixos-system-nixos-23.05pre-git/init console=tty0 console=ttyS0,115200 loglevel=4
kexec failed, dumping dmesg
kexec_load failed: Permission denied
entry       = 0x7be2f7790 flags = 0x3e0000
nr_segments = 7
segment[0].buf   = 0x7fdeadafec10
segment[0].bufsz = 0x70
segment[0].mem   = 0x100000
segment[0].memsz = 0x1000
segment[1].buf   = 0x7fdeadafe830
segment[1].bufsz = 0x1b8
segment[1].mem   = 0x101000
segment[1].memsz = 0x1000
segment[2].buf   = 0x7fdeadafe660
segment[2].bufsz = 0x30
segment[2].mem   = 0x102000
segment[2].memsz = 0x1000
segment[3].buf   = 0x7fde98823400
segment[3].bufsz = 0x152da240
segment[3].mem   = 0x7a5d25000
segment[3].memsz = 0x152db000
segment[4].buf   = 0x7fdeadb03060
segment[4].bufsz = 0x958e20
segment[4].mem   = 0x7bb000000
segment[4].memsz = 0x3190000
segment[5].buf   = 0x7fde98815430
segment[5].bufsz = 0x40c3
segment[5].mem   = 0x7be2f2000
segment[5].memsz = 0x5000
segment[6].buf   = 0x7fde9881a420
segment[6].bufsz = 0x70e0
segment[6].mem   = 0x7be2f7000
segment[6].memsz = 0x9000
+ echo kexec failed, dumping dmesg
+ dmesg
+ tail -n 100
...
[1603986.053260] ima: impossible to appraise a kernel image without a file descriptor; try using kexec_file_load syscall.
+ exit 1
+ cleanup
+ rm -rf /root/kexec/kexec/tmp.F8l8LcGdu7

Expected behavior

nixos-anywhere succeeds.

System information

• Host is NixOS (unstable), target is Ubuntu 23.04.

Is your feature request related to a problem? Please describe.

I'm attempting to push nix into my org.

We currently use coder for our development environments. We have a prototype which sets up everything we need using nix.

Some things, like the vs-code-server, need systemd to run.

Describe the solution you'd like

Instead of setting everything up w systemd units and running that at startup, I figured it would be easier to manage a nixOS system.

Describe alternatives you've considered

systemd + oneshot units which run the nix setup - annoying to set up.
nixOS vm - at this stage the company will not support this. If I had nixOS running in a container, they would likely switch to this after a few months.

Is your feature request related to a problem? Please describe.

We want to ensure that decommissioned hard-drives don't end up in the wrong hands by mistake. Especially on a remote machine where we don't know their processes.

Describe the solution you'd like

As part of the setup, it would be nice to enrol the machine into secure boot and allow full disk encryption.

Describe alternatives you've considered

Open up an SSH server in the initrd that allows typing the decryption key remotely. The main downside is that this requires manual intervention on reboots.

Additional context

https://twitter.com/blitzclone/status/1596108176914493440?s=20&t=ExP91clcY38ZUsnrrvtkfA

Describe the bug

When running nixos-anywhere, the script will use the nixpkgs version of openssh. This is problematic on non-NixOS based systems such as Fedora 37. Where openssh is packaged with extra features, leading to a crash loop due to the feature-sets clashing inside the system's packaged openssh.config files.

% nix run nixpkgs#openssh -- [email protected]
/etc/crypto-policies/back-ends/openssh.config: line 3: Bad configuration option: gssapikexalgorithms
/etc/crypto-policies/back-ends/openssh.config: line 7: Bad configuration option: rsaminsize
/etc/crypto-policies/back-ends/openssh.config: terminating, 2 bad configuration options

To Reproduce

Steps to reproduce the behavior:

1. Install an operating system such as Fedora 37.
2. Install nix trough the official means or through nix-community/nix-installers.
3. Attempt to run nix run nixpkgs#openssh on the system.
4. Similarly, the nix run github:numtide/nixos-anywhere breaks similarly due to the above package.

Expected behavior

It runs correctly as if it was on NixOS.

System information

Operating System: Fedora Linux 37 (Workstation Edition)
          Kernel: Linux 6.1.7-200.fc37.x86_64
    Architecture: x86-64

Additional context

None.

Is your feature request related to a problem? Please describe.

My server configuration is low, and the running speed is too slow, often reconnecting to ssh after kexec will time out. Or the network needs to be reconfigured (I don't know why), so ssh will time out and the installation will fail.

Describe the solution you'd like
Increase the number of ssh retries options or infinite retries

Describe alternatives you've considered
I hope after kexec I re-run nixos-anywhere it works fine. But it tends to stop here.

### Gathering machine facts ###

Additional context
None

Describe the bug

After I've installed with nixos-anywhere I reboot and the only disk I see is the installation disk.

To Reproduce

Steps to reproduce the behavior:

git clone https://github.com/fud/nixos-anywhere-test/
❯ nix run github:numtide/nixos-anywhere -- --flake .#fusion --build-on-remote [email protected]

Expected behavior

I expect to see the disks and they're bootable. I'm just not seeing any disks to boot. I'm not sure if it's a configuration issue or an issue with nixos-anywhere.

System information

I'm on an m1 trying to install on an x86 machine with --build-on-remote and it works perfectly apart from seeing the bootable disks.

I am definitely booting has UEFI

[root@nixos:~]# [ -d /sys/firmware/efi ] && echo UEFI || echo BIOS
UEFI

My partition information looks correct (to me)

[root@nixos:~]# parted /dev/sdc
GNU Parted 3.6
Using /dev/sdc
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) p
Model: ATA Samsung SSD 850 (scsi)
Disk /dev/sdc: 500GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number Start End Size File system Name Flags
1 1049kB 538MB 537MB fat32 disk-main1-ESP boot, esp
2 538MB 466GB 465GB disk-main1-ZFS
3 466GB 500GB 34.4GB linux-swap(v1) disk-main1-swap

Install log,

❯ nix run github:numtide/nixos-anywhere -- --flake .#fusion --build-on-remote [email protected]

Uploading install SSH keys

/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/tmp/tmp.w7sNRRiwMf/nixos-anywhere.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.
([email protected]) Password:

Number of key(s) added: 1

Now try logging into the machine, with: "ssh -o 'ConnectTimeout=10' -o 'UserKnownHostsFile=/dev/null' -o 'StrictHostKeyChecking=no' '[email protected]'"
and check to make sure that only the key(s) you wanted were added.

Gathering machine facts

Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.

Building disko script

Formatting hard drive with disko

Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.
Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.
umount: /mnt/boot unmounted
umount: /mnt/boot-fallback/efi unmounted
umount: /mnt/home (zroot/home) unmounted
umount: /mnt/var (zroot/var) unmounted
umount: /mnt (zroot/root) unmounted
++ realpath /dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X

• disk=/dev/sda
• lsblk --output-all --json
  ++ dirname /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate
• jq -r --arg disk_to_clear /dev/sda -f /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate.jq
• set -fu
• wipefs --all -f /dev/sda1
  /dev/sda1: 8 bytes were erased at offset 0x00000052 (vfat): 46 41 54 33 32 20 20 20
  /dev/sda1: 1 byte was erased at offset 0x00000000 (vfat): eb
  /dev/sda1: 2 bytes were erased at offset 0x000001fe (vfat): 55 aa
• zpool destroy -f zroot
• wipefs --all -f /dev/sda2
  /dev/sda2: 8 bytes were erased at offset 0x00029400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00027000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00025400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00025000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00024c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00024800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00024400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00024000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00023c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00023800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00023400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00023000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00022c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00022800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00022000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00021c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00069400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00067000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00065400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00065000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00064c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00064800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00064400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00064000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00063c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00063800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00063400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00063000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00062c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00062800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00062000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00061c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00061800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00061400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x00061000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa9400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa7000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa5400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa5000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa4c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa4800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa4400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa4000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa3c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa3800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa3400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa3000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa2c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa2800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa2000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa1c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa1800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa1400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50aa1000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae9400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae7000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae5400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae5000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae4c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae4800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae4400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae4000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae3c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae3800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae3400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae3000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae2c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae2800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae2000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae1c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae1800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae1400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sda2: 8 bytes were erased at offset 0x6c50ae1000 (zfs_member): 0c b1 ba 00 00 00 00 00
• umount -R '[SWAP]'
  umount: [SWAP]: not found
• swapoff /dev/sda3
• wipefs --all -f /dev/sda3
  /dev/sda3: 10 bytes were erased at offset 0x00000ff6 (swap): 53 57 41 50 53 50 41 43 45 32
  ++ zdb -l /dev/sda
  ++ sed -nr 's/ +name: '''(.*)'''/\1/p'
• zpool=rpool
• [[ -n rpool ]]
• zpool destroy -f rpool
  cannot open 'rpool': no such pool
• unset zpool
• wipefs --all -f /dev/sda
  /dev/sda: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
  /dev/sda: 8 bytes were erased at offset 0x7470c05e00 (gpt): 45 46 49 20 50 41 52 54
  /dev/sda: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
  ++ realpath /dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
• disk=/dev/sdd
• lsblk --output-all --json
  ++ dirname /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate
• jq -r --arg disk_to_clear /dev/sdd -f /nix/store/bxq686x89kkmkaamh9qnk8178gqvr4vn-disk-deactivate/disk-deactivate.jq
• set -fu
• wipefs --all -f /dev/sdd1
  /dev/sdd1: 8 bytes were erased at offset 0x00000052 (vfat): 46 41 54 33 32 20 20 20
  /dev/sdd1: 1 byte was erased at offset 0x00000000 (vfat): eb
  /dev/sdd1: 2 bytes were erased at offset 0x000001fe (vfat): 55 aa
• zpool destroy -f zroot
  cannot open 'zroot': no such pool
• wipefs --all -f /dev/sdd2
  /dev/sdd2: 8 bytes were erased at offset 0x00029400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00027000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00025400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00025000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00024c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00024800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00024400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00024000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00023c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00023800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00023400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00023000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00022c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00022800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00022000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00021c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00069400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00067000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00065400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00065000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00064c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00064800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00064400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00064000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00063c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00063800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00063400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00063000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00062c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00062800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00062000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00061c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00061800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00061400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x00061000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa9400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa7000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa5400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa5000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa4c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa4800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa4400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa4000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa3c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa3800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa3400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa3000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa2c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa2800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa2000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa1c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa1800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa1400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50aa1000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae9400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae7000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae5400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae5000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae4c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae4800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae4400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae4000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae3c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae3800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae3400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae3000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae2c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae2800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae2000 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae1c00 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae1800 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae1400 (zfs_member): 0c b1 ba 00 00 00 00 00
  /dev/sdd2: 8 bytes were erased at offset 0x6c50ae1000 (zfs_member): 0c b1 ba 00 00 00 00 00
• umount -R '[SWAP]'
  umount: [SWAP]: not found
• swapoff /dev/sdd3
• wipefs --all -f /dev/sdd3
  /dev/sdd3: 10 bytes were erased at offset 0x00000ff6 (swap): 53 57 41 50 53 50 41 43 45 32
  ++ zdb -l /dev/sdd
  ++ sed -nr 's/ +name: '''(.*)'''/\1/p'
• zpool=rpool
• [[ -n rpool ]]
• zpool destroy -f rpool
  cannot open 'rpool': no such pool
• unset zpool
• wipefs --all -f /dev/sdd
  /dev/sdd: 8 bytes were erased at offset 0x00000200 (gpt): 45 46 49 20 50 41 52 54
  /dev/sdd: 8 bytes were erased at offset 0x7470c05e00 (gpt): 45 46 49 20 50 41 52 54
  /dev/sdd: 2 bytes were erased at offset 0x000001fe (PMBR): 55 aa
  ++ mktemp -d
• disko_devices_dir=/tmp/tmp.bv48vKI9TE
• trap 'rm -rf "$disko_devices_dir"' EXIT
• mkdir -p /tmp/tmp.bv48vKI9TE
• device=/dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X
• imageSize=2G
• name=main1
• type=disk
• device=/dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X
• type=gpt
• sgdisk --new=1:0:+512M --change-name=1:disk-main1-ESP --typecode=1:EF00 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X
  Creating new GPT entries in memory.
  The operation has completed successfully.
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main1-ESP
• extraArgs=()
• declare -a extraArgs
• format=vfat
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=/boot
• type=filesystem
• mkfs.vfat /dev/disk/by-partlabel/disk-main1-ESP
  mkfs.fat 4.2 (2021-01-31)
• sgdisk --new=2:0:-32G --change-name=2:disk-main1-ZFS --typecode=2:8300 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X
  The operation has completed successfully.
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main1-ZFS
• pool=zroot
• type=zfs
• echo /dev/disk/by-partlabel/disk-main1-ZFS
• sgdisk --new=3:0:-0 --change-name=3:disk-main1-swap --typecode=3:8300 /dev/disk/by-id/ata-Samsung_SSD_850_EVO_500GB_S21GNXAG805611X
  The operation has completed successfully.
  Setting up swapspace version 1, size = 32 GiB (34358689792 bytes)
  no label, UUID=26db739b-2400-4ddf-a921-266a365f2c80
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main1-swap
• randomEncryption=1
• resumeDevice=
• type=swap
• mkswap /dev/disk/by-partlabel/disk-main1-swap
• device=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
• imageSize=2G
• name=main2
• type=disk
• device=/dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
• type=gpt
• sgdisk --new=1:0:+512M --change-name=1:disk-main2-ESP --typecode=1:EF00 /dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
  Creating new GPT entries in memory.
  The operation has completed successfully.
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main2-ESP
• extraArgs=()
• declare -a extraArgs
• format=vfat
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=/boot-fallback
• type=filesystem
• mkfs.vfat /dev/disk/by-partlabel/disk-main2-ESP
  mkfs.fat 4.2 (2021-01-31)
• sgdisk --new=2:0:-32G --change-name=2:disk-main2-ZFS --typecode=2:8300 /dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
  The operation has completed successfully.
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main2-ZFS
• pool=zroot
• type=zfs
• echo /dev/disk/by-partlabel/disk-main2-ZFS
• sgdisk --new=3:0:-0 --change-name=3:disk-main2-swap --typecode=3:8300 /dev/disk/by-id/ata-Samsung_SSD_860_EVO_500GB_S4BENS0N804509D
  The operation has completed successfully.
• udevadm trigger --subsystem-match=block
• udevadm settle
• device=/dev/disk/by-partlabel/disk-main2-swap
• randomEncryption=1
• resumeDevice=
• type=swap
• mkswap /dev/disk/by-partlabel/disk-main2-swap
  Setting up swapspace version 1, size = 32 GiB (34358689792 bytes)
  no label, UUID=61a60f7f-8d22-4e8b-a56b-1e6e51d993ca
• mode=mirror
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=
• name=zroot
• options=()
• declare -A options
• rootFsOptions=(['com.sun:auto-snapshot']='false' ['compression']='zstd')
• declare -A rootFsOptions
• type=zpool
• readarray -t zfs_devices
  ++ cat /tmp/tmp.bv48vKI9TE/zfs_zroot
• zpool create -f zroot -R /mnt mirror -O com.sun:auto-snapshot=false -O compression=zstd /dev/disk/by-partlabel/disk-main1-ZFS /dev/disk/by-partlabel/disk-main2-ZFS
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=/home
• name=home
• options=(['com.sun:auto-snapshot']='true')
• declare -A options
• type=zfs_fs
• zfs create -up zroot/home -o com.sun:auto-snapshot=true
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=/
• name=root
• options=(['com.sun:auto-snapshot']='true')
• declare -A options
• type=zfs_fs
• zfs create -up zroot/root -o com.sun:auto-snapshot=true
• mountOptions=('defaults')
• declare -a mountOptions
• mountpoint=/var
• name=var
• options=(['com.sun:auto-snapshot']='true')
• declare -A options
• type=zfs_fs
• zfs create -up zroot/var -o com.sun:auto-snapshot=true
• zfs snapshot zroot@blank
• set -efux
• zpool list zroot
• findmnt zroot/root /mnt/
• mount zroot/root /mnt/ -o X-mount.mkdir -o defaults -o zfsutil -t zfs
• findmnt /dev/disk/by-partlabel/disk-main1-ESP /mnt/boot
• mount /dev/disk/by-partlabel/disk-main1-ESP /mnt/boot -t vfat -o defaults -o X-mount.mkdir
• findmnt /dev/disk/by-partlabel/disk-main2-ESP /mnt/boot-fallback
• mount /dev/disk/by-partlabel/disk-main2-ESP /mnt/boot-fallback -t vfat -o defaults -o X-mount.mkdir
• swapon --show
  ++ readlink -f /dev/disk/by-partlabel/disk-main1-swap
• grep -q '^/dev/sda3 '
• swapon /dev/disk/by-partlabel/disk-main1-swap
• swapon --show
  ++ readlink -f /dev/disk/by-partlabel/disk-main2-swap
• grep -q '^/dev/sdd3 '
• swapon /dev/disk/by-partlabel/disk-main2-swap
• findmnt zroot/home /mnt/home
• mount zroot/home /mnt/home -o X-mount.mkdir -o defaults -o zfsutil -t zfs
• findmnt zroot/var /mnt/var
• mount zroot/var /mnt/var -o X-mount.mkdir -o defaults -o zfsutil -t zfs
• rm -rf /tmp/tmp.bv48vKI9TE

Building the system closure

trace: warning: system.stateVersion is not set, defaulting to 23.11. Read why this matters on https://nixos.org/manual/nixos/stable/options.html#opt-system.stateVersion.

Uploading the system closure

Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.

Installing NixOS

Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.
installing the boot loader...
setting up /etc...
updating GRUB 2 menu...
installing the GRUB 2 boot loader into /boot-fallback/efi...
Installing for x86_64-efi platform.
Installation finished. No error reported.
updating GRUB 2 menu...
installing the GRUB 2 boot loader into /boot/efi...
Installing for x86_64-efi platform.
/nix/store/3aw76z5l8fhd2vggsqwn49nzqkracwkh-grub-2.12-rc1/sbin/grub-install: error: failed to get canonical path of `/boot/efi'.
/nix/store/4rrnq7ysr3p7yqqfhlb351bb6slk10rv-install-grub.pl: installation of GRUB EFI into /boot/efi failed: No such file or directory
installation finished!
cannot unmount '/mnt/zroot': no such pool or dataset
Connection to 192.168.4.52 closed by remote host.

Waiting for the machine to become reachable again

Warning: Permanently added '192.168.4.52' (ED25519) to the list of known hosts.

Done!

Describe the bug

22 blocks
~/kexec/kexec/tmp.1FvK0gLD2M
+ popd
+ cat extra.gz
+ rm -r /root/kexec/kexec/tmp.1FvK0gLD2M
+ /root/kexec/kexec/kexec --load /root/kexec/kexec/bzImage --initrd=/root/kexec/kexec/initrd --command-line 'init=/nix/store/nf32aa02c3nj9qmd391vp8li1g9sazrx-nixos-system-nixos-22.11pre-git/init console=tty0 console=ttyS0,115200 nohibernate loglevel=4'
kexec_load failed: Invalid argument
entry       = 0x3ffd0760 flags = 0x3e0000
nr_segments = 5

To Reproduce

Tried to use https://github.com/tfc/nixos-anywhere-example/blob/main/single-gpt-disk-fullsize-ext4.nix with
hetzner-cloud flake on a QEMU vm

nix run github:numtide/nixos-anywhere -- [email protected] --flake .#cloud-vm

System had not enough ram, so a swapfile was created.

Also, I'm running this command from aarch64-linux (on a mac with Rosetta enabled, previously tested with building various x86_64 systems with no issues)

Expected behavior

Nixos installed

System information

Linux v7777777.local 5.15.0-43-generic #46-Ubuntu SMP Tue Jul 12 10:30:17 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

Describe the bug

[ 3099.239362] ima: impossible to appraise a kernel image without a file descriptor; try using kexec_file_load syscall.
[ 3895.765035] ima: impossible to appraise a kernel image without a file descriptor; try using kexec_file_load syscall.
kexec_load failed: Permission denied

To Reproduce

Steps to reproduce the behavior:

1. Use the simplest configuration and run --vm-test that works fine.
2. Try to ssh with the same config leads to this.

Expected behavior

Should kexec fails, I guess kexec_load_file should be tried ? This is due to IMA subsystem trying to be enforced on Azure machine I believe : https://kernsec.org/pipermail/linux-security-module-archive/2018-October/008951.html

It is due to the machine also being booted with SecureBoot.

System information

Debian Bookworm 12 : Linux dev 6.1.0-10-cloud-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.38-2 (2023-07-27) x86_64 GNU/Linux

Additional context

Specific issue to Azure VMs

A possible workaround may be (source):

To use kexec_file_load() system call, instead of kexec_load(), at kexec
command, '-s' option must be specified. See [2] for a necessary patch for
kexec-tools.

I will try this workaround and open a PR if that fixes it.

Describe the bug

ZFS pools are (apparently) not exported prior to reboot, and may fail to import on first boot.

To Reproduce

Steps to reproduce the behavior:

1. Use a disko config with a ZFS pool imported during early boot, like <disko/example/zfs.nix>
2. Set boot.zfs.forceImportRoot = false; in the NixOS config. This is “highly recommended” by NixOS' own docs
3. Run nixos-anywhere $target

Observed behaviour

System installs successfully, reboots, then initramfs fails to import the zpool.

Expected behavior

Successful import and boot.

System information

Current nixos-anywhere HEAD (e516d37)
Target was a Lenovo x230, booted on the current NixOS minimal install image (nixos-minimal-23.05.1950.fcc147b1e93-x86_64-linux.iso)

Additional context

Things work properly when running nixos-anywhere --no-reboot and manually running zpool export -a prior to rebooting.

Describe the bug

Although the console print done, when I reboot the machine in virtual machine, I could not enter the system correctly. It still shows the installation interface.

To Reproduce
This is my shell record: https://fars.ee/t/9A05.json

Expected behavior

It should enter the system correction.

System information

My nix config

https://github.com/god464/NeoDotfile

Describe the bug

If .ssh/authorized_keys is read-only on the target host, uploading our install key keeps failing in a loop.

Ran in this with a target host which is already running nixos (with the wrong disk layout, hence i want to run disko against it).
My authorized keys are managed via home-manager on this host and are therefore symlinks into the nix store and read-only.

To Reproduce

Steps to reproduce the behavior:

1. Run nixos-anywhere against a host where the targets users .ssh/authorized_keys is read-only

[...]
### Uploading install SSH keys ###
+ ssh-copy-id -i /tmp/tmp.kxhT60wlyz/nixos-anywhere.pub -o ConnectTimeout=10 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $host
/Users/phaer/.nix-profile/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/tmp/tmp.kxhT60wlyz/nixos-anywhere.pub"
/Users/phaer/.nix-profile/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/Users/phaer/.nix-profile/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
Warning: Permanently added $host' (ED25519) to the list of known hosts.
sh: line 1: .ssh/authorized_keys: Permission denied

Expected behavior

System information

Additional context

Describe the bug

When attempting to provision (or in my case re-provision) a host that has password-protected sudo, nixos-anywhere fails completely.

### Switching system into kexec ###
Pseudo-terminal will not be allocated because stdin is not a terminal.
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required

To Reproduce

Steps to reproduce the behavior:

1. Enable password on the account being connected to
2. Configure sudo to require password for actions
3. Run nixos-anywhere

Expected behavior

I expect to be prompted for a password so I can continue.

System information

Additional context

I tried all the pseudo-tty options, but none of them solve for this. I think it's a fundamental issue with the way the commands are passed to _ssh through stdin.

I found a few bugs just by using shellcheck: c5bf76c
I think this is must for this kind of projects.

Is your feature request related to a problem? Please describe.

Trying to use nixos-anywhere from my M1 Mac gives me the following error:

error: a 'x86_64-linux' with features {} is required to build '/nix/store/nvnbh63ik024vf686qh9cz29snm1ry5l-disko.drv', but I am a 'aarch64-darwin' with features {benchmark, big-parallel, nixos-test}

I'm new to nixos but was under the impression that this would work when using --build-on-remote.

The command i'm running is:

nix run github:numtide/nixos-anywhere -- --flake .#higgins --build-on-remote [email protected]

With the following code: https://github.com/micke/home-cluster/tree/18255d6428e2c5e5ec562701862ff5d133f3cd3d/install

Describe the solution you'd like

I'd like to be able to run nixos-anywhere from my mac.

Describe alternatives you've considered

I've also tried using the remote builder option without any more luck:

nix run github:numtide/nixos-anywhere --builders "ssh://10.0.10.8?user=root" -- --flake .#higgins --build-on-remote [email protected]

Additional context

Dear numtide Team,
first of all: I appreciate your great work and learn so much working with your tools.

Here comes the Question:
As far as I know root on zfs only works with keylocation=prompt. But this did not work with nixos-anywhere. So I went with keylocation = "file:///tmp/secret.key" and pass the key to nixos-anywhere with:

--disk-encryption-keys /tmp/secret.key /tmp/secret.key

After installation/before I reboot I do

$ umount -Rl /mnt
$ zpool export -a
$ reboot

On reboot the key is not found and the system refuses therefore to boot.

What ma I missing? Is there a way to use nixos-anywhere with root on encrypted zfs.

For reference: here are the relevant nix-files.

configuration.nix

# Edit this configuration file to define Options of your System. Help is available in the configuration.nix(5) man page
# and in the NixOS manual (accessible by running ‘nixos-help’).
{ config
, pkgs
, ...
}: {

  #nixpkgs.config.allowUnfree = true;
  #nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
  #  "vscode"
  #];

  boot ={
    loader = {
      # when installing toggle this to true
      efi ={
        canTouchEfiVariables = true;
        efiSysMountPoint = "/boot";
      };
      systemd-boot.enable = true;
    };
    #zfs.requestEncryptionCredentials = [ "zroot/root" ];
  };

  networking.hostName = "lombok";
  networking.hostId = "83b0a257";

  networking.networkmanager.enable = true;
  time.timeZone = null;
  services.geoclue2.enable = true;

  programs.vim.defaultEditor = true;

  environment.systemPackages = with pkgs; [
    firefox-wayland
    chromium
    vscodium
    mpv
    inkscape
    yt-dlp
    calibre
    ubuntu_font_family
    aspell
    aspellDicts.de
    aspellDicts.fr
    aspellDicts.en
    hunspell
    hunspellDicts.en-gb-ise
    signal-desktop
    gimp
    wl-clipboard
    poppler_utils
  ];

  fonts.enableDefaultFonts = true;

  documentation.doc.enable = false;
  documentation.man.enable = false;

  services.openssh.enable = true;
  services.printing = {
    enable = true;
    browsing = true;
    drivers = [ pkgs.gutenprint ];
  };
  sops.defaultSopsFile = ./secrets/secrets.yaml;
  system.stateVersion = "23.05";
}

disko.nix

{...}: {
  disko.devices = {
    disk.nvme = {
      type = "disk";
      device = "/dev/disk/by-id/nvme-Samsung_SSD_960_EVO_1TB_S3ETNB0J405286H";
      content = {
        type = "table";
        format = "gpt";
        partitions = [
          {
            name = "bootcode";
            start = "0";
            end = "1M";
            part-type = "primary";
            flags = ["bios_grub"];
          }
          {
            name = "efiboot";
            fs-type = "fat32";
            start = "1MiB";
            end = "1GiB";
            bootable = true;
            content = {
              type = "filesystem";
              format = "vfat";
              mountpoint = "/boot";
            };
          }
          {
            name = "swap";
            start = "1GiB";
            end = "17GiB";
            content = {
              type = "swap";
              randomEncryption = true;
            };
          }
          {
            name = "zroot";
            start = "17GiB";
            end = "100%";
            content = {
              type = "zfs";
              pool = "rpool";
            };
          }
        ];
      };
    };
    zpool = {
      rpool = {
        type = "zpool";
        rootFsOptions = {
          acltype = "posixacl";
          canmount = "off";
          checksum = "edonr";
          compression = "zstd";
          dnodesize = "auto";
          encryption = "aes-256-gcm";
          keyformat = "passphrase";
          # if you want to use the key for interactive login be sure there is no trailing newline
          # for example use `echo -n "password" > /tmp/secret.key`
          #keylocation = "file:///tmp/secret.key";
          keylocation = "prompt";
          mountpoint = "none";
          normalization = "formD";
          relatime = "on";
          xattr = "sa";
          "com.sun:auto-snapshot" = "false";
        };
        options = {
          ashift = "12";
          autotrim = "on";
        };

        datasets = {
          # zfs uses cow free space to delete files when the disk is completely filled
          reserved = {
            options = {
              canmount = "off";
              mountpoint = "none";
              reservation = "5GiB";
            };
            type = "zfs_fs";
          };
          home = {
            type = "zfs_fs";
            options.mountpoint = "legacy";
            mountpoint = "/home";
            options."com.sun:auto-snapshot" = "true";
            postCreateHook = "zfs snapshot rpool/home@empty";
          };
          persist = {
            type = "zfs_fs";
            options.mountpoint = "legacy";
            mountpoint = "/persist";
            options."com.sun:auto-snapshot" = "true";
            postCreateHook = "zfs snapshot rpool/persist@empty";
          };
          nix = {
            type = "zfs_fs";
            options.mountpoint = "legacy";
            mountpoint = "/nix";
            options = {
              atime = "off";
              canmount = "on";
              "com.sun:auto-snapshot" = "true";
            };
            postCreateHook = "zfs snapshot rpool/nix@empty";
          };
          root = {
            type = "zfs_fs";
            options.mountpoint = "legacy";
            mountpoint = "/";
            postCreateHook = ''
              zfs snapshot rpool/root@empty
              zfs snapshot rpool/root@lastboot
            '';
          };
        };
      };
    };
  };
}

hardware-configuration.nix

{ lib
, modulesPath
, ...
}: {
  imports = [
    (modulesPath + "/installer/scan/not-detected.nix")
  ];

  boot = {
    extraModulePackages = [ ];

    initrd = {
      availableKernelModules = [ "xhci_pci" "ahci" "nvme" "usbhid" "usb_storage" "sd_mod" "rtsx_pci_sdmmc" ];
      kernelModules = [ ];


      postDeviceCommands =
        #wipe / and /var on boot
        lib.mkAfter ''
          zfs rollback -r rpool/root@empty
      '';
    };

    kernelModules = [ "kvm-intel" ];

  fileSystems."/persist".neededForBoot = true;


  powerManagement.cpuFreqGovernor = lib.mkDefault "powersave";
}

zfs.nix

{ config, pkgs, lib, ... }:
# Other useful settings come from srvos's zfs module
{
  config = lib.mkIf config.boot.zfs.enabled {
    environment.systemPackages = [
      pkgs.zfs-prune-snapshots
    ];

    
    boot = {
      # Newest kernels might not be supported by ZFS
      kernelPackages = config.boot.zfs.package.latestCompatibleLinuxPackages;
      # ZFS does not support swapfiles, disable hibernate and set cache max
      kernelParams = [
        "nohibernate"
        "zfs.zfs_arc_max=17179869184"
      ];
      supportedFilesystems = [ "vfat" "zfs" ];
      zfs = {
        devNodes = "/dev/disk/by-id/";
        forceImportAll = true;
        removeLinuxDRM = pkgs.hostPlatform.isAarch64;
        requestEncryptionCredentials = true;
        #zfs.requestEncryptionCredentials = [ "rpool/root" ];
      };
    };
    
    services.zfs = {
      autoScrub.enable = true;
      trim.enable = true;
    };
    # Don't let zfs mount the the datasets, because of legacy mounting
    systemd.services.zfs-mount.enable = false;
  };

}

Thanks in advance.

Describe the bug

nixos-anywhere fails during the 'Gathering machine facts' phase if the shell of the remote user is fish (or presumably any other non-POSIX-compliant shell). The following error is thrown:

fish: Unsupported use of '='. In fish, please use 'set is_nixos $(if test -f /etc/os-release && grep -q ID=nixos /etc/os-release; then echo "y"; else echo "n"; fi)'.
is_nixos=$(if test -f /etc/os-release && grep -q ID=nixos /etc/os-release; then echo "y"; else echo "n"; fi)
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^

To Reproduce

Steps to reproduce the behavior:

1. Set up a user on some host with fish configured as the users's shell.
2. Run nixos-anywhere --flake my#host user@target

Expected behavior

Ideally, I'd like nixos-anywhere should try to detect the shell and optionally spawn bash/dash/sh/... to run its commands.

Alternatively, I'd like to be able to manually specify a shell to be used on the remote.

System information

I'm running revision 9df79870b04667f2d16f1a78a1ab87d124403fb7 on an aarch64-darwin host trying to target a x86_64-linux machine.

Additional context

Describe the bug

Hi,

I have just discovered this amazing project and triggered my interest in nixos.

I would like to read some input of the nix configuration from a file, like:

programs.bash.interactiveShellInit = builtins.readFile ./bash-config.sh;

But I get the error that the file cannot be found.

I have noticed that there is the option --extra-files but looking at the file nixos-anywhere.sh there are only 2 lines that reference it:

• 28
• 106

therefore seems this option is defined but discarded.

Thanks

To Reproduce

Steps to reproduce the behavior:

1. programs.bash.interactiveShellInit = builtins.readFile ./bash-config.sh;
2. nix run github:numtide/nixos-anywhere -- --flake .#server --extra-files ./bash-config.sh --debug [email protected]

Expected behavior

bash-config.sh file is available in the source

Is your feature request related to a problem? Please describe.
I want to deploy impure system and seems like it's impossible right now.

Describe the solution you'd like
Pass arguments to Nix as deploy-rs does like nixos-anywhere --flake .#nixos root@address -- --impure

The installation leaves me with a nix system but no configuration.nix or hardware-configuration.nix. What is the best way to create these files and keep them in sync with the original flake? I'd like to be able to have a "working configuration" of the server that I can restore at any time.

Describe the bug

nixos-anywhere show the following output:

Installation finished. No error reported.
installation finished!
-bash: line 10: syntax error near unexpected token `fi'

To Reproduce

Use nixos-anywhere to deploy an setup.

Expected behavior

No syntax error in nixos-anywhere

Additional context

The error messages was introduced in #160.

Apply the patch below resolve the syntax error and nixos-anywhere works again as desired.

--- a/src/nixos-anywhere.sh
+++ b/src/nixos-anywhere.sh
@@ -443,9 +443,7 @@ export PATH=\$PATH:/run/current-system/sw/bin
 mkdir -p /mnt/tmp
 chmod 777 /mnt/tmp
 nixos-install --no-root-passwd --no-channel-copy --system "$nixos_system"
-if command -v zpool >/dev/null; then
-  zpool export -a || : # we always want to export the zfs pools so people can boot from it without force import
-fi
+! command -v zpool >/dev/null || zpool export -a || : # we always want to export the zfs pools so people can boot from it without force import
 # We will reboot in background so we can cleanly finish the script before the hosts go down.
 # This makes integration into scripts easier
 nohup bash -c '${maybe_reboot}' >/dev/null &

Thank you a lot for this !

Is it possible to use this method to remote install Raspberry Pi (and generally different architectures than the client connecting to ssh host to install nixos on) from a x86_64 system?

Is your feature request related to a problem? Please describe.

It's hard to know if the script keeps working after we change it. There might be regressions.

Describe the solution you'd like

Some NixOS test spins up new machines and provisions them using nixos-remote.

Describe alternatives you've considered

Additional context

Is your feature request related to a problem? Please describe.

It would be useful to be able to bootstrap those servers using Terraform

Describe the solution you'd like

Add a terraform module sub-folder that invokes nixos-remote. Basically merge https://github.com/numtide/terraform-nixos-install into this repo.

Describe the bug

Try to nixos-anywhere an Ubuntu AWS Lightsail VPS and... it just reboot in Ubuntu :-[.

To Reproduce

Steps to reproduce the behavior:

1. Create an AWS Lightsail VPS (those are free for 3 months for new accounts I guess?)
2. Try to kexec it via nixos-anywhere.

Expected behavior

NixOS!!

System information

Ubuntu 22.04 LTS, 2GB of RAM, Xen domU.

Additional context

Describe the bug

I tried installing using the config listed here (which is as close as I could get to the config in the README while updating things to fit current disco and the hetzner disk config): https://github.com/timthelion/server-configs-la/blob/b89eded82217e70bbe1a43a7da30a5a88d960975/nixos/INSTALL_LOG2.txt#LL1C1-L1C1

Install seems to run without error but then I cannot ssh into the server after it completes.

To Reproduce

1. Create a Hetzner robot server.
2. Reboot into the recovery.
3. Clone repo above.
4. Run command nix --extra-experimental-features flakes --extra-experimental-features nix-command run github:numtide/nixos-any where -- --flake .#hetzner-cloud [email protected]
5. Wait a bit and try sshing into the server
6. Reboot the system using the hetzner admin
7. Wait a bit and try sshing into the server

Expected behavior

Sshing into the system after install works

Actual behavior

Ssh hangs

Is your feature request related to a problem? Please describe.

After using this tool for the first time and completing an install I was expecting to find a hardware-configuration.nix file somewhere but as far as I can tell there was no such file generated. I've installed nixos a few times with more traditional methods and always took that file and incorporated it into my source control.

Describe the solution you'd like

I'm thinking that this omission is deliberate, so I'd like to see the docs updated with an explanation for why there are not configs generated at /etc/nixos and what steps one might expect to need in order to have a valid config.

Describe alternatives you've considered

The other option would be to just generate the configs at /etc/nixos. The disko docs describe how to generate the hardware config without filesystems (since that conflicts with disko's disks-configs).