NixOS Installation Stumbling Blocks

Posted on Mon 04 September 2017 in lessons

Here are some issues I ran into installing NixOS and how I eventually got around them.

Setting up a static IP since DHCP wasn't available.

My VM was hosted in an oVirt cluster where DHCP wasn't working/configured, so the installation CD booted without a network. Here's how I manually configred a static IP:

ifconfig enp0s3 <my-static-ip> netmask <my-netmask>
route add default gw <gateway-ip>
echo "nameserver 8.8.8.8" >> /etc/resolv.conf

Partitioning the disk

I spent a lot of time messing with various partitioning schemes until I stumbled across one that worked. I didn't need disk encryption, and I didn't want to bother trying UEFI with ovirt, so here's what I ended up with.

  • A 20G disk split into /dev/sda1 and /dev/sda2
  • /dev/sda1 is a 400MB 'WIN VFAT32' partition (type 'b', not type '4' !!)
  • /dev/sda2 is a LVM partition with the rest of the space
  • For the LVM, /dev/vg/swap is an 8G swap partition and /dev/vg/root has the rest of the LVM parition

In retrospect, I think a lot of my partitioning pain may have been caused by trying to have /dev/sda1 set as a BIOS Parition (type '4'), since I suspect the BIOS partition has to be under 32M.

Also in retrospect, I see only 23M is actually used on the current /boot parition, so maybe 400MB was way too much and I should have gone with /dev/sda1 being 32M and type '4'. ¯\_(ツ)_/¯

I think I also ran into problems using fsck on the boot partition instead of fsck.vfat.

When the boot partition wasn't working, grub would fall into rescue mode and the various 'set prefix / set root / insmod' fixes like this one or this other one didn't work.

What did work here was booting the system with the install CD again, mounting /mnt/boot manually and seeing that failed, or that /mnt/boot contained gibberish after mounting, and then unmounting /mnt/boot and using testdisk to fix the partition type. Testdisk really saved the day.

Mounting the boot partition

Before running nixos-install, I had to also mount the boot partition under /mnt/boot:

> mount /dev/vg/root /mnt
> mkdir -p /mnt/boot
> mount /dev/sda1 /mnt/boot
> nixos-install

Verify the /mnt/etc/nixos/hardware-configuration.nix device paths

When I was messing with the disk partitioning, I rebuilt the /dev/sda1 partition a couple times. Apparently when you do that, you get new UUID for the device.

This meant the "/boot" file system in /mnt/etc/nixos/hardware-configuration.nix was using a device path that was no longer valid. I updated the file to point to the current /boot device and reran 'nixos-install'.

It looks like nixos-install isn't verifying the device paths are valid, since nixos-install ran OK with the invalid device paths.

Configuring a static IP in /mnt/etc/nixos/configuration.nix

Here's what I ended up adding to the configuration.nix file to set up static IP:

networking = {
    hostName = '<my hostname>';
    usePredictableInterfacenames = false;
    interfaces.eth0.ip4 = [{
        address= "<my ipv4 address>";
        prefixLength = <my netmask prefix>;
    }];
    defaultGateway = "<my gateway>"
    nameservers = [ "8.8.8.8" ];
 };

I also adding this boot setting:

boot.load.grub.device = "/dev/sda";

lessons nixos