Installing UEFI ZFS Root on Ubuntu 22.04

Before reading further you should know that Ubuntu has a ZFS setup option since 19.10. You should use it instead of the manual installation procedure unless you need something special. In my case that special something is the native ZFS encryption, UEFI boot, and custom partitioning I find more suitable for a single disk laptop.

After booting into Ubuntu desktop installation (via “Try Ubuntu” option) we want to open a terminal. Since all further commands are going to need root credentials, we can start with that.

sudo -i

The very first step should be setting up a few variables - disk, pool, host name, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system. I like to use upper-case for ZFS pool as that’s what will appear as password prompt. It just looks nicer and ZFS doesn’t care either way.

DISK=/dev/disk/by-id/^^ata_disk^^
POOL=^^Ubuntu^^
HOST=^^desktop^^
USER=^^user^^

General idea of my disk setup is to maximize amount of space available for pool with the minimum of supporting partitions. If you are not planning to have multiple kernels, decreasing boot partition size might be a good idea (512 MB is ok). This time I decided to also add a small swap partition. While hosting swap on top of the pool itself is a perfectly valid scenario, I actually found it sometime causes issues. Separate partition seems to be slightly better.

blkdiscard -f $DISK

sgdisk --zap-all                      $DISK
sgdisk -n1:1M:+63M  -t1:EF00 -c1:EFI  $DISK
sgdisk -n2:0:+960M  -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:+4096M -t3:8200 -c3:Swap $DISK
sgdisk -n4:0:0      -t4:BF00 -c4:Pool $DISK
sgdisk --print                        $DISK

Finally we’re ready to create system ZFS pool. Note that you need to encrypt it at the moment it’s created. And yes, I still like lz4 the best.

zpool create -o ashift=12 -o autotrim=on \
    -O compression=lz4 -O normalization=formD \
    -O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
    -O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
    -O canmount=off -O mountpoint=none -R /mnt/install $POOL $DISK-part4

On top of the pool we can create a root dataset.

zfs create -o canmount=noauto -o mountpoint=/ $POOL/root
zfs mount $POOL/root

Over time I went back and forth whether to use a separate home dataset or not. In this iteration, a separate dataset it is. :)

zfs create -o canmount=noauto -o mountpoint=/home $POOL/home
zfs mount $POOL/home
zfs set canmount=on $POOL/home

Assuming we’re done with datasets, we need to do a last minute setting change.

zfs set devices=off $POOL

Assuming UEFI boot, two additional partitions are needed. One for EFI and one for booting. Unlike what you get with the official guide, here I don’t have ZFS pool for boot partition but a plain old ext4. I find potential fixup works better that way and there is a better boot compatibility. If you are thinking about mirroring, making it bigger and ZFS might be a good idea. For a single disk, ext4 will do.

# yes | mkfs.ext4 $DISK-part2
# mkdir /mnt/install/boot
# mount $DISK-part2 /mnt/install/boot/

# mkfs.msdos -F 32 -n EFI -i 4d65646f $DISK-part1
# mkdir /mnt/install/boot/efi
# mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package. Starting this step, you must be connected to the Internet.

apt update && apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool comes next. This will take a while.

debootstrap $(basename `ls -d /cdrom/dists/*/ | grep -v stable | head -1`) /mnt/install/

We can use our live system to update a few files on our new installation.

echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials. Don’t worry if this returns errors - that just means you are not using wireless.

mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we’re ready to “chroot” into our new system.

mount --rbind /dev  /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys  /mnt/install/sys
chroot /mnt/install \
    /usr/bin/env DISK=$DISK USER=$USER \
    bash --login

Let’s not forget to setup locale and time zone.

locale-gen --purge "en\_US.UTF-8" # update-locale LANG=en\_US.UTF-8 LANGUAGE=en\_US # dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we’re ready to onboard the latest Linux image.

apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

apt install --yes zfs-initramfs cryptsetup keyutils grub-efi-amd64-signed shim-signed

Since data is encrypted, we might as well use random key to encrypt our swap.

echo "swap $DISK-part3 /dev/urandom \
    swap,cipher=aes-xts-plain64,size=256,plain" >> /etc/crypttab
cat /etc/crypttab

To mount EFI, boot, and swap partitions, we need to do some fstab setup.

echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "/dev/mapper/swap none swap defaults 0 0" >> /etc/fstab
cat /etc/fstab

We might as well activate the swap now.

/etc/init.d/cryptdisks restart && sleep 5
swapon -a

Now we get grub started and update our boot environment.

KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
    --recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it’ll take a considerable amount of time.

apt install --yes ubuntu-desktop-minimal

A short package upgrade will not hurt.

add-apt-repository universe
apt update && apt dist-upgrade --yes

If one is so inclined, /home directory can get a separate dataset too but I usually skip it these days. I just proceed to create the user, assign a few extra groups to it, and make sure its home has correct owner.

adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
echo "$USER ALL=NOPASSWD:ALL" > /etc/sudoers.d/$USER
passwd $USER

As install is ready, we can exit our chroot environment.

exit

And cleanup our mount points.

umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
umount /mnt/install/home
umount /mnt/install
zpool export -a

After the reboot you should be able to enjoy your installation.

reboot

PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 21.10 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, 19.04, and 18.10.