There is a newer version of this guide for Ubuntu 19.10.
As rumors of Ubuntu 19.04 including ZFS installer proved to be a bit premature, I guess it's time for a slight adjustment to my previous ZFS instructions.
Again, all this is just a derivation on ZFS-on-Linux project's instruction for older version.
As before, we first need to get into root prompt:
Terminalsudo -i
Followed by getting a few basic packages ready:
Terminalapt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs
Disk setup is quite simple with only two partitions:
Terminalsgdisk --zap-all /dev/disk/by-id/ata_disk
sgdisk -n3:1M:+511M -t3:8300 -c3:Boot /dev/disk/by-id/ata_disk
sgdisk -n2:0:+128M -t2:EF00 -c2:EFI /dev/disk/by-id/ata_disk
sgdisk -n1:0:0 -t1:8300 -c1:Data /dev/disk/by-id/ata_disk
sgdisk --print /dev/disk/by-id/ata_disk
I believe full disk encryption should be a no-brainer so of course we set up LUKS:
Terminalcryptsetup luksFormat -q --cipher aes-xts-plain64 --key-size 512 \
--pbkdf pbkdf2 --hash sha256 /dev/disk/by-id/ata_disk-part1
cryptsetup luksOpen /dev/disk/by-id/ata_disk-part1 system
Creating ZFS stays the same as before:
Terminalzpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 \
-O normalization=formD -O xattr=sa -O mountpoint=none system /dev/mapper/system
zfs create -o canmount=noauto -o mountpoint=/mnt/system/ system/root
zfs mount system/root
Getting basic installation on our disks follows next:
Terminaldebootstrap disco /mnt/system/
zfs set devices=off system
zfs list
And then we setup EFI boot partition:
Terminalyes | mkfs.ext4 /dev/disk/by-id/ata_disk-part3
mount /dev/disk/by-id/ata_disk-part3 /mnt/system/boot/
mkdir /mnt/system/boot/efi
mkfs.msdos -F 32 -n EFI /dev/disk/by-id/ata_disk-part2
mount /dev/disk/by-id/ata_disk-part2 /mnt/system/boot/efi
We need to ensure boot partition auto-mounts:
Terminalecho PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/ata_disk-part3) \
/boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1 >> /mnt/system/etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/ata_disk-part2) \
/boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1 >> /mnt/system/etc/fstab
cat /mnt/system/etc/fstab
Before we start using anything, we should prepare a few necessary files:
Terminalecho "hostname" > /mnt/system/etc/hostname
sed 's/ubuntu/hostname/' /etc/hosts > /mnt/system/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/system/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/system/etc/netplan/
If you are installing via WiFi, you might as well copy your credentials:
Terminalmkdir -p /mnt/system/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/system/etc/NetworkManager/system-connections/
With chroot we can get the first taste of our new system:
Terminalmount --rbind --make-rslave /dev /mnt/system/dev
mount --rbind --make-rslave /proc /mnt/system/proc
mount --rbind --make-rslave /sys /mnt/system/sys
chroot /mnt/system/ /bin/bash --login
Now we can update our software:
Terminalapt update
Immediately followed with locale and time zone setup:
Terminallocale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales
dpkg-reconfigure tzdata
Now we install Linux image and basic ZFS boot packages:
Terminalapt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs
Since we're dealing with encrypted data, our cryptsetup should be also auto mounted:
Terminalapt install --yes cryptsetup keyutils
echo "system UUID=$(blkid -s UUID -o value /dev/disk/by-id/ata_disk-part1) \
none luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
cat /etc/crypttab
Now we get grub started:
Terminalapt install --yes grub-efi-amd64
And update our boot environment again (seeing errors is nothing unusual):
Terminalupdate-initramfs -u -k all
And then we finalize our grup setup:
Terminalupdate-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
--bootloader-id=Ubuntu --recheck --no-floppy
Finally we get the rest of desktop system:
Terminalapt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes
We can omit creation of the swap
dataset but I always find it handy:
Terminalzfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
-o sync=always -o primarycache=metadata -o secondarycache=none system/swap
mkswap -f /dev/zvol/system/swap
echo "/dev/zvol/system/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume
If one is so inclined, /home directory can get a separate dataset too:
Terminalrmdir /home
zfs create -o mountpoint=/home system/home
Only remaining thing before restart is to create user:
Terminaladduser user
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo user
chown -R user:user /home/user
As install is ready, we can exit our chroot environment and reboot unmount our new environment. If unmount fails, just repeat it until it doesn't. :)
Terminalexit
umount -R /mnt/system
Finally we can correct root's mount point and reboot:
Terminalzfs set mountpoint=/ system/root
reboot
Assuming nothing went wrong, your UEFI system is now ready.
[2019-10-27: Added --make-rslave]
PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 21.10 and 20.04
PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, and 18.10.
What about doing a:
# zpool export system
# zpool import -N system
# zfs set mountpoint=/ system/root
before the reboot?
That could work; will try it out. :)
Unfortunately, doesn’t work. Export fails with unmount error.
it’s busy because you need to umount /mnt/system/boot, proc, dev,sys etc.
if anything of these fails to umount, try umount -l
After mounting dev, proc and sys they should be made rslaves:
mount –make-rslave /mnt/system/proc
mount –make-rslave /mnt/system/dev
mount –make-rslave /mnt/system/sys
This makes unmounting them less of a fresh hell. Also I’m not sure if there’s a reason for this, but you have the partition numbers backwards. Shouldn’t the ESP partition be 1 and the ZFS partition be 2?
ESP partition has to be physically at the beginning of disk. However, it can be any number between 1 to 128. It’s matter of personal preference which number you give it. I like to give number 1 to my biggest partition and work from there. However, all this would work with EFI as the first partition too, i.e.:
As long as
-part1
and-part2
suffixes are swapped too, there is no difference.This was awesome, thanks! I did this before with 16.04 and 18.04, and probably could have used the 18.04 directions for 19.04, but it was nie to have 19.04 specific directions.
Worked like a charm!
-Mark
Awesome for you to post this How-To for 19.04! This is my first attempt at ZFS on Root. I’ve followed the https://github.com/zfsonlinux/zfs/wiki/Ubuntu-18.04-Root-on-ZFS and tried https://postgres.men/os/linux/ubuntu-18-04-root-on-zfs/ but without success. In my case I have a Dell T3500 with legacy bios (non UEFI). It’s ripe for use as a VM server for Docker, etc. Everything works well up till I install grub, it fails to install using the “BIOS” option given in the tuts. Would you have any suggestions or adjustment to your install how-to?
Tony~
Which version of Ubuntu you’re installing. If it’s 19.10, that could be the reason why it’s not working as ISO doesn’t contain all files needed to correctly set it up. I found that installing 19.04 and then upgrading to 19.10 works without issue. I also have a bit older BIOS boot guide at https://www.medo64.com/2019/01/setting-up-encrypted-ubuntu-18-10-zfs-desktop/. Take partition setup from it and follow newer guides when it comes to everything else.
Thank you for your response!
I’m installing on 19.04. Since this will be a VM server for Docker services I will avoid a short term release. Prior to seeing your response, I’ve successfully gotten a bootable system. What I found is that I needed to add “GRUB_DISABLE_OS_PROBER=true” to the /dev/default/grub file. The “apt install -y grub-pc” was, of course, using the default /etc/default/grub file. After the change I then reissued the the following commands where I was asked if I wanted to keep the original file. I am looking at testing “apt install -y grub-pc -o Dpkg::Options::=–force-confdef” to see if I can automate the selection for a scripted installation:
apt reinstall -y grub-pc
update-initramfs -u -k all
update-grub
grub-install $DISK1 (where $DISK1 = /dev/disks/by-id/)
I’m now looking into why I am seeing the error “lz4 compression not supported” on boot. The system does boot and I am able to finalize the installation procedure. But errors are not something to ignore.
Oops, sent w/o entering my name/email
Tony~
Trying installing Ubuntu 19.10 with LUKS+ZFS but it fails at the `update-grub` step with this error:
/usr/sbin/grub-probe: error: failed to get canonical path of `system/root`.
Any suggestions?
There seems to be issue with chroot unmounting cdrom too early and thus not all paths needed for manual installation are at their places. I am trying to figure workaround but nothing seems to work in UEFI mode. I ended up installing 19.04 and then upgrading to 19.10. :(
After modifying procedure a bit, you can use it for 19.10 too. Link here.
note for Ubuntu 20.04: update-grub will not work properly unless you set system/root mountpoint to “/”. You may set it inside chroot, then remount root partition back read/write.
Tnx! Good to know!
PS: There is an updated guide for Ubuntu 20.04 which uses a slightly different procedure so it doesn’t suffer from the same issue.