Setting up Encrypted Ubuntu 18.10 ZFS Desktop

I have already explained how I deal with ZFS mirror setup on Ubuntu 18.10. But what about laptops that generally come with a single drive?

Well, as before basic instructions are available from ZFS-on-Linux project. However, they do have a certain way of doing things I don’t necessarily subscribe to. Here is my way of setting this up. As always, it’s best to setup remote access so you can copy/paste as steps are numerous.

As before, we first need to get into root prompt:

sudo -i

Followed by getting a few basic packages ready:

apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

We setup disks essentially the same way as in previous guide:

sgdisk --zap-all                 /dev/disk/by-id/^^ata_disk^^

sgdisk -a1 -n3:34:2047  -t3:EF02 /dev/disk/by-id/^^ata_disk^^
sgdisk     -n2:1M:+511M -t2:8300 /dev/disk/by-id/^^ata_disk^^
sgdisk     -n1:0:0      -t1:8300 /dev/disk/by-id/^^ata_disk^^

sgdisk --print                   /dev/disk/by-id/^^ata_disk^^
 …
 Number  Start (sector)    End (sector)  Size       Code  Name
    1         1050624        67108830   31.5 GiB    8300
    2            2048         1050623   512.0 MiB   8300
    3              34            2047   1007.0 KiB  EF02

Because we want encryption, we need to setup LUKS:

cryptsetup luksFormat -qc aes-xts-plain64 -s 512 -h sha256 /dev/disk/by-id/^^ata_disk^^-part1
cryptsetup luksOpen /dev/disk/by-id/^^ata_disk^^-part1 luks1

Unlike in the last guide, this time I want to have a bit of separation. Dataset system will contain the whole system, while data will contain only the home directories. Again, if you want to split it all, follow the original guide:

zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD \
    -O xattr=sa -O mountpoint=none rpool /dev/mapper/luks1
zfs create -o canmount=noauto -o mountpoint=/mnt/rpool/ rpool/system
zfs mount rpool/system

We should also setup the boot partition:

mke2fs -Ft ext2 /dev/disk/by-id/^^ata_disk^^-part2
mkdir /mnt/rpool/boot/
mount /dev/disk/by-id/^^ata_disk^^-part2 /mnt/rpool/boot/

Now we can get basic installation onto our disks:

debootstrap cosmic /mnt/rpool/
zfs set devices=off rpool
zfs list

Before we start using it, we prepare few necessary files:

cp /etc/hostname /mnt/rpool/etc/hostname
cp /etc/hosts /mnt/rpool/etc/hosts
cp /etc/netplan/*.yaml /mnt/rpool/etc/netplan/
sed '/cdrom/d' /etc/apt/sources.list > /mnt/rpool/etc/apt/sources.list

With chroot we can get the first taste of our new system:

mount --rbind /dev  /mnt/rpool/dev
mount --rbind /proc /mnt/rpool/proc
mount --rbind /sys  /mnt/rpool/sys
chroot /mnt/rpool/ /bin/bash --login

Now we can update our software and perform locale and time zone setup:

apt update

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we install Linux image and basic ZFS boot packages:

apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs

Since we’re dealing with encrypted data, our cryptsetup should be also auto mounted:

apt install --yes cryptsetup

echo "luks1 UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part1) none luks,discard,initramfs" >> /etc/crypttab
cat /etc/crypttab

And of course, we need to auto-mount our boot partition too:

echo "UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part2) /boot ext2 noatime 0 2" >> /etc/fstab
cat /etc/fstab

Now we get grub started (do select the WHOLE disk):

apt install --yes grub-pc

And update our boot environment again (seeing errors is nothing unusual):

update-initramfs -u -k all

And then we finalize our grup setup:

update-grub
grub-install /dev/disk/by-id/^^ata_disk^^

Finally we get the rest of desktop system:

apt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes

We can omit creation of the swap dataset but I always find it handy:

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput -o sync=always \
    -o primarycache=metadata -o secondarycache=none rpool/swap
mkswap -f /dev/zvol/rpool/swap
echo "/dev/zvol/rpool/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

And now is good time to swap our /home directory too:

rmdir /home
zfs create -o mountpoint=/home rpool/data

Now we are ready to create the user:

adduser -u 1002 ^^user^^
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo ^^user^^
chown -R ^^user^^:^^user^^ /home/^^user^^

Lastly we exit our chroot environment and reboot:

exit
reboot

You will get stuck after the password prompt as our mountpoint for system dataset is wrong. That’s easy to correct:

zfs set mountpoint=/ rpool/system
exit
reboot

Assuming nothing went wrong, your system is now ready.

Expanding Ext4 Volume on ZFS

Due to Dropbox’s idiotic decision to limit file system support drastically for no reason other than to piss people off, I have a small ext4 volume hosted on my ZFS pool.

Originally I made it a bit small (only 8 GB) and got Dropbox complaining. Had I created it as partition, enlarging it would be annoying task at best. However, having it exposed as ZFS block volume, resize was trivial.

First I simply increased volsize property and then told ext4 to simply use that additional space (resize2fs command):

sudo zfs set volsize=^^16G^^ ^^rpool/data/dropbox^^

sudo resize2fs ^^/dev/zvol/rpool/data/dropbox^^
 resize2fs 1.44.4 (18-Aug-2018)
 Filesystem at /dev/zvol/rpool/data/dropbox is mounted on /home/user/Dropbox; on-line resizing required
 old_desc_blocks = 1, new_desc_blocks = 2
 The filesystem on /dev/zvol/rpool/data/dropbox is now 4194304 (4k) blocks long.

Doesn’t get much easier.

Unreal Tournament 2004 Save Files (Windows 10 Edition)

As my son moved to another computer, one thing was left behind - his Unreal Tournament 2004 save file. Without it he would be just a lowly peasant climbing in the beginner ladders. With it, he is the overlord of the game.

Dramatic wording aside, he wanted his save files.

So I looked toward Internet and saw bunch of posts about moving CD key to another computer. As I got my UT2004 from GoG, that wasn’t my problem. Without an obvious solution on the first search results page, I gave up and though to my self: how hard can it be?

Well, for once these words were not the curse.

On his old computer I went into a virtual store - shady place where all programs writing to their own directories live:

"%USERPROFILE%\AppData\Local\VirtualStore\Program Files (x86)\GOG.com\Unreal Tournament 2004\Saves"

There I found a single file about 20K in size. Once I moved that file to the same location on the other computer I was done.

Booting Encrypted ZFS Mirror on Ubuntu 18.10

As I was setting up my new Linux machine with two disks, I decided to forgo my favorite Linux Mint and give Ubuntu another try. Main reason? ZFS of course.

Ubuntu already has a quite decent guide for ZFS setup but it’s slightly lacking in the mirroring department. So, here I will list steps that follow their approach closely but with slight adjustments as not only I want encrypted setup but also a proper ZFS mirror setup. If you need a single disk ZFS setup, stick with the original guide.

After booting into installation, we can go for Try Ubuntu and open a terminal. My strong suggestion would be to install openssh-server package first and connect to it remotely because that allows for copy/paste:

passwd
Changing password for ubuntu.``
 (current) UNIX password: ^^(empty)^^
 Enter new UNIX password: ^^password^^
 Retype new UNIX password: ^^password^^
 passwd: password updated successfully

sudo apt install --yes openssh-server

Regardless if you continue directly or you connect via SSH (username is ubuntu), the first task is to get onto root prompt and never leave it again. :)

sudo -i

To get the ZFS on, we need Internet connection and extra repository:

sudo apt-add-repository universe
apt update

Now we can finally install ZFS, partitioning utility, and an installation tool:

apt install --yes debootstrap gdisk zfs-initramfs

First we clean the partition table on disks followed by a few partition definitions (do change ID to match your disks):

sgdisk --zap-all /dev/disk/by-id/^^ata_disk1^^
sgdisk --zap-all /dev/disk/by-id/^^ata_disk2^^

sgdisk -a1 -n2:34:2047 -t2:EF02 /dev/disk/by-id/^^ata_disk1^^
sgdisk -a1 -n2:34:2047 -t2:EF02 /dev/disk/by-id/^^ata_disk2^^

sgdisk     -n3:1M:+512M -t3:EF00 /dev/disk/by-id/^^ata_disk1^^
sgdisk     -n3:1M:+512M -t3:EF00 /dev/disk/by-id/^^ata_disk2^^

sgdisk     -n4:0:+512M  -t4:8300 /dev/disk/by-id/^^ata_disk1^^
sgdisk     -n4:0:+512M  -t4:8300 /dev/disk/by-id/^^ata_disk2^^

sgdisk     -n1:0:0      -t1:8300 /dev/disk/by-id/^^ata_disk1^^
sgdisk     -n1:0:0      -t1:8300 /dev/disk/by-id/^^ata_disk2^^

After all these we should end up with both disks showing 4 distinct partitions:

sgdisk --print /dev/disk/by-id/^^ata_disk1^^
 …
 Number  Start (sector)    End (sector)  Size       Code  Name
    1         2099200        67108830   31.0 GiB    8300
    2              34            2047   1007.0 KiB  EF02
    3            2048         1050623   512.0 MiB   EF00
    4         1050624         2099199   512.0 MiB   8300

With partitioning done, it’s time to encrypt our disks and mount them (note that we only encrypt the first partition, not the whole disk):

cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 /dev/disk/by-id/^^ata_disk1^^-part1
cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha256 /dev/disk/by-id/^^ata_disk2^^-part1

cryptsetup luksOpen /dev/disk/by-id/^^ata_disk1^^-part1 luks1
cryptsetup luksOpen /dev/disk/by-id/^^ata_disk2^^-part1 luks2

Finally we can create our pool (rpool is a “standard” name) consisting of both encrypted devices:

zpool create -o ashift=12 -O atime=off -O compression=lz4 \
    -O normalization=formD -O xattr=sa -O mountpoint=/ -R /mnt/rpool \
    rpool mirror /dev/mapper/luks1 /dev/mapper/luks2

There is advantage into creating fine grained datasets as the official guide instructs, but I personally don’t do it. Having one big free-for-all pile is OK for me - anything of any significance I anyhow keep on my network drive where I have properly setup ZFS with rights, quotas, and all other goodies.

Since we are using LUKS encryption, we do need to mount 4th partition too. We’ll do it for both disks and deal with syncing them later:

mkdir /mnt/rpool/boot
mke2fs -t ext2 /dev/disk/by-id/ata_disk1-part4
mount /dev/disk/by-id/ata_disk1-part4 /mnt/rpool/boot

mkdir /mnt/rpool/boot2
mke2fs -t ext2 /dev/disk/by-id/^^ata_disk2^^-part4
mount /dev/disk/by-id/^^ata_disk2^^-part4 /mnt/rpool/boot2

Now we can finally start copying our Linux (do check for current release codename using lsb_release -a). This will take a while:

debootstrap ^^cosmic^^ /mnt/rpool/

Once done, turn off devices flag on pool and check if data has been written or we messed the paths up:

zfs set devices=off rpool

zfs list
 NAME    USED  AVAIL  REFER  MOUNTPOINT
 rpool   218M  29.6G   217M  /mnt/rpool

Since our system is bare, we do need to prepare a few configuration files:

cp /etc/hostname /mnt/rpool/etc/hostname
cp /etc/hosts /mnt/rpool/etc/hosts
cp /etc/netplan/*.yaml /mnt/rpool/etc/netplan/
sed '/cdrom/d' /etc/apt/sources.list > /mnt/rpool/etc/apt/sources.list

Finally we get to try our our new system:

mount --rbind /dev  /mnt/rpool/dev
mount --rbind /proc /mnt/rpool/proc
mount --rbind /sys  /mnt/rpool/sys
chroot /mnt/rpool/ /bin/bash --login

Once in our new OS, a few further updates are in order:

apt update

locale-gen --purge "^^en_US.UTF-8^^"
update-locale LANG=^^en_US.UTF-8^^ LANGUAGE=^^en_US^^
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we need to install linux image and headers:

apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Then we configure booting ZFS:

apt install --yes zfs-initramfs
echo UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk1^^-part4) /boot  ext2 noatime 0 2 >> /etc/fstab
echo UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk2^^-part4) /boot2 ext2 noatime 0 2 >> /etc/fstab

And disk decryption:

apt install --yes cryptsetup
echo "luks1 UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk1^^-part1) none luks,discard,initramfs" >> /etc/crypttab
echo "luks2 UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk2^^-part1) none luks,discard,initramfs" >> /etc/crypttab

And install grub bootloader (select both disks - not partitions!):

apt install --yes grub-pc

Followed by update of boot environment (some errors are ok):

update-initramfs -u -k all
 update-initramfs: Generating /boot/initrd.img-4.18.0-12-generic
 cryptsetup: ERROR: Couldn't resolve device rpool
 cryptsetup: WARNING: Couldn't determine root device

Now we update the grub and fix its config (only needed if you are not using sub-datasets):

update-grub
sed -i "s^root=ZFS=rpool/^root=ZFS=rpool^g" /boot/grub/grub.cfg

Now we get to copy all boot files to second disk:

cp -rp /boot/* /boot2/

With grub install we’re getting close to the end of story:

grub-install /dev/disk/by-id/^^ata_disk1^^
 Installing for i386-pc platform.
 Installation finished. No error reported.

grub-install /dev/disk/by-id/^^ata_disk2^^
 Installing for i386-pc platform.
 Installation finished. No error reported.

Now we install full GUI and upgrade whatever needs it (takes a while):

sudo apt-get install --yes ubuntu-desktop samba
apt dist-upgrade --yes

As this probably updated grub, we need to both correct config (only if we have bare dataset) and copy files to the other boot partition (this has to be repeated on every grub update):

sed -i "s^root=ZFS=rpool/^root=ZFS=rpool^g" /boot/grub/grub.cfg
cp -rp /boot/* /boot2/

Having some swap is always a good idea:

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput -o sync=always \
    -o primarycache=metadata -o secondarycache=none rpool/swap

mkswap -f /dev/zvol/rpool/swap
echo /dev/zvol/rpool/swap none swap defaults 0 0 >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

Almost there, it’s time to set root password:

passwd

And to create our user for desktop environment:

adduser ^^user^^
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo ^^user^^

Finally, we can reboot (don’t forget to remove CD) and enjoy our system:

exit
reboot

WordPress Not Showing Dialogs

Illustration

My WordPress installation is generally troubleless. Yes, there was an issue with upgrade or occasional missed post, but nothing that would annoy me too much. Except…

For quite a while I’ve been having trouble with the plugin detail page. Whenever I tried to open one, it would greet me with Refused to connect or something similar (depending on browser and its version). Mind you, everything would work but some annoyance would always linger.

Until one day I accidentally opened it in a separate tab. Wouldn’t you know it, the damn thing worked. That lead me to check console tools where I saw WordPress attempting to use iframe. Quick check of my web server configuration told me I had HTTP header X-Frame-Options set to "deny". To make it simple, I was blocking myself. Change to a slightly more permissive X-Frame-Options="sameorigin" solved it.

Minor issue that I have been ignoring for better part of the year solved by a pure accident. Doesn’t get better than that. :)