Symbolic Links on VirtualBox Share

If you are running VirtualBox with Linux guest under a Linux host, you will definitely need symbolic links at one point in time. And, while you can do links toward shared folders, you cannot create a link on shared folder itself. You will get something like:

ln -s /something /somethingelse
 ln: failed to create symbolic link '/somethingElse': Read-only file system

Well, fortunately it’s easy to enable connection on per share basis. Just run the following command on your host (replacing GuestName and ShareName with correct information):

VBoxManage setextradata '^^GuestName^^' VBoxInternal2/SharedFoldersEnableSymlinksCreate/^^ShareName^^ 1

If your machine is running, you will need to fully shut it down (no, restart doesn’t help) and you get to enjoy the full symbolic link glory.

Installing UEFI ZFS Root on Ubuntu 19.04

There is a newer version of this guide for Ubuntu 19.10.

As rumors of Ubuntu 19.04 including ZFS installer proved to be a bit premature, I guess it’s time for a slight adjustment to my previous ZFS instructions.

Again, all this is just a derivation on ZFS-on-Linux project’s instruction for older version.

As before, we first need to get into root prompt:

sudo -i

Followed by getting a few basic packages ready:

apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

Disk setup is quite simple with only two partitions:

sgdisk --zap-all                      /dev/disk/by-id/^^ata_disk^^

sgdisk -n3:1M:+511M -t3:8300 -c3:Boot /dev/disk/by-id/^^ata_disk^^
sgdisk -n2:0:+128M  -t2:EF00 -c2:EFI  /dev/disk/by-id/^^ata_disk^^
sgdisk -n1:0:0      -t1:8300 -c1:Data /dev/disk/by-id/^^ata_disk^^

sgdisk --print                        /dev/disk/by-id/^^ata_disk^^

I believe full disk encryption should be a no-brainer so of course we set up LUKS:

cryptsetup luksFormat -q --cipher aes-xts-plain64 --key-size 512 \
    --pbkdf pbkdf2 --hash sha256 /dev/disk/by-id/^^ata_disk^^-part1
cryptsetup luksOpen /dev/disk/by-id/^^ata_disk^^-part1 system

Creating ZFS stays the same as before:

zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 \
      -O normalization=formD -O xattr=sa -O mountpoint=none system /dev/mapper/system
zfs create -o canmount=noauto -o mountpoint=/mnt/system/ system/root
zfs mount system/root

Getting basic installation on our disks follows next:

debootstrap disco /mnt/system/
zfs set devices=off system
zfs list

And then we setup EFI boot partition:

yes | mkfs.ext4 /dev/disk/by-id/^^ata_disk^^-part3
mount /dev/disk/by-id/^^ata_disk^^-part3 /mnt/system/boot/

mkdir /mnt/system/boot/efi
mkfs.msdos -F 32 -n EFI /dev/disk/by-id/^^ata_disk^^-part2
mount /dev/disk/by-id/^^ata_disk^^-part2 /mnt/system/boot/efi

We need to ensure boot partition auto-mounts:

echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/^^ata_disk^^-part3) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1 >> /mnt/system/etc/fstab
echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/^^ata_disk^^-part2) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1 >> /mnt/system/etc/fstab
cat /mnt/system/etc/fstab

Before we start using anything, we should prepare a few necessary files:

echo "^^hostname^^" > /mnt/system/etc/hostname
sed 's/ubuntu/^^hostname^^/' /etc/hosts > /mnt/system/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/system/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/system/etc/netplan/

If you are installing via WiFi, you might as well copy your credentials:

mkdir -p /mnt/system/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/system/etc/NetworkManager/system-connections/

With chroot we can get the first taste of our new system:

mount --rbind --make-rslave /dev  /mnt/system/dev
mount --rbind --make-rslave /proc /mnt/system/proc
mount --rbind --make-rslave /sys  /mnt/system/sys
chroot /mnt/system/ /bin/bash --login

Now we can update our software:

apt update

Immediately followed with locale and time zone setup:

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we install Linux image and basic ZFS boot packages:

apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs

Since we’re dealing with encrypted data, our cryptsetup should be also auto mounted:

apt install --yes cryptsetup keyutils

echo "system UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part1) \
    none luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab

cat /etc/crypttab

Now we get grub started:

apt install --yes grub-efi-amd64

And update our boot environment again (seeing errors is nothing unusual):

update-initramfs -u -k all

And then we finalize our grup setup:

update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
    --bootloader-id=Ubuntu --recheck --no-floppy

Finally we get the rest of desktop system:

apt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes

We can omit creation of the swap dataset but I always find it handy:

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
    -o sync=always -o primarycache=metadata -o secondarycache=none system/swap
mkswap -f /dev/zvol/system/swap
echo "/dev/zvol/system/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too:

rmdir /home
zfs create -o mountpoint=/home system/home

Only remaining thing before restart is to create user:

adduser ^^user^^
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo ^^user^^
chown -R ^^user^^:^^user^^ /home/^^user^^

As install is ready, we can exit our chroot environment and reboot unmount our new environment. If unmount fails, just repeat it until it doesn’t. :)

exit
umount -R /mnt/system

Finally we can correct root’s mount point and reboot:

zfs set mountpoint=/ system/root
reboot

Assuming nothing went wrong, your UEFI system is now ready.

[2019-10-27: Added --make-rslave]

PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 21.10 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, and 18.10.

Appending to Certificate Bundle

When one needs to add an extra certificate to a certificate bundle, the first idea might be something like this:

cat example.pem >> /etc/ssl/certs/ca-bundle.crt

And that will work - if you are a root user. However, if you are just a sudoer, or God-forbid a cloud-user, you might find yourself up the creek without a paddle.

You see, sudo won’t simply do as it operates on the source and not the destination:

sudo cat example.pem >> /etc/ssl/certs/ca-bundle.crt
 -bash: /etc/ssl/certs/ca-bundle.crt: Permission denied

You might want to play with tee, but depending on your exact permissions that might fail also.

cat example.pem | sudo tee -a /etc/ssl/certs/ca-bundle.crt

However, one command that never fails is vi as it has quite a lot of code to work out writing to read-only files. And conveniently, we can script it too:

sudo ex +"r /opt/install/ca.pem" -scwq! /etc/ssl/certs/ca-bundle.crt

Yep, we essentially start vi, read the extra content into it, and then force-save it all.

Configuring Thumb Button on M720 Under Ubuntu

The most usable function I found for my mouse thumb button is actually Minimize window. When you combine it with Alt+Tab, it does wonders for fast window switching. Under Windows it’s trivial to configure this within Logitech Options. While not as easy in Linux, it’s actually not that hard either.

Since the Thumb button on most Logitech’s devices is not a mouse button but a key combination, we can use Ubuntu’s built-in Keyboard utility instead of remapping buttons. Or we can do it from command line equally easy:

gsettings set org.gnome.desktop.wm.keybindings minimize "['<Primary><Alt>Tab']"

If you really want to have quick switching, you might want to also disable animations:

gsettings set org.gnome.desktop.interface enable-animations false

With those two changes you have your thumb button configured.

PS: And yes, this works for MX Master too.

Remap M720 Mouse Buttons Under Ubuntu

M720 mouse has quite a few buttons. But remapping them under Linux is not necessarily the easiest thing in the world. For example, if one wants Forward and Backward buttons remapped to window left/right snap, there are a few things that need to be done manually.

First we need a few packages:

sudo apt install -y xbindkeys xautomation

Then we need to figure out which code our forward and backward keys are:

xev | grep -A 2 Button

In my case, these were 8 and 9 respectively.

Then we need to write mappings in ~/.xbindkeysrc:

"xte 'keydown Super_L' 'key Left' 'keyup Super_L'"
b:9

"xte 'keydown Super_L' 'key Right' 'keyup Super_L'"
b:8

And lastly, we need to restart xbindkeys:

killall xbindkeys 2>/dev/null
xbindkeys

If all went well, your buttons now know a new trick.

UEFI Install for Root ZFS Ubuntu 18.10

There is a newer version of this guide for Ubuntu 19.04.

Booting ZFS Ubuntu of MBR is story I already told. But what if we want an encrypted UEFI ZFS setup?

Well, it’s quite simple to previous steps and again just a derivation on ZFS-on-Linux project.

As before, we first need to get into root prompt:

sudo -i

Followed by getting a few basic packages ready:

apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

Disk setup is quite simple with only two partitions:

sgdisk --zap-all             /dev/disk/by-id/^^ata_disk^^

sgdisk -n2:1M:+511M -t2:EF00 /dev/disk/by-id/^^ata_disk^^
sgdisk -n1:0:0      -t1:8300 /dev/disk/by-id/^^ata_disk^^

sgdisk --print               /dev/disk/by-id/^^ata_disk^^
 Number  Start (sector)    End (sector)  Size       Code  Name
    1         1050624        67108830   31.5 GiB    8300
    2            2048         1050623   512.0 MiB   8300

I believe full disk encryption should be a no-brainer so of course we set up LUKS:

cryptsetup luksFormat -qc aes-xts-plain64 -s 512 -h sha256 /dev/disk/by-id/^^ata_disk^^-part1
cryptsetup luksOpen /dev/disk/by-id/^^ata_disk^^-part1 luks1

Creating ZFS stays the same as before:

zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD \
    -O xattr=sa -O mountpoint=none rpool /dev/mapper/luks1
zfs create -o canmount=noauto -o mountpoint=/mnt/rpool/ rpool/system
zfs mount rpool/system

Getting basic installation on our disks follows next:

debootstrap cosmic /mnt/rpool/
zfs set devices=off rpool
zfs list

And then we setup EFI boot partition:

mkdosfs -F 32 -n EFI /dev/disk/by-id/^^ata_disk^^-part2
mount /dev/disk/by-id/^^ata_disk^^-part2 /mnt/rpool/boot/

We need to ensure boot partition auto-mounts:

echo PARTUUID=$(blkid -s PARTUUID -o value /dev/disk/by-id/^^ata_disk^^-part2) /boot vfat noatime,nofail,x-systemd.device-timeout=5s 0 1 >> /mnt/rpool/etc/fstab
cat /mnt/rpool/etc/fstab

Before we start using anything, we should prepare a few necessary files:

cp /etc/hostname /mnt/rpool/etc/hostname
cp /etc/hosts /mnt/rpool/etc/hosts
cp /etc/netplan/*.yaml /mnt/rpool/etc/netplan/
sed '/cdrom/d' /etc/apt/sources.list > /mnt/rpool/etc/apt/sources.list

If you are dual-booting system with Windows, do consider turning off UTC BIOS time:

echo UTC=no >> /mnt/rpool/etc/default/rc5

With chroot we can get the first taste of our new system:

mount --rbind /dev  /mnt/rpool/dev
mount --rbind /proc /mnt/rpool/proc
mount --rbind /sys  /mnt/rpool/sys
chroot /mnt/rpool/ /bin/bash --login

Now we can update our software:

apt update

Immediately followed with locale and time zone setup:

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we install Linux image and basic ZFS boot packages:

apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs

Since we’re dealing with encrypted data, our cryptsetup should be also auto mounted:

apt install --yes cryptsetup

echo "luks1 UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part1) none luks,discard,initramfs" >> /etc/crypttab
cat /etc/crypttab

Now we get grub started:

apt install --yes grub-efi-amd64

And update our boot environment again (seeing errors is nothing unusual):

update-initramfs -u -k all

And then we finalize our grup setup:

update-grub
grub-install --target=x86_64-efi --efi-directory=/boot --bootloader-id=ubuntu --recheck --no-floppy

Finally we get the rest of desktop system:

apt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes

We can omit creation of the swap dataset but I always find it handy:

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput -o sync=always \
    -o primarycache=metadata -o secondarycache=none rpool/swap
mkswap -f /dev/zvol/rpool/swap
echo "/dev/zvol/rpool/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too:

rmdir /home
zfs create -o mountpoint=/home rpool/data

Only remaining thing before restart is to create user:

adduser ^^user^^
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo ^^user^^
chown -R ^^user^^:^^user^^ /home/^^user^^

As install is ready, we can exit our chroot environment and reboot:

exit
reboot

You will get stuck after the password prompt as our mountpoint for system dataset is wrong. That’s easy to correct:

zfs set mountpoint=/ rpool/system
exit
reboot

Assuming nothing went wrong, your UEFI system is now ready.

PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 21.10 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, and 19.04.

Exporting JSON via Awk

I wanted to process Unicode CSV file to extract the first two columns into JSON. With awk it seemed easy enough:

awk '
    BEGIN {
        FS=";"
        print "["
    }
    {
        print "  { \"code\": \"" $1 "\", \"description\": \"" $2 "\" },"
    }
    END {
        print "]"
    }
    ' UnicodeData.txt | less

This will give you ALMOST parsable output. One thing that will spoil it is the last “hanging” comma making the whole JSON invalid (albeit some parsers will still load it). And no, there is no way to tell awk to do something special with the last line as processing of the lines is done one-by-one and thus there is no telling which line is last at any give moment.

What we can do is tell awk to process lines with a single line delay:

awk '
    BEGIN {
        FS=";"
        print "["
    }
    NR>1 {
        print "    { \"code\": \"" code "\", \"description\": \"" description "\" },"
    }
    {
        code = $1
        description = $2
    }
    END {
        print "    { \"code\": \"" code "\", \"description\": \"" description "\" }"
        print "]"
    }
    ' UnicodeData.txt | less

This prints content starting from the second line (NR>1) and what we do in the main loop is just storing fields into our variables that’ll be read in the next iteration. Essentially what we have is a single line delay mechanism. To catch up with the last line we just print it out without trailing comma in END portion of our program.

Valid JSON at last.

Common Code for Files and Pipes in Rust

To read a file in Rust, the following code is easy enough to figure:

fn open(file_name: &str) -> Result<(), Error> {
    let input = File::open(file_name)?;
    let mut reader = io::BufReader::new(input);
    …

What if you want to read from standard input when file is not given? In many other languages you might solve this using interface. However, in Rust we haven’t got interface support. What we do have are traits.

fn parse(file_name: Option<&str>) -> Result<(), Error> {
    let input = match file_name {
        Some(file_name) => Box::new(File::open(file_name)?) as Box<Read>,
        None => Box::new(io::stdin()) as Box<Read>,
    };
    let mut reader = io::BufReader::new(input);
    …

As you can see, it reads almost exactly like an interface would, the only curiosity being explicit boxing you have to do.

To see the code in context, you can check IniEd and it’s implementation.

Local Resolve of Proxied SSH Host Names

If you need to use one SSH machine to jump over to other hosts, you know the drill. The easiest way is to simply edit ~/.ssh/config and add ProxyJump definitions:

Host tunnel
  HostName      ^^192.168.0.1^^
  User          ^^myname^^
  IdentityFile  ~/.ssh/^^id_rsa^^

Host 192.168.1.*
  ProxyJump     tunnel

Example file above essentially assumes you have your jump point at 192.168.0.1 and you are using it to get into machines in 192.168.1.0/24 network. To go into 192.168.1.100, you would simply use

ssh user@192.168.1.100

SSH is then going to use it’s config definitions to connect to tunnel machine first (192.168.0.1) and then to make another connection from that machine to the final destination (192.168.1.100). Easy enough.

However, what if we want names to be resolved too?

If you have DNS or those names defined on your jump point, all is ok. However, what if your jump point is not under your control or you are too lazy to keep /etc/hosts up-to-date on both your local machine and the jump one?

Well, you will see the following error message:

ssh user@myremotehost.internal
 ssh: Could not resolve hostname myremotehost.internal: Name or service not know

In that case, you will need ProxyCommand and dig magic in ~/.ssh/config to do local IP resolve.

Host *.^^internal^^
  ProxyCommand           ssh -W "[`dig +short %h`]:%p" tunnel

Example above will locally resolve all IPs for host names ending in .internal before using the resolved IP on the jump host.

Fixing Discord Tray on Ubuntu 18.10

On my Ubuntu 18.10 installation, I had a peculiar problem. While Discord did work, it didn’t appear in the tray. Not broken enough to warrant immediate action, but broken enough to annoy me.

A quick search on Internet narrowed the culprit to unset desktop environment variable. Simply setting XDG_CURRENT_DESKTOP environment variable to Unity fixed the issue altogether.

Armed with that knowledge, I first modified the application launcher for both desktop and startup:

sudo sed -i 's^Exec=/usr/share/discord/Discord^Exec=/usr/bin/env XDG_CURRENT_DESKTOP=Unity /usr/share/discord/Discord^' \
    /usr/share/discord/discord.desktop \
    ~/.config/autostart/discord-stable.desktop

To propagate the change, a quick update of desktop database was in order:

sudo update-desktop-database

Followed by termination of the existing Discord instances:

killall Discord ; killall Discord

Once I started Discord again, its icon appeared in its rightful place.