Lenovo P70 Fingerprint Reader Under Ubuntu

Fingerprint support under Linux is spotty at best. For many laptops one will never be able to use it. Fortunately, somebody went through the effort to make drivers for VFS7500 Touch Fingerprint Sensor used on P70.

Unfortunately instructions to install it on Ubuntu 20.04 are not really complete so I here is what worked for me. Newer Ubuntu steps should be reasonably similar if not same but I haven’t checked them from scratch.

First I installed a few packages. Since drivers use combination of snap and python, we obviously need them. GMP library is needed to as it’s dependency of pyfast library.

sudo apt-get install --yes snapd python3-pip libgmp-dev
sudo pip install pyusb pycrypto pyfast

Then we can follow original install procedure:

sudo snap install validity-sensors-tools
sudo snap connect validity-sensors-tools:raw-usb
sudo snap connect validity-sensors-tools:hardware-observe

With all in place we can finally initialize reader:

sudo validity-sensors-tools.initializer

Lastly, install libfprint TOD driver.

sudo add-apt-repository -u ppa:3v1n0/libfprint-vfs0090
sudo apt-get install --yes libfprint-2-tod-vfs0090

Now you should see Fingerprint Login in system’s Users applet. Once you enroll your one or more finger, you should be able to use it for login.

Manually Installing Ubuntu 21.04 on Surface Go

Now, one can install Ubuntu perfectly well onto Surface Go without any shenanigans. Just follow a guide on how to boot install USB and you’re golden. But I like my installations to be a bit special. :)

After booting into Ubuntu desktop installation one needs a root prompt. All further commands are going to need root credentials anyhow.

$ sudo -i

The very first step should be setting up a few variables - disk, host, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system.

DISK=/dev/disk/by-id/^^ata_disk^^
HOST=^^desktop^^
USER=^^user^^

Disk setup is really minimal. Please note you can actually reduce size of boot partition but that might get you in trouble if you start playing with low latency kernel. Some extra space will help here.

blkdiscard $DISK

sgdisk --zap-all                       $DISK
sgdisk -n1:1M:+47M -t1:EF00 -c1:EFI    $DISK
sgdisk -n2:0:+720M -t2:8300 -c2:Boot   $DISK
sgdisk -n3:0:0     -t3:8309 -c3:Ubuntu $DISK

sgdisk --print                         $DISK

I usually encrypt just the root partition as having boot partition unencrypted does offer advantages and having standard kernels exposed is not much of a security issue.

cryptsetup luksFormat -q --cipher aes-xts-plain64 --key-size 256 \
    --pbkdf pbkdf2 --hash sha256 $DISK-part3

Since crypt device name is displayed on every startup, for Surface Go I like to use host name here.

cryptsetup luksOpen $DISK-part3 ${HOST^}

Now we can prepare all needed partitions.

yes | mkfs.ext4 /dev/mapper/${HOST^}
mkdir /mnt/install
mount /dev/mapper/${HOST^} /mnt/install/

yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot/

mkfs.msdos -F 32 -n EFI $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package.

apt update ; apt install --yes debootstrap

And then we can get basic OS on the disk. This will take a while.

debootstrap $(basename `ls -d /cdrom/dists/*/ | head -1`) /mnt/install/

Our newly copied system is lacking a few files and we should make sure they exist before proceeding.

echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials:

mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we’re ready to “chroot” into our new system.

mount --rbind /dev  /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys  /mnt/install/sys
chroot /mnt/install \
    /usr/bin/env DISK=$DISK HOST=$HOST USER=$USER \
    bash --login

Let’s not forget to setup locale and time zone.

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we’re ready to onboard the latest Linux image.

apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

apt install --yes initramfs-tools cryptsetup keyutils grub-efi-amd64-signed shim-signed tasksel

Since we’re dealing with encrypted data, we should auto mount it via crypttab. If there are multiple encrypted drives or partitions, keyscript really comes in handy to open them all with the same password. As it doesn’t have negative consequences, I just add it even for a single disk setup.

echo "${HOST^} UUID=$(blkid -s UUID -o value $DISK-part3)  none \
    luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
cat /etc/crypttab

To mount boot and EFI partition, we need to do some fstab setup too:

echo "UUID=$(blkid -s UUID -o value /dev/mapper/${HOST^}) \
    / ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
cat /etc/fstab

Now we update our boot environment.

KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

sed -i "s/^GRUB_CMDLINE_LINUX_DEFAULT.*/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash \
    mem_sleep_default=deep\"/" /etc/default/grub
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
    --recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it’ll take a considerable amount of time.

tasksel install ubuntu-desktop-minimal

Short package upgrade will not hurt.

add-apt-repository universe
apt update ; apt dist-upgrade --yes

The only remaining task before restart is to create the user, assign a few extra groups to it, and make sure its home has correct owner.

adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
echo "$USER ALL=NOPASSWD:ALL" > /etc/sudoers.d/$USER
passwd $USER

Before finishing it up, I like to install Surface Go WiFi and backlight tracer packages. This will allow for usage of wireless once we boot into installed system and for remembering light level between plugged/unplugged states.

wget -O /tmp/surface-go-wifi_amd64.deb \
    https://www.medo64.com/download/surface-go-wifi_0.0.5_amd64.deb
apt install --yes /tmp/surface-go-wifi_amd64.deb

wget -O /tmp/backlight-tracer_amd64.deb \
    https://www.medo64.com/download/backlight-tracer_0.1.1_all.deb
apt install --yes /tmp/backlight-tracer_amd64.deb

As install is ready, we can exit our chroot environment.

exit

And unmount our disk:

umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}

After the reboot you should be able to enjoy your installation.

reboot

Once booted I like to setup suspend to react on power button and and to disable automatic brightness changes.

gsettings set org.gnome.settings-daemon.plugins.power button-power 'suspend'
gsettings set org.gnome.settings-daemon.plugins.power power-button-action 'suspend'
gsettings set org.gnome.settings-daemon.plugins.power ambient-enabled 'false'
gsettings set org.gnome.mutter experimental-features "['x11-randr-fractional-scaling']"

My preferred scale factor is 150% (instead of default 200%) but you’ll need to change that in settings manually.

HexDump's Illegal Seek

After I upgraded to Ubuntu 21.04, my TmpUsb script suddenly started reporting the following hexdump: stdin: Illegal seek.

Line causing issue was the one determining partition serial number:

dd if=/dev/sda bs=512 skip=1 count=1 | hexdump -s39 -n4 -e '4/1 "%02X"'

It seems that hexdump got a bit too stricter with its input parameters and now disallows skipping bytes in fifo stream. I haven’t investigated much but my guess is that skipping 39 bytes probably messes with its internal buffer. In any case, dd has no such issues so the same code can be done without skipping in hexdump.

dd if=/dev/sda bs=1 skip=551 count=4 | hexdump -n4 -e '4/1 "%02X"'`

The best part is that this is compatible with older versions too.

Dazed and Confused, but Trying to Continue

From time to time, I would see the following slightly poetic statement on my console.

Uhhuh. NMI received for unknown reason 31 on CPU 3.
Do you have a strange power saving mode enabled?
Dazed and confused, but trying to continue

The first Internet search brought me sadness and dismay - my hardware was failing. It took going a bit deeper to find that AMD servers have that issue quite often even in the absence of real hardware failure.

Solution? Disable darn C-States. It’s a server after all.

PS: And no, even if you want to keep C-States, reason 31 is nothing to worry about - it’s been happening on my system for 2 years before this and I had no issues with it. It’s just annoyance and nothing more.

Recording Both Microphone and Speaker Under Ubuntu 21.04

When one records audio under Linux, issue that quite a few applications have is recording both microphone input and headphone output. And that’s true for SimpleScreenRecorder, otherwise really good recording application.

However, Linux always has a way around those restrictions and those can be actually found on SimpleScreenRecorder pages if you look deep enough.

pactl load-module module-null-sink \
  sink_name=duplex_out sink_properties=device.description="\"Duplex\ Output\""
pactl load-module module-null-sink \
  sink_name=app_out sink_properties=device.description="\"Application\ Output\""
pactl load-module module-loopback source=app_out.monitor
pactl load-module module-loopback source=app_out.monitor sink=duplex_out
pactl load-module module-loopback sink=duplex_out
pactl set-default-sink app_out

Trick is to essentially create two new output devices (i.e. sinks). One of them (app_out) will just be a target toward which applications should direct their output. Magic happens with the second output (duplex_out) which combines application output and what comes from microphone.

Now when you record audio, you can just point application to Duplex Output and record both sides.

PS: To make these changes permanent, they can be entered into /etc/pulse/default.pa. Of course, quoting rules are bit different so adjust accordingly if you have a space in your description.

…
# Recording setup
load-module module-null-sink sink_name=duplex_out sink_properties=device.description="Duplex\ Output"
load-module module-null-sink sink_name=app_out sink_properties=device.description="Application\ Output"
load-module module-loopback source=app_out.monitor
load-module module-loopback source=app_out.monitor sink=duplex_out
load-module module-loopback sink=duplex_out
set-default-sink app_out

DevStack on VirtualBox Ubuntu 20.04

The first step for DevStack inside of VirtualBox is creating the virtual machine. There are two obvious changes that you need to make and those are increasing processor count and assigned memory as high as you can afford it. The other two are a bit more “sneaky”.

We really have to enable Nested VT-x/AMD-V under System, Processor and if we want to access our system we should really set network forwarding rules for port 80 (HTTP for Dashboard) and port 22 (SSH, optional but really helpful). I usually set them to be 56080 and 56022 respectively under my localhost but the actual numbers can be of your choosing. And yes, there are other ways to setup networking but NAT with forwarding is mine.

With the virtual machine set, the next step toward DevStack is installing OS. While official guidelines prefer Ubuntu 18.04, I like to go with a slightly newer Ubuntu 20.04 Server. Whole installation is essentially one big click-next event with the only non-default part being installation of OpenSSH.

Once OS is installed, I also like to add my user to password-less sudoers and do any needed updates:

echo "$USER ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/$USER

sudo apt update
sudo apt dist-upgrade --yes

And now finally we can follow the DevStack instructions with customized host IP (otherwise you’ll get “Could not determine host ip address” error) and admin password.

sudo useradd -s /bin/bash -d /opt/stack -m stack
echo "stack ALL=(ALL) NOPASSWD: ALL" | sudo tee /etc/sudoers.d/stack
sudo su - stack

git clone https://github.com/openstack-dev/devstack.git -b $STACK_BRANCH devstack/
STACK_BRANCH=stable/^^wallaby^^

cd devstack
cp samples/local.conf .
sed -i 's/#HOST_IP=w.x.y.z/HOST_IP=^^10.0.2.15^^/' local.conf
sed -i 's/ADMIN_PASSWORD=nomoresecret/ADMIN_PASSWORD=^^devstack^^/' local.conf
echo "#Enable heat services" >> local.conf
echo "enable_service h-eng h-api h-api-cfn h-api-cw" >> local.conf
echo "#Enable heat plugin" >> local.conf
echo "enable_plugin heat https://git.openstack.org/openstack/heat $STACK_BRANCH" >> local.conf

./stack.sh

Once done, GUI is available at http://localhost:56080/dashboard/.

Wait For Mountpoint

I have quite a few scripts running on my home server and they love writing on disk. They love it so much that, after reboot, they don’t necessarily wait for mount points to appear - they just start writing. Unfortunately, such eagerness also means that my other scripts mounting ZFS might find directory already in use and give up.

What I needed was a way to check if mount point is already there before starting with write. The easiest approach for me was using mountpoint command.

TEST_PATH=/test/directory
while(true); do  # wait for mount point
    mountpoint "$TEST_PATH" >/dev/null
    if [[ $? != 0 ]]; then
        sleep 1
        continue
    fi
    break
done

Script fragment above will check if given directory has something mounted and, if not, wait for 1 more second. Once test succeeds, it will break out of the infinite loop and proceed with whatever follows.

Easy enough.

One Taken Every Second

In order to keep an eye on my garage I placed a Wyze camera in it. So when I noticed one day that somebody has swiped my tool bag, I thought I’ll find the culprit quickly. Well, it was not meant to be.

I had recording turned on but only a 32 GB card in it. And I noticed tool bag missing only after two weeks or so. So any recording was already gone by the time I took a look. Since only other people that had access to the garage were Amazon delivery guys, I could narrow it down but not sufficiently to do anything about it.

So I went to look into a cheap solution to record images long term and RTSP immediately came to mind. Even better, Wyze cameras already support it (albeit via a special firmware).

My idea was to simply record an image every second and save it on my NAS using ffmpeg. While this was a simple task in theory, it proved to be a bit annoying to find parameters that would be stable enough. Most notably, sometime it would just stop ingesting new frames and thus require restart.

After testing a lot of different parameters, I came with the following code:

while (true); do
    ffmpeg \
        -probesize 1000000 \
        -analyzeduration 1000000 \
        -flags low_delay \
        -fflags discardcorrupt \
        -re \
        -rtsp_transport tcp \
        -stimeout 10000000 \
        -allowed_media_types video \
        -i rtsp://${CAMERA_USER}:${CAMERA_PASS}@${CAMERA_IP}/live \
        -f image2 \
        -strftime 1 \
        -vf fps=fps=1 \
        -async 1 \
        -vsync 1 \
        -threads 1 \
        -use_wallclock_as_timestamps 1 \
        "${BASE_DIRECTORY}/${CAMERA_IP}~%Y-%m-%d-%H-%M-%S.jpg"
    sleep 1
done

Using this setup ffmpeg will keep taking image every second. If it gets stuck, it will exit and then it’s on while to restart the capture again. One can then use a separate process to convert these files into a mjpeg file but that’s story for another day.

Inappropriate Ioctl for Device

After disconnecting a serial USB cable from my Ubuntu Server 20.04, I would often receive “Inappropriate ioctl for device” error when trying to redirect output to serial port.

stty -F /dev/ttyACM0 -echo -onlcr
 stty: /dev/ttyACM0: Inappropriate ioctl for device

Quick search yielded multiple results but nothing that actually worked for me. Most promising were restarting udev and manual driver unbind but they didn’t really solve anything end-to-end. The only solution was to reboot.

However, after a bit of playing with unloading drivers, I did find solution that worked. Unload driver, manually delete device, and finally load driver again.

modprobe -r cdc_acm
rm -f /dev/ttyACM0
modprobe cdc_acm

I am not sure why unloading driver didn’t remove device link itself, but regardless, I could finally get it to work without waiting for reboot.

Killing a Connection on Ubuntu Server 20.04

If you really want to kill a connection on a newer kernel Ubuntu, there is a ss command. For example, to kill connection toward 192.168.1.1 with dynamic remote port 40000 you can use the following:

ss -K dst 192.168.1.1 dport = 40000

Nice, quick, and it definitelly beats messing with routes and waiting for a timeout. This is assuming your kernel was compiled with CONFIG_INET_DIAG_DESTROY (true on Ubuntu).

To get a quick list of established connections for given port, one can use netstat with a quick’n’dirty grep:

$ netstat -nap | grep ESTABLISHED | grep ^^<port>^^