Speeding up BITS

If you deal with Microsoft Software Center you know the annoyance it can cause with extremely low download speeds. BITS protocol it uses was designed to be gentle toward your connection and avoid taking up bandwidth from anything else you might also be running.

However, sometime it can get overly cautious and slow down to a ridiculously low level even if you are not doing anything else with your computer. And good luck pulling those patches down if it gets into that mood.

Fortunately, you can tweak registry to give it a bit of a boost. Just go into regedit and navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\BITS. There add a DWORD named UseSystemMaximum and set it’s value to 1.

Alternatively you can get pre-prepared registry file import here (but do be very cautious when downloading registry meddling bits of the Internet).

You should see BITS drop their good guy mask and gobble all the bandwidth they can get. That will make your Software Center run much faster and maybe you get to install that darn patch within a year.

Removing Recording Time Limit on LX100

One of the rare things annoying me on my Panasonic LX100 is recording time limit. No matter how big SD you have, there is only 30 minutes you can record in one go. Yes, this is more than enough 99% of the time, but there are scenarios this disables, for example recording my occasional talk.

Well, reading through some DP Review forum threads I noticed discussion about service mode for the purpose of PAL/NTSC mode switcheroo and I wondered whether service mode offers anything else.

Well, to try it out, I followed instructions from the thread. First I kept Right, Display, and AE/AF Lock buttons while turning on the camera. Once camera has been turned on, I released all those buttons and swiched into Playback. Then I held Up and AE/AF Lock buttons while turning off the camera. If done correctly (and it took me a few tries), camera will flash yellow rectangle at you. Welcome to the service mode!

While in service mode, I suddenly got some limits increased:

As you can see, camera can now record MP4 in 4K limited only by card size and all AVCHD modes got a bump in recording length. Strangely, the same improvement is not visible in other MP4 modes as they all either retained their 30 minute or lower limits.

In case you find exclamation mark annoying, you can always exit service mode by setting both shutter speed and aperture to A. Then press Menu and Right key together and turn off the camera. That will present you with a strange menu where pressing Right, Right will end the service mode. Unfortunately that comes at the cost of your settings as that’ll do factory reset.

Me? I’m gonna keep my service mode on.

Fixing Discord Tray on Ubuntu 18.10

On my Ubuntu 18.10 installation, I had a peculiar problem. While Discord did work, it didn’t appear in the tray. Not broken enough to warrant immediate action, but broken enough to annoy me.

A quick search on Internet narrowed the culprit to unset desktop environment variable. Simply setting XDG_CURRENT_DESKTOP environment variable to Unity fixed the issue altogether.

Armed with that knowledge, I first modified the application launcher for both desktop and startup:

sudo sed -i 's^Exec=/usr/share/discord/Discord^Exec=/usr/bin/env XDG_CURRENT_DESKTOP=Unity /usr/share/discord/Discord^' \
    /usr/share/discord/discord.desktop \
    ~/.config/autostart/discord-stable.desktop

To propagate the change, a quick update of desktop database was in order:

sudo update-desktop-database

Followed by termination of the existing Discord instances:

killall Discord ; killall Discord

Once I started Discord again, its icon appeared in its rightful place.

Systemd Watchdog for Any Service

Making basic systemd service is easy. Let’s assume the simplest application (not necessarily even designed to be a service) and look into making it work with systemd.

Our example application will be a script in /opt/test/application with the following content:

#!/bin/bash

while(true); do
  date | tee /var/tmp/test.log
  sleep 1
done

Essentially it’s just never ending output of a current date.

To make it a service, we simply create /etc/systemd/system/test.service with description of our application:

[Unit]
Description=Test service
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
ExecStart=/opt/test/application
Restart=always
RestartSec=1

[Install]
WantedBy=multi-user.target

That’s all needed before we can start the service:

sudo systemctl start test

sudo systemctl status test
 ● test.service - Test service
    Loaded: loaded (/etc/systemd/system/test.service; disabled; vendor preset: enabled)
    Active: active (running)
  Main PID: 5212 (service)
     Tasks: 2 (limit: 4657)
    CGroup: /system.slice/test.service
            ├─5212 /bin/bash /opt/test/application
            └─5321 sleep 1

Systemd will start application and even perform restart if application fails. But what if we want it a bit smarter? What if we want a watchdog that’ll restart application not only when it’s process fails but also when some other health check goes bad?

While sytemd does support such setup, application generally should be aware of it and call watchdog function every now and then. Fortunately, even if our application doesn’t do that, we can use watchdog facilities via systemd-notify tool.

First we need to change three things in our service definition. One is changing type to notify, then changing executable to the wrapper script, and lastly defining the watchdog time.

In this example, if application doesn’t respond in 5 seconds, it will be considered failed. The new service definition in /etc/systemd/system/test.service can look something like this:

[Unit]
Description=Test service
After=network.target
StartLimitIntervalSec=0

[Service]
Type=^^notify^^
ExecStart=^^/opt/test/test.sh^^
Restart=always
RestartSec=1
TimeoutSec=5
WatchdogSec=^^5^^

[Install]
WantedBy=multi-user.target

Those watching carefully will note we don’t actually solve anything with this and that we just move all responsibility to /opt/test/test.sh wrapper.

It’s in that script we first communicate to sytemd when application is ready and later, in a loop, check for not only application PID but also for any other condition (e.g. certain curl response), calling systemd-notify if application proves to be healthy:

#!/bin/bash

trap 'kill $(jobs -p)' EXIT

/opt/test/service &
PID=$!

/bin/systemd-notify --ready

while(true); do
    FAIL=0

    kill -0 $PID
    if [[ $? -ne 0 ]]; then FAIL=1; fi

#    curl http://localhost/test/
#    if [[ $? -ne 0 ]]; then FAIL=1; fi

    if [[ $FAIL -eq 0 ]]; then /bin/systemd-notify WATCHDOG=1; fi

    sleep 1
done

Starting service now gives slightly different output:

sudo systemctl stop test

sudo systemctl start test

sudo systemctl status test
 ● test.service - Test service
    Loaded: loaded (/etc/systemd/system/test.service; disabled; vendor preset: enabled)
    Active: active (running)
  Main PID: 6406 (test.sh)
     Tasks: 4 (limit: 4657)
    CGroup: /system.slice/test.service
            ├─6406 /bin/bash /opt/test/test.sh
            ├─6407 /bin/bash /opt/test/application
            ├─6557 sleep 1
            └─6560 sleep 1

If we kill application manually (e.g. sudo kill 6407), systemd will pronounce service dead and start it again. It will do the same if any other check fails.

While this approach is not ideal, it does allow for easy application watchdog retrofitting.

Setting up Encrypted Ubuntu 18.10 ZFS Desktop

I have already explained how I deal with ZFS mirror setup on Ubuntu 18.10. But what about laptops that generally come with a single drive?

Well, as before basic instructions are available from ZFS-on-Linux project. However, they do have a certain way of doing things I don’t necessarily subscribe to. Here is my way of setting this up. As always, it’s best to setup remote access so you can copy/paste as steps are numerous.

As before, we first need to get into root prompt:

sudo -i

Followed by getting a few basic packages ready:

apt-add-repository universe
apt update
apt install --yes debootstrap gdisk zfs-initramfs

We setup disks essentially the same way as in previous guide:

sgdisk --zap-all                 /dev/disk/by-id/^^ata_disk^^

sgdisk -a1 -n3:34:2047  -t3:EF02 /dev/disk/by-id/^^ata_disk^^
sgdisk     -n2:1M:+511M -t2:8300 /dev/disk/by-id/^^ata_disk^^
sgdisk     -n1:0:0      -t1:8300 /dev/disk/by-id/^^ata_disk^^

sgdisk --print                   /dev/disk/by-id/^^ata_disk^^
 …
 Number  Start (sector)    End (sector)  Size       Code  Name
    1         1050624        67108830   31.5 GiB    8300
    2            2048         1050623   512.0 MiB   8300
    3              34            2047   1007.0 KiB  EF02

Because we want encryption, we need to setup LUKS:

cryptsetup luksFormat -qc aes-xts-plain64 -s 512 -h sha256 /dev/disk/by-id/^^ata_disk^^-part1
cryptsetup luksOpen /dev/disk/by-id/^^ata_disk^^-part1 luks1

Unlike in the last guide, this time I want to have a bit of separation. Dataset system will contain the whole system, while data will contain only the home directories. Again, if you want to split it all, follow the original guide:

zpool create -o ashift=12 -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD \
    -O xattr=sa -O mountpoint=none rpool /dev/mapper/luks1
zfs create -o canmount=noauto -o mountpoint=/mnt/rpool/ rpool/system
zfs mount rpool/system

We should also setup the boot partition:

mke2fs -Ft ext2 /dev/disk/by-id/^^ata_disk^^-part2
mkdir /mnt/rpool/boot/
mount /dev/disk/by-id/^^ata_disk^^-part2 /mnt/rpool/boot/

Now we can get basic installation onto our disks:

debootstrap cosmic /mnt/rpool/
zfs set devices=off rpool
zfs list

Before we start using it, we prepare few necessary files:

cp /etc/hostname /mnt/rpool/etc/hostname
cp /etc/hosts /mnt/rpool/etc/hosts
cp /etc/netplan/*.yaml /mnt/rpool/etc/netplan/
sed '/cdrom/d' /etc/apt/sources.list > /mnt/rpool/etc/apt/sources.list

With chroot we can get the first taste of our new system:

mount --rbind /dev  /mnt/rpool/dev
mount --rbind /proc /mnt/rpool/proc
mount --rbind /sys  /mnt/rpool/sys
chroot /mnt/rpool/ /bin/bash --login

Now we can update our software and perform locale and time zone setup:

apt update

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we install Linux image and basic ZFS boot packages:

apt install --yes --no-install-recommends linux-image-generic
apt install --yes zfs-initramfs

Since we’re dealing with encrypted data, our cryptsetup should be also auto mounted:

apt install --yes cryptsetup

echo "luks1 UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part1) none luks,discard,initramfs" >> /etc/crypttab
cat /etc/crypttab

And of course, we need to auto-mount our boot partition too:

echo "UUID=$(blkid -s UUID -o value /dev/disk/by-id/^^ata_disk^^-part2) /boot ext2 noatime 0 2" >> /etc/fstab
cat /etc/fstab

Now we get grub started (do select the WHOLE disk):

apt install --yes grub-pc

And update our boot environment again (seeing errors is nothing unusual):

update-initramfs -u -k all

And then we finalize our grup setup:

update-grub
grub-install /dev/disk/by-id/^^ata_disk^^

Finally we get the rest of desktop system:

apt-get install --yes ubuntu-desktop samba linux-headers-generic
apt dist-upgrade --yes

We can omit creation of the swap dataset but I always find it handy:

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput -o sync=always \
    -o primarycache=metadata -o secondarycache=none rpool/swap
mkswap -f /dev/zvol/rpool/swap
echo "/dev/zvol/rpool/swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

And now is good time to swap our /home directory too:

rmdir /home
zfs create -o mountpoint=/home rpool/data

Now we are ready to create the user:

adduser -u 1002 ^^user^^
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sambashare,sudo ^^user^^
chown -R ^^user^^:^^user^^ /home/^^user^^

Lastly we exit our chroot environment and reboot:

exit
reboot

You will get stuck after the password prompt as our mountpoint for system dataset is wrong. That’s easy to correct:

zfs set mountpoint=/ rpool/system
exit
reboot

Assuming nothing went wrong, your system is now ready.