Using iSCSI from Ubuntu 19.04

Samba has taken over most of my sharing but I still occasionally need iSCSI. Connecting to it from Ubuntu is straight enough. First step is of course installing necessary package:

sudo apt-get install open-iscsi

Next step is simply connecting to iSCSI and discovering the drive name within dmesg output:

sudo iscsiadm -m discovery -t sendtargets -p ^^192.168.1.1^^
 192.168.1.1:3260,1 iqn.2007-09.jp.ne.peach.istgt:somedisk0

sudo iscsiadm --mode node --targetname iqn.2007-09.jp.ne.peach.istgt:^^somedisk0^^ \
    --portal ^^192.168.1.1^^:3260 --login

sudo dmesg | grep "Attached SCSI disk" | tail -1
 [ 3203.813577] sd 2:0:0:0: [^^sda^^] Attached SCSI disk

Lastly, just use the normal mounting to access the disk:

sudo mkdir /mnt/scsi
sudo mount /dev/sda2 /mnt/iscsi

Installing MPLAB X IDE 5.25 on 64-bit Ubuntu 19.04

While I love seeing companies provide Linux version of their software, it still annoys me when they do it as lazily as possible. For example Microchip not only doesn’t provide repository for their software, making you download and install every version separately, but they also don’t provide up-to-date instructions. Their section describing Ubuntu 14.04 and above simply doesn’t work.

To install MPLAB X on newer Ubuntu one must first explicitly add architecture:

sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt install libc6:i386 libx11-6:i386 libxext6:i386 libstdc++6:i386 libexpat1:i386

Only once those are up you can proceed installing MPLAB IDE X itself:

tar -xvf MPLABX-v*-linux-installer.tar
chmod +x MPLABX-v*-linux-installer.sh
sudo ./MPLABX-v*-linux-installer.sh

Of course, installing compiler follows (XC8 in this case):

chmod u+x xc8-v*-full-install-linux-installer.run
sudo ./xc8-v*-full-install-linux-installer.run

And now MPLAB X IDE is installed.

PS: I won’t even go into the need for 32-bit compatibility libraries in 2019…

ZFS Pool for Virtual Machines

Running VirtualBox on ZFS pool intended for general use is not exactly the smoothest experience. Due to it’s disk access pattern, what works for all your data will not work for virtual machine disk access. Yes, you can play with record size and adding SLOG device but you can also go slightly different route. Add disk specifically for VirtualBox.

My testing has found that simple SSD with the following settings does wonders:

zpool create -o autoexpand=on -m /VirtualBox \
    -O compression=off -O recordsize=4K -O atime=off \
    -O utf8only=on -O normalization=formD -O casesensitivity=sensitive \
    VirtualBox /dev/diskid/^^DISK.eli^^

First of all, you don’t want compression. Not because data is not compressible but because compression can lead you to believe you have more space than you actually do. Even when you use fixed disk, you can run out of disk space just because some uncompressible data got written within VM. Due to copy-on-write architecture, you can still get into the trouble but exposure is greatly limited.

Ideally record size should match your expected load. In case of VirtualBox that’s 512 bytes. However, tracking 512 byte records takes so much metadata that 4K records are actually both more space efficient and perform better. Depending on your exact hardware you might find that going to 8K or even higher might hit the sweet spot. Testing is the only way to know for sure but 4K is a reasonable starting point.

All other options are just plumbing - of course you want UTF-8 and no access time tracking.

Now you can run VirtualBox without complicating your main data pool.

PS: This assumes that you have disks enumerated by diskid and fully encrypted.

PPS: I usually just spin up temporary virtual machines for testing and thus I don’t care much about them long term. If you plan to kick something up long-term, do consider mirrored ZFS.

Dual Boot Clock Shenanigans

Probably the most annoying thing when dual booting Windows and Linux is the clock. For the various reasons, Windows keeps BIOS clock in local time-zone while Linux prefers it as UTC. While this is not a problem in Reykjavík, it surely is everywhere else.

There are ways to make Windows run in UTC but they either don’t work with the latest Windows 10 or they require time synchronization to be turned off. As I value precise time, a solution on Linux side was needed.

Fortunately, Linux does offer setting for just this case. Just run the following command:

sudo timedatectl set-local-rtc 1 --adjust-system-clock

This will tell Linux to keep local clock in RTC. While this is not necessarily fully supported, I found it’s actually the only setting that reliably works when dual booting Windows 10.

PS: You might need a reboot or two before this takes effect.

Avoiding Select Audio Device Prompt on Dell XPS 15 under Ubuntu

As a headphone user I find nothing more annoying than computer asking me every single freaking time what exactly did I plug in. While Windows drivers for Dell XPS 15 audio do allow you to select default, one is not so lucky under Linux.

However, Linux being configurable to a fault does offer a workaround.

You can append the following options to the end of /etc/modprobe.d/alsa-base.conf, followed by a reboot:

options snd-hda-intel model=headset-mic

This will lie a bit to sound driver and stop the darn questions.

PS: You can also use this one liner:

echo "options snd-hda-intel model=headset-mic" | sudo tee -a /etc/modprobe.d/alsa-base.conf

Installing DropBox on ZFS

While I already wrote about expanding DropBox’s Ext4 volume on ZFS, I never actually wrote how to create one in the first place. I guess it’s time to fix that injustice.

First you need to create a volume of sufficient size. While you can just make it as big as your Dropbox allowance is, I would advise going with at least double of that. Not only this helps if you are doing ZFS snapshots (remember it’s copy-on-write) but it also helps if you are moving files around as Dropbox fully releases space only once the new files are created.

Whatever you decide, you need to create a volume and format it:

sudo zfs create -V 12G ^^pool^^/dropbox
sudo mkfs.ext4 /dev/zvol/^^pool^^/dropbox

Once volume is created, mounting the newly created volume within our user directory is in order:

mkdir /home/^^user^^/Dropbox
sudo mount /dev/zvol/^^pool^^/dropbox /home/^^user^^/Dropbox
sudo chown -R ^^user^^:^^user^^ Dropbox

Of course, to retain it between reboots one should add it to fstab:

echo "/dev/zvol/^^pool^^/dropbox /home/^^user^^/Dropbox ext4 defaults,_netdev 0 0" | sudo tee -a /etc/fstab

Do note the _netdev part as it ensures dropbox volume is mounted way after ZFS has already done so. Without it you might have a race condition and volume mounting might prevent subpools to be mounted under the same path.

Finally you can install Dropbox as you usually would. While it will complain about directory already being present, you can simply cancel directory selection and it will start syncing regardless.

Congratulations, your Dropbox is now on ZFS.

DaVinci Resolve and MP4 under Linux

If you try to import mp4 file to DaVinci Resolve under Linux, you will be greeted with audio-only experience if you’re lucky or nothing at all if you’re not. Unless you’re paying customer of DaVinci Resolve Studion, MP4 and AAC are not supported at all.

Conversion to an intermediate format is needed and good old ffmpeg can help here:

ffmpeg -i ^^input.mp4^^ \
    -c:v dnxhd -profile:v dnxhr_hq -pix_fmt yuv422p \
    -c:a pcm_s16le \
    -f mov ^^output.mov^^

Here I use DNxHR with HQ settings. Even this will generate file that’s much bigger than source. How much bigger depends on the source, but don’t be surprised to see it grow 16-fold or even higher.

Smaller alternative can be MPEG4 format but with a drop in quality even at its best settings:

ffmpeg -i ^^input.mp4^^ \
    -c:v mpeg4 -qscale:v 1 \
    -c:a pcm_s16le \
    -f mov ^^output.mov^^

These files can now be used as input to Resolve and editing can commence.

Once done with rendering a conversion to MP4 might be in order again. For this I use settings discussed on Video StackExchange. While the original discussion was for YouTube, I find their recommendations quite good as a starting point:

$ ffmpeg -i ^^render.mov^^ \
    -c:v libx264 -pix_fmt yuv420p -crf 16 \
    -force_key_frames 'expr:gte(t,n_forced/2)' -bf 2 \
    -vf yadif -use_editlist 0 \
    -movflags +faststart \
    -c:a aac -q:a 1 \
    -ac 2 -ar 48000 \
    -f mp4 ^^out.mp4^^

Mind you, all my cameras use YUV 4:2:0 and I don’t do any major editing so increasing YUV to 4:2:2 or even 4:4:4 makes no sense nor does increase to 10-bit resolution. Depending on which equipment you have, your mileage may vary.

Installing DaVinci Resolve on Ubuntu Dell XPS

When I first installed DaVinci Resolve on my Dell XPS under Ubuntu 19.04 I was greeted with stuck splash screen. What was I missing?

Well, the first issue was missing OpenCL. Fortunately that was easy to solve:

sudo apt install ocl-icd-opencl-dev

Other issue was missing nVidia driver. For some reason Resolve really dislikes nouveau driver. Therefore we have to upgrade driver a bit. I found that auto-install works perfectly for me:

sudo ubuntu-drivers autoinstall

Those who like to be more precise can always check which drivers are available and install only those:

ubuntu-drivers devices
 == /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0 ==
 modalias : pci:v000010DEd00001C8Dsv00001028sd000007BEbc03sc02i00
 vendor   : NVIDIA Corporation
 model    : GP107M [GeForce GTX 1050 Mobile]
 driver   : nvidia-driver-390 - distro non-free
 driver   : nvidia-driver-418 - distro non-free recommended
 driver   : xserver-xorg-video-nouveau - distro free builtin

sudo apt install nvidia-driver-418

With these two changes my DaVince Resolve can finally start properly.

PS: Interestingly, installing non-free nVidia driver solved Ubuntu getting stuck booting after incorrect password got entered.

Switching to Linux Desktop?

I’ve been playing with Linux/Unix servers for a long time now - both at work and home. I even switched almost completely to Linux at work - Outlook and Excel being the only reason why I still have Windows VM. However, at home I still used Windows 10 as my main OS. With Linux Desktop getting better and better while Windows 10 is getting more and more annoying, I figured I need to revisit installing Linux at home too.

And no, I didn’t go full-in. However, for the first time in the last 10 years I am dual booting again.

Ubuntu with ZFS was Linux of my choice primarily because of the ZFS support. Running ZFS on my NAS has been a pure joy (as much as any file system can be) and that’s what moved needle from Linux Mint toward Ubuntu. Other benefits include a huge install base and prepackaged software support.

While Linux is rather enjoyable, I don’t see Windows going away from my computer for a while. And the major reason is Visual Studio - there is simply nothing equivalent for C# development. Visual Studio Code does try but that’s like saying butcher is an equivalent of a surgeon. It’s simply a difference between IDE and really good text editor.

While running under Wine is an option, DipTrace still works better under Windows. And this goes for the whole “electronics tools” category. Whether it’s MPLAB (which actually has a Linux version), PicoScope (Linux version in beta), or just some stupid utility, making it work under Windows is simply easier with less issues present.

Another stumbling point was Vegas Movie Studio in particular and media editing in general. These applications are few and far between. I did find DaVinci Resolve to be actually extremely powerful and free alternative. It definitely has a steeper learning curve and does require a bit of getting used to. However, at this moment it actually doesn’t support H.264 and MP4 file format under Linux. :(

I am still looking for a good alternative to Paint.NET as Gimp doesn’t really work for me. It’s excellent software if you need to do a major work, but it’s really annoying when it comes to simple operations. As Windows 10 image viewer is pure shit, I didn’t need to search long to find a substitute.

WinRAR was another stupidly simple utility that doesn’t really work properly under Linux. Yes, you can get other archivers but there is nothing that really offers the same features; archive locking and preview without extracting to temporary directory being my favorites.

And lastly, there is nothing as good on Linux as Explorer. Yes, Nautilus does come close when it comes to pure file browsing but just trying to do something in Open and Save dialog brings tears to my eyes. No, there is no rename, full right-click menu, or even pasting file there. Nope, you can just open and save files. As I often remember things I want to adjust only when saving other file, this seems overly restricting.

End result is that for Visual Studio, electronics, and gaming I still use Windows. For all other purposes I simply switch to Linux as my preferred environment. It’s something I wouldn’t believe just a few years ago - oh, how times change.

PS: If you’re curious as to what I find annoying in Windows 10, here are the first couple of things coming to mind without any specific order: Windows update reboots my computer overnight time and time again, every major update randomly resets my settings, local account requires THREE damn password reminders, it keeps forgetting my damn network credentials (despite checking the “remember” checkbox), Start menu cannot find applications that are freshly installed, damn settings are distributed between multiple apps and programs with each holding some settings, it uses every opportunity to shove Edge and Skype app down my throat, and the list goes on.

Configuring HTTP/2 for WordPress on Ubuntu

I moved my website to Ubuntu just recently and big part of that was due to desire for HTTP/2. However, if you have a legacy site configuration, things might not be as straightforward as one would hope.

The first step, of course, is enabling HTTP/2 module:

a2enmod http2

Second step is adding HTTP/2 protocol definition to /etc/apache2/apache2.conf:

Protocols h2 h2c http/1.1
H2Direct on
H2ModernTLSOnly on

Followed by Apache’s restart:

systemctl restart apache2

In ideal world this would be it. But, despite Apache starting without error, a check via Developer Tools in browser showed I was still stuck with HTTP 1.1. Yep, as many other sites, mine used mpm_prefork module. Unfortunately, prefork is not compatible with HTTP/2 and PHP 7.2 (needed for WordPress) is not compatible with anything else.

The only additional package we need is PHP with FastCGI support:

apt-get install php7.2-fpm

Furthermore, we need some modules enabled and disabled:

a2dismod php7.2
a2dismod mpm_prefork
a2enmod mpm_event
a2enmod proxy_fcgi
a2enconf php7.2-fpm

Of course, addition to /etc/apache2/apache2.conf is needed too:

<Files "*.php">
   SetHandler "proxy:unix:/var/run/php/php7.2-fpm.sock|fcgi://localhost/"
</Files>

If you configured prefork same as me, you also need to remove it’s configuration. In my case StartServers, MinSpareServers, MaxSpareServers, MaxClients, and MaxRequestsPerChild settings had to go.

Of course, another Apache restart is upon us:

systemctl restart apache2

Congratulations! HTTP/2 should be working now.