Why My Serial Port Device Is Detected as a Mouse

When designing electronics, probably the easiest way to make it communicate with a computer is to connect it as a serial port. However, occasionally you might see your device detected as a serial mouse instead of a plain serial port. When this happens the device will obviously not work and you might even see your mouse cursor jump around.

This behavior is an artefact of how Windows detect a serial mouse since that was written way before plug’n’play times. Widnows will first raise DTR and RTS signals and then monitor if there’s an M or B character coming at 1,200 baud. If either character appears, congratulations - your device is now a mouse. And anything it sends will be processed as if it was a mouse action.

There’s a brute force solution of disabling serial mouse in registry and that would honestly be the easiest way around. However, what if you cannot modify the registry? What if you need to make your device work in the face of an unknown machine? Well, there are certain tactics you can employ to either make this behavior unlikely or avoid it altogether.

The simplest solution of not sending M or B will unfortunately not work as easily. Yes, if your device works at the 1,200 baud rate, avoiding those characters will be fine. But if your device works at faster baud rate (and it probably does) the other end receiving at 1,200 will misunderstand its output. Depending when the actual bit sample is taken and what your device is actually sending, you might see an offending character appear either every time or every 10 times. Even worse, the frequency of the issue might change depending on the computer or even the room temperature. So, not really a solution.

The second obvious choice is detecting DTR and RTS combination and pausing whatever output your device gives until that state passes. If you can afford this, it’s a great and surefire way to sort the issue out as DTR and RTS are not baud rate sensitive. The only problem happens when you need those signals for flow control or for something completely custom. Even if you’re not using them they’ll still “cost” you two pins.

The most practical way I found of stopping this issue is by having device not speak until spoken to. If you design your serial protocol so that a device must receive a valid command before it will send any data, it will pass serial mouse detection unharmed. And, if you really need a streaming output (a perfectly valid requirement), just use a command to turn it on. If you can additionally somehow get CRC check into your protocol, the world is your oyster.

Installing UEFI ZFS Root on Ubuntu 21.10

Before reading further you should know that Ubuntu has a ZFS setup option since 19.10. You should use it instead of the manual installation procedure unless you need something special. In my case that special something is the native ZFS encryption and custom partitioning I find more suitable for laptop.

After booting into Ubuntu desktop installation we want to get a root prompt. All further commands are going to need root credentials anyhow.

sudo -i

The very first step should be setting up a few variables - disk, pool, host name, and user name. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system. I like to use upper-case for ZFS pool as that’s what will appear as password prompt. It just looks nicer and ZFS doesn’t care either way.

DISK=/dev/disk/by-id/^^ata_disk^^
POOL=^^Ubuntu^^
HOST=^^desktop^^
USER=^^user^^

General idea of my disk setup is to maximize amount of space available for pool with the minimum of supporting partitions. If you are not planning to have multiple kernels, decreasing boot partition size might be a good idea (512 MB is ok).

blkdiscard -f $DISK

sgdisk --zap-all                     $DISK

sgdisk -n1:1M:+63M -t1:EF00 -c1:EFI  $DISK
sgdisk -n2:0:+960M -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:0     -t3:BF00 -c3:Root $DISK

sgdisk --print                       $DISK

Finally we’re ready to create system ZFS pool. Note that you need to encrypt it at the moment it’s created.

zpool create -o ashift=12 -o autotrim=on \
    -O compression=lz4 -O normalization=formD \
    -O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
    -O encryption=aes-256-gcm -O keylocation=prompt -O keyformat=passphrase \
    -O canmount=off -O mountpoint=none $POOL $DISK-part3

On top of this encrypted pool we could create our root set (as I did in previous guides) or just use default dataset itself. I found that actually works better for me. In either case, our new root file system will end up at /mnt/install.

zfs set canmount=noauto $POOL
zfs set mountpoint=/ $POOL

mkdir /mnt/install
mount -t zfs -o zfsutil $POOL /mnt/install

Assuming UEFI boot, two additional partitions are needed. One for EFI and one for booting. Unlike what you get with the official guide, here I don’t have ZFS pool for boot partition but a plain old ext4. I find potential fixup works better that way and there is a better boot compatibility. If you are thinking about mirroring, making it bigger and ZFS might be a good idea. For a single disk, ext4 will do.

yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot/

mkfs.msdos -F 32 -n EFI $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

To start the fun we need debootstrap package.

apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool is next. This will take a while.

debootstrap $(basename `ls -d /cdrom/dists/*/ | grep -v stable | head -1`) /mnt/install/

We can use our live system to update a few files on our new installation.

echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials. Don’t worry if this returns errors - that just means you are not using wireless.

mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Finally we’re ready to “chroot” into our new system.

mount --rbind /dev  /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys  /mnt/install/sys
chroot /mnt/install \
    /usr/bin/env DISK=$DISK POOL=$POOL USER=$USER \
    bash --login

Let’s not forget to setup locale and time zone.

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales

dpkg-reconfigure tzdata

Now we’re ready to onboard the latest Linux image.

apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by boot environment packages.

apt install --yes zfs-initramfs grub-efi-amd64-signed shim-signed tasksel

To mount boot and EFI partition, we need to do some fstab setup.

echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
cat /etc/fstab

Now we get grub started and update our boot environment.

KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -u -k $KERNEL

Grub update is what makes EFI tick.

update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
    --recheck --no-floppy

Finally we install out GUI environment. I personally like ubuntu-desktop-minimal but you can opt for ubuntu-desktop. In any case, it’ll take a considerable amount of time.

tasksel install ubuntu-desktop-minimal

Short package upgrade will not hurt.

apt dist-upgrade --yes

We can omit creation of the swap dataset but I personally find a small one handy.

zfs create -V 4G -b $(getconf PAGESIZE) -o compression=off -o logbias=throughput \
    -o sync=always -o primarycache=metadata -o secondarycache=none $POOL/Swap
mkswap -f /dev/zvol/$POOL/Swap
echo "/dev/zvol/$POOL/Swap none swap defaults 0 0" >> /etc/fstab
echo RESUME=none > /etc/initramfs-tools/conf.d/resume

If one is so inclined, /home directory can get a separate dataset too but I usually skip it these days. I just proceed to create the user, assign a few extra groups to it, and make sure its home has correct owner.

adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dip,lpadmin,plugdev,sudo $USER
passwd $USER

As install is ready, we can exit our chroot environment.

exit

And cleanup our mount points.

umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
umount /mnt/install
zpool export -a

After the reboot you should be able to enjoy your installation.

reboot

PS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 22.04 and 20.04

PPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, 19.04, and 18.10.

ZFS Pool on SSD

I am a creature of habit. Long time ago I found ZFS setup that works for me and didn’t change much since. But sometime I wonder if those settings still hold with SSDs in game. Most notably, are 4K blocks still the best?

Since I already “had” to update my desktop to Ubuntu 21.10, I used that opportunity to clear my disks and have it installed from scratch. And it would be a shame not to run some tests first on my XPG SX6000 Pro - SSD I use for pure data storage. After trimming this DRAM-less SSD, I tested the pool across multiple recordsize values and at ashift values of 12 (4K block) and 13 (8K block).

My goal was finding a good default settings for both bulk storage and virtual machines. Unfortunately, those are quite opposite requirements. Bulk storage benefits greatly from good sequential access while virtual machines love random IO more. Fortunately, with ZFS, one can accomplish both using two datasets with different recordsize values. But ashift value has to be the same.

Due to erase block sizes getting larger and larger, I expected performance to be better with 8K “sectors” (ashift=13) than what I usually used (ashift=12). But I was surprised.

First of all, results were all over the place but it seems that ashift=12 is still a valid starting point. It might be due to my SSD having smaller than expected erase page but I doubt it. My thoughts go more toward SSDs being optimized for the 4K load. And the specific SSD I used to test with is DRAM-less thus allowing any such optimizations to be even more visible.

Optimizations are probably also the reason for 128K performing so well in the random IO scenarios. Yes, for sequential access you would expect it, but for random access it makes no sense how fast it is. No matter what’s happening, it’s definitely making recordsize=128K still the best general choice. Regardless, for VMs, I created a sub-dataset with much smaller 4K records (and compression off) just to lower write-amplification a bit.

The full test results are in Google Sheets. For testing I used fio’s fio-rand-RW.fio and fio-seq-RW.fio profiles.

Overescaping By Default

Writing JSON has became trivial in C# and there’s no class I like better for that purpose than Utf8JsonWriter. Just look at a simple example:

var jsonUtf8 = new Utf8JsonWriter(Console.OpenStandardOutput(),
                                  new JsonWriterOptions() { Indented = true });
jsonUtf8.WriteStartObject();
jsonUtf8.WriteString("Test", "2+2");
jsonUtf8.WriteEndObject();
jsonUtf8.Flush();

This simple code will produce perfectly valid JSON:

{
  "Test": "2\u002B2"
}

While valid, you’ll notice this is slightly different than any other programming language would do. A single plus character became escape sequence \u002B.

In their eternal wisdom, .NET architects decided that, by default, JSON should be over-escaped and they “explained” their reasoning in the ticket. Essentially they did it out of abundance of caution to avoid any issues if someone puts JSON where it might not be expected.

Mind you, in 99% of cases JSON is used in HTTP body and thus doesn’t need this but I guess one odd case justifies this non-standard but valid output in their minds. And no, other JSON encoders don’t behave this way either. Only .NET as far as I can tell.

Fortunately, some time later, they also implemented what I (alongside probably 90% of developers) consider the proper JSON encoder which escapes just mandatory characters and leaves the rest of text alone. It just requires a small extra parameter.

var jsonUtf8 = new Utf8JsonWriter(Console.OpenStandardOutput(),
                                  new JsonWriterOptions() { Indented = true,
                                    ^^Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping^^});
jsonUtf8.WriteStartObject();
jsonUtf8.WriteString("Test", "2+2");
jsonUtf8.WriteEndObject();
jsonUtf8.Flush();

Using UnsafeRelaxedJsonEscaping is not unsafe despite it’s name; darn it, it’s not even relaxed as compared to the specification. It’s just a properly implemented JSON encoder without any extra nonsense thrown in.

Framework Expansion Board

One of most exciting recent developments in laptop world for me is definitely the framework laptop. A major component of that concept are its expansion cards. And, of course, you can build your own.

This repository is quite encompassing if you’re using KiCAD. However, for those who love nicer tools (ehm, DipTrace), it’s annoying to find that there is no board size specification in human readable format (and no, KiCAD XML is not). So I decided to figure it out.

To cut the long story short, here are the board outline points for the expansion card PCB:

• (0.0, 0.0)
• (26.0, 0.0)
• (26.0, 26.5)
• (25.0, 26.5)
• (25.0, 30.0)
• (17.7, 30.0)
• (17.7, 28.0)
• (16.0, 28.0)
• (16.0, 29.0)
• (10.0, 29.0)
• (10.0, 28.0)
• (8.3, 28.0)
• (8.3, 30.0)
• (1.0, 30.0)
• (1.0, 26.5)
• (0.0, 26.5)

In order to make it slightly nicer to handle, each corner is additionally rounded with a 0.3 mm radius.

And let’s not forget two holes at (1.7, 10.5) and (24.3, 10.5), both with a 2.2 mm diameter and 4.9 mm keepout region.

With that information in hand, one can create PCB board in any program they might prefer. Of course, I already did so for DipTrace and you download the files here.

And yes, PCB is just a first step in a development process. What I found the hardest is actually getting appropriate connectors for the enclosure as there’s not too much height to work with.

PS: No, I do not own framework laptop at this time. I am waiting for 15.6" model as 13.5" is simply too small for me when not used external monitor.