RS-485 Framework Expansion Card (FTDI Edition)

Boards for this project were sponsored by PCBWay (affiliate link).

Playing with electronics means dealing with a lot of protocols. Be it UART, RS-232, I2C, or CAN-bus, I have made a version to fit my Framework laptop. However, one popular protocol was strangely absent - RS-485. Well, it’s time to correct that mistake.

Since I successfully used JST XH with many Framework expansion cards before, connector decision was easy. However, deciding which pinout to use was a bit more troublesome as I needed 3 pins and common wisdom was that order is GND-A-B. However, my UART cards were using 3-pin setup with GND in middle. It would be too easy to forget which card was plugged in and get the most crucial pin wrong.

So, the only remaining choice was to have A-GND-B order but I really didn’t like it and honestly I am not sure why. It’s a good pinout allowing for excellent routability (since protection diodes have this pinout already). But no other RS-485 device has it. While moving GND pin to center would alleviate error rate among my framework expansion cards, I could see even myself accidentally connecting ground to the first pin.

At the end I decided to use 4-pin socket with the last pin pulled out. This would allow for standard GND-A-B setup while making connector distinctively different from my cards that use center pin for GND. Even better, GND signal here actually matches my CAN-bus card where that last pin is populated (albeit not used).

Since I still had bunch of SP485EN chips I used for another project, it was easy to select RS-485 transceiver. And honestly, while aged, it’s not a bad chip for the purpose. It’s available from multiple sources, it has low pin count in easily hand-solderable package, and at 5 Mbps it’s way faster than UART I was planning to pair it with.

Speaking of UARTs, there is one that you will find mentioned time and time again when it comes to RS-485 - FTDI FT232RL. And that’s for a good reason. It’s a single chip solution without the need for crystal, allows for reasonably high baud rates, and it has specialized TXDEN pin allowing for automatic flow control. While its pin count might be on a high side and comes with a bit of baggage, it really fits nicely for the purpose.

While FT232 does offer reasonably full RS-485 schematics, I added a few things to spice it all up. First of all, I added a MOSFET to keep RS-485 transceiver fully off when in USB suspend. I later did find this was not really necessary as SP485EN has a suspend mode that FT232 can nicely trigger, it does offer me a bit of flexibility for further testing.

Another thing I added were biasing resistors. In my own setups I found that this increases bus resiliency a lot and it only costs you 3 resistors. What value resistors? Well, that’s where it gets tricky. There are so many ways to calculate these resistor values and you need to know a lot about the exact setup in order to make it perfect. Since I wanted to use this device in many different networks, the best I could hope for is to get close enough. In this case this meant 130/750Ω combination. When combined with 120Ω termination on the other side of the bus, it should handle many scenarios.

For protection, I decided to use good old SM712 TVS diodes to keep SP485EN safe and a few fast fuses on both data lines and on the ground for catastrophe prevention. Ideally you might want to have an isolated interface between your laptop and any bus. However, I simply didn’t have enough space on board for discrete solution and solutions with integrated DC-DC converter were either way to expensive or unobtainable.

While fuses are not ideal replacement for isolation since bad things can still happen if you deal with wildly different potentials, things are not as bleak as it’s easy to insulate your laptop. Simply unplugging it from its power supply and running it of battery will do the trick and allow you safe interfacing.

With the overall design done, I turned to PCB fabrication and PCBWay. I decided to go with them due to previous good experiences when it came to Framework expansion boards. While they are simple enough, routing them properly is a bit tricky. Only once I already made an order did PCBWay come back and proposed a sponsorship in the form of free PCBs. And that jinxed it, I guess, since for the first time routing was not ok.

Per my understanding, most commonly fabs do board edge routing with a 1.6 mm or larger mill end. This is done for both speed and durability as smaller bits do tend to break more. Routing boards intended for type-C connector used here requires 1.0 mm or smaller bit in order to properly do the notches. If you check the bad board (green) and compare it to the good board (red), you’ll see how little difference there is.

In any case, that gave me an opportunity to deal with their helpdesk for the first time. After writing a ticket, I got a response in less than 24 hours despite them being in holiday time. After a short back-and-forth where I got to explain the exact problem, I had my updated order with extra notes attached. And yes, I selected a different color so I don’t mix them up.

When it comes to the PCB quality, there isn’t much I can say I haven’t said before. A default HASL finish is good enough and there were no leveling issues. If you want ENIG, that is an option albeit quoted price skyrockets when selected. Soldermask is decent and can handle lead-free soldering temperatures without issues. I usually go fora green color as it has the shortest lead time, but you can select other 5 colors at no extra cost (purple and matte colors are extra).

Silk screen is of decent quality and resolution even for small font sizes. And yes, PCBWay still adds their identification number on board by default with option to remove it requiring $1.50. For these boards I don’t really care since they won’t be visible to the end user but if you want to do user-facing panels, you might want to pay extra.

With all components soldered, I connected it to a Framework laptop and the experience was as good as you can get. One great advantage of FTDI chips is that they work flawlessly. I tested it on a few small RS-485 busses at varying speeds (up to 1 Mbaud) without any communication issues. Even under scope things were looking good.

But, is there a way to get rid of FTDI? Well, we can discuss that in the next instalment of the RS-485 Framework expansion card saga. :)

As always, you can find project on GitHub

Hashing It Out

For a while now I had a selection of CRC algorithms in my library. It offered support for many CRC-8, CRC-16, and CRC-32 variants, all inheriting from a bit clunky HashAlgorithm base class. It wasn’t an ideal choice as hashes and checksums are different beasts but it did a job.

With .NET 7 out, we finally got NonCryptographicHashAlgorithm base class to inherit from and that one is much better at dealing with CRC nuances. Adjustment of algorithms was easy enough but testing has shown one issue. Output of Microsoft’s CRC-32 class and one I have created was exactly reversed. For example, if an input would resut in 0x1A2B3C4D output from my class, Microsoft’s class would return 0x4D3C2B1A. Yep, we selected a different endianess.

I originally created my CRC classes with microcontroller communication in mind. Since most of microcontrollers are from the big-endian world, my classes were designed to output bytes in big-endian fashion. On other hand, Microsoft designed their CRC-32 class for communication on x86 platform. And that one is little-endian.

After giving it a lot of thought, I decided to go the Microsoft route and output result in native endianess if using GetCurrentHash method from NonCryptographicHashAlgorithm base class. Main reason was to have the same overall behavior whether someone uses my, Microsoft’s, or any other class. That said, I my classes always had an “escape hatch” method that outputs a number (HashAsUInt32) that can be then converted to any endianess.

In any case, if you need any CRC-8, CRC-16, or CRC-32 calculations with custom polynomials, do check out Medo.IO.Hashing library.

Looping Ansible on Per-user Basis

When I moved my home network to Ansible, I had one problem repeating in many roles. How to do some action for every user?

For example, take setitng up a default .SSH configuration or Bash profile. While setting it for known users is easy enough, I wanted a role to setup things automatically for any user present on the system. Since searches of existing solutions didn’t fully apply to my solution, I decided to roll my own.

The first part is getting list of users and here is where good old passwd comes into play. However, one needs to filter it first for all accounts that cannot log into the system.

grep -v -e '/nologin$' -e '/bin/false$' -e '/bin/sync$' /etc/passwd

In Ansible, collecting this data would look something like this:

- name: Find all users
  shell: "grep -v -e '/nologin$' -e '/bin/false$' -e '/bin/sync$' /etc/passwd"
  register: grep_users_result
  changed_when: false
  failed_when: false

With user list in hand, we can now include the other task and forward it data it needs. In my case, that would be user name and its directory.

- name: Setup user
  include_tasks: peruser.yml
  vars:
    user_name: "{{ item.split(':').0 }}"
    user_directory: "{{ item.split(':').5 }}"
  with_items: "{{ grep_users_result.stdout_lines | list }}"

Finally, we can use those variables in a separate task file:

- name: "Setup .ssh directory ()"
  file:
    path: "/.ssh"
    state: directory
    owner: ""
    group: ""
    mode: "0700"

Easy peasy.

Native ZFS Encryption Speed (Ubuntu 22.10)

There is a newer version of this post.

I guess it’s that time of year when I do ZFS encryption testing on the latest Ubuntu. Is ZFS speed better, worse, or the same?

Like the last time, I did testing on Framework laptop with i5-1135G7 processor and 64GB of RAM. The only change is that I am writing a bit more data during the test this time. Regardless, this is still mostly an exercise in relative numbers. Overall, the procedure is still basically the same as before.

With all that out of way, you can probably just look at the 22.04 figures and be done. While there are some minor differences between 22.10 and 22.04, there is nothing big enough to change any recommendation here. Even ZFS version reflects this as we see only a tiny bump from zfs-2.1.2-1ubuntu2 to zfs-2.1.5-1ubuntu2.

ZFS GCM is still the fastest when it comes to writing and it wins over LUKS by a wide margin. It was a surprise when I saw it with 22.04 and it’s a surprise still. The surprise is not that ZFS is fast but why the heck is LUKS so slow. When it comes to reading speed, LUKS is still slightly faster but not by a wide margin.

With everything else the same, I would say ZFS GCM is a clear winner here with LUKS coming close second if you don’t mind slower write speed. I would expect both to be indistinguishable when it comes to real-world scenarios most of the time.

Using CCM encryption doesn’t make much sense at all. While encryption speed seems to benefit from disabling AES (part of which I suspect to be an artefact of my test environment), it’s just a smidgen faster than GCM. Considering that any upgrade in future is going to bring AES instruction set, I would say GCM is the way forward.

Of course, the elephant in the room is the fact the native ZFS encryption doesn’t actually cover all metadata. The only alternative that can help you with that is running ZFS on top of LUKS encryption. I honestly go back and forth between two with current preference being toward LUKS on laptop and the native ZFS encryption on servers where encrypted send/receive is a killer feature.

You can take a peek at the raw data and draw your own conclusions. As always, just keep in mind that these are just limited synthetic tests intended just to give you ballpark figure. Your mileage may vary.

Encrypted ZFS on Ubuntu 22.10

Ubuntu 22.10 in release notes brought one unpleasant news: “The option to install using zfs as a file system and encryption has been disabled due to a bug”. The official recommendation is to “simply” install Ubuntu 22.04 and upgrade from there. However, if you are willing to go over a lot of manual steps, that’s not necessary.

Those familiar with my previous installation guides will note two things. First, steps below will use LUKS encryption instead of the native ZFS option. I’ve been going back and forth on this one as I do like native ZFS encryption but in setups with hibernation enabled (and this one will be one of those), having both swap with LUKS and native ZFS would require user to type password twice. Not a great user experience.

Second, my personal preferences will “leak through”. This guide will clearly show dislike of snap system and use of way too big swap partition to facilitate hibernation under even the worst-case scenario. Your preferences might vary and thus you might want to adjust guide as necessary. The important part is disk and boot partition setup, everything else is just extra fluff.

Without further ado, let’s proceed with the install.

After booting into Ubuntu desktop installation (via “Try Ubuntu” option) we want to open a terminal. Since all further commands are going to need root credentials, we can start with that.

sudo -i

The very first step should be setting up a few variables - disk, hostname, and username. This way we can use them going forward and avoid accidental mistakes. Just make sure to replace these values with ones appropriate for your system. I like to use upper-case for ZFS pool as that’s what will appear as password prompt. It just looks nicer and ZFS doesn’t care either way.

DISK=/dev/disk/by-id/<diskid>
HOST=<hostname>
USER=<username>

For my setup I want 4 partitions. First two partitions will be unencrypted and in charge of booting. While I love encryption, I decided not to encrypt boot partition in order to make my life easier as you cannot integrate boot partition password prompt with the later data password prompt thus requiring you to type password twice. Both swap and ZFS partitions are fully encrypted.

Also, my swap size is way too excessive since I have 64 GB of RAM and I wanted to allow for hibernation under the worst of circumstances (i.e., when RAM is full). Hibernation usually works with much smaller partitions but I wanted to be sure and my disk was big enough to accommodate.

Lastly, while blkdiscard does nice job of removing old data from the disk, I would always recommend also using dd if=/dev/urandom of=$DISK bs=1M status=progress if your disk was not encrypted before.

blkdiscard -f $DISK
sgdisk --zap-all                      $DISK
sgdisk -n1:1M:+127M -t1:EF00 -c1:EFI  $DISK
sgdisk -n2:0:+896M  -t2:8300 -c2:Boot $DISK
sgdisk -n3:0:+65G   -t3:8200 -c3:Swap $DISK
sgdisk -n4:0:0      -t4:8309 -c4:ZFS  $DISK
sgdisk --print                        $DISK

Once partitions are created, we want to setup our LUKS encryption.

cryptsetup luksFormat -q --type luks2 \
    --cipher aes-xts-plain64 --key-size 512 \
    --pbkdf argon2i $DISK-part4
cryptsetup luksFormat -q --type luks2 \
    --cipher aes-xts-plain64 --key-size 512 \
    --pbkdf argon2i $DISK-part3

Since creating encrypted partition doesn’t mount them, we do need this as a separate step. Notice I use host name as the name of the main data partition.

cryptsetup luksOpen $DISK-part4 $HOST
cryptsetup luksOpen $DISK-part3 swap

Finally, we can setup our ZFS pool with an optional step of setting quota to roughly 80% of disk capacity. Adjust exact value as needed.

zpool create -o ashift=12 -o autotrim=on \
    -O compression=lz4 -O normalization=formD \
    -O acltype=posixacl -O xattr=sa -O dnodesize=auto -O atime=off \
    -O canmount=off -O mountpoint=none -R /mnt/install ${HOST^} /dev/mapper/$HOST
zfs set quota=1.5T ${HOST^}

I used to be fan of using just a main dataset for everything but these days I use more conventional “separate root dataset” approach.

zfs create -o canmount=noauto -o mountpoint=/ ${HOST^}/Root
zfs mount ${HOST^}/Root

And a separate home partition will not be forgotted.

zfs create -o canmount=noauto -o mountpoint=/home ${HOST^}/Home
zfs mount ${HOST^}/Home
zfs set canmount=on ${HOST^}/Home

With all datasets in place, we can finish setting the main dataset properties.

zfs set devices=off ${HOST^}

Now it’s time to format swap.

mkswap /dev/mapper/swap

And then boot partition.

yes | mkfs.ext4 $DISK-part2
mkdir /mnt/install/boot
mount $DISK-part2 /mnt/install/boot

And finally EFI partition.

mkfs.msdos -F 32 -n EFI -i 4d65646f $DISK-part1
mkdir /mnt/install/boot/efi
mount $DISK-part1 /mnt/install/boot/efi

At this time, I also like to disable IPv6 as I’ve noticed that on some misconfigured IPv6 networks it takes ages to download packages. This step is both temporary (i.e., IPv6 is disabled only during installation) and fully optional.

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.lo.disable_ipv6=1

To start the fun we need debootstrap package. Starting this step, you must be connected to the Internet.

apt update && apt install --yes debootstrap

Bootstrapping Ubuntu on the newly created pool comes next. This will take a while.

debootstrap $(basename `ls -d /cdrom/dists/*/ | grep -v stable | head -1`) /mnt/install/

We can use our live system to update a few files on our new installation.

echo $HOST > /mnt/install/etc/hostname
sed "s/ubuntu/$HOST/" /etc/hosts > /mnt/install/etc/hosts
sed '/cdrom/d' /etc/apt/sources.list > /mnt/install/etc/apt/sources.list
cp /etc/netplan/*.yaml /mnt/install/etc/netplan/

If you are installing via WiFi, you might as well copy your wireless credentials. Don’t worry if this returns errors - that just means you are not using wireless.

mkdir -p /mnt/install/etc/NetworkManager/system-connections/
cp /etc/NetworkManager/system-connections/* /mnt/install/etc/NetworkManager/system-connections/

Atlast, we’re ready to “chroot” into our new system.

mount --rbind /dev  /mnt/install/dev
mount --rbind /proc /mnt/install/proc
mount --rbind /sys  /mnt/install/sys
chroot /mnt/install \
    /usr/bin/env DISK=$DISK USER=$USER \
    bash --login

With our newly installed system running, let’s not forget to setup locale and time zone.

locale-gen --purge "en_US.UTF-8"
update-locale LANG=en_US.UTF-8 LANGUAGE=en_US
dpkg-reconfigure --frontend noninteractive locales
dpkg-reconfigure tzdata

Now we’re ready to onboard the latest Linux image.

apt update
apt install --yes --no-install-recommends linux-image-generic linux-headers-generic

Followed by the boot environment packages.

apt install --yes zfs-initramfs cryptsetup keyutils grub-efi-amd64-signed shim-signed

Now it’s time to setup crypttab so our encrypted partitions are decrypted on boot.

echo "$HOST UUID=$(blkid -s UUID -o value $DISK-part4) none \
    luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
echo "swap UUID=$(blkid -s UUID -o value $DISK-part3) none \
    swap,luks,discard,initramfs,keyscript=decrypt_keyctl" >> /etc/crypttab
cat /etc/crypttab

To mount all those partitions, we need also some fstab entries. The last entry is not strictly needed. I just like to add it in order to hide our LUKS encrypted ZFS from file manager.

echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part2) \
    /boot ext4 noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "PARTUUID=$(blkid -s PARTUUID -o value $DISK-part1) \
    /boot/efi vfat noatime,nofail,x-systemd.device-timeout=5s 0 1" >> /etc/fstab
echo "UUID=$(blkid -s UUID -o value /dev/mapper/swap) \
    none swap defaults 0 0" >> /etc/fstab
echo "/dev/disk/by-uuid/$(blkid -s UUID -o value /dev/mapper/$HOST) \
    none auto nosuid,nodev,nofail 0 0" >> /etc/fstab
cat /etc/fstab

If hibernation is desired, a few settings need to be added.

sed -i 's/.*AllowSuspend=.*/AllowSuspend=yes/' \
    /etc/systemd/sleep.conf
sed -i 's/.*AllowHibernation=.*/AllowHibernation=yes/' \
    /etc/systemd/sleep.conf
sed -i 's/.*AllowSuspendThenHibernate=.*/AllowSuspendThenHibernate=yes/' \
    /etc/systemd/sleep.conf
sed -i 's/.*HibernateDelaySec=.*/HibernateDelaySec=13min/' \
    /etc/systemd/sleep.conf

Since we’re doing laptop setup, we can also setup lid switch. I like to set it as suspend-then-hibernate.

sed -i 's/.*HandleLidSwitch=.*/HandleLidSwitch=suspend-then-hibernate/' \
    /etc/systemd/logind.conf
sed -i 's/.*HandleLidSwitchExternalPower=.*/HandleLidSwitchExternalPower=suspend-then-hibernate/' \
    /etc/systemd/logind.conf

Lastly, I adjust swappiness a bit.

echo "vm.swappiness=10" >> /etc/sysctl.conf

And then, we can get grub going. Do note we also setup booting from swap (hibernation support) here too.

sed -i "s/^GRUB_CMDLINE_LINUX_DEFAULT.*/GRUB_CMDLINE_LINUX_DEFAULT=\"quiet splash \
    nvme.noacpi=1 \
    RESUME=UUID=$(blkid -s UUID -o value /dev/mapper/swap)\"/" \
    /etc/default/grub
update-grub
grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=Ubuntu \
    --recheck --no-floppy
update-initramfs -u -k all

Next step is to create boot environment.

KERNEL=`ls /usr/lib/modules/ | cut -d/ -f1 | sed 's/linux-image-//'`
update-initramfs -c -k $KERNEL

Finally, we can install our desktop environment.

apt install --yes ubuntu-desktop-minimal

Once installation is done, I like to remove snap and banish it from ever being installed.

apt remove --yes snapd
echo 'Package: snapd'    > /etc/apt/preferences.d/snapd
echo 'Pin: release *'   >> /etc/apt/preferences.d/snapd
echo 'Pin-Priority: -1' >> /etc/apt/preferences.d/snapd

Since Firefox is only available as snapd package, we can install it manually.

add-apt-repository --yes ppa:mozillateam/ppa
cat << 'EOF' | sed 's/^    //' | tee /etc/apt/preferences.d/mozillateamppa
    Package: firefox*
    Pin: release o=LP-PPA-mozillateam
    Pin-Priority: 501
EOF
apt update && apt install --yes firefox

While at it, I might as well get Chrome too.

cd /tmp
wget --inet4-only https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
apt install ./google-chrome-stable_current_amd64.deb

For Framework Laptop I use here, we need one more adjustment due to Dell audio needing a special care. Note that owners of Gen12 boards need a few more adjustments.

echo "options snd-hda-intel model=dell-headset-multi" >> /etc/modprobe.d/alsa-base.conf

Of course, we need to have a user too.

adduser --disabled-password --gecos '' $USER
usermod -a -G adm,cdrom,dialout,dip,lpadmin,plugdev,sudo,tty $USER
echo "$USER ALL=NOPASSWD:ALL" > /etc/sudoers.d/$USER
passwd $USER

I like to add some extra packages and do one final upgrade before calling it done.

add-apt-repository --yes universe
apt update && apt dist-upgrade --yes

It took a while but we can finally exit our debootstrap environment.

exit

Let’s clean all mounted partitions and get ZFS ready for next boot.

umount /mnt/install/boot/efi
umount /mnt/install/boot
mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export -a

With reboot, we should be done and our new system should boot with a password prompt.

reboot

Once we log into it, we need to adjust boot image and test hibernation. If you see your desktop after waking it up, all is good.

sudo update-initramfs -u -k all
sudo systemctl hibernate

If you get Failed to hibernate system via logind: Sleep verb "hibernate" not supported go into BIOS and disable secure boot (Enforce Secure Boot option). Unfortunately, secure boot and hibernation still don’t work together but there is some work in progress to make it happen in future. At this time, you need to select one or the other.

PS: If you already installed system with secure boot and hibernation is not working, run update-initramfs -c -k all and try again.

PPS: There are versions of this guide using the native ZFS encryption for other Ubuntu versions: 22.04, 21.10, and 20.04.

PPPS: For LUKS-based ZFS setup, check the following posts: 20.04, 19.10, 19.04, and 18.10.