About Time

Illustration

As we approach yet another biyearly daylight savings time change, there came a news about politicians actually doing something smart for a change. European Parliament voted to keep the same UTC offset the whole year round. If this decision is followed through, gone are the days of hunt for every clock to update it an hour back or forward before the next change comes.

And yes, of course it is not as simple as turning the daylight change off. First it needs to get to the European Commission that traditionally likes to avoid implementing anything. Then all member states need to agree when to do it. Then decision needs to be postponed multiple times. Then everybody will try to implement the decision at the very last moment. You know, the usual.

However, for the first time in forever there is some hope. I can already see myself talking to my grandchildren about the dark times when people all around the world changed their time by hour (or less - damn you Lord Howe Island), for reasons long forgotten, and at the time every country decided upon themselves.

Grandchildren will probably just say that grandpa is crazy and that this could have never been. And then they will go out to play in the UTC world…

Fix NAS4Free's Non-Working Google SMTP After Upgrade

Illustration

A few days after I updated NAS4Free to 11.1.0.4 (revision 5109), I noticed my e-mail reports were not arriving any longer. After making sure my script was working, I went onto checking my e-mail settings - first by sending a test e-mail.

There I received not really helpful message: “Failed to send test email. Please check the log files.” Further check in log showed equaly useless “root: Failed to send test email to: mail@example.com.” I guess it was time for the command line.

The easiest way to test e-mail is probably running msmtp directly from SSH and see what its issue is:

echo "Subject: Test" |& msmtp mail@example.com
 msmtp: account default from /usr/local/etc/msmtprc: cannot use tls_trust_file with tls_certcheck turned off

If you check that file, the offending line is indeed there:

cat /usr/local/etc/msmtprc
 account default
 domain example.com
 host smtp.gmail.com
 port 587
 protocol smtp
 from mail@example.com
 auth login
 user mail@example.com
 password 123
 tls on
 tls_starttls on
 tls_certcheck off
 !!tls_trust_file!! /usr/local/etc/ssl/cert.pem
 syslog LOG_MAIL

The easiest fix for this issue is to enable TLS Server Certificate Check in GUI. This will actually add a bit of security and there are no real downsides if you use Google’s SNMP as I do.

If you are using a custom e-mail server with self-signed certificate, removing offending line in post-startup script is your best bet:

sed -i -- '/^tls_trust_file/d' /usr/local/etc/msmtprc

Of course, proper solution would be for GUI to omit TLS trust file configuration when certificate check if off. Due to unrelated changes this seems to be already fixed in msmtp.in source file. Next version will have it all sorted out. :)

[2018-02-16: Issue is fixed in version 11.1.0.4.5127.]

[2018-07-22: NAS4Free has been renamed to XigmaNAS as of July 2018]

GELI With XTS-AES or AES-CBC?

If you are using GELI to encrypt disks on FreeNAS or NAS4Free, there are multiple algorithms you can use. However, in practice, for me this always becomes a choice between AES-XTS and AES-CBC. Since every modern processor had AES primitives supported in hardware, these two offer performance other algorithms cannot reach.

The current industry standard for disk encryption is XTS-AES and that is mode I usually default to. It has been made specifically to facilitate full-disk encryption and has no known attacks (other than malleability attack common to practically all such algorithms).

Using AES-CBC is not as crazy as it seems. While there is a known “watermark” attack, GELI is not vulnerable at it uses CBC-ESSIV. And, while it is still vulnerable against malleability attack, it’s more resilient than XTS to some (albeit rather unrealistic) attacks while sharing CBC’s “friendliness” to altering bits on per-block level.

In the absence of any new attack, you are probably fractionally more secure with XTS but realistically both XTS-AES and AES-CBC with ESSIV can be considered equivalent and well-suited for full-disk encryption. That said, as security is not concern, are there performance differences between those two?

And there are. At least on NAS4Free albeit I believe the same applies across the BSD family - including FreeNAS.

My ZFS mirror on i3-4010U used XTS-AES-128 for encryption and its median write speed was 197 MB/s. The same mirror had its write speed increased to 389 MB/s once both disks were set to use AES-CBC-128. Interestingly, changing the encryption mode on a single disk to CBC while the other was still using XTS also improved performance - all the way to 384 MB/s. Mind you there are no security benefits running two disks in different encryption modes. I just did it for fun. :)

My Atom C2558 server using software encryption with XTS-AES only wrote at 110 MB/s. Once both physical drives used AES-CBC, speed jumped to 162 MB/s. Again, changing only a single drive in mirror to CBC increased write speed to 157 MB/s.

The same server using hardware encryption could write 160 MB/s when both drives encrypted using XTS-AES. Both drives using AES-CBC achieved again 160 MB/s. Once more, the curiosity was half/half situation with write speed at 162 MB/s.

Based on these results it is obvious that AES-CBC performs much better than XTS-AES in some cases or both have the same speed at worst. The fact changing a single drive to CBC already brought 95% of improvement tells me that CPU was really straining with XTS despite hardware AES-NI support. On the slower CPU difference was non-existent albeit this was more related to its general slowness of AES-NI on Atom than to anything algorithm specific.

Intel’s own tests on the same generation (albeit vastly more powerful processors) shows that XTS implementation can actually be faster than CBC - especially when multiple cores can “exploit” XTS’ inherent parallelism.

This just shows that performance is highly dependent on everything in the pipeline - especially software support - and not just hardware. And one should be careful to occasionally retest if old assumptions are still valid. For example, the next version of BSD crypto library could improve performance of XTS to surpass CBC.

I personally will still stick with XTS-AES. Encryption and decryption speeds, while not exhilarating, are more than adequate for sharing files over Samba connections. While AES-CBC will surely enjoy support in the future, XTS is the one standardized for full-disk encryption. Unless a security issue is discovered, support for it will only get better with time.


PS: Malleability attack on AES-XTS and AES-CBC is actually not that problematic for my setup as ZFS includes data checksum. To be completely sure, for sensitive applications, one should think about using SHA-256 instead of default Fletcher-16 checksum.

PPS: GELI actually doesn’t use ESSIV as you would find it on Linux. However, its implementation is pretty much equivalent to it in both security and performance.

PPPS: For curious, I used the following command to test write speed, repeating the measurement 5 times:

dd if=/dev/zero of=/mnt/Temp/Test.tmp bs=4096 count=1048576

PPPPS: You can also get rough idea which encryption method is faster by running OpenSSL speed test:

openssl speed -evp aes-128-xts
openssl speed -evp aes-128-cbc

[2018-07-22: NAS4Free has been renamed to XigmaNAS as of July 2018]

Configuring Classless Static Route Option

If you want to push routes to your client, the easiest way to do so would be adding a classless static route (DHCP option 121) as defined in RFC 3442. Every router has their way of setting these but usually they have one thing in common - you must do so manually. And yes, if you make a single mistake, your Internet connectivity will be lost.

Issue of easy entry has bothered me for long enough to actually do something about it. Below find classless static route option calculator. Just enter routes you want and you will get their hexadecimal representations.

NetworkGateway
Default
DHCP option 121:
OpenSense/Ubiquiti notation:
Mikrotik code:

[2019-12-13: Updated script to have default route first (workaround for Ubuntu 19.10 Server).] [2020-12-26: Added OpenSense/Ubiquiti notation.] [2022-07-22: Fixed to allow for /32 network.]

Change Default Shell in NAS4Free

Albeit I love almost everything about NAS4Free and his cousin FreeNAS, I can never get adjusted to its shell choice. It might be that tcsh is an awesome shell, but I am much more accustomed to bash.

Standard FreeBSD (and Linux) approach is to use chsh command. However, that command is not present in NAS4Free. Fortunately, there is an alternative choice.

Command pw offers that and much more. To change shell, we simply execute the following command:

pw user mod root -s /bin/bash

While we cannot make this default, we can add it under System, Advanced, Command Scripts. If we add this command as Post Init script, the next login will greet us with bash prompt.

[2018-07-22: NAS4Free has been renamed to XigmaNAS as of July 2018] [2018-08-13: This change does result loss of console menu. There is a slightly different method without that downside.]