My Encrypted ZFS Setup

For my Nas4Free-based NAS I wanted to use full-disk encrypted ZFS in a mirror configuration across one SATA and one USB drive. While it might not be optimal for performance, ZFS does support this scenario.

On booting Nas4Free I discovered my disk devices were all around the place. To identify which one is which, I used diskinfo:

 diskinfo -v ada0
 ada0
         512             # sectorsize
         2000398934016   # mediasize in bytes (1.8T)
         3907029168      # mediasize in sectors
         4096            # stripesize
         0               # stripeoffset
         3876021         # Cylinders according to firmware.
         16              # Heads according to firmware.
         63              # Sectors according to firmware.
         S34RJ9AG212718  # Disk ident.

Once I went through all drives (USB drives are named da*), I found my data disks were at ada0 and da2. To avoid any confusion in the future and/or potential re-enumeration if I add another drive, I decided to give them a name. SATA disk would be known as disk0 and USB one as disk1:

glabel label -v disk0 ada0
 Metadata value stored on /dev/ada0.
 Done.

glabel label -v disk1 da2
 Metadata value stored on /dev/da2.
 Done.

Do notice that you lose the last drive sector for the device name. In my opinion, a small price to pay.

On top of the labels we need to create encrypted device. Beware to use labels and not the whole disk:

geli init -e AES-XTS -l 128 -s 4096 /dev/label/disk0
geli init -e AES-XTS -l 128 -s 4096 /dev/label/disk1

As initialization doesn’t really make devices readily available, both have to be manually attached:

geli attach /dev/label/disk0
geli attach /dev/label/disk1

With all things dealt with, it was time to create the ZFS pool. Again, be careful to attach inner device (ending in .eli) instead of the outer one:

zpool create -f -O compression=on -O atime=off -O utf8only=on -O normalization=formD -O casesensitivity=sensitive -m none Data mirror label/disk{0,1}.eli

While both SATA and USB disk are advertised as the same size, they do differ a bit. Due to this we need to use -f to force ZFS pool creation (otherwise we will get “mirror contains devices of different sizes” error). Do not worry for data as maximum available space will be restricted to a smaller device.

I decided that pool is going to have the compression turned on by default, there will be no access time recording, it will use UTF8, it will be case sensitive (yes, I know…) and it won’t be “mounted”.

Lastly I created a few logical datasets for my data. Yes, you could use a single dataset, but quotas make handling of multiple ones worth it:

zfs create -o mountpoint=/mnt/Data/Family -o quota=768G Data/Family
zfs create -o mountpoint=/mnt/Data/Install -o quota=256G Data/Install
zfs create -o mountpoint=/mnt/Data/Media -o quota=512G Data/Media

As I am way too lazy to login after every reboot, I also saved my password into the password.zfs file on the TmpUsb self-erasable USB drive. A single addition to System->Advanced->Command scripts as a postinit step was need to do all necessary initialization:

/etc/rc.d/zfs onestop ; mkdir /tmp/TmpUsb ; mount_msdosfs /dev/da1s1 /tmp/TmpUsb ; geli attach -j /tmp/TmpUsb/password.zfs /dev/label/disk0 ; geli attach -j /tmp/TmpUsb/password.zfs /dev/label/disk1 ; umount -f /tmp/TmpUsb/ ; rmdir /tmp/TmpUsb ; /etc/rc.d/zfs onestart

All this long command does is mounting of the FAT12 drive containing the password (since it was recognized as da1 its first partition was at da1s1) and uses file found there for attaching encrypted devices. Small restart of ZFS subsystem is all that is necessary for pool to reappear.

As I wanted my TmpUsb drive to be readable under Windows, it is not labeled and thus manual script correction might be needed if further USB devices are added.

However, for now, I had my NAS box data storage fully up and running.

Other ZFS posts in this series:

• NUC NAS
• Adding cache to ZFS pool
• Creating a ZFS backup machine

NUC NAS

Every few years I update my home NAS server and try to do the best within my restrictions.

First condition is that it has to use the hardware I already have. Yes, I might buy something new so that I free up the existing HW for its bright NAS future, but I don’t want to buy something specific for the NAS. While I am sure there are prebuilt systems that are much better than what I am planning, I am not building NAS only for my data. I am also building it to learn and have fun.

As second condition data has to be reasonably safe. That doesn’t exclude a single drive NAS setup - I’ve been running one for last two years. However, together with a backup process, it has to allow for a full hardware loss while keeping data loss at the minimum. It also has to cover for the remote backup - even if it is just an HDD I keep carrying with me. And I do not have a lot of data on my NAS - currently all things I hold dear are under 1 TB in size.

It also has to be physically small enough I could take it on a plane within my clothes (good padding is important). As I am currently in the US on a non-permanent visa, that scenario is as likely as any hard drive failure. Cheap bastard in me definitely doesn’t want to pay hundreds of dollars for shipping if I can just snuggle NAS in my luggage.

Last condition is that data has to be encrypted at rest. While NAS is at home I might make some things easy on me (e.g., auto-decryption at startup) but it has to be possible to keep data encrypted during transport. I am not saying TSA might be stealing stuff from luggage, I just want to be cautious.

All these things taken into consideration, I decided to use my old Intel NUC D34010WYKH as a new data storage. It is a two-core (4 logical processors) i3 device running at 1.7 GHz accompanied by 8 GB RAM and enough room for one SSD (mSATA) and one small 2.5" HDD (SATA). This nicely covered using old hardware (this was my ex-HTPC) and a small size.

For OS I decided upon XigmaNAS as it supports ZFS and it can be installed on an USB drive thus leaving my other drives fully available for data. I did consider FreeNAS as OS but NAS4Free just felt better. With ZFS I also had option of using FreeBSD or Solaris but I decided not to deal with OS updates myself. And yes, I know Linux supports both ZFS and its deranged brother BTRFS, but there are too many issues with getting either to work without issues.

As you could deduce, ZFS is going to be in charge of all data with the encryption taken care of by GELI. I did lose a bit of comfort as encryption makes web management a bit more difficult but, once scripts are in place, you don’t need GUI anyhow. To allow for quick disabling of auto-decryption I would use TmpUsb drives with auto-delete. If server gets stolen this would ensure nobody can get my data.

As I wanted to have a mirror and NUC has enough place only for one 2.5" 2 TB drive, I decided to have an external 2 TB USB 3.0 drive as its partner. To make backup work I would sync daily snapshots to another local machine (manual dual boot) and to the other at a remote site. In addition to this, I would also do the weekly backup on an external USB.

Let me be the first to say I know this setup is far from the ideal with two obvious (and big) faults. The first one is not having the ECC RAM as this diminishes data security ZFS has to offer. It is not a catastrophe but not what you might want for your NAS either. Second is the need for 2.5" drives due to NUC’s size. Those drives are more expensive, offer less capacity, and are slower than the bigger 3.5" brethren. This is made even worse by having an external USB drive as a part of the pool as this is making the performance worse than it should be. And let’s not even go thinking about accidental unplugging…

Regardless of all its limitations, I believe this setup is going to work well for my purpose. If everything else fails it will at least give me endless hours of scripting fun needed to make all this work.

Other ZFS posts in this series:

• My encrypted ZFS setup
• Adding cache to ZFS pool
• Creating a ZFS backup machine

Seattle Code Camp 2015 - Done

A day has been a way too short today at Seattle Code Camp. With quite a few interesting talks and folks it passed a way too quickly. Many parallel tracks (nine!) ensured that there will be at least couple of talks not seen - especially if you like to discuss stuff with people.

Unlike last year, this year talk, this year I had a fortune to give two talks.

Git in the enterprise environment session, as name implies, dealt with the slightly different philosophy of the source control within a large corporations. Interactivity during talk ensured discussion to spill throughout the day and that is the best any speaker could wish. As promised, here are PowerPoint slides but for many things you had to be there. :)

Second session was more of an Q&A discussion about working with the .NET under Linux. Although originally discussion format was selected due to the lack of time to prepare examples (I somehow missed that two of my talks went in the conference program - doh!) I had a lot of fun and a really positive feedback. I am definitely thinking about intentionally doing the same next year - relaxed discussion with audience definitely got my blood flowing. PowerPoint slides are a bit raw but there is a link or two in them.

Great thanks to both organizers and sponsors and see you next year.

PuTTY Doesn't Work With NAS4Free

Every few years there comes the time to refresh my NAS hardware and the choice usually falls upon the latest NAS4Free installation. As I do fair amount of customizing, this means that SSH access is mandatory. With version NAS4Free 10 I stumbled upon trouble. My trusty PuTTY could not connect and there was no obvious reason why.

Only potential culprit I could find was in NAS4Free using DSA keys but PuTTY has been supporting those for ages so that was obviously not a full story. And I could connect from Linux so it was really PuTTY doing funny stuff and not misconfiguration. As I wanted my project to go further I decided to find PuTTY’s replacement. And that search pretty much boiled my choices down to two.

First candidate was MobaXterm. Not only that this replaces PuTTY but it also offers much better session management and an reasonable tabbed interface. However, it has $70 price tag attached. Yes, there is a free version too but its restrictions make it unsuitable for anybody dealing with SSH regularly. Call me a cheap bastard but I don’t want to give that kind of money for SSH client. All other functionality MobaXterm has is a nice touch and might make it worth that money, but I didn’t really have any use for it.

Another program worth considering was mRemoteNG. While this one also worked well as a SSH client toward my NAS4Free machine and it does come as a free download I found that its interface was simply too annoying to deal with. Yes, I would use it in a pinch but for the most time it was making me think MobaXterm might be worth it.

And then I went onto PuTTY’s page and saw there was a new release available (0.65). Guess what? That release worked without a hitch. Yes, sane person would check for a new version before spending time testing the replacement but I got so used to PuTTY developed in a lazy fashion that I honestly didn’t expect a new version to be there.

So, after a long search I came back to PuTTY and its abysmal session management. And I couldn’t be happier about it.

Cinemark Cinema and the Waste of Time

Occasionally I visit a cinema with family - this time it happened due to Inside Out - a decent family movie with a couple of good jokes. Not the best I’ve seen but not the worst way to spend Saturday night either.

After getting tickets, buying overpriced snacks we’ve took our places and watched various commercials, trailers, and messages about the evils of the mobile phone. To quote them: “… we want our customers to enjoy their movie - FREE from distraction …” I share the view that mobile phones are distractions during movie and I do believe in silencing them completely. What I don’t get is why stop there? Why not remove the other distractions?

Why do I need to go through 10 commercials? Why is there at least 5 trailers before the movie I actually came to see? Why the heck movie scheduled for 7:25 starts at 7:40? This so reminds me of the unskippable FBI warnings on DVDs. Why the heck are you molesting your paying customers? Is there anybody really thinking those warnings work? Is there anybody really thinking pre-movie trailers do anything?

Frankly these days I avoid going to cinema almost completely. Only exception are animated movies that come recommended by friends as worthy of the big screen. All other stuff I simply ignore until it arrives on Amazon or Netflix. If it doesn’t arrive there timely I usually forget about it altogether and I don’t think I lose much.

I do love watching a movie on the big screen in the cinema. I even love their overpriced popcorn. But I simply cannot handle amounts of rubbish one has to consume before actually getting to see the movie. And with passing years it is only getting worse.

PS: And coming late is not the solution as in States seats are given on first come, first served basis.