My Backup ZFS Machine (Correction)

Before going on vacation I finished setting up my new ZFS backup machine, initialized first replication, and happily gone to see the big hole.

When I remotely connected to my main machine a few days later, I’ve found my sync command has failed before finishing. Also I couldn’t connect to my backup server. Well, that was unfortunate but I had enough foresight to get it connected via smart plug so I did power-off/power-on dance. My system booted and I restarted replication. I checked on it a few days later, to find it stuck again. Rinse-repeat. And the next day too. And the one after that…

Why? I have not idea as I was connected only remotely and I literally came home on the last day when I could return it to Amazon. Since I did raise a case with Supermicro in regards to video card error (5 beeps) which seemed hardware related my suspicions were definitely pointing only in direction of motherboard issue. I know memory was fine as I tested it thoroughly in another machine and power supply is happily working even now.

For my peace of mind I needed something that would allow me not only to reboot machine but to also access its screen and keyboard directly without any OS involvement. Variants are known under different names and slightly different execution. Whether it is KVM, iLO, AMT, or IPMI.

So I decided to upgrade to more manageable Supermicro A1SRi-2558F. With its C2558 processor (4 cores) and quad LAN it was definitely an overkill for my purpose but it was the cheapest IPMI-capable board I could find at $225 (compared to $150 for X10SBA-L). Unfortunately for my budget its ECC requirement meant adding another $35 for ECC RAM. And of course, different layout made my 6" right-angle SATA cable useless so now they decorate my drawer.

Board itself is really full of stuff with total of six USB ports (four are USB 3.0), one of which was even soldered on motherboard for internal USB needs. Having four gigabit ports is probably useless as Atom is unlikely to be able to drive them all at full speed but I guess it does allow for more relaxed network configuration. Moreover two SATA3 and four SATA2 just scream NAS. And rear bracket on my 1U case fits rear IO perfectly. Frankly, the only thing missing is HDMI albeit IPMI greatly reduces chance of ever needing it.

Total difference in system cost was $100 and it gave me a rock-solid experience (hasn’t crashed a single time in more than a month). Here is updated shopping list:

Setting Up Private Internet Access on Mint, 2018 Edition

I have already written about getting Private Internet Access running on Linux Mint back in 2016. Main reason is that with Linux Mint 18, not all DNS changes are properly propagated.

As OpenVPN client is installed by default these days, we only need to download PIA’s OpenVPN configuration files. More careful ones will notice these files are slightly different than recommended default. These have VPN server IP instead of DNS name. While this might cause long term issues if that IP ever changes, it does help a lot with firewall setup as we won’t need to poke a hole for DNS over our eth0 adapter.

From downloaded archive select .ovpn file with desired destination (usually going with one closest to you gives the best results) and also get both .crt and .pem file. Copy them all to your desktop and we’ll use them later for setup. Yes, you can use any other directory too - this is just one I prefer.

With this done we can go into configuring VPN from Terminal window (replacing username and password with actual values):

sudo mv ~/Desktop/*.crt /etc/openvpn/
sudo mv ~/Desktop/*.pem /etc/openvpn/
sudo mv ~/Desktop/*.ovpn /etc/openvpn/client.conf

sudo sed -i "s*ca *ca /etc/openvpn/*" /etc/openvpn/client.conf
sudo sed -i "s*crl-verify *crl-verify /etc/openvpn/*" /etc/openvpn/client.conf

sudo echo "auth-user-pass /etc/openvpn/client.login" >> /etc/openvpn/client.conf
sudo echo "mssfix 1400" >> /etc/openvpn/client.conf
sudo echo "dhcp-option DNS 209.222.18.218" >> /etc/openvpn/client.conf
sudo echo "dhcp-option DNS 209.222.18.222" >> /etc/openvpn/client.conf
sudo echo "script-security 2" >> /etc/openvpn/client.conf
sudo echo "up /etc/openvpn/update-resolv-conf" >> /etc/openvpn/client.conf
sudo echo "down /etc/openvpn/update-resolv-conf" >> /etc/openvpn/client.conf

unset HISTFILE
echo '^^username^^' | sudo tee -a /etc/openvpn/client.login
echo '^^password^^' | sudo tee -a /etc/openvpn/client.login
sudo chmod 500 /etc/openvpn/client.login

Now we can test our VPN connection:

sudo openvpn --config /etc/openvpn/client.conf

Assuming that this last step ended with Initialization Sequence Completed, we just need to verify whether this connection is actually used and I’ve found whatismyipaddress.com quite helpful here. Just check if IP detected there is different then IP you usually get without VPN.

Stop the test connection using Ctrl+C so we can configure automatic startup and test it.

echo "AUTOSTART=all" | sudo tee -a /etc/default/openvpn
sudo reboot

Once computer has booted and you are satisfied with VPN configuration, you can think about firewall and disabling default interface when VPN is not active. This means allowing traffic only on tun0 interface (VPN) and allowing only port 1198.

sudo ufw reset
sudo ufw default deny incoming
sudo ufw default deny outgoing
sudo ufw allow out on tun0
sudo ufw allow out on `route | grep '^default' | grep -v "tun0$" | grep -o '[^ ]*$'` proto udp to `cat /etc/openvpn/client.conf | grep "^remote " | grep -o ' [^ ]* '` port 1198
sudo ufw enable

Assuming all went well, VPN should be happily running.

Visual Studio 15.3

Behind Visual Studio’s slight version bump from 15.2 to 15.3, we have a major update.

First of all, .NET Core is finally here accompanied with .NET Standard 2.0. Both have greatly increased API coverage and hope is that they will help with greater acceptance of the whole new (open-source) ecosystem.

In addition there is C# 7.1 which is first time Microsoft updated a language in a minor Visual Studio version. True, there are not many changes to the language itself (albeit async Main was longed for) but it signals a new direction of decoupling Visual Studio, C#, .NET Standard releases.

I hope this .NET Standard 2.0 will do what .NET 2.0 did back in 2005 and unite fragmented developers around a new common denominator.

More details at Channel 9.

Adding Mirrored Disk to Existing ZFS Pool

Great thing about ZFS is that even with a single disk you get some benefits - data integrity being the most important. And all ZFS commands work perfectly well, for example status:

zpool status
   pool: Data.Tertiary``
  state: ONLINE``
 config:``
         NAME                   STATE     READ WRITE CKSUM``
         Data.Tertiary          ONLINE       0     0     0``
           diskid/DISK-XXX.eli  ONLINE       0     0     0``

However, what if one disk is not sufficient any more? It is clear zpool add can be used to create striped pool for higher speeds. And it is clear we can add another device to make a three way mirror. But what if we want to convert solo disk to mirror configuration?

Well, in that case we can get creative with attach command giving it both disks as an argument:

zpool attach Data.Tertiary ^^diskid/DISK-XXX.eli^^ ^^diskid/DISK-YYY.eli^^

After a few seconds, our mirror is created with all our data intact:

zpool status
   pool: Data.Tertiary
  state: ONLINE
 status: One or more devices is currently being resilvered.  The pool will
         continue to function, possibly in a degraded state.
 action: Wait for the resilver to complete.
 config:
         NAME                     STATE     READ WRITE CKSUM
         Data.Tertiary            ONLINE       0     0     0
           mirror-0               ONLINE       0     0     0
             diskid/DISK-XXX.eli  ONLINE       0     0     0
             diskid/DISK-YYY.eli  ONLINE       0     0     0  (resilvering)

PS: Yes, I use encrypted disks from /dev/diskid/ as I used them in previous ZFS examples. If you want plain devices, just use ada0 and companions instead.

Garmin Vivoactive

I am a fan of my Pebble Steel. However, in between Pebble not existing as a company anymore and my own Pebble dying I had to make a decision. Do I get last of the Pebble as replacement or I try something new?

My two requirements were to have long battery life (five days is bare minimum) and reasonable cost (less than $200). Here Pebble shined and Garmin VivoActive fits reasonably well. It has seven days battery life (assuming you avoid GPS) and, if you are willing to wait, you can find it on discount for $150 (I got mine at B&H).

Let me start by saying that Garmin is a sport watch. One that includes GPS and heart strap. I tried activity tracking for a few of my walks and it is interesting to say the least. I never tried heart strap though. All that said, I didn’t buy this sport watch for its sport functionality. I bought it because of its smartwatch functionality. So my review (spoilers!) might seem a bit harsh but note I am not reviewing the whole watch but just a certain aspect of its usage.

Lets start with the physical shape. Garmin is a tiny bit bigger than Pebble Steel but also much thinner. Both did fit equally well on my wrist but Pebble did look bigger and more intimidating with its steel construction than “plasticky” Garmin. Bezel scuffs are more visible on Garmin and you can see irregularities on black surface much more readily than on steel.

In regards to screen area, they are pretty much the same with Pebble having a vertical display while Garmin opted for horizontal mounting. Garmin does have color screen but I wouldn’t count it as a positive thing as Pebble’s black&white display offers better readability. Unfortunately Pebble also moved to color screen with later models so this excellent screen is exception and not a rule.

Pebble has all its keys on side while Garmin opted to have two side buttons, two on screen, and a touch sensitive screen. While touch sensitive screen is not necessarily a bad thing, Garmin’s execution where support for certain gestures is highly dependent on application and quite inconsistent over platform leaves a lot to be desired.

Charging is done on side pins for Pebble while Garmin has USB connection on bottom. Garmin’s charger is much nicer but you do need to take your watch off to use it. As an interesting tidbit, it is magnetic so you can have it stuck to any metallic surface. This might be side-effect of using magnet to hold the watch and it doesn’t really interfere with anything. I would dare to say some people might even find it useful to keep charger from moving. You can comfortable charge Pebble Steel without taking watch of your wrist but this is the last model allowing for that. All newer Pebbles have chargers at bottom making charging while on wrist uncomfortable at best.

Pebble is quite annoying with its bluetooth-loading application system. If your phone dies you are stuck to a single application that was last running and that’s it. If you need other application - e.g. compass - tough luck. Garmin works similar how Pebble used to. It has certain amount of memory on watch for applications (unlike Pebble’s fixed slots) and you fill it to your liking. As I need only a few things on my watch, I didn’t have problem with that system - I actually prefer it. However, poor selection of half-baked applications in Store is disappointment. Heck, even finding a nice, visible, uncomplicated watchface is daunting task between all “Hello World” apps. It sorta reminds me of Windows Store. Utter lack of applications and even those available are subpar.

Mail functionality is baked-in and it will show you notifications if all goes right. It will only show a few lines by default with option to see more. On Pebble you just push up/down buttons and your done. On Garmin you need to do ancient incantations while stroking screen in hope it will show you more. If you succeed, you will also see just a line or two because some idiot decided to include big X icon beneath it despite having a back button literally 2 millimeters further. It is a total disregard for the limited screen real estate.

But that is not the biggest worry with Garmin as for no apparent reason you might stop receiving e-mails. And it is not necessarily that something is wrong with your bluetooth as you can have Twitter notifications appearing during the same time. And that is the best case scenario as bluetooth will stop and start working depending on Jupiter’s tidal cycle. It makes having smartwatch pointless for me as I have to check my phone regardless. I never had these problems with Pebble.

And let’s not forget Garmin can remind you to move every hour or so if you are inactive. I wanted to like this feature. I really did. But two things are wrong with it. First of all, it doesn’t properly detect when I am sitting vs laying down. So I would get warnings to move my butt even as I was just about to fall asleep. And warning is the same vibration as for message. So you can never be sure if it is message or movement command. And then you walk a bit and you reach your daily goal. This uses the long vibration. Just enough that you think you are getting the phone call. Why so annoying?

Also, despite having the hardware to do so, Garmin also opted not to turn on light when you flick your wrist. That means, if you are in low lightning or god-forbid bed, there is no other way to see screen than to get your other hand to backlight button. Why? It is just a minor simple feature that is actually supported on Garmin’s higher-end watches. They just chose not to implement it here.

Speaking of not implementing stuff, forget any GPS navigation functionality. And I am not talking about mapping - it is obvious that there is not enough memory for that. A simple application allowing to save a few points of interest would go a long way. Or just using damn GPS to sync time. Not at all times as battery is limited resource but when asked to. It is kinda ironic that watch capable of receiving GPS signal fully depends on damn mobile phone for its accuracy.

For my non-sport use case, Garmin’s Vivoactive is a failure. After using it for four months, I simply switched to a normal wrist watch and I’m happier for it. And that is the highest insult you can give to any smartwatch.