EEPROM Access Routines For PIC18

While I was playing with CAN bus, I needed to write something to EEPROM memory - to keep it safe between reboots. Imagine my surprise when calls to eeprom_read and eeprom_write returned errors:

warning: unsupported: The Read_b_eep routine is no longer supported. Please use the MPLAB X MCC.
warning: unsupported: The Busy_eep routine is no longer supported. Please use the MPLAB X MCC.
warning: unsupported: The Write_b_eep routine is no longer supported. Please use the MPLAB X MCC.

It seems PIC18 series doesn’t support old EEPROM routines that worked on PIC16 but, at the same time, my PIC is old enough not to be properly supported in the suggested MCC. I guess it was time to write PIC18 version of these functions myself.

Reading over documentation for PIC18F25K80, I was pleasantly surprised that EEPROM writing hasn’t really changed. My assembly code I wrote back in PIC16F84 days was quite close to what’s needed these days too. The only significant change was expanding EEADR to allow for more than 256 bytes. Yes, there are some shenanigans with setting EEPGD/CFGS bits but that’s done only once.

My replacement for eeprom_read function was similarly named eeprom18_read and it’s really a drop-in replacement. Not to be the one to dwell in past, I opted to change my ASM code to XC8 flavored C. It’s essentially just setting up EEPROM address, followed by read request, a short pause to get the data, and then reading the data itself.

unsigned char eeprom18_read(unsigned int offset) {
  EECON1bits.EEPGD = 0; // accesses data EEPROM memory
  EECON1bits.CFGS = 0;  // accesses data EEPROM memory

  EEADRH = offset >> 8;
  EEADR = offset;

  EECON1bits.RD = 1; // initiates an EEPROM read
  Nop();             // it can be read after one NOP instruction

  return EEDATA; }

Writing was a bit more involved but not too much so. After setting EEPROM address and data, it was important to allow EEPROM write and disable interrupts as ancient EEPROM unlock sequence (0x55 0xAA) must not be interrupted. After write is initiated, code will wait for it to complete and then restore interrupts. For safety it will also disable further writes to EEPROM until it’s needed again.

void eeprom18_write(unsigned int offset, unsigned char value) {
  EECON1bits.EEPGD = 0;  // accesses data EEPROM memory
  EECON1bits.CFGS = 0;   // accesses data EEPROM memory

  EEADRH = offset >> 8;
  EEADR = offset;

  EEDATA = value;

  unsigned char oldGIE = GIE;  // interrupts be disabled during this code segment
  GIE = 0;

  EECON1bits.WREN = 1;  // allows write cycles

  EECON2 = 0x55;  // write sequence unlock
  EECON2 = 0xAA;  // write sequence unlock

  EECON1bits.WR = 1;     // initiates a data EEPROM erase/write cycle
  while(EECON1bits.WR);  // waits for write cycle to complete

  EECON1bits.WREN = 0;  // disable write
  GIE = oldGIE;         // restore interrupts
}

Encrypted ZFS (A Slightly Parallel Edition)

Initial encryption of ZFS pool does require a bit of work - especially when it comes to initial disk randomization. Yes, you could skip it but then encrypted bits are going to stick out. It’s best to randomize it all before even doing anything ZFS related.

The first problem I had with the old setup was the need to start randomizing each disk separately. Since operation takes a while (days!), this usually resulted in me starting all dd commands concurrently thus starving it of resources (mostly CPU for random number generation).

As my CPU can generate enough random data to saturate two disks, it made sense to use parallelize xargs using the serial number (diskid) of each disk as an input. While using /dev/sd* would work, I tend to explicitly specify disks serial number as it’s not destructive if ran on the wrong machine. I consider it a protection against myself. :)

The final command still takes ages but it requires only one window and it will take care to keep only two disks busy at a time:

echo "^^DISK-ID-123^^ ^^DISK-ID-456^^ ^^DISK-ID-789^^ ^^DISK-ID-012^^" | \
  tr ' ' '\n' | xargs -I '{}' -P 2 \
  dd if=/dev/urandom of=/dev/diskid/{} bs=1M

After drives are “cleaned”, I do the encryption (one-by-one this time):

echo "^^DISK-ID-123^^ ^^DISK-ID-456^^ ^^DISK-ID-789^^ ^^DISK-ID-012^^" | \
  tr ' ' '\n' | xargs -I '{}' \
  geli init -e AES-XTS -l 128 -s 4096 '/dev/diskid/{}'

There used to be times when I encrypted each disk with a separate password and that’s still a bit more secure than having a single one. However, with multiple passwords comes a great annoyance. These days I only have a single password for all the disks in the same pool. It makes my life MUCH easier.

In theory, somebody cracking one disk will immediately get access to all my data but in practice it makes no big difference. If somebody decrypted one disk, they either: found a gaping hole in Geli and/or underlying encryption and thus the other disks will suffer the same fate and there’s nothing I can do; or they intercepted one of my keys. As I always use all the keys together, chances are that intercepting one is the same effort as intercepting them all. So I trade a bit of security for a major simplification.

Now we get to attach all encrypted drives:

echo "^^DISK-ID-123^^ ^^DISK-ID-456^^ ^^DISK-ID-789^^ ^^DISK-ID-012^^" | \
  tr ' ' '\n' | xargs -I '{}' \
  geli attach '/dev/diskid/{}'

And the final step is creating ZFS pool, using RAIDZ2 and allowing for loss of two disks before data is compromised:

zpool create \
  -o autoexpand=on -m none -O compression=gzip-7 -O atime=off \
  -O utf8only=on -O normalization=formD -O casesensitivity=sensitive \
  ^^Data^^ raidz2 \
  /dev/diskid/^^DISK-ID-123^^.eli  /dev/diskid/^^DISK-ID-456^^.eli /dev/diskid/^^DISK-ID-789^^.eli /dev/diskid/^^DISK-ID-012^^.eli

And that’s it - pool is ready for all the data you can throw at it.


PS: Yes, I am still using Geli - native ZFS encryption didn’t find its way to FreeBSD yet.

PPS: If machine goes down, it is enough to re-attach Geli disks followed by restart of the ZFS daemon:

echo "^^DISK-ID-123^^ ^^DISK-ID-456^^ ^^DISK-ID-789^^ ^^DISK-ID-012^^" | \
  tr ' ' '\n' | xargs -I '{}' \
  geli attach '/dev/diskid/{}'

/etc/rc.d/zfs onestart

Failed to Load SELinux Policy

After a failed yum upgrade (darn low memory) I noticed my CentOS NTP server was not booting anymore. Look at console showed progress bar still loading but pressing Escape showed the real issue: Failed to load SELinux policy, freezing.

The first thing in that situation is to try booting without SELinux and the easiest way I found to accomplish this was pressing e on boot menu and then adding selinux=0 at the end of line starting with linux16. Continuing boot with Ctrl+X will load CentOS but with SELinux disabled.

As I actually don’t run my public-facing servers without SELinux, it was time to fix it. Since I didn’t have package before, I installed selinux-policy-targeted but I would equally use reinstall if package was already present. In any case, running both doesn’t hurt:

sudo yum install -y selinux-policy-targeted
sudo yum reinstall -y selinux-policy-targeted

Finally we need to let system know SELinux should be reapplied. This can be done by creating a special .autorelabel file in the root directory followed by a reboot:

sudo touch /.autorelabel
sudo reboot

During reboot SELinux will reapply all labeling it needs and we can enjoy our server again.

XigmaNAS Booting Directly Into Shell

Illustration

One of my XigmaNAS machines had a curious issue. Upon boot, it wouldn’t display the console menu. After boot it would go directly to the bash prompt - no password asked. While practical, this was completely insecure as anyone capable of connecting monitor and keyboard would get a full access.

Upon checking configuration, culprit was obvious - I changed the default shell. As console menu execution is part of .cshrc configuration file and bash ignores that file, the console menu was forever lost.

I definitely didn’t like that.

Since I really wanted bash prompt but also preferred to have the console menu (that can be disabled), I decided upon a slightly different shell selection approach without system-wide consequences . Simply adding exec bash command at the end of .cshrc works nicely without the nasty side-effects The following PostInit script will do the trick:

printf '\\nif ($?prompt) then\\n\\texec bash\\nendif\\n' >> /root/.cshrc

And yes, check if we’re working with interactive shell ($?prompt part) is necessary because commands executed without terminal (e.g. directly over SSH or cron jobs) will fail otherwise.

ZFS Record Size For Backup Machine

Illustration

When it came to setup my remote backup machine, only three things were important: use of 4K disks, two disk redundancy (raidz2), and a reasonably efficient storage of variously sized files. Reading around internet lead me to believe volblocksize tweaking was what I needed.

However, unless you create zvol, that knob is actually not available. The only available property impacting file storage capacity is recordsize. Therefore I decided to try out a couple record sizes and see how storage capacity compares.

For the purpose of test I decided to create virtual machine with six extra 20 GB disks. Yes, using virtual machine was not ideal but I was interested in relative results and not the absolute numbers so this would do. And mind you, I wasn’t interested in speed but just in data usage so again virtual machine seemed like a perfect environment.

Instead of properly testing with real files, I created 100000 files that were about 0.5K, 33000 files about 5K, 11000 files about 50K, 3700 files about 500K, 1200 files about 5M, and finally about 400 files around 50M. Essentially, there were six file sizes with each set being one decade bigger but happening only a third as often. The exact size for each file was actually randomly chosen to ensure some variety.

After repeating test three times with each size and for both 4 and 6 disk setup, I get the following result:

4 disks6 disks
Record sizeUsedAvailableUsedAvailable
4K-061,55717,064
8K-061,17117,450
16K34,0084,19131,22347,398
32K34,0254,17331,21347,408
64K31,3006,89931,26847,353
128K31,2766,92331,18047,441
256K30,7197,48131,43247,189
512K31,0697,13031,81446,807
1024K30,9207,27931,71446,907

Two things of interest to note here. The first one is that small record size doesn’t really help at all. Quantity of metadata needed goes well over available disk space in the 4-disk case and causes extremely inefficient storage for 6 disks. Although test data set has 30.2 GB, with overhead occupancy goes into the 60+ GB territory. Quite inefficient.

The default 128K value is actually quite well selected. While my (artificial) data set has shown a bit better result with the larger record sizes, essentially everything at 64K and over doesn’t fare too badly.

PS: Excel with raw data and script example is available for download.

PPS: Yes, the script generates the same random numbers every time - this was done intentionally so that the same amount of logical space is used with every test. Do note that doesn’t translate to the same physical space usage as (mostly due to transaction group timing) a slightly different amount of metadata will be written.