Image.FromStream Doesn't Always Preserve Exif Data

Reading Exif files within C# is easy. For example, if you want to know camera model, it is enough to simply loop through image’s properties and read it:

private static string GetCameraModel(Image image) {
    foreach (var property in image.PropertyItems) {
        if (property.Id == ExifTag.CameraModel) { //0x0110
            return ASCIIEncoding.ASCII.GetString(property.Value).Trim('\0');
        }
    }
    return null; //no model found
}

However, if you want to do anything with the file you cannot simply load it with Image.FromFile as .NET keeps image open all the way until disposal. Easy enough, just load image with Image.FromStream, like this:

Image image;
using (var stream = new FileStream(this.File.FullName, FileMode.Open, FileAccess.Read)) {
    image = Image.FromStream(stream);
}
return GetCameraModel(image);

Image loaded in this manner will not contain any Exif - just the image data. The default stream loader just tossed everything that’s not picture away. If you want to preserve picture, you need to explicitly tell it:

Image image;
using (var stream = new FileStream(this.File.FullName, FileMode.Open, FileAccess.Read)) {
    image = Image.FromStream(stream, useEmbeddedColorManagement: true, validateImageData: true);
}
return GetCameraModel(image);

Now you can read all the Exif data you can handle. :)


PS: Interestingly, it seems that on Windows 7 it works either way.

Random on 8-bit PIC

When dealing with 8-bit microcontrollers one doesn’t always have all facilities taken for granted on a computer. One such example is a simple random number. While all computers can easily provide you with random number at will, unless you have a heavily specialized PIC, real random numbers are out of your reach. However, there is a way to get sufficiently close to make it work for most non-cryptographic purposes.

The answer is in linear-feedback shift registers. Assuming polynomials are carefully chosen, it is relatively easy to get non-repeating 255 byte sequences on 8-bit PIC. Yes, they are not random but for the most purposes they are random enough.

There are many types of LFSRs, Fibonacci’s being the most famous and probably the one implemented most often due to its simple hardware structure. However, for the PIC device, Galois has similar enough properties with a much simpler software structure.

General Galois function might looks like this:

#define GALOIS_POLYNOMIAL 0b11010100; //x^8+x^7+x^5+x^3+1
uint8_t randomState;

uint8_t getRandomByte() {
    unsigned bit = randomState & 0x01;
    randomState >>= 1;
    if (bit) { randomState ^= GALOIS_POLYNOMIAL; }
    return randomState;
}

This function essentially takes whatever data has in previous step and, using one of the maximum sequence length polynomials, calculates the next random byte. However, if you run it as such you might or might not get anything other than 0. Why? Well, pesky sequence has one blind spot - number 0. You can get any 255 byte sequence assuming it does not contain 0. If data becomes 0, it will stay 0.

Knowing this, one might be tempted to simply initialize data to some number and call it a day. Indeed, that would work for many applications that use 8-bit PIC. However, there are two things to have in mind: sequence will repeat every 255 bytes and random number sequence will be exactly the same on every startup.

To deal with the issue of repeating sequences, we need a counter that will simply reset the sequence after it “runs out of bytes”. As LFSR “circles” back to the original value after going thru all permutations (assuming polynomial is selected well), the easiest approach is to remember first value and reinitialize inner state once that value comes up again.

Second problem is a bit harder but not impossible. While the hardware random source would be nice, we can also do without it. My favorite approach is to have 8-bit timer running (e.g. TMR4) and then execute something that doesn’t have deterministic time - for example any user interaction (e.g. button press), writing to EEPROM (if PIC has one), or just waiting for PLL lock:

//setup unused timer that runs of the clock
T4CONbits.T4CKPS = 0; no prescale
T4CONbits.T4OUTPS = 0; //no postscale
T4CONbits.TMR4ON = 1; turn on timer 4 (or any other timer)

PLLEN = 1; //enable PLL
while (!OSCCONbits.OSTS); //wait for PLL lock

If function takes a slightly different amount of clock cycles every time, your timer will be in reasonably random state every time you check it and you can use it as a seed for generating random sequence - just don’t forget to ensure it is not zero.

Thus, our simple generator of random numbers could look something like this:

#define GALOIS_POLYNOMIAL 0b11010100; //x^8+x^7+x^5+x^3+1
uint8_t randomState = 0;
uint8_t randomStart = 0;

uint8_t getRandomByte() {
    if (randomState == randomStart) {
        randomState = (randomState ^ TMR4 ^ polynomial) | 0x80; //ensure non-zero at the cost of LSB
        randomStart = randomState;
    }

    unsigned bit = randomState & 0x01;
    randomState >>= 1;
    if (bit) { randomState ^= GALOIS_POLYNOMIAL; }
    return randomState;
}

If you run it for a while, you’ll notice something a bit disturbing. While start of the sequence is random, sequence is always the same. We need a way to make the sequence a bit more random. The cheap way to change this is to simply loop through all maximum length polynomial terms and thus extending our sequence from 255 to 4080 bytes.

Or, to break a pattern, you can even make switches in middle of sequence to make it a bit less obvious:

const uint8_t POLYNOMIALS[] = { 0x8E, 0x95, 0x96, 0xA6, 0xAF, 0xB1, 0xB2, 0xB4,
                                0xB8, 0xC3, 0xC6, 0xD4, 0xE1, 0xE7, 0xF3, 0xFA };
const uint8_t POLYNOMIALS_COUNT = 16;
uint8_t polynomial = 0xB8;

uint8_t randomState = 0;
uint8_t randomIndex = 0;

uint8_t getRandomByte() {
    if (randomIndex == 0) {
        randomIndex = (TMR4 >> 6) + 1; //generate number 1 to 4
        randomState = (randomState ^ TMR4 ^ polynomial) | 0x80; //ensure non-zero at the cost of LSB
        polynomial = POLYNOMIALS[TMR4 & 0x0F]; //select next polynomial
    }
    randomIndex--;

    unsigned bit = randomState & 0x01;
    randomState >>= 1;
    if (bit) { randomState ^= polynomial; }
    return randomState;
}

Realistically, there is only so much you can do without any random entropy while keeping function reasonably fast and I think this final function gives a reasonable result.


PS: Again, do not use this for anything security related - for that you need real random numbers, not pseudo-random sequences.

PPS: Jason Sachs has a nice article going a bit deeper in details about LFSRs.

PPPS: If you really want to expand random sequence, think about using 16-bit state and polynomial. Albeit the price of 16-bit computations on a 8-bit microprocessor is non-negligible (albeit not devastating), your use case might warrant such step.

Seattle Code Camp 2018

Illustration

Registrations for Seattle Code Camp 2018 are open. If you come on September 15th you can hear me speak about cryptography failures. All intention is for this to be a lightweight and funny talk but let’s see how funny crosses the language barrier. :)

If you attend, feel free to say hi, whether you attend my talk or not. :)

Installing Git on Embedded XigmaNAS

Having Git on NAS server comes in handy and I was quite annoyed when my workaround that worked on NAS4Free 11.1 stopped working with the advent of XigmaNAS 11.2. Well, I guess it was time to make some adjustments.

I won’t get into how the whole process works as I already explained it previously. Suffice to say it stopped working on 11.2 due to even tighter disk space restrictions it brought. While originally I just didn’t have space to install software, this time there wasn’t enough space to even get needed packages.

So I adjusted my unionfs creation process to include two extra directories (/var/db/pkg/ and /var/cache/pkg/):

rm -rf /mnt/.unionfs/* |& sed '/^$/d' |& sed -e 's/^/  /'
for LOCATION in "usr/local" "var/db/pkg" "var/cache/pkg"
do
  if mount -t unionfs | grep "/mnt/.unionfs/$LOCATION" >/dev/null
  then
    echo -en "${ESCAPE_OUTPUT}"
    mount -t unionfs |& grep "/mnt/.unionfs/$LOCATION" | sed -e 's/^/  /'
    echo -en "${ESCAPE_RESET}"
  else
    echo -en "${ESCAPE_ISSUE}"
    mkdir -p "/mnt/.unionfs/$LOCATION/" |& sed '/^$/d' |& sed -e 's/^/  /'
    mkdir -p "/$LOCATION/"
    mount_unionfs "/mnt/.unionfs/$LOCATION/" "/$LOCATION/" |& sed '/^$/d' |& sed -e 's/^/  /'
    echo -en "${ESCAPE_RESET}"

    echo -en "${ESCAPE_OUTPUT}"
    mount -t unionfs |& grep "/mnt/.unionfs/$LOCATION" | sed -e 's/^/  /'
    echo -en "${ESCAPE_RESET}"
  fi
done

Adjusting initialization script with this allowed me to install Git once again.

EEPROM Access Routines For PIC18

While I was playing with CAN bus, I needed to write something to EEPROM memory - to keep it safe between reboots. Imagine my surprise when calls to eeprom_read and eeprom_write returned errors:

warning: unsupported: The Read_b_eep routine is no longer supported. Please use the MPLAB X MCC.
warning: unsupported: The Busy_eep routine is no longer supported. Please use the MPLAB X MCC.
warning: unsupported: The Write_b_eep routine is no longer supported. Please use the MPLAB X MCC.

It seems PIC18 series doesn’t support old EEPROM routines that worked on PIC16 but, at the same time, my PIC is old enough not to be properly supported in the suggested MCC. I guess it was time to write PIC18 version of these functions myself.

Reading over documentation for PIC18F25K80, I was pleasantly surprised that EEPROM writing hasn’t really changed. My assembly code I wrote back in PIC16F84 days was quite close to what’s needed these days too. The only significant change was expanding EEADR to allow for more than 256 bytes. Yes, there are some shenanigans with setting EEPGD/CFGS bits but that’s done only once.

My replacement for eeprom_read function was similarly named eeprom18_read and it’s really a drop-in replacement. Not to be the one to dwell in past, I opted to change my ASM code to XC8 flavored C. It’s essentially just setting up EEPROM address, followed by read request, a short pause to get the data, and then reading the data itself.

unsigned char eeprom18_read(unsigned int offset) {
  EECON1bits.EEPGD = 0; // accesses data EEPROM memory
  EECON1bits.CFGS = 0;  // accesses data EEPROM memory

  EEADRH = offset >> 8;
  EEADR = offset;

  EECON1bits.RD = 1; // initiates an EEPROM read
  Nop();             // it can be read after one NOP instruction

  return EEDATA; }

Writing was a bit more involved but not too much so. After setting EEPROM address and data, it was important to allow EEPROM write and disable interrupts as ancient EEPROM unlock sequence (0x55 0xAA) must not be interrupted. After write is initiated, code will wait for it to complete and then restore interrupts. For safety it will also disable further writes to EEPROM until it’s needed again.

void eeprom18_write(unsigned int offset, unsigned char value) {
  EECON1bits.EEPGD = 0;  // accesses data EEPROM memory
  EECON1bits.CFGS = 0;   // accesses data EEPROM memory

  EEADRH = offset >> 8;
  EEADR = offset;

  EEDATA = value;

  unsigned char oldGIE = GIE;  // interrupts be disabled during this code segment
  GIE = 0;

  EECON1bits.WREN = 1;  // allows write cycles

  EECON2 = 0x55;  // write sequence unlock
  EECON2 = 0xAA;  // write sequence unlock

  EECON1bits.WR = 1;     // initiates a data EEPROM erase/write cycle
  while(EECON1bits.WR);  // waits for write cycle to complete

  EECON1bits.WREN = 0;  // disable write
  GIE = oldGIE;         // restore interrupts
}