Serial Number in Intel Hex

While doing an electronic project in MPLAB, I found it necessary to create an unique serial number for each device I program in order to have a proper USB functionality. Something as simple and often needed as this has to be already solved. Or so I thought.

Oh, I found solutions and solutions for this particular issue. Most of them were only useful with their device programmers - and I wanted something to work inside of MPLAB. Some other worked by modifying source code - a big no-no when you do source control. Some required serial number placed at an exact location - annoying (and volatile) to calculate in my case because USB serial number is a part of a bigger structure. Commercial solutions were also out of question… With each visited link I grew more firm in decision to roll out my own (“me hammer, problem nail” approach).

However, simple replacement is not necessarily an easy thing to do. You see, in world of microcontrollers Intel Hex format is the ruler. Without getting into details, format has two characteristics that make it suitable for firmwares. It splits data into fixed length blocks which is really nice when you have to do chip programming in blocks. Additionally every row is checksumed so data corruption can be avoided. Both those characteristics conspire against us to do simple file change. Not only that we cannot just simply change data because of checksum but we often cannot even simply find it because it can span across the rows.

So, it was a scripting time. Because of annoyingly difficult parsing, simple CMD was out of question. Only other choices were PowerShell or a completely custom program. While custom program would probably offer easiest development, I didn’t want to embed only executable. And embedding sources would mean that I would need Visual Studio on each machine I want to compile this at - to high cost in practicality. Only other choice really was PowerShell - environment that exists on every Windows 7 and higher machine.

Requirements were straightforward. I wanted something that would be a single call (Microchip’s MPLAB can call only single external build command). It had to replace file in-place (so that I could continue using PICkit 3 programming procedure). It had to change file as little as possible (e.g. by not removing extra headers). It had to support ASCII random (USB MSD specifications). And, lastly, it had to be capable of producing a valid output hex without checksum shenanigans.

I won’t dwelve into boring details and explain each part of code but it suffices to say that it loads whole file in memory, searches for placeholder and then replaces that placeholder with random value, adjusting the checksum as it goes. End result is the file that gets pushed onto the device.

Here is an example command (Project Properties, Conf, Building, Execute this line after build):

PowerShell.exe -ExecutionPolicy RemoteSigned -File ${ProjectDir}\..\Setup\HexReplace.ps1 **${ProjectDir}\dist\default\production\Source.production.hex** **197901281815** -**AsciiHexRandom** -Destination2 **${ProjectDir}\..\Binaries\TmpUsb.hex**

First part we might as well ignore because it is only preparing terrain for script by calling PowerShell. First meaningful argument is a location of an intermediate Intel hex file followed by a placeholder value. Since my device is USB based, I opted to have a valid serial number (197901281815) as a placeholder - makes live easier if script fails. Most common approach is to use something that won’t appear by accident (various number of X characters seem to be popular choice). Anything unique will do.

Switch AsciiHexRandom ensures that our randomness will be limited to numbers 0 to 9 and letters A to F, as required by MSD Bulk Only Transport 1.0 specification. If that parameter gets omitted random binary (0 to 255) will be used. Which one is better actually depends on your use case.

Last pièce de résistance is -Destination2 argument. That ensures script will not only change file but additionally copy it to a different place. It is not strictly necessary but I find it useful for archiving purposes. You can omit it without any consequence.

And that is my way of generating random serial number from within MPLAB project. Try it out and see whether it fits your needs.

PS: Script itself was not designed but grown. As such it is full of unoptimized code (just check function that does replacement) and it treats memory as kids treat candy. But realistically we are speaking about file that won’t be bigger than couple hundreds KB in the worse case scenario. I could not justify spending time I would need to make it “proper” when it already runs (unoptimized) within a fraction of a second.

PPS: Yes, I have cheated by embedding C# code inside of PowerShell.

PPPS: If something goes wrong, add -debug argument. That will give you a bit more information to go with.

OSH Park Just Got Bigger

Last few years it got really easy to have your PCB manufactured in small quantities. All manufacturers shared similar price but each had its own peculiarities.

I find it very sad that one such manufacturer, BatchPCB, is closing its doors. It is not that I used them often. It is not that they were anything special. It is not even that they were cheap(er). It is just that I liked the choice.

Good news is that they were taken over by OSH Park. BatchPCB users can expect lower prices on smaller than 20 cm2 boards, they can expect ENIG finish (aka “gold”), and they can expect free shipping within USA.

Dream purple.

More Ohms Please

One of first things that you might learn as electronic hobbyist is that you should have 20 mA running through your indicator LEDs. For example, if you have 5 V power supply and your LED has voltage drop of 2V at 20 mA just give it 150 ohm resistor and you are golden.

And that is wrong. This recipe was valid 20 years ago but it is not something you should follow these days. If you have 20 mA coursing through LED, that is BRIGHT as hell. When you work in dark, every look at such device blinds you for minutes.

Don’t misunderstand me, I like to see some indication that device is alive but 1 mA or less should do the trick. These new LEDs are extremely efficient with their output and, while they still specify their current at 20 mA (or even higher), they will be bright enough at lower currents also.

PS: This rant is only about indicator LEDs; if your device needs to light-up the room, so be it.

Curious UART on 16F1516

Lately I have started using PIC16F1516 as my jelly bean PIC. It is cheap, has lot of memory (whooping 512 bytes) and it is fairly well feature-equipped. Best of all, it is almost completely compatible with my old favorite PIC16F1936.

Yes, PIC16F1936 has EEPROM built-in and both feature list and speed are superior. However, not all projects need all features and speed nor is lack of EEPROM something catastrophic.

As always I started programming by copy/pasting old code. Small adjustments were all that was needed. I copy pasted my UART routines and created simple echoing program on chip. And chip stayed quiet.

After some debugging it was obvious that both my timing routines and UART formulas are correct. And code was same as what I used on PIC16F1936:

void uart_init(int desiredBaudRate) {
    TRISC7 = 1; //RX
    TRISC6 = 0; //TX
    SPBRG  = (unsigned char)(_XTAL_FREQ / desiredBaudRate / 16 - 1);
    BRG16  = 0;    //8-bit
    BRGH   = 1;    //high speed
    SYNC   = 0;    //asynchronous mode
    SPEN   = 1;    //serial port enabled
    TXEN   = 1;
    CREN   = 1;
    asm("CLRWDT");
}

In moment of desperation I turned my attention to datasheet. And there, under Asynchronous Transmission Set-up chapter it said “Set the RXx/DTx and TXx/CKx TRIS controls to ‘1’.”

In all other PICs (I had opportunity to play with) TRIS for TX pin either does not matter or you set it to output. For some reason designers of this PIC decided that TX pin needs to be input in order for USART to work. Quite a trap.

Only change that was required was setting TRISC6 to 1 and my board became alive:

void uart_init(int desiredBaudRate) {
    TRISC7 = 1; //RX
    TRISC6 = 1; //TX
    SPBRG  = (unsigned char)(_XTAL_FREQ / desiredBaudRate / 16 - 1);
    BRG16  = 0;    //8-bit
    BRGH   = 1;    //high speed
    SYNC   = 0;    //asynchronous mode
    SPEN   = 1;    //serial port enabled
    TXEN   = 1;
    CREN   = 1;
    asm("CLRWDT");
}

PS: Do not forget to set ANSELC = 0.

How to Store Settings Without EEPROM

Having customizable setting in your PIC program is easy. Just define it as a constant and let XC8 handle everything else. If you need to change it during runtime there is bunch of code out there dealing with EEPROM access. But what if you want to store something in PIC without EEPROM?

In my case I needed to store two settings: Counter and Unit. Hardware device was already built around PIC without EEPROM so there was no choice but to use self-programming feature that almost all newer PICs share.

Idea is simple. Reserve some space in program memory by making some data constant (e.g. const char mySetting = 3;). Every time you access this variable PIC will return what it had stored there at time of programming. Our trick is to change those values by reprogramming code as it runs.

First complication comes from fact that you cannot just write in flash willy-nilly. Before each write you need to erase whole block of (usually) 64 bytes. Unless you are willing to write very complicated code that also means that you will always need to occupy whole block regardless of how many bytes you want to actually store.

Second caveat is that you can only erase whole block. That means that you cannot let compiler decide upon location of your data. You must select fixed location and that might give you :0: warning: segment "__SETTINGS_PROGRAM_text" (200-23F) overlaps segment "intcode" (8-273). Fortunately solution is really easy - just move your constant somewhere else. Of course that implies that you have enough free flash memory laying around.

Third issue might be that this operation typically takes few milliseconds which is ages in microcontroller terms. To make things worse interrupts must be disabled during almost all that time. Any precise time keeping is out of question.

However, if you wiggle your way around all these obstacles, you get nice storage for times when you just cannot afford EEPROM for one reason or another.

Full source follows:

#define _SETTINGS_FLASH_RAW { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 } //reserving space because erase block is block 32-word (64-bytes)
#define _SETTINGS_FLASH_LOCATION 0x0400

#define _SETTINGS_STORE_EMPTY { 0, 0 }
#define _SETTINGS_STORE_BYTES 2

#define _SETTINGS_REENABLE_GIE_AFTER_WRITE     1

const unsigned char _SETTINGS_PROGRAM[] @ _SETTINGS_FLASH_LOCATION = _SETTINGS_FLASH_RAW;

void _settings_write(int index, unsigned char data) {
    unsigned char newProgram[] = _SETTINGS_STORE_EMPTY;
    for (int i=0; i<_SETTINGS_STORE_BYTES; i++) {
        newProgram[i] = _SETTINGS_PROGRAM[i];
    }
    newProgram[index] = data;

    GIE = 0;

    EEPGD = 1; //point to Flash program memory
    CFGS = 0; //access Flash program memory
    WREN = 1; //enable write to memory

    TBLPTR = _SETTINGS_FLASH_LOCATION;
    FREE = 1; //enable block Erase operation
    #asm //erase block
        MOVLW 55h
        MOVWF EECON2 ; write 55h
        MOVLW 0AAh
        MOVWF EECON2 ; write 0AAh
    #endasm
    WR = 1; //start erase (CPU stall)

    TBLPTR = _SETTINGS_FLASH_LOCATION;
    for (int i=0; i<_SETTINGS_STORE_BYTES; i++) {
        TABLAT = newProgram[i];
        asm("TBLWT*+");
        #asm
            MOVLW 55h
            MOVWF EECON2 ; write 55h
            MOVLW 0AAh
            MOVWF EECON2 ; write 0AAh
        #endasm
        WR = 1; //start program (CPU stall)
    }
    WREN = 0; //disable write to memory

    if (_SETTINGS_REENABLE_GIE_AFTER_WRITE) { GIE = 1; }
}


#define _SETTINGS_INDEX_COUNTER  0

char  settings_getCounter() {
    return _SETTINGS_PROGRAM[_SETTINGS_INDEX_COUNTER];
}

void settings_setCounter(char data) {
    _settings_write(_SETTINGS_INDEX_COUNTER, data);
}


#define _SETTINGS_INDEX_UNIT 1

char  settings_getUnit() {
    return _SETTINGS_PROGRAM[_SETTINGS_INDEX_UNIT] % 3;
}

void settings_setUnit(char data) {
    _settings_write(_SETTINGS_INDEX_UNIT, data);
}

It would be best if you would keep this code in separate file (e.g. settings.h) and just include it from your main program file instead of pasting all this directly.

PS: If your program does not use interrupts change _SETTINGS_REENABLE_GIE_AFTER_WRITE to 0.

PPS: Do notice that flash memory was not intended as this kind of storage medium. Writing is much slower than EEPROM (because of erase) and endurance is much lower. Use it only in dire straits and be gentle.

PPS: This code is for PIC18F* series of micro-controllers. Same general principle works on PIC16F* but names of some registers might differ and additional NOPs might be required. Do check instruction manual.

XC8 and Fixup Overflow Referencing Psect BssBANK1

For one PIC microcontroller program I had to use assembly code for some critical timing. Rest of code was just plain XC8 C code and everything compiled (and performed) without any issue. So I decided to share code with another project of mine.

Suddenly I got bunch of warnings all saying almost same thing: wait.h:19: error: fixup overflow referencing psect bssBANK1 (0xA2) into 1 byte at 0xB3E/0x2 -> 0x59F (dist/default/production\Source.production.obj 136/0x3E). Program did work (fixup did what its name says) but each compilation would pepper output with warnings - unacceptable.

It was easy to find a culprit:

unsigned char wait_asm_W; //copy of W
...
#asm
    BANKSEL _wait_asm_W
    MOVWF   _wait_asm_W
    ...
#end asm

Address of an variable was used inside assembly code. Original program had this variable in bank0 and thus no warning were issues. This project had slightly different allocations and variable ended up in bank1 (e.g. 0xA2). Since assembly instruction can only deal with 7-bit registers, address was outside of allowed range.

Once solution is forcing variable into bank0:

bank0 unsigned char wait_asm_W; //copy of W
...
#asm
    BANKSEL _wait_asm_W
    MOVWF   _wait_asm_W
    ...
#end asm

This works but only after change to project’s compiler flags (Properties, XC8 global options, XC8 compiler, Option categories: Optimizations, Address qualifiers: Require). Just thought of having to remember to change project properties each time I reuse this code made me wince. Yes, it was a solution but not what I wanted.

What I wanted is way to transform offending value 0xA2 to non-problematic 0x22. Quick bit-masking solved that issue and warnings were gone:

unsigned char wait_asm_W; //copy of W
...
#asm
    BANKSEL _wait_asm_W
    MOVWF   _wait_asm_W & 0x7F
    ...
#end asm

Omega Is Not Ohm?

Whenever I had to enter resistance I would use Greek capital letter omega (Ω, U+03A9). It is how I was thought since high school and I never questioned it. That is until I accidentally saw that there is an actual ohm sign (Ω, U+2126). All these years I was doing it wrong. Or did I?

Unicode is known for lot of things but logic is not one of them. Although special ohm sign is available and it is clearly named as such, its use is not encouraged. To quote:

The ohm sign is canonically equivalent to the capital omega, and normalization would remove any distinction. Its use is therefore discouraged in favor of capital omega.

In other words: We created special character but we changed our minds since. Please do not use it.

Well, aside for awareness of this Unicode curiosity, I gained absolutely nothing. Omega I used before and omega I shall continue to use.

Long live 42 Ω.

Elusive CONFIG3H Setting

For toy project of mine I wanted to test PIC18F13K50. Nice fast PIC, lot of RAM, some ROM and USB connectivity. All that while working on good old 5V.

First order of business was to set configuration bits. Code should be familiar to anyone who used XC8:

#pragma config HFOFST = OFF
#pragma config MCLRE  = OFF

Unfortunately that did not work. I could set all bits other than those in CONFIG3H. And thus I could not make MCLR pin behave as normal input. Since PCB was already done, I could not just opt to use other pin. I had to have it my way.

I also tried to use:

__CONFIG(_CONFIG3H, HFOFST_OFF & MCLRE_OFF);

Hell, I even tried

 __PROG_CONFIG(3, 0x00);
```]

Nothing worked.

And than I noticed a curious thing. This PIC has CONFIG2H register followed by CONFIG3H. There is no CONFIG3L. Could that be an issue?

On a hunch I went to edit `18f13k50.cfgdata` (in `C:\Program Files\Microchip\xc8\v1.01\dat\`). There I added single line (highlighted) just before CONFIG3H declaration:

```plain
CWORD:300004:00:00:CONFIG3L
CWORD:300005:88:88:CONFIG3H
CSETTING:8:HFOFST:HFINTOSC Fast Start-up bit
CVALUE:8:ON:HFINTOSC starts clocking the CPU without waiting for the oscillator to stablize.
CVALUE:0:OFF:The system clock is held off until the HFINTOSC is stable.
CSETTING:80:MCLRE:MCLR Pin Enable bit
CVALUE:80:ON:MCLR pin enabled; RA3 input pin disabled
CVALUE:0:OFF:RA3 input pin enabled; MCLR disabled

With that simple change all was well and configuration bits were set.

And it works!

P.S. This workaround is valid for all other 18F family PICs that have CONFIG3L missing (e.g. PIC18F1220, PIC18F14K50…).

P.P.S. This should be solved in XC 1.10. Let’s hope. :)

[2012-08-11: As of XC8 1.10 this issue is fixed.]

Selecting a LDO Voltage Regulator

I am in process of creating new USB board for my personal nefarious purposes and I needed nice 3.3V voltage regulator. Usually I just use LD1117DT33TR. It has decent maximum current, works up to 15V and it comes in nice DPAK package that helps to keep it cool. However, for this occasion, it was just too big. I wanted something smaller.

While simplest thing would be to pick one that would suit me in this project. However, I always like to spend a bit more time in order to get a chip that I can reuse in bunch of other projects. Here were things that I desired:

• 3.3 V
• at least 50 mA
• SMD
• LDO
• easily available in Croatia
• very simple
• preferably cheap

First I searched on RS Croatia site for all LDO 3.3V regulators in SOT-23 package. Sorting them by price gave me few candidates. They were AP7313, TS9011SCX, TC1014, SP6201EM5 and MCP1700. All others that were cheap enough were variations of same basic devices. Quick search on DigiKey didn’t find TS9011SCX so it was automatically disqualified. High cost removed SP6201EM5. If I am searching for jelly bean component, I want to have one that is always available and very cheap.

Another odd-man out was TC1014. I haven’t had anything against it’s specifications. They were good enough and in some parts better than chips that stayed in competition. However, it had five leads. I wanted my final choice to be as simple as possible. That left two SOT-23-3 devices in race: AP7313 and MCP1700.

Both these devices satisfy all my needs. And then some. Both have short-circuit protection, thermal protection and both are stable with 1 μF ceramic (nice!) output capacitor. For most parameters where MCP1700 was slightly better they were close enough not to really matter. Great example would be quiescent current. While on first glance there is huge difference in favor of MCP1700 (40x) that only matters in case of extremely optimized battery-powered device. Since almost all devices I do revolve around computer, it is just parameter that I do not care about. For all jelly bean 5 to 3.3 V LDO purposes they are equivalent.

After spending some time going back and forth I decided to use MCP1700T-3302E/TT. Reason was stability. While both regulators work with ceramic capacitors, AP7313 requires output capacitor ESR to be 20 to 200 mΩ. This does cover wide range of ceramics, but for MCP1700 is even better. It’s output capacitor can have ESR anywhere from 0 to 2000 mΩ. That range covers pretty much all capacitor types that I would find on my shelf.

If something comes up where AP7313 would be great fit (e.g. no minimum load), I am fine with that also. You see, AP7313 comes in two different footprints. In case of AP7313-33SRG-7 footprint is same as one that MCP1700 uses. You can mix and match them based on cost and availability as much as your heart desires (just watch for those capacitors!). And that gives real lesson of this post: whenever you can try not to limit yourself to single supplier.

Smoothing

If you are making some measuring device with display it is always a challenge to select proper rate of refresh. Usually your measurement takes only small amount of time and it is very hard to resist updating display after each one. I saw number of devices that have displays that are just too fast to read.

Slowing rate at which measurement is taken is almost always beneficial for both user comfort and battery life. And that is valid solution, especially if value is relatively stable. However, if measurement fluctuates a bit, that results in jumps between values.

To cure that you should be doing averaging. If your measurement takes 10ms to complete, you can do 10 of them, average the result and still have quite a decent 10/second refresh rate. This is probably solution gets most use but it is not the only one.

My favorite way of slowing display is simplified weighted average. Between two measurements one that is current always carries more weight than newer one. Exact weight is matter of trial but I found small numbers like 23% work the best.

To clarify it a bit, let’s say that we have measurement of 10 and measurement of 20. Our new “average” value will become 12.3. If third measurement is also 20, value becomes 14.1, then 15.4 and so on. Value keeps getting closer and closer to real reading but speed with which it does that is very limited.

If you have measurements that are relatively stable this method works almost like an average. If value jumps occasionally this method will smooth such temporary change. That gives much nicer end user feeling as far as measurement goes. And since we are doing this at much faster rate than actually showing data, if permanent jump does occur, user will see such change relatively quickly.

Code for this might look like:

float value = measure();
while (1) {
    showValue(value);
    for (int i=0; i<10; i++) {
        float newValue = measure();
        value = value + (newValue - value) * 0.23; //to smooth it a little
        value = (int)(value * 1000.0) / 1000.0; //rounding
        //do other stuff
    }
}

In this particular code, we show a value to user as soon as we can (to enhance perceived speed). After that we average next 10 values (each new one is given 23% of consideration). Then we display new average to user. Rinse and repeat. Optional rounding step additionally limits small changes.

This code is not that good if measurement takes a long time. E.g. If you have one measurement per second you will find it takes eternity to change a value. For such cases you are probably better off just displaying current measurement to user. Or, if some smoothing is required, using higher values (e.g. 0.79 or similar).

P.S. This method might not work as expected for your measurements. Do test first.

P.P.S. This is intended for human display only. If you are logging values, it is probably best to write measurements without any adjustment. If you average them before writing, you are losing details.

P.P.P.S. If you are doing averaging in full-blown desktop application, ignore this code. You can use proper moving average (linear, exponential, weighted…) that allows for much greater control. This method is just a workaround to get similar results when working on memory-limited PIC.