Automatic Mangling

Mikrotik routers give you a lot of control over traffic. If you stick with IPv4, it’s possible to nicely subdivide your network and use queues to traffic-shape any user’s bandwidth based on IP address. But things are not as easy when you allow IPv6. Suddenly your simple queues are essentially broken and tracking users becomes quite a lot more difficult.

For exactly that purpose, you have firewall mangle rules. In its simplest form, you just use MAC address as a condition to mark your connection and then you use that connection mark to mark each packet. In Mikrotik’s lingo, that would be something like this:

/ip firewall mangle add
    chain=forward
    src-mac-address=^^12:34:56:78:90:12^^
    connection-state=new
    action=mark-connection
    new-connection-mark=^^some-mark^^

/ip firewall mangle add
    chain=forward
    connection-mark=^^some-mark^^
    action=mark-packet
    new-packet-mark=^^some-mark^^
    passthrough=no

Now you can use this packet mark in a simple queue as a condition instead of the subnet or IP. If you go further and create these mange rules in /ipv6 section as you did for /ip, you have yourself an easy way to track traffic even on per-MAC resolution.

The real challenge then becomes entering all these rules into the router. In order to fill this, it’s best to have a script as executing all these commands manually is really error prone. I like to keep this in a file looking something like this:

12:34:56:78:90:ab user1-mark
23:45:67:89:0a:bc user1-mark
34:56:78:90:ab:cd user2-mark

Using this I don’t necessarily track or limit traffic per device but per user. As long as all user’s MAC addresses result in the same mark, queue doesn’t really care.

I mark all automatic entries with !AUTO! in the comment field. Any time I want to update firewall rules, I make sure to delete the old ones first:

OLD_V4_ENTRIES=`ssh router.home "/ip firewall mangle print without-paging" | grep ";;; !AUTO!" | awk '{print $1}' | xargs | tr ' ' ','`
if [[ "$OLD_V4_ENTRIES" != "" ]]; then
    ssh router.home "/ip firewall mangle remove numbers=$OLD_V4_ENTRIES"
fi

OLD_V6_ENTRIES=`ssh router.home "/ipv6 firewall mangle print without-paging" | grep ";;; !AUTO!" | awk '{print $1}' | xargs | tr ' ' ','`
if [[ "$OLD_V6_ENTRIES" != "" ]]; then
    ssh router.home "/ipv6 firewall mangle remove numbers=$OLD_V6_ENTRIES"
fi

With old entries gone, we can add new entries using something like this:

cat ^^~/MAC.dat^^ | grep -v '^#' | while read ENTRY; do
    MAC=`echo "$ENTRY" | xargs | cut -sd' ' -f1`
    MARK=`echo "$ENTRY" | xargs | cut -sd' ' -f2`
    if [[ "$MAC" != "" ]] && [[ "$MARK" != "" ]]; then
        echo "$MAC $MARK"
        ssh -n router.home "/ip firewall mangle add chain=forward src-mac-address=$MAC connection-state=new action=mark-connection new-connection-mark=$MARK comment=\"!AUTO!\""
        ssh -n router.home "/ip firewall mangle add chain=forward connection-mark=$MARK action=mark-packet new-packet-mark=$MARK passthrough=no comment=\"!AUTO!\""
        ssh -n router.home "/ipv6 firewall mangle add chain=forward src-mac-address=$MAC connection-state=new action=mark-connection new-connection-mark=$MARK comment=\"!AUTO!\""
        ssh -n router.home "/ipv6 firewall mangle add chain=forward connection-mark=$MARK action=mark-packet new-packet-mark=$MARK passthrough=no comment=\"!AUTO!\""
    fi
done

Script above will go over the whole file, skipping all the comments, and assuming the first field in line is MAC address and the second is the mark you want to assign it. Then it will just add mangle pair to both IPv4 and the IPv6 firewall rules.

The last step is to force firewall to clear the existing connections so it can mark them with the new marks.

ssh -n router.home "/ip firewall connections remove [find]"
ssh -n router.home "/ipv6 firewall connections remove [find]"

Only thing now is to create simple queue based on each mark. And yes, that can be automated too. :)

Yield, Don't Stop

One of the first bad habits you pick up when riding a bicycle is not observing stop signs. Yes, you will (hopefully) check there is no traffic but I rarely see anybody come to the full stop if nobody is around. And I was regularly guilty of the same.

If you live in Washington state this is no longer infraction. As of October 1st, Safety Stop law turns all stop signs into yields if you’re on a bicycle.

While I am doubtful about it increasing safety, I am sure it won’t decrease it and it will make bike ride much smoother - even when cops are around. It’s about the time. :)

Stripping Diacritics in Qt

As someone dealing with languages different than English, I quite often need to deal with diacritics. You know, characters such as č, ć, đ, š, ž, and similar. Even in English texts I can sometime see them, e.g. voilà. And yes, quite often you can omit them and still have understandable text. But often you simply cannot because it changes meaning of the word.

One place where this often bites me is search. It’s really practical for search to be accent insensitive since that allows me to use English keyboard even though I am searching for content in another language. Search that would ignore diacritics would be awesome.

And over the time I implemented something like that in all my apps. As I am building a new QText, it came time to implement it in C++ with a Qt flavor. And, unlike C#, C++ was not really built with a lot of internationalization in mind.

Solution comes from non-other than now late Michael S. Kaplan. While his blog was deleted by Microsoft (great loss!), there are archives of his work still around - courtesy of people who loved his work. His solution was in C# (that’s how I actually rembered it - I already needed that once) and it was beautifully simple. Decompose unicode string, remove non-spacing mark characters, and finally combine what’s left back to a unicode string.

In Qt’s C++, that would be something like this:

QString stripDiacritics(QString text) {
    QString formD = text.normalized(QString::NormalizationForm_D);

    QString filtered;
    for (int i = 0; i < formD.length(); i++) {
        if (formD.at(i).category() != QChar::Mark_NonSpacing) {
            filtered.append(formD.at(i));
        }
    }

    return filtered.normalized(QString::NormalizationForm_C);
}

Poured Potatoes (Užljevak)

Užljevak is one of my favorite childhood foods. Due to it’s simplicity and the low cost, this was on our table at least once or twice a week. It’s a peasant food using only pantry ingredients and it’s quite forgiving when it comes to preparation.

This recipe is one of the simplest užljevak recipes I know but it’s definitely not the only one. My mother alone used a few different recipes and you’ll find probably every Bosnian family has another few. They all will taste similar though so trying one will give you an idea what you’re working with.

Ingredients

Quantities given are just for the orientation purpose and can be fudged quite a lot while still getting an excellent result.

• 600 g Russet potatoes (about 3 big ones)
• 300 g Flour (about 14 spoons)
• 1 Yellow onion
• 3 dcl Water
• 2 dcl Milk
• 1 dcl Oil
• 25 g Salt (1½ tablespoon)
• 1.5 g Black pepper (⅔ teaspoon)

Instructions

Peel potatoes and cut them into small square pieces (0.5 cm side). Leave them in a cold water for about 10 minutes and then drain the water to remove (some) starch. While potatoes are soaking, finely dice onion. The finer, the better.

Mix all the ingredients (including oil) together and you should end with a runny mixture similar to what you would use for crepes (a slightly less dense than a pancake mix). You can fine tune it by adding flour or water but the end result will be good as long as you are in a ballpark. Leave it alone for 10 minutes.

Use that time to preheat the oven to 220 °C (425 °F) with a well-oiled wide pan inside. You can also go with parchment paper but in that case preheat only pan and place parchment only when you are ready to pour.

Once oven is at temperature get the pan out (place parchment paper if that’s your groove) and pour the mixture into the hot pan. It should be about 2-3 cm in height. Be very careful as pan will be really hot. Once poured, get it immediately into the oven.

Bake for 1 hour and then turn on a broiler for another 5 minutes. Once browned, it’s ready to go out.

Let it rest for 10 minutes, cut it into square pieces, and enjoy it with some yogurt on side.

PS: Some would say the only way to eat užljevak iz with homemade sour milk. Getting yogurt is not quite the same taste but it’s definitely easier.

Renaming Master Branch to Main

GitHub is making a major change to the default branch name. As of October 1st, the default branch will be called main instead of master. While this is done just for the new repositories and the official recommendation is to wait until the end of year for the existing ones, I was never the one to follow the rules.

To locally change the name of the branch, you just need to move it.

git branch -m master main

Next step is telling GitHub you have a new branch:

git push -u origin main

If you go to GitHub now, youl’ll see both main and master present with master still being the default. To change this you’ll need to go into the repository settings and switch default branch there.

Only once that step is done, you can delete master branch forever.

git push origin --delete master

Now the existing repository now has main as the default branch name.

As you can see, currently this process is a bit involved and I am sure that GitHub will automate it reasonably soon. You might want to wait with local renames until they do. My plan is to update branch names as I push updates to my projects. Active repositories will get the update sooner while some old repositories might stay with master forever.

That’s all fine and dandy but what about the new repositories? Well, there’s a setting for that too. Just adjust init.defaultBranch Git property.

git config --global init.defaultBranch main

And now you’re ready to roll.

PS: I will not get into politics whether this change was necessary or not. As a Croat, I will never fully understand the slavery and the emotional impact having the master branch might have. For me this change is more of pragmatism. The exact name doesn’t matter much to me and main is a better choice anyhow.