My Resolve Dashcam Workflow

As I moved to Resolve I was forced to change my Vegas Movie Studio dashcam processing workflow a bit. Not only you cannot use MP4 under Linux at all, but MP4 presents challenges to the free Resolve under Windows too.

The first step I take for all dashcam footage is to convert it using ffmpeg to DNxHR LB. Not only it’s a well-supported intermediary codec that increases performance significantly, but it also get’s rid of any nonsense my dashcam puts in the clip. And 36 Mbps is more than enough for anything my dashcam can throw at it. Instead of converting clip-by-clip, I opted to merge them all into a single file - that’s the reason behind weird syntax:

ls *.MP4 | awk '{print "file \x27" $1 "\x27"}' | ffmpeg \
    -f concat -safe 0 -protocol_whitelist pipe,file -i - \
    -c:v dnxhd -profile:v dnxhr_lb -q:v 1 -pix_fmt yuv422p -an \
    ^^dashcam.mov^^

Once all these videos are imported into Resolve, I go over them removing any clip portions when car is not moving. For any stops where state around car changes (e.g. waiting for traffic light), I use smooth cut to transition from one state to another. Other than that, I leave footage as is.

Once I’m done with editing I export the whole video into DNxHR SQ VBR. If I hadn’t done any editing, exporting to DNxHR LB would be fine as generational loss is quite acceptable. However, with all smooth cuts I’ve made, a temporary bump in video quality is beneficial. Especially since this is not the final output.

As I don’t expect to edit these clips again, the final output is H.264 as it’s size savings cannot be ignored. I usually use two-pass encoding with 6 Mbps average rate. You can use veryslow preset to increase quality at the cost of speed but improvement is minimal so I simply go with the default of medium:

ffmpeg -i ^^render.mov^^ \
   -c:v libx264 -pix_fmt yuv420p -b:v 6M \
   -an -y -pass 1 -f mp4 ^^render^^.mp4

ffmpeg -i ^^render.mov^^ \
   -c:v libx264 -pix_fmt yuv420p -b:v 6M \
   -an -y -pass 2 -f mp4 ^^render^^.mp4

rm ffmpeg2pass-0.log*

And that’s it - final video is similar enough in quality while not taking extreme amounts of disk space.

PS: I am not using H.265 at this time because I find it even more trouble to work with than H.264 is. I might think about it in the future as support for it increases.

Black Frames in Garmin Dashcam Video

Editing video files produced by my Garmin dashcam is sometimes really annoying. Not only it splits every darn trip into 60 second chunks forcing you to edit bunch of small files, but it will also occasionally produce video with the first frame completely black. At least that’s how it looks in Vegas Movie Studio 15 and, while annoying, was easy enough to remove. In DaVinci Resolve these videos would have a few seconds worth of corrupted data and that’s a bigger problem.

As I’m moving most of my video editing to Resolve due to it’s cross-platform capabilities, I decided to figure it out. One way to see what’s going on with MP4 is by looking at video file in MediaInfo.

The first confusing thing was that MP4 contained only streams with ID 2 and 3. What happened to stream with ID 1 is anybody’s guess. The second source of confusion was that all streams combined amounted to smidgen over 60 MB. The whole video file was more than 70 MB. While one can expect MP4 container format to take some space, overhead is generally measured in KB - not MB.

However, both these things were present in both valid and invalid video file. It took going into Advanced mode to reveal more curiosities. At last it let me know where remaining 10 MB were - in the header. And more interestingly it has shown multiple stream size calculations for video stream. File that contained black frame had one of it’s six video stream sizes listed as 59.97 MB while fully working file had all video stream sizes set to 60.00 MB.

Either due to crappy encoder or bad coding, Garmin not only bloats header to unreasonable level but it can also miscalculate stream stream sizes. Because MP4 contains stream size data at multiple places, it was dependent on decoder whether error would be noticeable or not.

Knowing I am dealing with the corrupt container and seemingly correct stream (albeit one frame shorter), I decided to simply repackage MP4 without recompression using ffmpeg:

ffmpeg -i ^^input.mp4^^ -c:v copy -c:a copy ^^fixed.mp4^^

This copies both video and audio stream (irrelevant if there is no audio stream) into a new file. For normal videos this results in a direct stream copy. Videos where one frame was corrupted end up with 59.967 seconds worth of frames. Essentially the broken frame will be removed. And this repackaging solved the black frame issue for both Vegas Movie Studio and DaVinci Resolve.

Unfortunately, while DaVinci Resolve did recognize files now, exported result had a stutter. For some reason all these files were recognized as 15 fps. And no, this wasn’t due to stream copy as original videos were misidentified too. It took me a while to give up and ask the question about it on Blackmagic forum only to find out I stumbled upon a bug.

As a workaround before bug is resolved, I went onto converting the stream to DNxHD LB codec:

ffmpeg -i ^^input.mp4^^ -c:v dnxhd -profile:v dnxhr_lb -pix_fmt yuv420p -c:a copy ^^fixed.mov^^

Not only this also removes invalid frames but it also helps editing speed as DNxHD is much more CPU-friendly format.

Being too lazy to deal with these files on case-by-case basis, a bit of Bash magic to repackage multiple files comes in handy:

mkdir out
find . -type f -name '*.MP4' -exec \
    ffmpeg -i {} -c:v dnxhd -profile:v dnxhr_lb -q:v 1 -pix_fmt yuv422p -c:a copy \
    out/{}.mov \;

Lamy 2000 + Signo 207

I love Uni-ball’s Signo 207 pens. I find ink really nice, reasonably waterproof, and beyond everything else, beautifully looking. If I am not using fountain pen you’ll find Signo 207 in my hand. Due to fountain pens being a bit finicky when transporting them around in backpack I would say that Signo became my de-facto work pen.

However, there is one thing I do not like - pen body. While pen design is definitely not ugly, it’s not what I would call gorgeous either. It’s semi-transparent barrel with nothing really to show inside and “chrome” details just draw eyes to seams that can be seen in clip tag. It’s 307 successor comes in a bit nicer understated body but it only comes in three colors. And, although nicer, I wouldn’t call 307 body anything special except refinement of decent looking pen.

But do you know which design I like? Lamy 2000. It’s really stealthy pen that looks good even on a second look. It has really light Makrolon (read: fancy fiberglass) body with a snap-cap hiding nice brushed steel top. While not perfect (damn snap-cap dimples) I find both overall shape and form make it a pleasure to write with. But alas, it uses Lamy’s proprietary M63 refills. While they are not bad, they’re not Signo - especially when it comes to color selection. Is there any way to combine the two?

I started with Lamy 2000 rollerball (not ballpoint - that one takes different refill) and took a look at how refill fits. Lamy refill ended being a few millimeters longer flaring into a slightly different tip shape and with a slightly different tip diameter. If you talk just about numbers those refills look really different but looking at them in person there is simply something telling you they are “close enough”.

First thing you’ll notice if you put UMR-87 (aka Signo 207) refill into Lamy 2000 rollerball is that it’s definitely too short and you cannot even get it to show it’s head. Easy solution for this is to simply fill the cap. I found that M3 screw with a nut screwed all the way to the top works wonderfully. Not only it takes enough space (3.25 mm effective length) to actually push refill far enough to poke its head but it also fits into both cap and refill without permanent damage so you can always switch back if needed.

But that doesn’t make refill feel good as it rattles while you try to write. Since the whole front part of refill’s body is different enough, it needs something to provide enough purchase M63 gets with its shape. For that I found spring from Signo 207 does wonders. With that spring in the top there is enough force pushing Signo refill back and doing really good way of centering its tip that it actually feels and looks like the original refill. Even tip diameters being 0.2 mm narrower makes no difference.

That said, if you have a heavy hand, you might feel a bit side-to-side movement as tip does have a bit more play due to smaller diameter. If that happens, a bit of transparent tape should make it good enough. However, I didn’t have to go that far.

In any case, I can now enjoy comfort and looks of Lamy 2000 with my favorite gel refill.

PS: The same should work with Pilot G2 or really almost any euro-style refill.

Roll Over Baby

Some people were scared of year 2000 bugs. Some people dread NTP rollover in 2036. Some are scared of Unix epoch rollover in 2038. And some are swearing at their GPS because of 2019 rollover.

Realistically, if your GPS was manufactured in this century, chances are that you won’t even notice anything happened. No user action is needed nor expected. If you have old GPS that hasn’t received update in a while - congratulations - you have yourself a brick.

You see, due to a quirk of design, original GPS message format has a 10-bit week number. That gives you 1024 weeks or about 20 years before it needs to roll back to 0. As GPS needs correct time not only for display but also for determining actual position, if clock is off by 20 years or so, navigation is going to be almost as bad as using Apple Maps.

On a bright side, every GPS I ever owned (going back to original Garmin eTrex) is actually quite capable of dealing with this rollover without breaking a sweat - even with their original firmware. This is not a first rodeo for any decent manufacturer and thus, baring some unexpected bugs, a normal user can be oblivious to this problem.

If you need to deal with old industrial equipment that was installed under Roman emperor and last had its firmware upgraded in middle ages, you might want to heed warnings and check anything that’s really critical. But my guess all such users were aware of this for a long time now.

PS: Since newer GPS devices already have support for 13-bit week format giving you about 150 years between rollovers, I doubt anybody will be worried for next 10-bit rollover in 2038. Anyhow, for that year, we have a bigger fish to fry.

PS: Actually, on most of GPS devices only consequence of non-update firmware is just a wrong date.

Removing Recording Time Limit on LX100

One of the rare things annoying me on my Panasonic LX100 is recording time limit. No matter how big SD you have, there is only 30 minutes you can record in one go. Yes, this is more than enough 99% of the time, but there are scenarios this disables, for example recording my occasional talk.

Well, reading through some DP Review forum threads I noticed discussion about service mode for the purpose of PAL/NTSC mode switcheroo and I wondered whether service mode offers anything else.

Well, to try it out, I followed instructions from the thread. First I kept Right, Display, and AE/AF Lock buttons while turning on the camera. Once camera has been turned on, I released all those buttons and swiched into Playback. Then I held Up and AE/AF Lock buttons while turning off the camera. If done correctly (and it took me a few tries), camera will flash yellow rectangle at you. Welcome to the service mode!

While in service mode, I suddenly got some limits increased:

As you can see, camera can now record MP4 in 4K limited only by card size and all AVCHD modes got a bump in recording length. Strangely, the same improvement is not visible in other MP4 modes as they all either retained their 30 minute or lower limits.

In case you find exclamation mark annoying, you can always exit service mode by setting both shutter speed and aperture to A. Then press Menu and Right key together and turn off the camera. That will present you with a strange menu where pressing Right, Right will end the service mode. Unfortunately that comes at the cost of your settings as that’ll do factory reset.

Me? I’m gonna keep my service mode on.

WordPress Not Showing Dialogs

My WordPress installation is generally troubleless. Yes, there was an issue with upgrade or occasional missed post, but nothing that would annoy me too much. Except…

For quite a while I’ve been having trouble with the plugin detail page. Whenever I tried to open one, it would greet me with Refused to connect or something similar (depending on browser and its version). Mind you, everything would work but some annoyance would always linger.

Until one day I accidentally opened it in a separate tab. Wouldn’t you know it, the damn thing worked. That lead me to check console tools where I saw WordPress attempting to use iframe. Quick check of my web server configuration told me I had HTTP header X-Frame-Options set to "deny". To make it simple, I was blocking myself. Change to a slightly more permissive X-Frame-Options="sameorigin" solved it.

Minor issue that I have been ignoring for better part of the year solved by a pure accident. Doesn’t get better than that. :)

LineageOS in the Case of Vendor Image Mismatch

I few days ago, after I updated my Nexus 5x with the latest LineageOS, I was faced with the following message:

A vendor image mismatch has been detected. Typically this means image is out of date. Please ensure your vendor image matches OPM7.181205.00

What this rather scary message essentially tells you is that LineageOS update was built on newer version of your phone’s image than the one you have installed. For me that meant a visit to Google’s firmware page and download of factory image for Nexus 5x.

Assuming one wants to keep LineageOS and not downgrade to the official firmware, we have to go into downloaded zip file and find another zip file within. It’s in that inner file one can find vendor.img.

Place vendor.img in the same folder you already have platform tools in (I just assumed you have one) and get the phone into fastboot. I personally love Advanced restart functionality withing Developer menu and I simply reboot phone by pressing power key a second or two and selecting Bootloader. However, one can also do it from platform tools command prompt:

adb devices
 List of devices attached
 00b94424d9a02666        device

adb reboot bootloader

If there is no device listed at all, make sure you have USB Debugging turned on in the Developer menu.

Once your phone gets into bootloader, we simply need to upload vendor.img followed by a final reboot:

fastboot flash vendor vendor.img
 Sending 'vendor' (190332 KB)                       OKAY [  4.299s]
 Writing 'vendor'                                   OKAY [  3.098s]
 Finished. Total time: 7.431s

fastboot reboot
 Rebooting
 Finished. Total time: 0.009s

Pesky message should be gone until a next vendor image comes along.

Familiar Column Selection in Visual Studio Code

If you ever dealt with any advanced text editor, you are probably aware of column (a.k.a. block selection). You press Shift+Alt and then either use mouse or arrow keys to have a bit unusual block selection. While not needed often, it’s invaluable when it comes to dealing with text in columns.

Visual Studio Code does support it but, of course, there are minor issues. First of all, unlike almost any other editor (including Visual Studio!), shortcut is actually Shift+Ctrl+Alt. Fortunately this can be fixed either by manually remapping key bindings for column selection or by simply installing Visual Studio Keymap extension.

While that sorts out column selection key shortcut, it still leaves one annoying problem - if you move cursor in any direction while multiple lines are selected, you will see multiple cursors move - instead of more usual selection cancellation. Fortunately, you can add a few key bindings in keybindings.json to deal with that issue:

[
    {
        "key": "left",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    },
    {
        "key": "right",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    },
    {
        "key": "up",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    },
    {
        "key": "down",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    },
    {
        "key": "pageup",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    },
    {
        "key": "pagedown",
        "command": "cancelSelection",
        "when": "editorHasMultipleSelections && textInputFocus"
    }
]

Now you can enjoy block selection that works properly. :)

DRM Exceptions

A few days ago The Library of Congress has published Exemption to Prohibition on Circumvention of Copyright Protection Systems for Access Control Technologies. To make a long document short, you get to bypass a bit of DMCA (Digital Millennium Copyright Act) rules.

Most of the media talk about playing older games and fixing consoles or jail-breaking your phone but that’s not the full scope. For example, there is a security research exception. Without this exception any company ending as butt of a joke could sue the security researcher. And, the way how DMCA was written, they would prevail. Mind you, they still get to sue you, but now their victory is unlikely.

For Nth year in row, these rules also plug a hole in e-book accessibility. For example, without this exception, blind people would depend on the mercy of DRM-protected content producers. With this exception, they can use software of their choice to help them read and, if software has to break DRM to do it, so be it.

Lastly, one important category is fixing your vehicles. Quite a few manufacturers (John Deer comes first to mind but they are not alone) have been using DRM as a way to prevent you from fixing your vehicle yourself. There is quite a lot of revenue to get if you can block those pesky independent repairmen. Well, at least now they cannot use DRM to do this.

However, it’s not all good news as these provisions expire every three years and thus there is always a possibility of “LoC giveth, LoC taketh away” situation in the future. And just having right to DRM circumvention doesn’t mean shit if you still cannot get replacement parts and/or any replacement parts you do obtain are potentially seized.

But it is a good step forward.

Seattle Code Camp - Cryptography Failures

As you read this another Seattle Code Camp talk is behind me and its time to share the slides.

Due to way how I structure my presentations, just slides alone will probably not work for you. However, if you still want to proceed or you were at presentation and you want slides for links and resources contained within, feel free to download them here.