Batch Optimizing Images

The same image can be saved in multitude of ways. Whether it is camera phone or editing application, usually goal is to save image quickly without caring for each and every byte. I mean, is it really important if image is 2.5 MB or 2.1 MB? Under most circumstances bigger file is written more quickly and slightly bigger size is perfectly acceptable compromise.

However, if you place the image on a website, this suddenly starts to matter. If your visitors are bandwidth-challenged, it makes a difference between the load time measured in seconds or tenths of seconds. However, if you start optimizing, you can spend way too much time dealing with this. If you are lazy like me and don’t want to change your flow too much, there is always an option to save unoptimized files now and optimize later.

For optimizing images I tend to stick with two utilities: OptiPNG for PNG and jpegoptim for JPEG files. Both of them do their optimizations in a completely lossless fashion. This might not bring you the best savings, especially for JPEG images, but it has one great advantage - if you run optimization over the already optimized images, there will be no harm. This means you don’t need to track what files are already optimized and which need work. Just run the tools every once in a while and you’re golden.

I created the following script to go over each image and apply optimizations:

@ECHO OFF

SET  EXE_OPTIPNG="\Tools\optipng-0.7.5\optipng.exe"
SET EXE_JPEGTRAN="\Tools\jpegoptim-1.4.3\jpegoptim.exe"

SET    DIRECTORY=.\pictures

ECHO = OPTIMIZE PNG =
FOR /F "delims=" %%F in ('DIR "%DIRECTORY%\*.png" /B /S /A-D') do (
    ECHO %%F
    DEL "%%F.tmp" 2> NUL
    %EXE_OPTIPNG% -o7 -silent -out "%%F.tmp" "%%F"
    MOVE /Y "%%F.tmp" "%%F" > NUL
    IF ERRORLEVEL 1 PAUSE && EXIT
)

ECHO.

ECHO = OPTIMIZE JPEG =
FOR /F "delims=" %%F in ('DIR "%DIRECTORY%\*.jpg" /B /S /A-D') do (
    ECHO %%F
    %EXE_JPEGTRAN% --strip-all --quiet "%%F"
    IF ERRORLEVEL 1 PAUSE && EXIT
)

And yes, this will take ages. :)

Incremental Mercurial Clone

One both advantage and disadvantage of the distributed source control is repository containing the whole history. Upon the first clone, when all data must be downloaded, this can turn into an exercise in futility if you are on a lousy connection. Especially when, in my case, downloading a huge SVN-originating Mercurial repository multi-GB in size. As connection goes down, all work has to be repeated.

Game got boring after a while so I made following script for incremental updates:

@ECHO OFF

SET SOURCE=https://example.org/BigRepo/
SET REPOSITORY=MyBigRepo

IF NOT EXIST "%REPOSITORY%" (
    hg --debug clone %SOURCE% "%REPOSITORY%" --rev 1
)

SET XXX=0
FOR /F %%i IN ('hg tip --cwd "%REPOSITORY%" --template {rev}') DO SET XXX=%%i

:NEXT
SET /A XXX=XXX+1

:REPEAT
ECHO.
ECHO === %XXX% === %DATE% %TIME% ===
ECHO.

hg pull --cwd "%REPOSITORY%" --debug --rev %XXX% --update
SET EXITCODE=%ERRORLEVEL%
ECHO.
IF %EXITCODE% GTR 0 (
    SET FAILED=%EXITCODE%
    hg recover --cwd "%REPOSITORY%" --debug
    SET EXITCODE=%ERRORLEVEL%
    ECHO.
    ECHO ======= FAILED WITH CODE %FAILED% =======
    IF %EXITCODE% GTR 0 (
        ECHO ======= FAILED WITH CODE %EXITCODE% =======
    ) else (
        ECHO === SUCCESS ===
    )
    GOTO REPEAT
) else (
    ECHO.
    ECHO === SUCCESS ===
)

GOTO NEXT

Script first clones just a first revision and then incrementally asks for revisions one at a time. If something goes wrong, recovery is started following by yet another download. Simple and effective.

Forcing Compiled .NET Application to 32-Bit

.NET application compiled with Any CPU as a target and without “Prefer 32-bit” set, will run in 64 bits whenever it can. If application is developed and tested in that manner, this is actually a good thing. Even if you don’t care about a vastly more memory you can use, you should care about the fact Windows Server these days exists only in 64-bit flavor. Yes, with prefer 32-bit checked, your application is going to be capable of running on it. However, on all development machines you will run it in 32 bits and thus find some errors only once your application is running 64-bit on a (headless) server. Every developer should run his code in 64-bit. No excuses.

Saying that, if you stumble across a signed Any CPU .NET application that used to work on 32-bit OS just fine but stopped working with a move to 64 bits, you have a problem. Even if your environment does support 32-bit computing, stubborn code will hit bug again and again. If application was unsigned, you might go the route of editing binary directly. With signed binaries you’ll have to be a bit more sneaky.

One trick is to re-configure .NET loader:

C:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\Ldr64.exe SetWow
 loading kernel32...done.
 retrieved GetComPlusPackageInstallStatus entry point
 retrieved SetComPlusPackageInstallStatus entry point
 Setting status to: 0x00000000
 SUCCESS

However, this is a machine-wide setting and requires administrator access.

Another way is cheating the system and creating a loader application with settings you want (e.g. x86). Then load destination assembly and simply start it:

var targetAssembly = Assembly.LoadFrom("AnyCpuApplication.exe");
targetAssembly.EntryPoint.Invoke(null, null);

As “proxy application” is 32-bit, .NET loaded will load assembly into its domain with the same settings and our problem is solved.

Example code is available.

Media Feature Pack for Windows 10 N

Illustration

Coming from the Europe, Windows N is not foreign to me. For unfamiliar, N is a designation for Windows without Media Player. For most of installations I don’t have any issue with this as I will install a media player of my own choice.

However, there are some scenarios where you might want to have Windows Media Player around. For example, if a program is using Windows Media control for playback, there is no alternative. For exactly those cases Microsoft does provide a Media Feature Pack adding all the bits N version took out.

So, when I had to do this on Windows 10, I knew the drill. Simple search for “Windows 10 Media Feature Pack” returned a link to download pages and short reboot later… I still had an issue.

As download page mentioned KB3010081, I checked there and saw that “In Windows 10 1511, this update is superseded by update 3099229 for the November 2015 release of Media Feature Pack.” Easy, I went to KB3099229, tried download, and kissed 404 right on the lips.

Fortunately, knowing the KB article number, I was able to use Windows 10 Update Archive (not official Microsoft site) and figure Windows10.0-KB3099229-x64.msu was the droid file I was looking for. And another reboot later, Windows Media Player was installed with all annoyances it brings.

Let’s recapitulate. Microsoft’s own official download page for Media Feature Pack has a link to an executable that doesn’t work with the latest Windows release. In the knowledge base they do include information about a newer installation but that executable is unreachable. The only way to get a file is to go to independent site and download it from there…

Twofish in C#

As one of the five AES competition finalists, Twofish algorithm is well known and appreciated. Yes, Rijndael ended up being AES at the end but, unlike other finalists, Twofish is still widely used.

As I tried to read some Twofish-encrypted files I’ve noticed one sad fact - C# support is severely lacking. Yes, there is one reasonably good free implementation on CodeProject but it doesn’t really play nicely with CryptoStream due to lack of any padding support. So, since I had some free time on my hands, I’ve decided to roll my own implementation.

I won’t go too much into the details - do check the code yourself. Suffice to say it can be inserted wherever SymmetricAlgorithm can be used. It supports CBC and ECB encryption modes with no, zero, or PKCS#7 padding. You can chose if you are going to deal with encryption yourself or decide to be at the mercy of CryptoStream.

Basis for this code was Twofish reference implementation in C. Yes, I am aware recommendation is to use the optimized version instead but I find reference code much more readable and it made porting so much easier. I did however use MDS table lookup from the optimized version as it pretty much doubles the speed. Some time in the future I might consider optimizing it further but, as I have no pressing need, it might be a while.

It should be reasonably easy to follow Twofish code and only confusion will probably arise from the use of DWord structure with the overlapping fields. Yes, you can have the exactly same functionality with bitwise operations but overlapping structure does help performance quite a bit (around 25%). As I am using unoptimized reference code, one might argue implementation is not a speed champ to start with. However I believe this actually makes code even a bit more readable so I saw no reason to resist the temptation.

Probably the same amount of time I’ve spent on code was spent on setting up the test environment. While the basic test cases for full block transformations were covered by the official test vectors, the more expansive test cases with padding were pretty much non-existent. And let’s not even get into the Monte Carlo tests and all the insanity brewing within.

In any case, download is available (or check a GitHub link). Those a bit more interested in validity can check tests too.