Programming in C#, Java, and god knows what not

Multi-mercurial

I believe that ideal setup for Mercurial is one project per repository. Most of time this is really good setup but occasionally it will cause problems. Whenever I need to do same thing on multiple repositories (e.g. push) there is an issue. Better said, there was an issue before I created HgAll script.

Script is written in PowerShell and does two things. First it traverses all directories bellow current (or defined) one and finds all mercurial repositories. Then it forwards whatever is on command line as parameter to hg command line utility. In effect it executes same command to each repository it finds.

HgAll [StartingFolder] [commands]

HgAll C:\Source summary
HgAll push all

First example will start search from C:\Source and execute “hg summary” command for any Mercurial folder found. Second example will start in current directory and execute “hg push all”.

You can download script here (or take a peek at source bellow):

$StartupLocation = (Get-Location -PSProvider FileSystem).ProviderPath

$Folder = $Null;
$Command = ""
foreach ($arg in $args) {
    if ($Folder -eq $Null) {
        if ((Test-Path -Path $arg -PathType Container) -eq $True) {
            $Folder = (Resolve-Path -Path $arg).Path
            $Command = ""
        } else {
            $Folder = $StartupLocation
            $Command = $arg
        }
    } else {
        if ($Command.length -ne 0) { $Command += " " }
        $Command += $arg
    }
}

if ($Command.length -eq 0) {
    $Command = "summary"
}

function Traverse([string]$path) {
    Write-Debug $path
    $dirs = Get-ChildItem $path | Where {$_.psIsContainer -eq $true}

    if ($dirs -ne $Null) {
        foreach ($dir in $dirs) {
            $hgsubdir = Join-Path -Path $dir.FullName -ChildPath .hg
            if ((Test-Path -Path $hgsubdir -PathType Container) -eq $True) {
                $hgdir = $dir.FullName
                if ($Command.length -ne 0) {
                    Set-Location $hgdir
                    Write-Host
                    $result = Invoke-Expression "hg $Command 2>&1"
                    if (($LASTEXITCODE -ne 0) -or (!$?)) {
                        Write-Host $dir.Name
                        foreach ($line in $result) {
                            Write-Warning $line
                        }
                    } else {
                        Write-Output $dir.Name
                        foreach ($line in $result) {
                            Write-Host $line
                        }
                    }
                }
            } else {
                Traverse($dir.FullName)
            }
        }
    }
}


try {
    Traverse($Folder)
} finally {
    Set-Location $StartupLocation
}

Not Your Normal Random

Illustration

When I test with random data I usually tend to use normal distribution. This particular time I didn’t want that. I wanted to simulate sensor readings and that means that I cannot have values going around all willy-nilly. I wanted sensors to read their default value most of time with occasional trips to edge areas. This is code I ended up with:

private static double GetAbnormalRandom() {
    double rnd = Rnd.NextDouble();
    double xh = 0.5 - rnd;
    double xs = xh * xh * 2;
    double x = 0.5 + Math.Sign(xh) * xs;
    if (x < 1) { return x; } else { return 0.5; }
}

2) This is just standard random number. Minimum value is 0 and maximum is LOWER than 1 (it can also be written as [0,1)).

3) I than moved this number to minimum of -0.5 and maximum lower than 0.5 ([-0.5,0.5)).

4) In order to maximize our mean value, I decided upon square function. It gives us number ranging from 0 to less than 0.5 (0.52 gives 0.25 and multiplication by 2 moves this back to 0.5).

5) Everything could stop here sine I have my distribution already available in step 4. However, I wanted to get everything back into [0,1) range. It is just matter of adding 0.5 to whatever number we got in step 4. Sign operation is here to ensure that half of numbers go to less than 0.5 and half of them go above.

6) Of course all this math is causing rounding to our floating point numbers. In rare occasions that can cause our step 4 to generate positive range number that is equal to 0.5. It is very rare (approximately 1 in 1000000) and we can just add it in middle.

And this is it. Function will most probably return values around 0.5 with sharp drop in probability for values further toward edge (see graph).

For full code alongside some statistics you can download source code.

ToolStrip as Main Menu

Let’s take small project as an example. You need two or three options and you cram everything as root menu item. Since you like your users and your users like pictures, you additionally create toolbar with same options. Final toolbar has one-to-one mapping with menu. Now question is why do we need menu in first place?

Personally I see only one reason - keyboard accessibility. Toolbar might be nice but it will not allow us to use pure keyboard interface in order to select items. However, solution is simple. First we set KeyPreview=true on form itself and then we just handle Alt key:

private void MainForm_KeyDown(object sender, KeyEventArgs e) {
    switch (e.KeyData) {
        case Keys.Alt | Keys.Menu: {
            menu.Select();
            menuNew.Select();
            e.SuppressKeyPress = true;
            e.Handled = true;
        } break;
    }
}

This code acts upon Alt key press. First we need to select ToolStrip control and then we select first item (New, in this case) within it. If we just use Select on item it will look same, but keyboard arrows will not move selection between items.

Downside to this is handling shortcut keys. While MenuItem will allow us to set Ctrl+N as shortcut key same is not true for ToolStripButton. All shortcuts should be handled explicitly now (within MainForm_KeyDown event):

···
case Keys.Control | Keys.N:
    menuNew_Click(null, null);
    e.SuppressKeyPress = true;
    e.Handled = true;
    break;
···

Full code example is available here.

Not All Files Can Be Embedded

Quite a lot of helper files needed for average applications are best stored as resource. This way separation of data and code is kept on logical level but everything gets stored in one executable so nothing can be lost.

In order to make some file resource only thing needed is selecting “Properties” from context menu and setting “Build Action” to “Embedded Resource”. Reading this from code is equally easy (assuming program is called MyProgram and resource file is called Test.txt):

Stream myStream = Assembly.GetExecutingAssembly().GetManifestResourceStream("MyProgram.Test.txt");

It is easy as it can be and I was very surprised when I could not get this to work. I added file “File.hr.xml” to project and I embedded it correctly but GetManifestResourceStream method returned null for it. Documentation says that this can only happen when resource is not there and this was impossible in my case.

In order to confirm this I looped through all resources I had embedded:

foreach (string name in Assembly.GetExecutingAssembly().GetManifestResourceNames()) {
    Debug.WriteLine(name);
}

To my surprise documentation was correct, my file simply wasn’t there. However, in my bin folder there was subfolder named “hr” with “MyProgram.resources” file inside.

Then it hit me. Problem was in multi-dotted extension with valid language code as first part. These files are understood by Visual Studio as being language specific and reading them from another culture is not possible (ok, it is possible but not easy).

Since I really wanted this resource to be available regardless of localization, solution was simple. I just renamed file to “File-HR.xml” and it magically appeared in resource list.

Mercurial Over Both Http and Https

My last post about Mercurial left us with Mercurial server listening to https-only requests. This is probably best solution security-wise but there might be valid reasons for having it on http also (e.g. performance within local network).

Solution lies in editing “/etc/apache2/sites-available/hg” to have two configurations - one for http and one for https:

NameVirtualHost *:80
NameVirtualHost *:443

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
    RewriteEngine on
    RewriteRule (.*) /srv/hg/cgi-bin/hgweb.cgi/$1
</VirtualHost>

<VirtualHost *:443>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
    RewriteEngine on
    RewriteRule (.*) /srv/hg/cgi-bin/hgweb.cgi/$1
    SSLEngine on
    SSLOptions +StrictRequire
    SSLCertificateFile /srv/hg/https.crt
    SSLCertificateKeyFile /srv/hg/https.key
</VirtualHost>

After quick Apache restart your Mercurial will answer both http and https requests.

Mercurial Over Https

My guide on making Mercurial server left us with http as a protocol of choice. This might be ok in local network but https would be more comfortable choice. This post starts with already running Mercurial server on Ubuntu.

Apache does come with https module which has to be enabled:

sudo a2enmod ssl
 Enabling module ssl.
 See /usr/share/doc/apache2.2-common/README.Debian.gz on how to configure SSL and create self-signed certificates.
 Run '/etc/init.d/apache2 restart' to activate new configuration!

Before restarting Apache we need to create few things (key, certificate signing request, certificate) and easiest way is to create it ourself (write whatever you want for organization details):

openssl genrsa -out https.key 1024
 Generating RSA private key, 1024 bit long modulus
 ..........++++++
 ......++++++
 e is 65537 (0x10001)

openssl req -new -key https.key -out https.csr
 You are about to be asked to enter information that will be incorporated
 into your certificate request.
 What you are about to enter is what is called a Distinguished Name or a DN.
 There are quite a few fields but you can leave some blank
 For some fields there will be a default value,
 If you enter '.', the field will be left blank.
 -----
 Country Name (2 letter code) [AU]: 
 State or Province Name (full name) [Some-State]: 
 Locality Name (eg, city) []: 
 Organization Name (eg, company) [Internet Widgits Pty Ltd]: 
 Organizational Unit Name (eg, section) []: 
 Common Name (eg, YOUR name) []: 
 Email Address []: 

 Please enter the following 'extra' attributes
 to be sent with your certificate request
 A challenge password []: 
 An optional company name []: 

openssl x509 -req -days 36500 -in https.csr -signkey https.key -out https.crt
 Signature ok
 subject=...
 Getting Private key

Once keys are created we must enter following lines in “/etc/apache2/sites-available/hg” (I left other stuff for clarity):

NameVirtualHost *
<VirtualHost *>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
    RewriteEngine on
    RewriteRule (.*) /srv/hg/cgi-bin/hgweb.cgi/$1
    SSLEngine on
    SSLOptions +StrictRequire
    SSLCertificateFile /srv/hg/https.crt
    SSLCertificateKeyFile /srv/hg/https.key
</VirtualHost>

With this we are ready for restart:

/etc/init.d/apache2 restart
 * Restarting web server apache2
     [warn] NameVirtualHost *:80 has no VirtualHosts
 ... waiting [warn] NameVirtualHost *:80 has no VirtualHosts

After these changes Mercurial is listening ONLY on https. Since we made self-signed certificate, browser will complain about verification, but that is quite normal.

P.S. To use both http and https, read some more.

From SVN to Mercurial

Moving from SVN to Mercurial is obvious if you don’t want history. Folder copy/paste action is more than enough. If we want to get whole history, that is where things get more involved. This guide is written with assumption of Ubuntu installation of Mercurial. If you are using something other, do adjust a little (yes, it also works on Windows).

First thing that we need is “python-subversion” package installed. For some reason I could not find this through Ubuntu Software Center. It worked from Synaptic Package Manager without issues. Afterward we need to enable Mercurial’s convert extension. Just edit “/home/josip/.hgrc” (do change folder if your name is not Josip) and add these lines to it:

[extensions]
hgext.convert=

Last thing to do is actual conversion:

mkdir TestRepo

hg convert svn://192.168.0.3/TestRepo TestRepo
 initializing destination TestRepo repository
 scanning source...
 sorting...
 converting...
 32 
 31
 ...
 1 
 0

cd TestRepo

hg update
 49 files updated, 0 files merged, 0 files removed, 0 files unresolved

Your Mercurial repository awaits. :)

P.S. If folder with Mercurial repository is empty, you probably forgot to do “hg update”.

How Hard Can It Be to Create ListView

Illustration

In WPF data binding is fairly consistent, I would even dare to say best that I ever saw. It is easy to do fairly complex things at some cost to way how simpler things are done. Internet is full of examples how to do simple WPF data binding but it surprisingly lacks examples for doing background updates. I tried here to make simplest example that will work with dynamic updates.

Everything revolves around ObservableCollection. In my example this class is inherited within ItemViewCollection class and all “filling” should be done to Items property. I opted to do so with hard-coded data but you can imagine it being done with data from database or any other source.

...
this.Items.Clear();
this.Items.Add(new ItemViewData("Primary weapons", 1000));
this.Items.Add(new ItemViewData("Secondary weapons", 1000));
this.Items.Add(new ItemViewData("Shields", 1000));
this.Items.Add(new ItemViewData("Torpedos", 1000));
this.Items.Add(new ItemViewData("Item " + DateTime.Now.ToLongTimeString(), 1000));
this.OnCollectionChanged(new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Reset));
...

Once data is shown, it will not be refreshed unless OnCollectionChanged method gets called. Only then update will propagate around. Above we can see how whole collection should be refreshed once Items get cleared and filled again. If you see that ListView does not respond to your changes (e.g. you added some items) there is chance you forgot to call it.

Data itself is stored within ItemViewData class. That class implements INotifyPropertyChanged interface and thus it enables update notifications for selected properties (e.g. Energy and ImageIndex). You could update all properties but updating only those that you actually changed will improve performance. Again, if you changed property and your ListView does not reflect it, you forgot to do notification.

public void SetEnergy(decimal newEnergy) {
    if (newEnergy < this.Energy) {
        this.ImageIndex = 1; //down
    } else if (newEnergy > this.Energy) {
        this.ImageIndex = 2; //up
    } else {
        this.ImageIndex = 0; //same
    }
    this.Energy = newEnergy;
    this.OnPropertyChanged(new PropertyChangedEventArgs("Energy"));
    this.OnPropertyChanged(new PropertyChangedEventArgs("ImageIndex"));
}

Code that handles data binding is in following (abreviated) XAML:

...
    <Window.Resources>
        <ObjectDataProvider x:Key="parts" ObjectType="{x:Type local:ItemViewCollection}" MethodName="GetInstance" />
    </Window.Resources>

    <ListView Name="List" ItemsSource="{Binding}" DataContext="{Binding Source={StaticResource parts}}">
    ...
    </ListView>
...

What really happens here?

ListView uses binding to “parts” data provider in order to call GetInstance method within item collection (ItemViewCollection). Whatever this method returns get’s treated as rows for ListView control. Our code is responsible for putting right stuff inside (in FillItems method) and for doing it in correct thread (thus Invoke code there). This is all there is to it.

To make things more interesting I added small method (BackUpdate within App.xaml.cs) that updates some values every two seconds (INotifyPropertyChanged.OnPropertyChanged gets triggered here) and every 20 seconds everything gets reloaded from start (thus need for ObservableCollection.OnCollectionChanged).

It might seem complicated when you look at it for first time but once you see full example code these code fragments should make some sense. :)

Nicer Mercurial URL

I already went through motions of installing Mercurial server on Ubuntu. However, that left us with nasty looking URL with hgweb.cgi in it (e.g. http://192.168.0.2/hgweb.cgi/TestRepo/). Can we make it nicer?

We need to edit “/etc/apache2/sites-available/hg” in order to add two highlighted lines (I kept all other lines for clarity):

NameVirtualHost *
<VirtualHost *>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
    RewriteEngine on
    RewriteRule (.*) /srv/hg/cgi-bin/hgweb.cgi/$1
</VirtualHost>

Next thing to do is enabling Apache’s mod_rewrite module and restarting apache:

sudo a2enmod rewrite
 Enabling module rewrite.
 Run '/etc/init.d/apache2 restart' to activate new configuration!

sudo /etc/init.d/apache2 restart
  * Restarting web server apache2                                                                                                                              apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.0.2 for ServerName
 [Tue Feb 01 09:42:15 2011] [warn] NameVirtualHost *:80 has no VirtualHosts
  ... waiting apache2: Could not reliably determine the server's fully qualified domain name, using 192.168.0.2 for ServerName
 [Tue Feb 01 09:42:16 2011] [warn] NameVirtualHost *:80 has no VirtualHosts

If everything went fine you should be access your repositories without hgweb.cgi, e.g. http://192.168.0.2/TestRepo/. Not a big change but it does look much better.