Private Mercury

Sharing source with Mercurial is not hard. There is quite a good guide at Martin’s Blog and, indeed, first part of this post will mostly follow his setup.

Sharing sources with password authentication is still not hard but (at least from my perspective) is not obvious. This post will document my efforts of creating private and password protected Mercurial storage.

This procedure is tested on Ubuntu 10.04.1 LTS but I would expect it to work on older versions as well.

First we need to install it’s package.

sudo apt-get install mercurial
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 The following extra packages will be installed:
   mercurial-common
 Suggested packages:
   qct vim emacs kdiff3 tkdiff meld xxdiff python-mysqldb python-pygments
 The following NEW packages will be installed:
   mercurial mercurial-common
 0 upgraded, 2 newly installed, 0 to remove and 63 not upgraded.
 Need to get 1,182kB of archives.
 After this operation, 4,956kB of additional disk space will be used.
 Do you want to continue [Y/n]? Y
 Get:1 http://hr.archive.ubuntu.com/ubuntu/ lucid/universe mercurial-common 1.4.3-1 [1,131kB]
 Get:2 http://hr.archive.ubuntu.com/ubuntu/ lucid/universe mercurial 1.4.3-1 [50.7kB]
 Fetched 1,182kB in 3s (388kB/s)
 Selecting previously deselected package mercurial-common.
 (Reading database ... 124142 files and directories currently installed.)
 Unpacking mercurial-common (from .../mercurial-common_1.4.3-1_all.deb) ...
 Selecting previously deselected package mercurial.
 Unpacking mercurial (from .../mercurial_1.4.3-1_i386.deb) ...
 Processing triggers for man-db ...
 Setting up mercurial-common (1.4.3-1) ...
 Setting up mercurial (1.4.3-1) ...
 Creating config file /etc/mercurial/hgrc.d/hgext.rc with new version
 Processing triggers for python-support ...

Create location for Mercurial repositories at /srv/hg with cgi-bin at subdirectory:

sudo mkdir -p /srv/hg/cgi-bin
sudo cp /usr/share/doc/mercurial-common/examples/hgweb.cgi /srv/hg/cgi-bin/

Additionally we need “/srv/hg/cgi-bin/hgweb.config” (do not forget to sudo) with following lines:

[collections]
/srv/hg/ = /srv/hg/

In newer Mercurial installations you also need to edit “/srv/hg/cgi-bin/hgweb.cgi” in order to fix config parameter. Just change example config line with:

config = "/srv/hg/cgi-bin/hgweb.config"

Next thing to do is installing apache web server:

sudo apt-get install apache2
 Reading package lists... Done
 Building dependency tree
 Reading state information... Done
 The following extra packages will be installed:
   apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1
   libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libssl0.9.8
 Suggested packages:
   apache2-doc apache2-suexec apache2-suexec-custom
 The following NEW packages will be installed:
   apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common
   libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap
 The following packages will be upgraded:
   libssl0.9.8
 1 upgraded, 9 newly installed, 0 to remove and 62 not upgraded.
 Need to get 6,343kB of archives.
 After this operation, 10.2MB of additional disk space will be used.
 Do you want to continue [Y/n]? Y
 Get:1 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main libssl0.9.8 0.9.8k-7ubuntu8.1 [3,015kB]
 Get:2 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libapr1 1.3.8-1build1 [116kB]
 Get:3 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1 1.3.9+dfsg-3build1 [85.4kB]
 Get:4 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1-dbd-sqlite3 1.3.9+dfsg-3build1 [27.1kB]
 Get:5 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1-ldap 1.3.9+dfsg-3build1 [25.1kB]
 Get:6 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2.2-bin 2.2.14-5ubuntu8.2 [2,622kB]
 Get:7 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2-utils 2.2.14-5ubuntu8.2 [159kB]
 Get:8 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2.2-common 2.2.14-5ubuntu8.2 [290kB]
 Get:9 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2-mpm-worker 2.2.14-5ubuntu8.2 [2,366B]
 Get:10 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2 2.2.14-5ubuntu8.2 [1,484B]
 Fetched 6,343kB in 14s (440kB/s)
 Preconfiguring packages ...
 (Reading database ... 124530 files and directories currently installed.)
 Preparing to replace libssl0.9.8 0.9.8k-7ubuntu8 (using .../libssl0.9.8_0.9.8k-7ubuntu8.1_i386.deb) ...
 Unpacking replacement libssl0.9.8 ...
 Setting up libssl0.9.8 (0.9.8k-7ubuntu8.1) ...
 Processing triggers for libc-bin ...
 ldconfig deferred processing now taking place
 Selecting previously deselected package libapr1.
 (Reading database ... 124530 files and directories currently installed.)
 Unpacking libapr1 (from .../libapr1_1.3.8-1build1_i386.deb) ...
 Selecting previously deselected package libaprutil1.
 Unpacking libaprutil1 (from .../libaprutil1_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package libaprutil1-dbd-sqlite3.
 Unpacking libaprutil1-dbd-sqlite3 (from .../libaprutil1-dbd-sqlite3_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package libaprutil1-ldap.
 Unpacking libaprutil1-ldap (from .../libaprutil1-ldap_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package apache2.2-bin.
 Unpacking apache2.2-bin (from .../apache2.2-bin_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2-utils.
 Unpacking apache2-utils (from .../apache2-utils_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2.2-common.
 Unpacking apache2.2-common (from .../apache2.2-common_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2-mpm-worker.
 Unpacking apache2-mpm-worker (from .../apache2-mpm-worker_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2.
 Unpacking apache2 (from .../apache2_2.2.14-5ubuntu8.2_i386.deb) ...
 Processing triggers for man-db ...
 Processing triggers for ufw ...
 Processing triggers for ureadahead ...
 ureadahead will be reprofiled on next reboot
 Setting up libapr1 (1.3.8-1build1) ...
 Setting up libaprutil1 (1.3.9+dfsg-3build1) ...
 Setting up libaprutil1-dbd-sqlite3 (1.3.9+dfsg-3build1) ...
 Setting up libaprutil1-ldap (1.3.9+dfsg-3build1) ...
 Setting up apache2.2-bin (2.2.14-5ubuntu8.2) ...
 Setting up apache2-utils (2.2.14-5ubuntu8.2) ...
 Setting up apache2.2-common (2.2.14-5ubuntu8.2) ...
 Enabling site default.
 Enabling module alias.
 Enabling module autoindex.
 Enabling module dir.
 Enabling module env.
 Enabling module mime.
 Enabling module negotiation.
 Enabling module setenvif.
 Enabling module status.
 Enabling module auth_basic.
 Enabling module deflate.
 Enabling module authz_default.
 Enabling module authz_user.
 Enabling module authz_groupfile.
 Enabling module authn_file.
 Enabling module authz_host.
 Enabling module reqtimeout.
 Setting up apache2-mpm-worker (2.2.14-5ubuntu8.2) ...
  * Starting web server apache2
 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
 Setting up apache2 (2.2.14-5ubuntu8.2) ...
 Processing triggers for libc-bin ...
 ldconfig deferred processing now taking place

We need new configuration for web interface of our repositories (“/etc/apache2/sites-available/hg”) with following content:

NameVirtualHost *
<VirtualHost *>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin/
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
</VirtualHost>

Lines under Location are ones that ensure privacy of our repository.

We can now disable default web site and enable new one (and we can ignore all warnings) together with changes of ownership and rights:

sudo chown -R www-data /srv/hg

sudo chmod a+x /srv/hg/cgi-bin/hgweb.cgi

sudo a2dissite default
 Site default disabled.
 Run '/etc/init.d/apache2 reload' to activate new configuration!

sudo a2ensite hg
 Enabling site hg.
 Run '/etc/init.d/apache2 reload' to activate new configuration!

sudo /etc/init.d/apache2 reload
  * Reloading web server config apache2
 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
 [warn] NameVirtualHost *:80 has no VirtualHosts

If we try to access “http://localhost” now, we should be greeting with password prompt.

Thing that we are missing is “/srv/hg/.htpasswd” file. We can create all users we need with htpasswd command:

sudo htpasswd -c /srv/hg/.htpasswd testuser
 New password:
 Re-type new password:
 Adding password for user testuser

All further users are then added with slightly modified command (notice that -c is missing):

sudo htpasswd /srv/hg/.htpasswd testuser2
 New password:
 Re-type new password:
 Adding password for user testuser2

After creating repository itself

sudo hg init /srv/hg/TestRepo

we must also create “/srv/hg/TestRepo/.hg/hgrc” file with following content:

[web]
push_ssl=false
allow_push=testuser

This allows for using http (instead of https) and allows access to our “testuser” (if there are no restricturons, just put * for user name). Very last step in setup is actually allowing apache to use our repository for writing. Easiest thing to do here is just transferring ownership to it:

sudo chown -R www-data /srv/hg/TestRepo

Finally we can use “http://192.168.0.2/hgweb.cgi/TestRepo/” for pushing and pulling data from any Mercurial client.

P.S. To use https, check second post of a series.

DNS Problems With FreeNAS

It may come as a surprise to quite a few guys that know me, but I use FreeNAS as my main file server. I used version 0.6 for quite a while but I switched to 0.7 few weeks ago.

At one point in time I noticed that one of my scripts is not working anymore. That script checked my external IP address and displayed it on character LCD. I could not say when exactly it stopped working since I didn’t checked it for a while.

Little bit of digging discovered that I had problems with host name resolving:

host ^^example.dyndns.org^^
0.0.0.0 connect: Address family not supported by protocol family
0.0.0.0 connect: Address family not supported by protocol family
example.dyndns.org A record not found, try again

Then I remembered. Few days after upgrading server to new FreeNAS edition, I also changed configuration from DHCP to static IP. That meant that DNS servers were not set anymore. All I needed to do is to manually set DNS addresses and script would be working once more.

Only problem - no manual setting of DNS is available. Since I know a thing-or-two about command line interface, I went to add missing entries to /etc/resolv.conf from there. Unfortunately, that file gets overwritten after every reboot.

Since I didn’t want to depend on power grid to keep my server alive, I devised a cunning plan. If you go into System -> Advanced menu, there is option to set commands to be executed after every reboot. I just added new postinit command:

echo "nameserver 8.8.8.8" >> /etc/resolv.conf

That line just appended DNS server definition to file. Once that line is there, all DNS resolving works once more.

Problem solved.

P.S. After all this effort, I found place where DNS can be setup in more user friendly manner. To my surprise it is under System -> General menu.

FreeNAS on Alix1d

I always had a thing for file servers. Usually, I would just take some old machine, throw few hard drives inside and consider it done. However, as I got married, noise of computers became issue. I was just something I didn’t notice before - all my “servers” were noisy like hell. In order to keep wife happy, I decided to invest a little into small machine without any fans. My final decision fell onto PC Engine’s alix1d system board with box1c. While this is not most powerful machine, I considered it’s 500 MHz CPU and 256 MB of RAM enough for a task at hand. I just slammed small 2.5" hard drive inside of box and it was ready to serve.

My first idea was to install Windows Home Server, but there were two problems with it - MSDN subscription didn’t cover it at the time and it’s system requirements (1 GHz processor / 512 MB of memory) were too high for this machine to handle. After some testing, my initial installation was done with Windows Server 2003 and I had it going for quite a while (more than a year actually). However, Windows were not lively on this machine (probably has something to do with slow processor and not enough memory). Since that machine was used only to share files, my though were on giving it a little boost by installing Windows XP on it.

At that time I was preparing setup for one Hyper-V server and in order to do file sharing I decided on FreeNAS. This free solution was quite good for NAS tasks at hand especially because it could fit inside of 128 MB of RAM. That way, I would give it enough to work with and impact on other Hyper-V machines would be minimal. It took me a while to notice that it could be solution for my problem also.

Since my machine has support for Compact Flash, I decided to install FreeNAS on it. With Windows 2003 on hard drive, this would also mean that my original installation will be left intact and a backup solution in case of any problems.

I decided on version 0.7 RC1 which was latest version that I already tested to work in Hyper-V so that gave me confidence to try it on real machine. I was surprised with CD booting process that was painfully slow and after a while it would just stop at:

Using device=/dev/fd0 fstype=msdos to store configuration.

It would stay there no matter how long you wait. Upon pressing Ctrl+Alt+Del it would complain about parsing errors in config.xml and go into shutdown. My first thought was that there is some error on CD. However, same thing repeated with other CD’s with different versions. Only than it occurred to me that I should run it in safe mode.

Once I selected safe mode, CD install was still slow, but it worked. However same issue occurred with booting process on installed version. Only difference was that it was left hanging at:

Starting devd.

Safe mode still helped but since this machine was to be left unattended, it wasn’t really a solution.

Next step was to disable things in BIOS one-by-one and culprit was found in ACPI power management. Once that was turned off, booting process went without a hitch.

I have it installed for a month now and I must say I am satisfied. Temperature of hard drive went from 50°C to 40°C just because of using flash media as system drive and having data drive sleep most of time. While level of comfort is little bit different (scripting in tcsh vs. programming in C#) and, although sometime I miss my remote desktop, this little guy does it’s duties well.

Hyper-V Drivers in Linux

Microsoft has shared his Hyper-V integration components with Linux community. That contribution to Linux Driver Project will make it possible to have integration between any Linux and Hyper-V that was previously reserved for Suse and RedHat.

In worst-case scenario this will enable other distributions to work faster while under Hyper-V.

You can see more details on Windows Virtualization Blog.