It's Not Natty, It Is Nutty

On my “standard” work day I interact with Unix a lot so it seemed like a logical solution to have Linux on my machine. I used Ubuntu 10.10 and world was beautiful. Yes, some things did bother me but it worked. And after work I could always enjoy Windows 7.

Two days ago I upgraded to newest Ubuntu (11.04, nicknamed Natty). After upgrade boot seemed a little slow so I decided to do what I should have done first time - clean install. I spent one full day on fresh installation of Nutty and here are some problems I had in span of eight hours:

• If your secondary monitor is on left, it is almost impossible to pinpoint single dot at which Unity launcher will appear.
• Turning off auto-hide for Unity launcher causes it to be over all maximized windows and to hide their left side.
• Applications closed without warning and without any form of crash dialog. Since this includes even calculator, I would dare to suggest that new Unity interface is to blame.
• Once system started swapping it took Ctrl+Alt+F1 and top command to find that compiz took whooping 4.5 GB of RAM (out of 6 GB).
• LibreCalc died every five minutes while editing relatively simple document. Recovery worked in almost 50% of cases.
• Laptop could not wake from sleep.
• Network connection was breaking every half an hour or so.
• Booting took ages - easily double the time that 10.10 needed.
• … do notice that there was a lot of smaller problems still …

I started using Linux it in nineteens with Slackware and went through my share of different distributions. I was never completely satisfied and people working with me know that I curse Linux a lot because of small bugs. But this is first time ever that I was actually unable to do my work. At the end I just rebooted into Windows XP (company issued) and did my work there.

My next step will be to take out this Natty piece of shit from laptop and to go back to Ubuntu 10.10.

P.S. I actually liked search interface in Unity. Unfortunately everything else sucks.

Cleaning Up

What to do if your script needs to kill all processes that it started? Just kill everything that has same parent as your current shell ($$):

#!/bin/bash
…
kill `ps -ef | awk '$3 == '$$' {print $2}'`

Extracting Sparse Tar

I had to move some files from Unix. File was big, I had small USB drive - one thing led to another…

GZipped tar was obvious solution. In addition to that a friend of mine recommended to also use --sparse argument with it. Theory behind sparse files tells that block of 0 should be saved extremely efficiently thus making my file smaller even before zipping part gets involved. This made my command look like “tar cfzS somefile.tar.gz somefile”. It all worked as advertised.

Next day I got to extract this on Windows. My trusty WinRAR had no idea how to proceed. I just got “The archive is corrupt” message. My next efforts went into searching for Win32 version of tar. Since GNU tools like to be small and concentrated, of course this was not sufficient - I needed Win32 GZip also. Notice that I might be wrong here and there might be Win32 tar somewhere with everything integrated - I just haven’t found it.

Since (on Win32) extracting this tar.gz needed temporary files, I did it in two steps: first with gzip (gzip -d < somefile.tar.gz > somefile.tar) and then with tar (tar xSf somefile.tar). Even with all this, file was just too small.

After testing few more programs I gave up and recreated this archive without --sparse option. It ends up that size difference (with compression on) is not that high after all but final result is much more portable.

Here are tools I used:

• tar (site)
• gzip (site)

2032

After annual maintenance of power grid in my neighborhood with few hours without power, my trusty file server went down. It wasn’t first time that it went down. It was first time it stayed down.

This was alix1d embedded PC with FreeNAS running on it so my first thoughts went to file corruption. And I was right, there was some file corruption, but nothing that simple fsck could not solve. However, boot process still had issues.

I will not detail everything that I tried. It is sufficient to say that I wasted whole day playing with this thing. As last resort I decided to reinstall system.

As I went into BIOS to set my boot device, I noticed that my BIOS password is missing. As I went through setting, everything seemed to be on default. And default is not state you wish your alix1d board to be in.

FreeNAS has some issues with ACPI on this board. It will just not boot if you have it turned on. And I had it turned on in my BIOS. Fixing was easy - just turn it OFF. All that wasted time amounted to issue I already knew.

Reason why BIOS settings were changed was simple CR2032 battery. It usually keeps BIOS settings nice and fresh but mine was dead. Any power outage would cause same issues. It was accident waiting to happen.

I checked old invoices and it happens that this system is only two years old. I find it quite peculiar that battery is already gone. There is something that is drinking battery like mad on this motherboard.

Anyhow, everything works perfectly with new battery. I only hope that I will remember this issue when everything fails again in two years. :)

Private Mercury

Sharing source with Mercurial is not hard. There is quite a good guide at Martin’s Blog and, indeed, first part of this post will mostly follow his setup.

Sharing sources with password authentication is still not hard but (at least from my perspective) is not obvious. This post will document my efforts of creating private and password protected Mercurial storage.

This procedure is tested on Ubuntu 10.04.1 LTS but I would expect it to work on older versions as well.

First we need to install it’s package.

sudo apt-get install mercurial
 Reading package lists... Done
 Building dependency tree       
 Reading state information... Done
 The following extra packages will be installed:
   mercurial-common
 Suggested packages:
   qct vim emacs kdiff3 tkdiff meld xxdiff python-mysqldb python-pygments
 The following NEW packages will be installed:
   mercurial mercurial-common
 0 upgraded, 2 newly installed, 0 to remove and 63 not upgraded.
 Need to get 1,182kB of archives.
 After this operation, 4,956kB of additional disk space will be used.
 Do you want to continue [Y/n]? Y
 Get:1 http://hr.archive.ubuntu.com/ubuntu/ lucid/universe mercurial-common 1.4.3-1 [1,131kB]
 Get:2 http://hr.archive.ubuntu.com/ubuntu/ lucid/universe mercurial 1.4.3-1 [50.7kB]
 Fetched 1,182kB in 3s (388kB/s)
 Selecting previously deselected package mercurial-common.
 (Reading database ... 124142 files and directories currently installed.)
 Unpacking mercurial-common (from .../mercurial-common_1.4.3-1_all.deb) ...
 Selecting previously deselected package mercurial.
 Unpacking mercurial (from .../mercurial_1.4.3-1_i386.deb) ...
 Processing triggers for man-db ...
 Setting up mercurial-common (1.4.3-1) ...
 Setting up mercurial (1.4.3-1) ...
 Creating config file /etc/mercurial/hgrc.d/hgext.rc with new version
 Processing triggers for python-support ...

Create location for Mercurial repositories at /srv/hg with cgi-bin at subdirectory:

sudo mkdir -p /srv/hg/cgi-bin
sudo cp /usr/share/doc/mercurial-common/examples/hgweb.cgi /srv/hg/cgi-bin/

Additionally we need “/srv/hg/cgi-bin/hgweb.config” (do not forget to sudo) with following lines:

[collections]
/srv/hg/ = /srv/hg/

In newer Mercurial installations you also need to edit “/srv/hg/cgi-bin/hgweb.cgi” in order to fix config parameter. Just change example config line with:

config = "/srv/hg/cgi-bin/hgweb.config"

Next thing to do is installing apache web server:

sudo apt-get install apache2
 Reading package lists... Done
 Building dependency tree       
 Reading state information... Done
 The following extra packages will be installed:
   apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common libapr1
   libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap libssl0.9.8
 Suggested packages:
   apache2-doc apache2-suexec apache2-suexec-custom
 The following NEW packages will be installed:
   apache2 apache2-mpm-worker apache2-utils apache2.2-bin apache2.2-common
   libapr1 libaprutil1 libaprutil1-dbd-sqlite3 libaprutil1-ldap
 The following packages will be upgraded:
   libssl0.9.8
 1 upgraded, 9 newly installed, 0 to remove and 62 not upgraded.
 Need to get 6,343kB of archives.
 After this operation, 10.2MB of additional disk space will be used.
 Do you want to continue [Y/n]? Y
 Get:1 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main libssl0.9.8 0.9.8k-7ubuntu8.1 [3,015kB]
 Get:2 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libapr1 1.3.8-1build1 [116kB]
 Get:3 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1 1.3.9+dfsg-3build1 [85.4kB]
 Get:4 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1-dbd-sqlite3 1.3.9+dfsg-3build1 [27.1kB]
 Get:5 http://hr.archive.ubuntu.com/ubuntu/ lucid/main libaprutil1-ldap 1.3.9+dfsg-3build1 [25.1kB]
 Get:6 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2.2-bin 2.2.14-5ubuntu8.2 [2,622kB]
 Get:7 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2-utils 2.2.14-5ubuntu8.2 [159kB]
 Get:8 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2.2-common 2.2.14-5ubuntu8.2 [290kB]
 Get:9 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2-mpm-worker 2.2.14-5ubuntu8.2 [2,366B]
 Get:10 http://hr.archive.ubuntu.com/ubuntu/ lucid-updates/main apache2 2.2.14-5ubuntu8.2 [1,484B]
 Fetched 6,343kB in 14s (440kB/s)                                               
 Preconfiguring packages ...
 (Reading database ... 124530 files and directories currently installed.)
 Preparing to replace libssl0.9.8 0.9.8k-7ubuntu8 (using .../libssl0.9.8_0.9.8k-7ubuntu8.1_i386.deb) ...
 Unpacking replacement libssl0.9.8 ...
 Setting up libssl0.9.8 (0.9.8k-7ubuntu8.1) ...
 Processing triggers for libc-bin ...
 ldconfig deferred processing now taking place
 Selecting previously deselected package libapr1.
 (Reading database ... 124530 files and directories currently installed.)
 Unpacking libapr1 (from .../libapr1_1.3.8-1build1_i386.deb) ...
 Selecting previously deselected package libaprutil1.
 Unpacking libaprutil1 (from .../libaprutil1_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package libaprutil1-dbd-sqlite3.
 Unpacking libaprutil1-dbd-sqlite3 (from .../libaprutil1-dbd-sqlite3_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package libaprutil1-ldap.
 Unpacking libaprutil1-ldap (from .../libaprutil1-ldap_1.3.9+dfsg-3build1_i386.deb) ...
 Selecting previously deselected package apache2.2-bin.
 Unpacking apache2.2-bin (from .../apache2.2-bin_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2-utils.
 Unpacking apache2-utils (from .../apache2-utils_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2.2-common.
 Unpacking apache2.2-common (from .../apache2.2-common_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2-mpm-worker.
 Unpacking apache2-mpm-worker (from .../apache2-mpm-worker_2.2.14-5ubuntu8.2_i386.deb) ...
 Selecting previously deselected package apache2.
 Unpacking apache2 (from .../apache2_2.2.14-5ubuntu8.2_i386.deb) ...
 Processing triggers for man-db ...
 Processing triggers for ufw ...
 Processing triggers for ureadahead ...
 ureadahead will be reprofiled on next reboot
 Setting up libapr1 (1.3.8-1build1) ...
 Setting up libaprutil1 (1.3.9+dfsg-3build1) ...
 Setting up libaprutil1-dbd-sqlite3 (1.3.9+dfsg-3build1) ...
 Setting up libaprutil1-ldap (1.3.9+dfsg-3build1) ...
 Setting up apache2.2-bin (2.2.14-5ubuntu8.2) ...
 Setting up apache2-utils (2.2.14-5ubuntu8.2) ...
 Setting up apache2.2-common (2.2.14-5ubuntu8.2) ...
 Enabling site default.
 Enabling module alias.
 Enabling module autoindex.
 Enabling module dir.
 Enabling module env.
 Enabling module mime.
 Enabling module negotiation.
 Enabling module setenvif.
 Enabling module status.
 Enabling module auth_basic.
 Enabling module deflate.
 Enabling module authz_default.
 Enabling module authz_user.
 Enabling module authz_groupfile.
 Enabling module authn_file.
 Enabling module authz_host.
 Enabling module reqtimeout.
 Setting up apache2-mpm-worker (2.2.14-5ubuntu8.2) ...
  * Starting web server apache2
 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
 Setting up apache2 (2.2.14-5ubuntu8.2) ...
 Processing triggers for libc-bin ...
 ldconfig deferred processing now taking place

We need new configuration for web interface of our repositories (“/etc/apache2/sites-available/hg”) with following content:

NameVirtualHost *
<VirtualHost *>
    ServerAdmin webmaster@localhost
    DocumentRoot /srv/hg/cgi-bin/
    <Directory "/srv/hg/cgi-bin/">
        SetHandler cgi-script
        AllowOverride None
        Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
        Order allow,deny
        Allow from all
    </Directory>
    ErrorLog /var/log/apache2/hg.log
    <Location />
        AuthType Basic
        AuthName "Mercurial"
        AuthUserFile  /srv/hg/.htpasswd
        Require valid-user
    </Location>
</VirtualHost>

Lines under Location are ones that ensure privacy of our repository.

We can now disable default web site and enable new one (and we can ignore all warnings) together with changes of ownership and rights:

sudo chown -R www-data /srv/hg

sudo chmod a+x /srv/hg/cgi-bin/hgweb.cgi

sudo a2dissite default
 Site default disabled.
 Run '/etc/init.d/apache2 reload' to activate new configuration!

sudo a2ensite hg
 Enabling site hg.
 Run '/etc/init.d/apache2 reload' to activate new configuration!

sudo /etc/init.d/apache2 reload
  * Reloading web server config apache2
 apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
 [warn] NameVirtualHost *:80 has no VirtualHosts

If we try to access “http://localhost” now, we should be greeting with password prompt.

Thing that we are missing is “/srv/hg/.htpasswd” file. We can create all users we need with htpasswd command:

sudo htpasswd -c /srv/hg/.htpasswd testuser
 New password: 
 Re-type new password: 
 Adding password for user testuser

All further users are then added with slightly modified command (notice that -c is missing):

sudo htpasswd /srv/hg/.htpasswd testuser2
 New password: 
 Re-type new password: 
 Adding password for user testuser2

After creating repository itself

sudo hg init /srv/hg/TestRepo

we must also create “/srv/hg/TestRepo/.hg/hgrc” file with following content:

[web]
push_ssl=false
allow_push=testuser

This allows for using http (instead of https) and allows access to our “testuser” (if there are no restricturons, just put * for user name). Very last step in setup is actually allowing apache to use our repository for writing. Easiest thing to do here is just transferring ownership to it:

sudo chown -R www-data /srv/hg/TestRepo

Finally we can use “http://192.168.0.2/hgweb.cgi/TestRepo/” for pushing and pulling data from any Mercurial client.

P.S. To use https, check second post of a series.

DNS Problems With FreeNAS

It may come as a surprise to quite a few guys that know me, but I use FreeNAS as my main file server. I used version 0.6 for quite a while but I switched to 0.7 few weeks ago.

At one point in time I noticed that one of my scripts is not working anymore. That script checked my external IP address and displayed it on character LCD. I could not say when exactly it stopped working since I didn’t checked it for a while.

Little bit of digging discovered that I had problems with host name resolving:

host ^^example.dyndns.org^^
0.0.0.0 connect: Address family not supported by protocol family
0.0.0.0 connect: Address family not supported by protocol family
example.dyndns.org A record not found, try again

Then I remembered. Few days after upgrading server to new FreeNAS edition, I also changed configuration from DHCP to static IP. That meant that DNS servers were not set anymore. All I needed to do is to manually set DNS addresses and script would be working once more.

Only problem - no manual setting of DNS is available. Since I know a thing-or-two about command line interface, I went to add missing entries to /etc/resolv.conf from there. Unfortunately, that file gets overwritten after every reboot.

Since I didn’t want to depend on power grid to keep my server alive, I devised a cunning plan. If you go into System -> Advanced menu, there is option to set commands to be executed after every reboot. I just added new postinit command:

echo "nameserver 8.8.8.8" >> /etc/resolv.conf

That line just appended DNS server definition to file. Once that line is there, all DNS resolving works once more.

Problem solved.

P.S. After all this effort, I found place where DNS can be setup in more user friendly manner. To my surprise it is under System -> General menu.

FreeNAS on Alix1d

I always had a thing for file servers. Usually, I would just take some old machine, throw few hard drives inside and consider it done. However, as I got married, noise of computers became issue. I was just something I didn’t notice before - all my “servers” were noisy like hell. In order to keep wife happy, I decided to invest a little into small machine without any fans. My final decision fell onto PC Engine’s alix1d system board with box1c. While this is not most powerful machine, I considered it’s 500 MHz CPU and 256 MB of RAM enough for a task at hand. I just slammed small 2.5" hard drive inside of box and it was ready to serve.

My first idea was to install Windows Home Server, but there were two problems with it - MSDN subscription didn’t cover it at the time and it’s system requirements (1 GHz processor / 512 MB of memory) were too high for this machine to handle. After some testing, my initial installation was done with Windows Server 2003 and I had it going for quite a while (more than a year actually). However, Windows were not lively on this machine (probably has something to do with slow processor and not enough memory). Since that machine was used only to share files, my though were on giving it a little boost by installing Windows XP on it.

At that time I was preparing setup for one Hyper-V server and in order to do file sharing I decided on FreeNAS. This free solution was quite good for NAS tasks at hand especially because it could fit inside of 128 MB of RAM. That way, I would give it enough to work with and impact on other Hyper-V machines would be minimal. It took me a while to notice that it could be solution for my problem also.

Since my machine has support for Compact Flash, I decided to install FreeNAS on it. With Windows 2003 on hard drive, this would also mean that my original installation will be left intact and a backup solution in case of any problems.

I decided on version 0.7 RC1 which was latest version that I already tested to work in Hyper-V so that gave me confidence to try it on real machine. I was surprised with CD booting process that was painfully slow and after a while it would just stop at:

Using device=/dev/fd0 fstype=msdos to store configuration.

It would stay there no matter how long you wait. Upon pressing Ctrl+Alt+Del it would complain about parsing errors in config.xml and go into shutdown. My first thought was that there is some error on CD. However, same thing repeated with other CD’s with different versions. Only than it occurred to me that I should run it in safe mode.

Once I selected safe mode, CD install was still slow, but it worked. However same issue occurred with booting process on installed version. Only difference was that it was left hanging at:

Starting devd.

Safe mode still helped but since this machine was to be left unattended, it wasn’t really a solution.

Next step was to disable things in BIOS one-by-one and culprit was found in ACPI power management. Once that was turned off, booting process went without a hitch.

I have it installed for a month now and I must say I am satisfied. Temperature of hard drive went from 50°C to 40°C just because of using flash media as system drive and having data drive sleep most of time. While level of comfort is little bit different (scripting in tcsh vs. programming in C#) and, although sometime I miss my remote desktop, this little guy does it’s duties well.

Hyper-V Drivers in Linux

Microsoft has shared his Hyper-V integration components with Linux community. That contribution to Linux Driver Project will make it possible to have integration between any Linux and Hyper-V that was previously reserved for Suse and RedHat.

In worst-case scenario this will enable other distributions to work faster while under Hyper-V.

You can see more details on Windows Virtualization Blog.