Linux, Unix, and whatever they call that world these days

Cloning My Website

One disadvantage of having a reliable hosting provider is that you tend to forget about backups. In my ten years with Plus hosting there was not a single case of data loss. Regardless I wanted to go “better safe than sorry” route and make automated backups. And, while I am at it, I might as well use it to make “production replica” environment for testing.

My website environment is based on CentOS 6, MySQL database, and Apache. Ideally I would need exactly the same environment. But in this case I decided to upgrade all components to latest versions. In case of Linux that meant going with CentOS 7.1 (minimal).

Installation for CentOS is as simple as it gets. I basically click on Next until there is no button to press. :) Only possibility of error is forgetting to enable the Ethernet network adapter - not a catastrophic mistake; just annoying one. Once install was done, additional packages were in order:

yum install httpd mariadb-server php php-mysql rsync

To connect to my website I created new SSH keys:

ssh-keygen -b 4096

I appended newly created .ssh/id_rsa.pub key to .ssh/authorized_keys on my web server. That meant I could login and copy files without any passwords - great for scripting.

Setting up MySQL/MariaDB came next. It is just a basic setup followed by user and database creation:

mysql_install_db
chown -R mysql:mysql /var/lib/mysql/
service mariadb start
chkconfig mariadb on
mysql -e "CREATE USER '^^mydbuser_wp^^'@'localhost' IDENTIFIED BY '^^mydbpassword^^';"
mysql -e "CREATE DATABASE ^^mydatabase_wordpress^^"

For data replication (after making sure /home/myuser directory exists) I created simple /root/replicate.sh script with following content:

#!/bin/bash

ssh ^^myuser^^@^^myhost.com^^ "mysqldump -u ^^mydbuser_wp^^ -p^^mydbpassword^^ --opt ^^mydatabase_wordpress^^" > /var/tmp/mysql.dump
mysql ^^mydatabase_wordpress^^ < /var/tmp/mysql.dump
rm /var/tmp/mysql.dump

scp -r ^^myuser^^@^^myhost.com^^:/home/^^myuser^^/* /home/^^myuser^^
#rsync -avz -e ssh ^^myuser^^@^^myhost.com^^:/home/^^myuser^^ /home/^^myuser^^

First three lines ensure I have a fresh MySQL database import and SCP is tasked with file copy. Better approach would be rsync but I kept getting Out of memory errors. As my site is not huge, I opted for dummy copy instead of troubleshooting.

Once I ran script once to verify all is working as expected, i added it to crontab (crontab -e) so it runs every midnight:

…
00 00 * * * /root/replicate.sh

For Apache I edited /etc/httpd/conf/httpd.conf file to change its root:

…
DocumentRoot "^^/home/myuser/public_html^^"
<Directory "^^/home/myuser/public_html^^">
    AllowOverride None
    # Allow open access:
    Require all granted
</Directory>
…
<IfModule mime_module>
    …
    ^^AddType text/html .php .phps^^
</IfModule>
…

Opening filewall was next task:

firewall-cmd --permanent --zone=public --add-service=http
firewall-cmd --permanent --zone=public --add-service=https
firewall-cmd --reload

sudo iptables -I INPUT -p tcp --dport 80 -j ACCEPT
service iptables save

And all remaining was to start Apache:

chown -R apache:apache ^^/home/myuser/public_html^^
restorecon -R ^^/home/myuser/public_html^^
service httpd start
chkconfig httpd on

PS: Do notice that I didn’t describe security setup for this machine. Unless there are some mitigating circumstances, you pretty much want your backup as secure as a real thing.

Internal IPv6 Router Advertisement

I have fully functioning IPv6 network at home for a while now. I used to do tunneling but for last year or so, I have native IPv6 to my home router. And everything works really nice. Problem arises when I am not at home and I want to do some IPv6 development and I want some proper (other than link-local) IPv6 addresses assigned. There are many ways to deal with this, but I found that virtual IPv6 router works the best for me.

For this I started with preparation of the VirtualBox machine (any virtualization environment will do) for the 64-bit Linux Red Hat. Only change to the defaults was enabling the second network interface (either Bridged or Internal). That was the network I intend to use for the IPv6 router advertisement.

After installing the CentOS 7.0 Minimal Install Image onto the newly created virtual machine, there are a few manual steps needed.

First was actually enabling DHCP so additional packets can be installed. After verifying which network interfaces are present using the ip addr (I had enp0s3 and enp0s8), I edited both network startup files (/etc/sysconfig/network-scripts/ifcfg-enp0s3 and /etc/sysconfig/network-scripts/ifcfg-enp0s8) by changing the ONBOOT setting to yes followed by a network restart:

service network restart
 Restarting network (via systemctl):    [  OK  ]

With Internet working, I was ready for installing IPv6 Router Advertisement Daemon:

yum install radvd
 …
 Complete!

In /etc/radvd.conf I added the following content:

interface enp0s8
{
    AdvSendAdvert on;
    MaxRtrAdvInterval 5;
    prefix 2001:db8:42::/64;
};

This is followed by service start (and enabling it on boot):

systemctl start radvd.service
systemctl enable radvd.service
…

Now any IPv6 capable network adapter sharing the same network with second VirtualBox network interface will get our defined IPv6 prefix.

PS: Advertisement time is intentionally kept very short. I find it useful to get quick unsolicited router advertisement messages for the demonstration purposes.

Sharp Video Play

Making a minimal C# video player in the Linux ought to be easy. I mean, both platforms are mature and something so basic should not present a problem.

GStreamer C# bindings seemed as a perfect choice at first. All needed was to “make” it. Simple enough - just use autogen.sh followed by make and enjoy the fruits of your labor.

That was a plan until I started the dependency dance. First I had to make GtkSharp myself since my latest library versions were too low for GStreamer. Then I had to get GStreamer. Then I had to get GStreamer plugins. And then I gave up.

Between compiling, downloading, and resolving dependencies I have installed around 50 packages, manually compiled multiple libraries, and I was still on the “just one more” stage. Call me spoiled, but I find this unacceptable.

Fortunately Linux is full of choices and my next try was with the MPlayerControl. It too needed a few packets on the top of a clean Mint 17 installation but nowhere close to the dependency hell:

sudo apt-get install mono-complete monodevelop mplayer git

After that I had to get the latest MPlayerControl source:

git clone https://github.com/majorsilence/MPlayerControl MPlayerControl

And that was it. Shockingly, example within the source was working perfectly albeit it was not the smallest one they could make. I created my own sample form (with a reference to the LibMPlayerCommon) to play a single file as simply as possible:

using System;
using System.Collections.Generic;
using System.IO;
using System.Windows.Forms;
using LibMPlayerCommon;

namespace TestMedia {
  public class MainForm : Form {
    private MPlayer mPlayer;

    public MainForm() {
      this.WindowState = FormWindowState.Maximized;
      this.mPlayer = new MPlayer(this.Handle.ToInt32(), MplayerBackends.GL2);

      this.Load += delegate(object sender, EventArgs e) {
      this.Text = this.fileQueue.Peek();
        this.mPlayer.Play("~/somefilename.mp4");
      };
      this.mPlayer.VideoExited += delegate(object sender, MplayerEvent e) {
        this.Close();
      };
    }
  }
}

While GStreamer more complete solution by far, I found MPlayerControl infinitely easier to use and setup.

End-to-end Encryption

Illustration

Well, with all these NSA revelations it had to happen. Not only that Google is thinking about easing encryption in GMail but we got a pretty nice early code drop in a form of a Google Chrome extension.

Extension is called End-To-End and it will help you use OpenPGP encryption for your e-mails. Since it is a really early code drop, Google intentionally made it a bit difficult to install. First of all, you’ll need to compile thing yourself.

For that you would need any Linux machine (e.g. Mint 17). Step-by-step instructions are really good and work flawlessly after you install Git and Subversion:

sudo apt-get install git
sudo apt-get install subversion

After going through all steps, the extension is ready for deployment under /end-to-end/javascript/crypto/e2e/extension. To load it into the Chrome, go to Tools, Extensions, Load unpacked extension. I all went good you will see an additional icon next to the address bar.

Click on that icon will allow you access to Options where you can import key if you already have one (RSA or ECDSA). If you don’t have a key, you can provide your e-mail have one created (only ECDSA). From that moment on you can create a new message by clicking that same magical button.

Currently extension is not really polished. Sending mail still requires a few manual steps, e.g. opening mail window yourself, decryption is not automatic, there is no public key lookup… but this is still probably best solution I have seen for a web mail. Best of all, it works with essentially any web mail - not just GMail.

While there is still a lot of work remaining to make this a comfortable solution, first steps are promising. Mail encryption is a bit hard by design but I could see myself explaining all necessary steps to someone with basic computer knowledge and having them send encrypted e-mail within minutes. To me that means that biggest issue is resolved. All other stuff is just an icing on the cake.

P.S. Those who don’t have a Linux machine (or don’t want to deal with compile) can download unpacked extension here. But do notice that this extension is alpha and I probably won’t bother updating these binaries as the source gets updates.

[2014-12-17: Project source is now available on GitHub.]

Setting Up Private Internet Access on Mint

Illustration

When I published post about OpenVPN on CentOS most common comment was "How to do that on some more user-friendly distribution. Well, you don’t get much more friendlier than Mint 16 (Petra).

Before starting with anything we need to install OpenVPN package. This is done via Menu, Administration, Software Manager. Just type OpenVPN and install first thing you get back (yep, this is great piece of security advice ;)).

First we can do the easy stuff. Download PIA’s OpenVPN configuration files and extract it to directory of your choice. I kept them in /home/MyUserName/pia.

Next easy step is setting up DNS resolving. For that we go to Menu, Preferences, Network Connections. Just click edit on connection you are using and go to IPv4 Settings tab. Change Method to Automatic (DHCP addresses only). Under DNS servers enter 209.222.18.222 209.222.18.218 (PIA’s DNS).

All other commands are to be executed in terminal and most of them require root privileges. It might be best if you just become root for a while:

su - root

Next step is getting configuration in place (replace username and password with yours):

cp /home/MyUserName/pia/ca.crt /etc/openvpn/ca.crt
cp /home/MyUserName/pia/crl.pem /etc/openvpn/crl.pem
cp /home/MyUserName/pia/US\ Midwest.ovpn /etc/openvpn/client.conf

sed -i "s*ca ca.crt*ca /etc/openvpn/ca.crt*" /etc/openvpn/client.conf
sed -i "s*crl-verify crl.pem*crl-verify /etc/openvpn/crl.pem*" /etc/openvpn/client.conf

echo "auth-user-pass /etc/openvpn/login.pia" >> /etc/openvpn/client.conf
echo "mssfix 1400" >> /etc/openvpn/client.conf

echo "^^username^^" > /etc/openvpn/login.pia
echo "^^password^^" >> /etc/openvpn/login.pia

chmod 500 /etc/openvpn/login.pia

Now we can test our connection (after we restart network in order to activate DNS changes):

/etc/init.d/networking restart
openvpn --config /etc/openvpn/client.conf

Assuming that this last step ended with Initialization Sequence Completed, we just need to verify whether this connection is actually used. I found whatismyipaddress.com quite helpful here. If you see some mid-west town on map, you are golden (assuming that you don’t actually live in US mid-west :)).

Now you can stop test connection via Ctrl+C in order to properly start it. In addition, you can specify it should start on each system startup:

service openvpn start
echo "AUTOSTART=all" >> /etc/default/openvpn

Lastly you can think about firewall and disabling default interface when VPN is not active. Of course, in order for things to work, we still need to allow port 1194 toward our VPN server and DNS:

ufw default deny incoming
ifw default deny outgoing
ufw allow out on tun0
ufw allow out 1194/udp
ufw allow out on eth0 to 209.222.18.222 port 53 proto udp
ufw allow out on eth0 to 209.222.18.218 port 53 proto udp
ufw enable

And that is all.

PS: It is not a déjà vu, article is really close to how it was set on CentOS.

[2016-10-01: Removed auth-nocache directive as it didn’t play nice with auth-user-pass.]

Android X86

Illustration

My Android phone has died on me last week. Suddenly I had an issue. Bunch of stuff I had on it was not available anywhere else. Not on Windows and especially not on crusty Nokia 6300 I used as a replacement. I had to get another Android. My choice fell onto running Android x86 in VirtualBox.

First step was creating virtual machine. Upon creation I selected Linux 2.6, increased memory to 1024, and got 8 GB disk ready. After machine was created, I disabled absolute pointing device (System) and changed network adapter type to PCnet-FAST III (Network).

I started machine with Android x86 live CD (android-x86-4.3-20130725) and went for installation. There I was met with slightly annoying partition creation (New, Primary, Bootable, Write, Quit) and disk format selection (ext3). I installed GRUB and went for writable system partition. One reboot later (don’t forget to remove DVD) and my new Android device is ready.

Before I could do anything I had to go to Machine menu and disable mouse pointer integration (; remember host key for exit) in order to click-through setup options. And then I noticed that my network wasn’t working and screen was a bit weird size-wise. So I rebooted and selected Debug mode upon next boot.

There I went on to edit /android/system/etc/init.sh (remember vi) and I added new line with netcfg eth0 dhcp near the end (just before return 0). For fixing up graphics, solution was in editing /mnt/grub/menu.lst (vi again) and appending vga=842 (34A) to kernel line. This gave me resolution of 1152x864x16 and that one worked perfectly for me.

Short reboot later (reboot -f) my new Android was ready.

Setting Up Private Internet Access on CentOS 6.4

Illustration

I am a big fan of Private Internet Access. It gives you anonymous and secure connection to Internet. I personally value my privacy and thus I find such VPN service very valuable.

Under Windows and huge variety of alternate platforms (Android, iOS, Ubuntu, …) installation is very simple and it hardly ever fails. But some platforms don’t come with instructions. Unfortunately one of them is CentOS. Fortunately, setting it all up is not that hard.

First we can do the easy stuff. Download PIA’s OpenVPN configuration files and extract it to directory of your choice. I kept them in /home/MyUserName/pia.

Next easy step is setting up DNS resolving. For that we go to System, Preferences, Network Connections. Just click edit on connection you are using and go to IPv4 Settings tab. Change Method to Automatic (DHCP addresses only). Under DNS servers enter 209.222.18.222 209.222.18.218 (Private Internet Access DNS).

All other commands are to be executed in terminal and most of them require root privileges. It might be best if you just become root for a while:

su - root

CentOS repositories are not known for their extensive software collection. But we can always add a repository of our choice:

wget http://dl.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
rpm -Uvh epel-release-6-8.noarch.rpm

This repository has OpenVPN package that we need:

yum install openvpn

Next step is getting configuration in place (replace username and password with yours):

cp /home/MyUserName/pia/ca.crt /etc/openvpn/ca.crt
cp /home/MyUserName/pia/US\ Midwest.ovpn /etc/openvpn/client.conf
echo "auth-user-pass /etc/openvpn/login.pia" >> /etc/openvpn/client.conf
echo "username" > /etc/openvpn/login.pia
echo "password" >> /etc/openvpn/login.pia

Now we can test our connection (after we restart network in order to activate DNS changes):

service network restart
openvpn --config /etc/openvpn/client.conf

Assuming that this last step ended with Initialization Sequence Completed, we just need to verify whether this connection is actually used. I found whatismyipaddress.com quite helpful here. If you see some mid-west town on map, you are golden (assuming that you don’t actually live in US mid-west).

Now you can stop test connection via Ctrl+C in order to properly start it. In addition, you can specify it should start on each system startup:

service openvpn start
chkconfig openvpn on

And that is all.

CentOS 6.4 and VirtualBox Additions

When you install CentOS 6.4 in VirtualBox, quite quickly you might be annoyed by a lack of a mouse integration. Usual cure in form of VM guest additions simply fails with

Building the main Guest Additions module   [FAILED]

Fortunately this message comes with some additional information which points to lack of compiler and kernel headers. Easiest way to install them is in terminal:

su - root
yum install gcc
yum install kernel-devel-`uname -r`

After this you can retry guest additions installation and you should see better results.

PS: This method probably works for RedHat also.

PostgreSQL on CentOS

It is not really a possibility to be Windows-only developer these days. Chances are that you will end up connecting to one Linux server or another. And best way to prepare is to have some test environment ready. For cold days there is nothing better than once nice database server.

For this particular installation I (again) opted for minimal install of CentOS 6.3. I will assume that it is installed (just bunch of Nexts) and your network interfaces are already set (e.g. DHCP).

First step after this is actually installing PostgreSQL:

yum install postgresql postgresql-server
 Complete!

Next step is initializing database:

service postgresql initdb
 Initializing database:                                     [  OK  ]

Start service and basic setup is done:

chkconfig postgresql on
service postgresql start
 Starting postgresql service:                               [  OK  ]

Next step is allowing TCP/IP connections to be made. For that we need to edit postgresql.conf:

su - postgres
vi /var/lib/pgsql/data/postgresql.conf

There we find listen_addresses and port parameters and un-comment them (along with small change from all to *):

listen_addresses = '*'
port = 5432

While we are at it, we might add all hosts as friends in pg_hba.conf (yes, don’t do this in production):

vi /var/lib/pgsql/data/pg_hba.conf

Add following line at the bottom:

host    all         all         0.0.0.0/0             trust

Finish up editing and restart service

exit
 logout
/etc/init.d/postgresql restart
 Stopping postgresql service:                               [  OK  ]
 Starting postgresql service:                               [  OK  ]

Quick check with psql is in order (notice that \q is used for exit):

psql -h 192.168.56.101 -U postgres -d postgres
 psql (8.4.13)
 Type "help" for help.
\q

If external connections are needed, we must handle firewall. And easiest way to do this is disabling it. For production environment this is a big no-no. For simple testing of virtual machine it will suffice:

/etc/init.d/iptables stop
chkconfig iptables off

And with this we are ready to accept clients.

Simplest LDAP Server

One application I am working on needed LDAP authorization support. In order to test before actually deploying it I decided to create local LDAP server in virtual machine.

I decided to use CentOS minimal install as starting point. It is extremely small distribution to start with and it allows for virtual machine with only 256 MB of RAM (although it needs 512 MB in order to install, go figure).

Installation of CentOS is uneventful. Just go next, next, next and it is done. Although it might be wise to skip media check since it takes ages. In matter of minutes OS will boot up and then the fun starts.

Since we will need network access for both using machine as LDAP server and for getting packages of the Internet, we need network access. Getting it to work is as easy as writing ifup eth0. In order to make these changes permanent just edit /etc/sysconfig/network-scripts/ifcfg-eth0 and change line starting with ONBOOT with ONBOOT="yes". It is as easy (if you disregard annoyance of vi editor).

Now we need to install our directory server. First install package (answer y to everything):

yum install 389-ds-base

And then run setup (answer yes to first two questions and just use default for others):

setup-ds.pl

That should leave us with values totally unsuitable for anything but for testing (which is exactly what we want):

Computer name ...............: //localhost.localdomain//
System User .................: //nobody//
System Group ................: //nobody//
Directory server network port: //389//
Directory server identifier .: //localhost//
Suffix ......................: //dc=localdomain//
Directory Manager DN ........: //cn=Directory Manager//

Quick search will prove that our directory server is up and running

ldapsearch -h 127.0.0.1 -x -b "dc=localdomain"
 ...
 # search result
 search: 2
 result: 0 Success
 # numResponses: 10
 # numEntries: 9

Well, now we are ready to add our first user. In order to do this just create user.ldif file with following content:

dn: uid=jdoe,ou=People,dc=localdomain
objectClass: top
objectClass: person
objectClass: organizationalPerson
objectClass: inetOrgPerson
uid: jdoe
cn: John Doe
displayName: John Doe
givenName: John
sn: Doe
userPassword: test

Not all these attributes are mandatory but I find this to be minimum acceptable set for my use. This is not enough if you want to use LDAP server for logons but it is enough for basic password checking. We add user with:

ldapadd -h 127.0.0.1 -x -D "cn=Directory Manager" -W -f user.ldif
 adding new entry "uid=jdoe,ou=People,dc=localdomain"

If something is messed up, just delete the user and add it again:

ldapdelete -h 127.0.0.1 -x -D "cn=Directory Manager" -W "uid=jdoe,ou=people,dc=localdomain"
ldapadd -h 127.0.0.1 -x -D "cn=Directory Manager" -W -f user.ldif
 adding new entry "uid=jdoe,ou=People,dc=localdomain"

Yes, there is an ldapmodify operation but I find it better to start with clean slate during testing.

Another test to verify that our user authentication works and we are good. Password asked here is not your root LDAP password but password of an user (test in my example):

ldapsearch -h 127.0.0.1 -x -D "uid=jdoe,ou=People,dc=localdomain" -W -b "ou=people,dc=localdomain" "uid=jdoe"
 dn: uid=jdoe,ou=People,dc=localdomain
 objectClass: top
 objectClass: person
 objectClass: organizationalPerson
 objectClass: inetOrgPerson
 uid: jdoe
 cn: John Doe
 displayName: John Doe
 givenName: John
 sn: Doe
 search: 2
 result: 0 Success

Congratulations, you have just made your first LDAP authorization.

Since, in current state, our LDAP cannot talk with outside world, we can think of dropping firewall (not something that you should do in production environment):

iptables -F INPUT
service iptables save

And last step would be to ensure that our directory server gets started as soon as machine is booted up:

chkconfig dirsrv on

With this LDAP test server configuration is done.