Slow Ansible Response Using External IP

As I was playing with Ansible’s ad-hoc commands, I’ve noticed that one host was struggling. Any command I tried took 15 seconds more for it than for any other host.

ansible all -a date

Adding IP to /etc/hosts actually solved that trouble which strongly indicated toward issue actually being connected to DNS setup. But, why the hell was my temporary lab even using DNS? It was just a bunch of virtual machines and I definitely didn’t want to add all hosts by name. I needed another solution…

After a bit of investigation, I discovered the actual culprit - SSH daemon was trying to resolve IP address. For which purpose you ask? Just to write host name to its log.

The final solution was easy - just adding UseDNS no to /etc/ssh/sshd_config:

echo -e "\nUseDNS no" | sudo tee -a /etc/ssh/sshd_config'
sudo systemctl restart sshd

VPN-only Internet Access on Linux Mint 18.3 Via Private Internet Access

Setting up Private Internet Access VPN is usually not a problem these days as Linux version is readily available among the supported clients. However, such installation requires GUI. What if we don’t want or need one?

For setup to work independently of GUI, one approach is to use OpenVPN client usually installed by default. Also needed are PIA’s IP-based OpenVPN configuration files. While this might cause issues down the road if that IP changes, it does help a lot with security as we won’t need to poke an unencrypted hole (and thus leak information) for DNS.

From the downloaded archive extract .crt and .pem files followed by your choice of .ovpn file (usually going with the one physically closest to you). Copy them all to your desktop to be used later. Yes, you can use any other directory - this is just the one I’ll use in example commands below.

Rest of the VPN configuration needs to be done from the Bash (replacing username and password with actual values):

sudo mv ~/Desktop/*.crt /etc/openvpn/
sudo mv ~/Desktop/*.pem /etc/openvpn/
sudo mv ~/Desktop/*.ovpn /etc/openvpn/client.conf

sudo sed -i "s*ca *ca /etc/openvpn/*" /etc/openvpn/client.conf
sudo sed -i "s*crl-verify *crl-verify /etc/openvpn/*" /etc/openvpn/client.conf

sudo echo "auth-user-pass /etc/openvpn/client.login" >> /etc/openvpn/client.conf
sudo echo "mssfix 1400" >> /etc/openvpn/client.conf
sudo echo "dhcp-option DNS 209.222.18.218" >> /etc/openvpn/client.conf
sudo echo "dhcp-option DNS 209.222.18.222" >> /etc/openvpn/client.conf
sudo echo "script-security 2" >> /etc/openvpn/client.conf
sudo echo "up /etc/openvpn/update-resolv-conf" >> /etc/openvpn/client.conf
sudo echo "down /etc/openvpn/update-resolv-conf" >> /etc/openvpn/client.conf

unset HISTFILE
echo '^^username^^' | sudo tee -a /etc/openvpn/client.login
echo '^^password^^' | sudo tee -a /etc/openvpn/client.login
sudo chmod 500 /etc/openvpn/client.login

To test VPN connection execute:

sudo openvpn --config /etc/openvpn/client.conf

Assuming test was successful (i.e. resulted in Initialization Sequence Completed message), we can further make sure data is actually traversing VPN. I’ve found whatismyipaddress.com quite helpful here. Just check if IP detected is different then IP you usually get without VPN.

Stop the test connection using Ctrl+C and proceed to configure OpenVPN’s auto-startup:

echo "AUTOSTART=all" | sudo tee -a /etc/default/openvpn
sudo reboot

Once computer has booted and no further VPN issues have been observed, you can also look into disabling the default interface when VPN is not active. Essentially this means traffic is either going through VPN or not going at all.

Firewall rules are to allow data flow only via VPN’s tun0 interface with only encrypted VPN traffic being allowed on port 1198.

sudo ufw reset
sudo ufw default deny incoming
sudo ufw default deny outgoing
sudo ufw allow out on tun0
sudo ufw allow out on `route | grep '^default' | grep -v "tun0$" | grep -o '[^ ]*$'` proto udp to `cat /etc/openvpn/client.conf | grep "^remote " | grep -o ' [^ ]* '` port 1198
sudo ufw enable

This should give you quite secure setup without the need for GUI.

Monitoring Certificate Expiration

Once you get Let’s Encrypt certificate setup, there are two more things needed. First one is setting up renewal as our certificates don’t last more than 90 days. The second one is often overlooked - actually monitoring how long before certificate expires. If anything prevents your certificate renewing, you definitely want to know it.

My approach to this problem is introducing an extra step in my daily e-mail report (I will assume here you have one setup already). This bash code will connect to a server, enumerate all certificates within /etc/letsencrypt/ directory, extract their name, and give an extra warning if certificate is expiring in less than 15 days.

Without the further ado, here is the code excerpt:

NOW=`date +%s`

PEMS=`ssh ^^myuser^^@^^myserver.example.com^^ find /etc/letsencrypt/ -name "cert.pem" -print`
for PEM in $PEMS
do
  NAME=`echo $PEM | rev | cut -d'/' -f2 | rev`
  EXPIRY_RAW=`ssh ^^myuser^^@^^myserver.example.com^^ openssl x509 -enddate -noout -in "$PEM" | cut -d= -f 2`
  EXPIRY=`date -jf "%b %d %T %Y %Z" "$EXPIRY_RAW" "+%s"`
  REMAINING=$(( EXPIRY - NOW ))
  REMAINING_DAYS=$(( REMAINING / 86400 ))

  if (( REMAINING_DAYS >= 15 ))
  then
    echo "• $NAME expires in $REMAINING_DAYS days"
  else
    if (( REMAINING_DAYS < 0 ))
    then
      echo "‼ $NAME expiry cannot be determined"
    else
      echo "‼ $NAME expires in $REMAINING_DAYS days"
    fi
  fi
done

TimeSpan and HH:mm

Time printing and parsing is always “fun” regardless of language but, for most of time C#, actually has it much better than one would expect. However, sometime one can get surprised by small details.

One application had the following (extremely simplified) code writing time entries:

var text = someDate.ToString("HH:mm", CultureInfo.InvariantCulture);

Other application had the (also simplified) parsing code:

var someTime = TimeSpan.ParseExact(text, "HH:mm", CultureInfo.InvariantCulture);

And all I got was System.FormatException: 'Input string was not in a correct format.'

Issue was not in text, all entries were valid time. Nope, issue was in a slight difference between custom DataTime and custom TimeSpan specifiers.

Reading documentation it’s really easy to note the first error as these is a clear warning: “The custom TimeSpan format specifiers do not include placeholder separator symbols.” In short that means we cannot use colon (:) as a TimeSpan separator. Nope, we need to escape it using backslash (\) character. I find this one mildly annoying as not implementing time separators into TimeSpan seems like a pretty basic functionality.

But there is also a second error that’s be invisible if you’re not watching carefully. TimeSpan has no knowledge of H specifier. It understands only its lowercase brother h. I would personally argue this is wrong. If any of those two had to be used, it should have been H as it denotes time more uniquely.

In any case, the corrected code was as follows:

var someTime = TimeSpan.ParseExact(text, "hh\\:mm", CultureInfo.InvariantCulture);

Ideally one would always want to use the same format for output as the input but just because semantics of one class are the same as the other for your use case, you cannot count of format specifier to be the same.

Webalizer With a Dash of Logrotate

Recently I decided to stop using Google Analytics for my website traffic analysis. Yes, it’s still probably the best analytics tool out there but I actually don’t care about details that much. The only thing I care about is trend - is site getting better or worse - and nothing much else. For that purpose, a simple Webalizer setup would do.

My Webalizer setup was actually as close to plain as it gets. The only trick up my sleeve was that I had a separate configuration file for each of my sites. Of course, since I use logrotate to split my Apache traffic logs, I also needed to add a bit of prerotate action into the /etc/logrotate.conf to ensure I don’t miss any entries.

My first try was this:

…
  prerotate
    for FILE in /srv/configs/**/webalizer.conf
    do
      /usr/bin/webalizer -c $FILE
    done
  endscript
…

And, while this did wonders from command line, it seemed to do absolutely nothing when executed by logrotate itself. Reason for its misbehavior was that logrotate (and crontab) uses sh as its shell and not bash.

To get around this, a Posix-compatible command is needed:

…
  prerotate
    find /srv/configs/ -name "webalizer.conf" -exec /usr/bin/webalizer -c {} \;
  endscript
…

This now executes just before my Apache log file gets rotated-out and Webalizer gets one last chance to catch-up with its stats.