KVM live migration with sanlock using virsh

Sanlock helps you avoid screwing things up by starting the same virtual machine on multiple hosts, which will quickly mangle the guest file systems. But you may quickly find KVM live migration no longer works from the virt-manager GUI.

I was in shock when I first saw this migration error from virt-manager!

virt manager error dialog

Internal error cannot use migrate v2 protocol with lock manager sanlock? But alas, all is not lost.

Migration from the virsh command shell works just fine for me. And it’s quicker and more efficient than doing it pointy-clicky style from a gui anyway. If i’m migrating a guest from one host to another, it’s usually because I need to shut down the host hardware. I’m not migrating just one guest, i’m migrating one at a time.

With the shell method, I repeat the migration command in a loop and move them all, one after another automatically. Monday migrate, tuesday migrate, everybody. It’ll get in your bones!

root@host1~# for x in 1 2 3 4 5 6 7 8; do
virsh migrate –live guest$x qemu+ssh://host2/system
done

When I first set up sanlock for qemu locking, I saw errors during migration of several hosts that were already up and running with local locking before sanlock locking was implemented.

error: Failed to inquire lock: No such process

There is no impact to the guest or for users connected to the guest. The end result is the migration doesn’t complete and I found the guest still running on the original host. Other guests migrated without error.

A scheduled reboot of the guests allowed migration to proceed. No reboot or disruption to the existing host was required at all.

Mount SMB on Windows 7 Home Premium

microsoft technet screenshot
Most guides telling you how to mount smb CIFS/Samba shares on Linux to mount on Windows 7 will point you to adjust settings in Administrative Tools -> Local Policies.

Windows 7 Home Premium does not have the Local Policies MMC snap-in. Therefore you cannot use that tool to change the NTLMv2 security settings.

Instead of messing with a snap-in at all, just open the registry editor and set the LmCompatibilityLevel explicitly.

It’s described here, buried in the Lsa control set on technet.microsoft.com.

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Lsa

Add a new DWORD (32-bit) Value, “LmCompatibilityLevel”

Set the value to 1 for the widest compatibility, or a higher value if you want to restrict session security to some combination of only NTLMv2, NTLM, LM.

See the technet page for the chart with descriptions of all the levels:

http://technet.microsoft.com/en-us/library/cc960646.aspx

VM Guest as a NTPD Time Source

blue analog clockIf all I wanted to do was fix the clock on a VM guest, I would have stopped running ntp on the guest and left it running on the underlying physical host. Simple.

But what if you want to use a VM guest as a ntpd time source?! Reasons might be because you’re migrating from a physical server to a VM and don’t have access to the guests to redirect them to another host for whatever reason.

I know this problem is caused in different ways for each virtualization platform. And while my specific problem was using KVM, this should avoid the problem on all of them.

If your host system uses Time Stamp Counter (TSC)

# grep -m1 constant_tsc /proc/cpuinfo
flags  : fpu vme de pse tsc blah blah blah...
constant_tsc
# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
tsc

and Virtual Machine guests use kvm-clock

# cat /sys/devices/system/clocksource/clocksource0/current_clocksource
kvm-clock
# dmesg | grep clock
...
[    0.056001] kvm-clock: cpu 11, msr 0:11975701, secondary cpu clock
[    0.388036] Switching to clocksource kvm-clock
[    0.637349] rtc_cmos 00:01: setting system clock to 2012-01-21 19:18:21 UTC

Do not use ntp on the guest!

But what if you must use a guest as a time source? If the physical host synchronizes to a good Internet time source and the VM guest uses itself (127.0.0.1), what would happen? Well it would still be fighting the clock. So that’s not going to work. I was just hoping to avoid spike messages showing up in client ntp logs, but the clock can skew drastically in either direction.

iptables MASQUERADE to the rescue!

Stop running ntpd on the guest and forward all ntp requests that come in to a physical host serving NTP. You need two rules minimum:

iptables -t nat -A PREROUTING -i eth0 -p udp -m udp --dport 123 -j DNAT --to-destination $HOST
iptables -t nat -A POSTROUTING -o eth0 -p udp -m udp --dport 123 -j MASQUERADE

If you have two interfaces, you can forward the traffic from one network to the other this way too. Just change the -i eth0 to match the other network interface and then allow forwarding:

sysctl net.ipv4.conf.eth0.forwarding=1
sysctl net.ipv4.conf.eth1.forwarding=1

Even if you only have one interface and the ntp server is on the same network, the masquerade should still work.

You should really limit forwarding to ntp for your source and destination too. Default policies of ACCEPT for iptables are bad if you don’t have an explicit rule to drop everything not handled by a higher rule.

# forwarding for ntp requests to your ntp server
iptables -A FORWARD -p udp -d 10.9.8.7/32 --dport 123 -j ACCEPT
# forwarding for responses from your ntp server
iptables -A FORWARD -p udp -d 192.168.1.0/24 --dport 123 -j ACCEPT

Linux Mint is the best Operating System available today

This distribution of Linux is the best setup available today for the desktop.

Since Mint is based on Ubuntu or Debian depending on your choice, the installation and setup is simple and easy. You can even install it while running in live mode or from inside Windows using Ubuntu’s wubi installer.

Where Ubuntu and Fedora have recently shunned almost everyone with their new half-developed unity desktop, this distro shines through. I really enjoy the LXDE environment over Gnome, but the choice is yours to pick and choose what you like to use.

Choice is one of many reasons why I really like this distribution over the other popular flavours right now.

I share the same dissatisfaction for Unity and Gnome 3 with most of the community. While this is the main reason for jumping ship and crashing the party, there are some really simple things I like, like Dropbox.

You dont have to go and download it to start using it, its right there in the start menu. But not installed yet so its not taking up space. This is one of the many reasons why maintaining compatibility with Ubuntu is a good thing. Click on it to start it, and tells you it needs to download. One more click and you’re off.

Once its done, just log in with your account or make a new one, and thats it. I wish all services worked that well.

I think Microsoft is heading in the right direction with swipe gestures for unlocking the desktop, but I hope they take a hint from the recent Gnome debacle and make it configurable. I want the option to jump right into desktop mode without having to bother with metro at all if that’s what I like. For desktop users, forcing you to use metro will be a deal breaker. Remember how bad ME and Vista were?

The sad part is Microsoft doesn’t have a history of giving you much choice in anything and their track record for anything mobile is horrible.

But enough about Windows, especially since is not even available yet. Mint is the choice for desktops today. From the simple, straightforward installer to the fact that everything just works the way it should, it picks up where Ubuntu fails.

Install VirtualBox Guest Additions on Linux Mint

I used to like Ubuntu, before they simultaneously dumbed down the user interface and complicated the administration. The current direction of Ubuntu and Fedora is just plain bad, but that’s a story for another day. Let’s look at our wonderful Ubuntu replacement, Linux Mint.
green lxde logo
When installing Mint, you get a choice of desktop environments. LXDE is fast, lightweight, and uses GTK, which I’m already very familiar with. It’s not bloated like Gnome. It’s everything Gnome wishes it was. If you use Gnome, stop right now and do yourself a favor, install LXDE. Even if you’re hooked on another distribution, there are usually alternative versions available such as Fedora’s LXDE spin. Or just go through package installation of .deb or .rpm files using your favorite package manager. You’ll never look back.

The only problem i’ve had with LXDE and Mint is that running as a guest in VirtualBox, the guest additions will not install properly. I tried mounting the additions from the host and installing them, no dice. I tried installing from the software repositories using the aptitude update manager, no dice.

After attempting an installation from either method, open a terminal and go to the source directory that you just installed. Run make and make install. Voila! Now restart the guest and enjoy the seamless mouse and keyboard integration.

So it didn’t work out of the box for me, but a quick recompile did the trick. No moving around files or manipulating configurations are necessary, just recompile for the running kernel and you’re in business.

Write simple netsnmp apps in Python

Here’s a couple of different ways you can use netsnmp in Python.

I had a hard time finding documentation, and what I did find was old and outdated. I figured most of it out just by playing around with the library.

#!/usr/bin/env python
import netsnmp

string = 'public'
ver = 1
port = 161
host = '192.168.1.1'

# uptime using method 1
bind1 = netsnmp.Varbind('sysUpTime.0')
# 1 minute load using method 2
bind2 = netsnmp.Varbind('.1.3.6.1.4.1.2021.10.1.3.1')

snmpget = netsnmp.snmpget(bind1,
                    Version=ver,
                    RemotePort=port,
                    DestHost=host,
                    Community=string)
uptime_seconds = snmpget[0]
print uptime_seconds

list = ( bind1, bind2 )
x = netsnmp.Session(DestHost=host,
                    Version=ver,
                    RemotePort=port,
                    Timeout=400000,
                    Retries=5,
                    Community=string)
output_list = x.get(list)
if not output_list:
    print "FAILED TO CONNECT!!!"
    sys.exit(1)

if output_list[0]:
    uptime = output_list[0]

if output_list[1]:
    load1 = output_list[1]

python blue and yellow logoNext I wrote a class to wrap it up as an AWN applet. If you’ve never heard of AWN or haven’t tried the avant-window-navigator you should definitely check it out and consider continuing development on it. It was the best app bar available; very pretty. It fit my needs at the time anyway.

I replaced the bottom gnome-panel with it. If you remove everything except the Launcher/Taskmanager applet and add the Show Desktop applet, it directly replaces gnome-panels functionality completely.

Gnome 3 Fallback Desktop Better than Gnome 3 itself

I don’t like the Gnome 3 desktop.

angry dude cartoon character
If all you use a computer for is facebook and instant messaging, you’ll probably love it. Gnome 3 desktop hides everything from you under multiple layers of mouse clicks in order to try to simplify the user experience. But what it actually does is over categorize everything!

I don’t like to have to constantly click-click-click to get to where I’m going. I don’t care how easy it is to respond to instant messages as they pop up along the bottom of my screen, I don’t like this new interface at all! This would work just fine on a tablet pc, but I’m on a desktop or full laptop most of the time.

If you’re like me, you try out new .iso image distributions in virtual machines. It’s easier to install, reinstall, and play with different settings all while easily resetting back to a fresh installation with a click of the revert-to-snapshot button.

The only downfall is that I don’t get to fully utilize the pow-wah of my graphics cards. I’m currently using a pair of NVidia GTX 470′s, but inside a Virtual Machine the host is using a slow Virtualbox software driver. Gnome 3 doesn’t even attempt to play with my slow non-3D accelerated graphics.

So it drops to a fallback desktop… that rocks!!

I like Gnome 3′s Fallback Desktop!

Ahh that warm fuzzy comfy feeling I get from the familiar surrounding of Gnome panels along the top and bottom of my desktop. But it’s not just because I’m used to it and know where everything is, it’s the usability. It’s all right there just like before, only now it looks more polished.

In order to try out the fallback desktop, you’ll have to go dig around in the new interface.

1. Click “Activities”
2. Change to the “Applications” tab near the top.
3. Click “System Tools” along the right, near the bottom.
4. Click the button “System Settings”, in the middle area.

Timeout!!!
Do you see how this is going? Jump around here, hunt and peck there. Am I supposed to be feeling productive just because I had to jump all over the place to get a single action performed? Continuing on…

5. System Info
6. Graphics
7. Forced Fallback

The image i’m playing with is the Fedora 15 iso available at http://www.gnome3.org/tryit.html . Your Gnome 3 fallback desktop may not have panels already started, but the methods are the same as Gnome 2. Take a look in ~/.gconf and ~/.gnome2 for the xml files too.

File Check Hash Generator – Recursive Tripwire

finger pointing at security textYou can use this to check to see if anyone has modified, updated, upgraded, added, or removed any files on your system. After you’ve configured a system the way you want it, dump hash files for all the important directories, /etc, /bin, /usr/local, etc., or just dump the whole thing. Move the output to another system. Now if you want to check to see if something has changed, you can hash the file(s) in question and grep for the hash.

A directory like /etc has many subdirectories with subdirectories of their own – not a problem. When the script encounters a directory, it recursively calls itself so it will parse all child directories. Skipping special files should avoid the problem of probing char files, proc, and other gotchas. know it could be better. There’s things like pid files that are useless to hash.

This was just a quick stab at it. Feel free to adapt this to your own needs as you see fit.

Bash script:

#!/bin/bash
md5sum=/usr/bin/md5sum # hash algorithm to use
mkdir=/bin/mkdir
indir=${1} # base input directory to start hashing files
outfile=${2} # full path of output file

if [ "${indir}" == "" -o "${outfile}" == "" ]; then
  echo "Usage: $0  "
  echo "  ex: $0 /etc /root/etc.hash"
  exit 1
fi

for x in `ls "${indir}"`; do
  if [ -d ${indir}/$x ]; then # is a dir
    echo "[ Recursively hashing ${indir}/$x ]"
    $0 ${indir}/$x ${outfile} # pass new path in
    if [ $? != 0 ]; then # recursive call failed, die
      echo "Could not hash ${indir}/$x"
      exit 1
    fi
  else # is not a dir
    if [ -f ${indir}/$x ]; then # regular files only
      ${md5sum} "${indir}/$x" >> "${outfile}"
    fi
  fi
done

exit 0