Managing KVM instances

At May First/People Link we have been using KVM for several years now and recently I have been running KVM instances on my local laptop.

I’m pleased to see all the work that has gone into libvirt, which seems like a robust and full-featured suite of tools for managing many virtualization technologies, including KVM. However, we don’t use it at May First/People Link for a number of reasons. The most pressing is that it runs all virtual guests as the same user, but also because it offers far more features than we need (such as graphical access to virtual server, which we don’t need since none of our guest servers run X).

On May First/People Link hosts, we are using a relatively simple set of bash scripts (accessible via git at git://lair.fifthhorseman.net/~dkg/kvm-manager). These scripts re-use many tools we are already familiar with to build and launch kvm guests. Each guests runs as a dedicated non-provileged user, with a console available using screen, and the kvm process is managed using runit. Since our admins are familiar with these tools already, the learning curve involved is much less steep.

Despite the relative simplicity of kvm-manager, it was still more complicated and involved than I wanted on my laptop. Additionally, I wanted to fully understand every piece of the puzzle and separating out user privileges wasn’t important to me.

So - I wrote the a bash script to launch virtual servers. It assumes you are using logical volume manager.

Some editing required if you want to re-use it. You can presuse it below or download it.

# !/bin/bash
# Manage a virtual server

# This script assumes you are using Logical Volume Manager (LVM)
#
# There are changes to your system that you need to make once to get this
# system working. Once you have made these changes you are done.  There are
# other steps you have to take everytime you add a new guest to your
# system.
#
# ONE TIME CHANGES
#
# Install necessary packages 
#
# sudo apt-get install qemu-kvm screen bridge-utils dnsmasq
#
# For networking to properly work, your kernel must allow packet forward.
#
# You can enable packet forwarding by adding the file /etc/sysctl.d/local.conf
# with the contents:
#
# # used for networking kvm instance
# net.ipv4.ip_forward=1
#
# When you restart your computer, this change will effect. Or, you can run the
# following command to get it to take effect right away:
#
# sudo -i    # to become root
# echo 1 > /proc/sys/net/ipv4/ip_forward
# exit    # to return to being a normal user
#
# You must modify your /etc/network/interfaces file and add the following
# stanza:
#
# auto virbr0 
# iface virbr0 inet static 
#  address 10.11.13.1 
#  netmask 255.255.255.0 
#  pre-up brctl addbr virbr0 
#  post-down brctl delbr virbr0
#
# Then, bring up the interface with:
# 
# sudo ifup virbr0
# 
# Next configure dnsmasq by creating the file: /etc/dnsmasq.d/local with the following
# content:
#
# interface=virbr0 
# dhcp-range=10.11.13.2,10.11.13.100,1h
#
# Restart dnsmasq for the changes to take place:
#
# sudo service dnsmasq restart
#
# In order to run your virtual server as a non-privileged user (e.g. your
# normal user) you will need to make a change to your system so that your
# newly created logical volume (and all future logical volumes) will be
# owned by your user. 
# 
# Add a file called /etc/udev/rules.d/92-kvm.rules with the following line
# (change "jamie" to the group you are running as and vg_kermit0 to the
# name of your volume group). If you are not sure the name of your volume
# group type: sudo vgs.
#
# ACTION=="change", SUBSYSTEM=="block", ATTR{dm/name}=="vg_kermit0-*_root", GROUP="jamie"
#
# Lastly, you will need to download a Debian installer ISO to your file
# system.
# 
# Example command to download the full installer (CD 1):
# wget http://cdimage.debian.org/debian-cd/6.0.5/amd64/iso-cd/debian-6.0.5-amd64-CD-1.iso
# 
# Example command to download the net installer (smaller download, will use 
# the Internet to download needed packages):
#
# wget http://cdimage.debian.org/debian-cd/6.0.5/amd64/iso-cd/debian-6.0.5-amd64-netinst.iso
#
# REPEAT ONCE FOR EACH GUEST
#
# Now, create a new logical volume for your virtual server.  You need to
# repeat this step for every virtual server you create. This example
# assumes that you are creating a virtual server named gonzo and that your
# logical volume group name is vg_kermit0 (if you are not sure what your
# logical volume group is named, try typing the command: sudo vgs).
#
# sudo lvcreate --name gonzo_root --size 15GB vg_kermit0
#
# Finally, assuming your ISO is stored in /home/jamie/ISOs/debian.iso,
# type: 
#
# ./vlaunch gonzo start /home/jamie/ISOs/debian.iso 
#
# and you are ready to go. 
# 
# This form of the command will start your virtual server with the Debian
# installer passed to it and you should be prompted through the
# installation.
#
# After you have installed Debian, you can start it with simply:
#
# vlaunch gonzo
#
# If you want to clean up the networking devices created (after you have
# shutdown your virtual server) you can do that with:
#
# vlaunch gonzo cleanup
#
# Be sure to edit the variables below to match your system:

bridge=virbr0
user=jamie
vg=vg_animal0
# Change graphic to 0 if you want to launch this via screen
graphic=1
# Change configure_nat to 0 if you want to handle your nat creation
# on your own (e.g. via /etc/network/ifup.d/
configure_nat=1

# Modify server memory here. Depending on how much memory you have for your entire
# system you may want to raise or lower this number
mem=512

# Create a function that will echo a variable passed in and then exit the script
die () {
  printf "$1\n"
  exit 1
}

# This is the function that will be called if we are starting a virtual server
function start() {
  if [ "$configure_nat" -eq 1 ]; then
    # Get the name of the current network device
    dev=$(ip route | grep ^default | grep -oE "dev [a-z0-9]+" | sed "s/dev //")

    if [ -n "$dev" ]; then
      # Flush the nat table to avoid duplicates
      sudo iptables --table nat -F

      # Create a NAT (network address translation rule)
      sudo iptables --table nat -A POSTROUTING ! -d 127.0.0.1/8 --out-interface "$dev" -j MASQUERADE
    else
      printf "I could not determine your network device. Not configuring NAT.\n"
    fi
  fi
  lvname="${vg}-${server}_root"

  # Trigger udev to ensure we have proper ownership of the block device.
  sudo udevadm trigger --subsystem-match=block --attr-match=dm/name="$lvname"

  lv="/dev/mapper/$lvname"
  [ ! -e "$lv" ] && die "Can't find $lv"

  sudo modprobe -v tun || die "Failed to modprobe tun module"

  # Create network device if it doesn't already exist.
  ip tuntap | grep "$tap" >/dev/null || sudo ip tuntap add dev "$tap" mode tap user "$user" || die "Failed to create device $tap"
  # Bring up device if it's not already up.
  ip link | grep " $tap " >/dev/null || sudo ip link set "$tap" up || die "Failed to set $tap to up"
  # Add the device to the bridge so it get use the upstream network connections.
  /sbin/brctl show | grep "$tap" > /dev/null || sudo brctl addif "$bridge" "$tap" || die "Failed to add tap to bridge"

  # Launch kvm.
  screen=
  nographic=
  if [ "$graphic" -eq 0 ]; then
    # Launch with -nographic in a screen session
    screen="screen -S $server"
    nographic="-nographic"
  fi

  if [ "$command" = "show" ]; then
    printf "Here is the command that would be executed:\n"
    echo kvm -drive "file=$lv,if=virtio,id=hda,format=raw" -m "$mem" -device "virtio-net-pci,vlan=1,id=net0,mac=$mac,bus=pci.0" -net "tap,ifname=$tap,script=no,downscript=no,vlan=1,name=hostnet0" $cdarg $nographic 
  else
    $screen kvm -drive "file=$lv,if=virtio,id=hda,format=raw" -m "$mem" -device "virtio-net-pci,vlan=1,id=net0,mac=$mac,bus=pci.0" -net "tap,ifname=$tap,script=no,downscript=no,vlan=1,name=hostnet0" $cdarg $nographic || die "Failed to start kvm" 
  fi
}

# This is the function we will call to cleanup.
function cleanup() {
  read -p "Please shutdown the host first then hit any key to continue..."
  sudo brctl delif "$bridge" "$tap"
  sudo ip link set "$tap" down
  sudo ip tuntap del mode tap dev "$tap"
}

# Start the main program logic.
# The first argument passed to the script is $1 - resave as the variable $server
server="$1"

# Second argument is the command
command="$2"
# If no command is passed, assume we are starting.
if [ -z "$command" ]; then
  command=start
fi

# By default cdarg variable is left empty
cdarg=

# If a third variable is passed, it means they are passing an ISO image.
if [ -n "$3" ]; then
  # Make sure it exists.
  [ ! -f "$3" ] && die "Third argument should be path to cd iso. Can't find that path."
  cdarg="-cdrom $3"
fi


# Generate reproducible mac address.
mac="$(printf "02:%s" "$(printf "%s\0%s" "$(hostname)" "${server}" | sha256sum | sed 's/\(..\)/\1:/g' | cut -f1-5 -d:)" )"
tap="${server}0"

if [ "$command" = "start" ] || [ "$command" = "show" ]; then
  start "$command"
elif [ "$command" = "cleanup" ]; then
  cleanup
else
  die "Please pass start or cleanup as first argument. You passed: $command"
fi