| Chapter 4. Helper Utilities | ||
|---|---|---|
 |  | |
| Chapter 4. Helper Utilities | ||
|---|---|---|
| | | |
Table of Contents
Up until now most of the commands we've been using have been very low-level. Just the section on resizing images [34] is about 8 pages of this book (depending on what format you're reading it in). Let's get real here: it's not pragmatic to run ten commands when one or two will suffice. Luckily for us some very helpful utilities exist.
This section will introduce those utilities. I'll highlight some key features in each, show demos, and tell you where you can find more information. Let's get started by introducing our new heros using their official descriptions.
libguestfs is a way
to create, access and modify disk images. You can look
inside disk images, modify the files they contain, create
them from scratch, resize them, and much more. It's
especially useful from scripts and programs and from the
command line.
The “Virtual Machine Manager” application (virt-manager for short package name) is a desktop user interface for managing virtual machines. It presents a summary view of running domains, their live performance & resource utilization statistics. The detailed view graphs performance & utilization over time. Wizards enable the creation of new domains, and configuration & adjustment of a domain's resource allocation & virtual hardware. An embedded VNC client viewer presents a full graphical console to the guest domain.
libguestfs make managing
virtual disks (and machines) a lot simpler. Included is a C
library (with bindings available for Perl, Python, Ruby, Java,
OCaml, PHP, Haskell, Erlang, Lua and C#), as well as a collection
of 34 utilities (at the time of writing).
I won't even attempt to cover all of it's features in this
book. Instead, I'll go over some of the most useful utilities. For
more information on libguestfs you should go to the
project website
[35]
where they have a complete 250 page manual fully describing all
aspects of libguestfs.
The guestmount program can be used to mount virtual machine filesystems and other disk images on the host. It uses libguestfs for access to the guest filesystem, and FUSE (the "filesystem in userspace") to make it appear as a mountable device. | ||
| --man 1 guestmount | ||
TODO: What about LVM bits spanning across multiple disks?
foo
This tool allows you to discover filesystems, partitions, logical volumes, and their sizes in a disk image or virtual machine. | ||
| --man 1 virt-filesystems | ||
virt-filesystems is the Sherlock Holmes [36] of virtual disk management. What delights me most about virt-filesystems is how well it integrates with LVM (Logical Volume Manager) to show you LVM device paths. This tool is most useful in combination with other tools, such as virt-resize, virt-sparsify, or guestmount.
TODO: link to related sections
virt-rescue is like a Rescue CD, but for virtual machines, and without the need for a CD. virt-rescue gives you a rescue shell and some simple recovery tools which you can use to examine or rescue a virtual machine or disk image. | ||
| --man 1 virt-rescue | ||
virt-resize is a tool which can resize a virtual machine disk, making it larger or smaller overall, and resizing or deleting any partitions contained within. | ||
| --man 1 virt-resize | ||
virt-sparsify is a tool which can make a virtual machine disk (or any disk image) sparse a.k.a. thin-provisioned. This means that free space within the disk image can be converted back to free space on the host. | ||
| --man 1 virt-sparsify | ||
Depending on your deployment strategy the virt-sparsify command could potentially save you a lot of disk space. This is especially the case in “cloud”-type setups where new machines are commonly created from a single “golden-master” image. If you're making copies of any disk image then you need to make sure that you aren't unnecessarily wasting space on your disk.
That's where virt-sparsify comes in. When you sparsify a disk image (or any other file for that matter) you're potentially reducing the number of blocks on the backing storage volume [37] which are allocated to the disk image. This frees up room on the backing volume for storing other files. Sparsifying a disk image is only effective as there is space that can be freed. More on that next.
![[Important]](images/important.png) | Important |
|---|---|
Sparsifying files doesn't divorce you from the inherent size limitations of your backing storage volume. I.e., you can not expect to fill two 100GiB sparse disk images with data if the volume they're stored on is only 50GiB. |
In this example I'll sparsify the disk image we cloned from a thumbdrive earlier in the book [38] . Let's start by using the qemu-img info subcommand [39] to see how much total space the disk image has allocated to it presently:
# qemu-img info ./thumb_drive.raw image: ./thumb_drive.raw file format: raw virtual size: 966M (1012924416 bytes) disk size: 914M
The output you want to take note of here is on the disk
size line: 914M. For a thumb drive that only had two
small text files on it, we sure are wasting a lot of space. Let's
attempt to reclaim that space with
virt-sparsify. We'll call
virt-sparsify with two parameters, the first is
the source disk name, thumb_drive.raw, and
the second is the name of the sparsified disk image we're going to
create, thumb_drive_sparse.raw:
Example 4.1. Sparsify a disk image
# virt-sparsify ./thumb_drive.raw ./thumb_drive_sparse.raw Create overlay file to protect source disk ... Examine source disk ... 100% [...] 00:00 Fill free space in /dev/vda1 with zero ... 100% [...] --:-- Fill free space in /dev/vda2 with zero ... 100% [...] --:-- Copy to destination and make sparse ... Sparsify operation completed with no errors. Before deleting the old disk, carefully check that the target disk boots and works correctly.
As you can see, the output from virt-sparsify is straightforward and easy to grok [40] . Right away we can tell that virt-sparsify is protecting our source file from undesired modifications by creating an “overlay file”. Next it examines the disk, identifying entities such as partition tables, LVM volumes, and space to be freed. Then the freeable space is zeroed out.
Remember that we haven't modified the source image yet! All of the potential changes were made to an overlay file. The final step to sparsify the file is combining the delta present in the overlay file with the source file and writing the result out to disk. Observe the following important note from the virt-sparsify man page:
| Important |
|---|---|
virt-sparsify may require up to 2x the virtual size of the source disk image (1 temporary copy + 1 destination image). This is in the worst case and usually much less space is required. |
Let's use qemu-img info again and examine the
sparsified disk image
(thumb_drive_sparse.raw). Recall that we're
primarily concerned with the disk size field
and that the starting size was 914M:
# qemu-img info ./thumb_drive_sparse.raw image: ./thumb_drive_sparse.raw file format: raw virtual size: 966M (1012924416 bytes) disk size: 6.2M
From this we can see that after the image was sparsified the allocated space is only 6.2M. That's a net savings of 907.8M! Don't let this result give you unreasonable expectations though. This example demonstrated an ideal case, where the source disk was virtually 100% empty to begin with.
virt-sparsify has other options available as well. For example, it can convert between formats (e.g., vdmk to qcow2), ignore specific filesystems on the source disk, or enable compression in qcow2 images. Read the man page for a complete description of all the available options.
[35] libguestfs homepage: http://libguestfs.org/
[36] Sherlock Holmes is a fictional detective. Read some of the books online for free on Project Gutenberg: http://www.gutenberg.org/ebooks/1661
[37] The phrase “backing storage volume” refers to the actual storage device which the disk image is saved on.
[39] We could also use the ls -lsh
command. The -s option prints the
allocated size (actually used space), and
the -h options prints sizes in “human
readable” formats, e.g., 915M or 4.0K
[40] grok - verb - "To understand. Connotes intimate and exhaustive knowledge". Source: http://www.catb.org/jargon/html/G/grok.html
| | | |
| 3.2. Devices and Partitions |  | 4.2. virt manager |