(July 2017)
virsh is the primary linux CLI tool for dealing with virtual machines.
What is virsh? How does it fit into linux virtualization?
libvirtd daemon, and the virsh CLI utility) for managing virtual machines (and storage pools and some network stuff).virsh command-line utility is one way of interacting with libvirt.See VIRSH(1).
# virsh list --all
Id Name State
----------------------------------------------------
- debian shut off
- freebsd-512ram-1cpu-20gb shut off
- openbsd shut off
- openbsd59 shut off
- rhel7 shut off
Edit a guest/domain configuration:
# virsh edit my-vm
Starting, shuting down, rebooting, suspending/pausing, resuming/unpausing, and pulling the plug on a domain:
# virsh start my-vm
# virsh shutdown my-vm
# virsh reboot my-vm
# virsh suspend my-vm
# virsh resume my-vm
# virsh destroy my-vm
Set a guest to autostart or not autostart:
# virsh autostart my-vm
# virsh autostart --disable my-vm
# virsh list --autostart
Take a snapshot of a domain, list snaphots, restore/revert to a snapshot, and delete a saved snapshot:
# virsh snapshot-create my-vm
# virsh snapshot-list my-vm
# virsh snapshot-restore my-vm snapshot-1
# virsh snapshot-delete my-vm snapshot-2
http://libvirt.org/formatstorage.html
Libvirt provides storage management on the physical host through storage pools and volumes.
A storage pool is a quantity of storage set aside by an administrator, often a dedicated storage administrator, for use by virtual machines. Storage pools are divided into storage volumes either by the storage administrator or the system administrator, and the volumes are assigned to VMs as block devices.
For example, the storage administrator responsible for an NFS server creates a share to store virtual machines’ data. The system administrator defines a pool on the virtualization host with the details of the share (e.g. nfs.example.com:/path/to/share should be mounted on /vm_data). When the pool is started, libvirt mounts the share on the specified directory, just as if the system administrator logged in and executed ‘mount nfs.example.com:/path/to/share /vmdata’. If the pool is configured to autostart, libvirt ensures that the NFS share is mounted on the directory specified when libvirt is started.
Once the pool is started, the files in the NFS share are reported as volumes, and the storage volumes’ paths may be queried using the libvirt APIs. The volumes’ paths can then be copied into the section of a VM’s XML definition describing the source storage for the VM’s block devices.
https://libvirt.org/storage.html
Although all storage pool backends share the same public APIs and XML format, they have varying levels of capabilities. Some may allow creation of volumes, others may only allow use of pre-existing volumes. Some may have constraints on volume size, or placement.
The top level tag for a storage pool document is ‘pool’. It has a single attribute type, which is one of dir, fs, netfs, disk, iscsi, logical, scsi (all since 0.4.1), mpath (since 0.7.1), rbd (since 0.9.13), sheepdog (since 0.10.0), gluster (since 1.2.0), zfs (since 1.2.8) or vstorage (since 3.1.0).
# virsh pool-list --all --details
Name State Autostart Persistent Capacity Allocation Available
----------------------------------------------------------------------------------
data-libvirt running yes yes 468.45 GiB 179.47 GiB 288.98 GiB
default running yes yes 422.38 GiB 213.00 GiB 209.39 GiB
Downloads running yes yes 422.38 GiB 213.00 GiB 209.39 GiB
# virsh pool-info virt
Name: virt
UUID: aa52fbc0-2088-43ce-9618-3a7993cb45e4
State: running
Persistent: yes
Autostart: yes
Capacity: 468.45 GiB
Allocation: 234.22 GiB
Available: 234.23 GiB
# virsh vol-list --pool virt-lvm-pool
Name Path
------------------------------------------------------------------------------
home /dev/falstaff-vg/home
lv-openbsd-30gb /dev/falstaff-vg/lv-openbsd-30gb
lv-pi2-test /dev/falstaff-vg/lv-pi2-test
root /dev/falstaff-vg/root
swap_1 /dev/falstaff-vg/swap_1
# virsh pool-define-as test --type dir --target /data/virt
# virsh pool-start test
# virsh pool-autostart test
# virsh pool-dumpxml virt
<pool type='dir'>
<name>test</name>
<uuid>aa52fbc0-2088-43ce-9618-3a7993cb45e4</uuid>
<capacity unit='bytes'>502996557824</capacity>
<allocation unit='bytes'>251489689600</allocation>
<available unit='bytes'>251506868224</available>
<source>
</source>
<target>
<path>/data/virt</path>
<permissions>
<mode>0755</mode>
<owner>1000</owner>
<group>1000</group>
</permissions>
</target>
</pool>
# virsh pool-destroy test
Pool test destroyed
# virsh pool-create /etc/libvirt/storage/test.xml
Pool test created from /etc/libvirt/storage/test.xml
# virsh pool-destroy test
Pool test destroyed
# virsh pool-undefine test
Pool test has been undefined
# virsh find-storage-pool-sources logical
<sources>
<source>
<device path='/dev/md0'/>
<name>vg0</name>
<format type='lvm2'/>
</source>
</sources>
# sudo vgscan
Reading all physical volumes. This may take a while...
Found volume group "vg0" using metadata type lvm2
# vgs --all
VG #PV #LV #SN Attr VSize VFree
vg0 1 7 0 wz--n- 2.73t 1.91t
$ ls /dev | grep vg0
vg0
# virsh pool-define-as --name vg0 --type logical --target /dev/vg0
# virsh pool-start vg0
Pool vg0 started
# virsh pool-autostart vg0
Pool vg0 marked as autostarted
# virsh pool-list --all --details
Name State Autostart Persistent Capacity Allocation Available
---------------------------------------------------------------------------
default running yes yes 110.00 GiB 9.70 GiB 100.30 GiB
vg0 running yes yes 2.73 TiB 841.56 GiB 1.91 TiB
# virsh vol-create-as vg0 myvm.img 10G
# virsh vol-delete --pool vg0 myvm.img
# lvs
LV VG Attr LSize Pool Origin Data% Move Log Copy% Convert
2012r2 vg0 -wi-a--- 80.00g
myvm.img vg0 -wi-a--- 10.00g
lvroot vg0 -wi-ao-- 111.76g
lvswap vg0 -wi-ao-- 29.80g
smbshare vg0 -wi-ao-- 450.00g
virtbackup vg0 -wi-ao-- 320.00g
# vgs
VG #PV #LV #SN Attr VSize VFree
vg0 1 8 0 wz--n- 2.73t 1.63t
VIRT-INSTALL(1)
--os-variant OS_VARIANT
Optimize the guest configuration for a specific operating system (ex. 'fedora18', 'rhel7', 'winxp').
While not required, specifying this options is HIGHLY RECOMMENDED, as it can greatly increase performance by specifying virtio among other guest tweaks.
Use the command "osinfo-query os" to get the list of the accepted OS variants.
# apt-get install libosinfo-bin
$ osinfo-query os
Short ID | Name | Version | ID
----------------------+----------------------------------------------------+----------+-----------------------------------------
centos7.0 | CentOS 7.0 | 7.0 | http://centos.org/centos/7.0
debian9 | Debian Stretch | 9 | http://debian.org/debian/9
debiantesting | Debian Testing | testing | http://debian.org/debian/testing
fedora25 | Fedora 25 | 25 | http://fedoraproject.org/fedora/25
freebsd11.0 | FreeBSD 11.0 | 11.0 | http://freebsd.org/freebsd/11.0
openbsd5.8 | OpenBSD 5.8 | 5.8 | http://openbsd.org/openbsd/5.8
rhel6.8 | Red Hat Enterprise Linux 6.8 | 6.8 | http://redhat.com/rhel/6.8
rhel7.0 | Red Hat Enterprise Linux 7.0 | 7.0 | http://redhat.com/rhel/7.0
rhel7.1 | Red Hat Enterprise Linux 7.1 | 7.1 | http://redhat.com/rhel/7.1
rhel7.2 | Red Hat Enterprise Linux 7.2 | 7.2 | http://redhat.com/rhel/7.2
ubuntu16.04 | Ubuntu 16.04 | 16.04 | http://ubuntu.com/ubuntu/16.04
ubuntu14.04 | Ubuntu 14.04 LTS | 14.04 | http://ubuntu.com/ubuntu/14.04
win10 | Microsoft Windows 10 | 10.0 | http://microsoft.com/win/10
win2k12 | Microsoft Windows Server 2012 | 6.3 | http://microsoft.com/win/2k12
win2k12r2 | Microsoft Windows Server 2012 R2 | 6.3 | http://microsoft.com/win/2k12r2
win2k3 | Microsoft Windows Server 2003 | 5.2 | http://microsoft.com/win/2k3
win2k3r2 | Microsoft Windows Server 2003 R2 | 5.2 | http://microsoft.com/win/2k3r2
win2k8 | Microsoft Windows Server 2008 | 6.0 | http://microsoft.com/win/2k8
win2k8r2 | Microsoft Windows Server 2008 R2 | 6.1 | http://microsoft.com/win/2k8r2
win7 | Microsoft Windows 7 | 6.1 | http://microsoft.com/win/7
win8 | Microsoft Windows 8 | 6.2 | http://microsoft.com/win/8
win8.1 | Microsoft Windows 8.1 | 6.3 | http://microsoft.com/win/8.1
winvista | Microsoft Windows Vista | 6.0 | http://microsoft.com/win/vista
winxp | Microsoft Windows XP | 5.1 | http://microsoft.com/win/xp
# virt-install \
--name=centos7 \
--disk pool=data-libvirt,cache=none,format=qcow2,size=16 \
--os-variant=centos7.0 \
--cdrom /home/paulgorman/Downloads/CentOS-7-x86_64-Minimal-1611.iso \
--vcpus=2 \
--ram=1024 \
--graphics spice \
--network bridge=br0
In the guest, edit /etc/default/grub, and append this to the GRUB_CMDLINE_LINUX=“” value: console=tty0 console=ttyS0,115200.
On Red Hat, run grub2-mkconfig -o /etc/grub2.cfg, and reboot.
On Debian, run update-grub, and reboot.
# virsh console centos7-test
# virsh domiflist centos7
Interface Type Source Model MAC
-------------------------------------------------------
vnet0 bridge br0 virtio 52:54:00:80:ef:da
# nmap -sn 10.0.1.0/24 | grep -i -B2 52:54:00:80:ef:da
Nmap scan report for 10.0.1.134
Host is up (-0.087s latency).
MAC Address: 52:54:00:80:EF:DA (QEMU virtual NIC)
(Or ip nei show dev br0?)
# virsh attach-disk myguest /var/lib/libvirt/images/file.img vdb --cache none
host-a~# virsh shutdown my-vm
host-a~# virsh dumpxml my-vm > my-vm.xml
host-a~# scp my-vm.xml host-b:
host-a~# scp /var/lib/libvirt/images/my-vm.qcow2 host-b:/var/lib/libvirt/images/
host-a~# virsh undefine my-vm
host-a~# rm /var/lib/libvirt/images/my-vm.qcow2
host-b~# virsh define my-vm.xml
host-b~# virsh start my-vm
Live migration should be possible, even without shared storage.
Without shared storage, it’s necessary to pre-define identical storage at the destination.
Know the root password for the destination (is there no way to “sudo” this? https://libvirt.org/remote.html).
(So, remote root ssh logins need to be enabled?!)
It may be necessary to open TCP ports 49152-49215 on the destination (or without opening the ports by passing the --tunnelled flag, if available).
host-b# iptables -A INPUT -p tcp --match multiport --dports 49152:49215 -j ACCEPT
host-a~# virsh vol-list vg0
Name Path
-----------------------------------------
myvm.img /dev/vg0/myvm.img
doorentry /dev/vg0/doorentry
lvroot /dev/vg0/lvroot
lvswap /dev/vg0/lvswap
postoffice-data /dev/vg0/postoffice-data
postoffice-os /dev/vg0/postoffice-os
virtbackup /dev/vg0/virtbackup
wolf /dev/vg0/wolf
host-b~# sudo virsh vol-create-as vg0 myvm.img 10G
host-a~# virsh migrate --copy-storage-all --persistent --live my-vm qemu+ssh://host-b/system
A migrate a non-running guest:
host-a~# virsh migrate --copy-storage-all --persistent --offline my-vm qemu+ssh://host-b/system
# apt-get install spice-client-gtk
# virsh domdisplay my-vm
spice://127.0.0.1:5900
$ spicy --uri=spice://127.0.0.1:5900
# virsh vncdisplay my-vm
:1
(“:1” seems to mean 5901.)
Once we know the guest console port, SSH forwarding works for Spice or VNC too.
$ ssh -L <local port>:localhost:<hypervisor port> hypervisor.example.com
$ ssh -L 5900:localhost:5900 hypervisor.example.com
$ spicy --uri=spice://127.0.0.1:5900
Or, through a jump host:
$ ssh -A -t jump.example.com -L 7900:localhost:6900 ssh -A -t hypervisor.example.com -L 6900:localhost:5900
$ spicy --uri=spice://127.0.0.1:7900
$ vncviewer 127.0.0.1::7900
# apt-get install virt-top
“3” shows block devices.
Installing the network driver is straightforward.
Shut off the VM. (I guess we could add a second interface to the live VM with virsh attach-interface.)
Change the network interface model from “e1000” (or “rtl8139” or whatever) to “virtio”. Do this via virsh edit myvm.
Spin up the VM.
Attach the VirtIO driver ISO to the image, and add the new drivers from inside the guest.
Installing the VirtIO disk bus driver is slightly less straightforward, but not a huge deal. We need to trick Windows into loading the driver by adding a small, temporary volume of the “VirtIO” type.
# virsh vol-create-as vg0 stub 1G
# virsh attach-disk doorentry /dev/vg0/stub vdc --config
# virsh detach-disk doorentry hdc --config
# virsh vol-delete stub --pool vg0
(Newer versions of virsh attach-disk can specify the bus explicitly with a --targetbus virtio flag. Older version only inferred the bus type based on the target name, like “hda”, “sdb”, or “vdc”.)
# virsh undefine --nvram --remove-all-storage --delete-snapshots myvm