Moving a VM from ESXi to KVM has two separate problems. The first is the disk: VMware’s VMDK has to become something QEMU reads well, usually raw or qcow2. The second is the guest: an operating system installed on VMware’s virtual hardware expects a PVSCSI or LSI controller, a VMXNET3 NIC and VMware Tools, and it has to boot on virtio devices instead. qemu-img solves the first problem. virt-v2v tries to solve both. This post covers where each one stops and what tends to break.

VMDK is a family of formats#

“VMDK” covers several layouts. QEMU’s vmdk driver can create monolithicSparse (the default), monolithicFlat, twoGbMaxExtentSparse, twoGbMaxExtentFlat and streamOptimized, and it can also read the layouts ESXi uses on VMFS.

Layout Where you see it Notes
Descriptor + flat extent Thick and thin disks on an ESXi datastore: web01.vmdk (small text file) and web01-flat.vmdk (the data) Point tools at the descriptor, not the -flat file. The flat extent is raw data with no header.
Sparse (monolithicSparse, twoGbMaxExtentSparse) Workstation and Fusion, older exports Grows as data is written.
Snapshot deltas (vmfsSparse, seSparse) web01-000001.vmdk after a snapshot QEMU follows the chain; its seSparse support is read-only. Consolidate snapshots first if you can.
streamOptimized Inside OVA/OVF exports Compressed and written sequentially for transfer. Use it as a conversion source, not as a running disk.

An OVA is an uncompressed tar archive (tar tf VM.ova lists its contents), usually holding an OVF descriptor and one or more stream-optimized VMDKs.

Converting the disk with qemu-img#

For a single disk this is all it takes:

# Inspect first: format, virtual size, backing chain
qemu-img info --backing-chain web01.vmdk

# VMDK to qcow2, with a progress bar
qemu-img convert -p -f vmdk -O qcow2 web01.vmdk web01.qcow2

# VMDK to raw, for an LVM volume, Ceph RBD or a block device
qemu-img convert -p -f vmdk -O raw web01.vmdk /dev/vg0/web01

Some details are worth knowing. qemu-img convert skips runs of zeroes (4 KiB by default, set by -S), so a thin source stays thin in a sparse destination. -S 0 turns that off and fully allocates the target. -m sets the number of parallel coroutines (8 by default). -W allows out-of-order writes, which the docs recommend only for preallocated targets such as raw block devices. qemu-img measure -O qcow2 web01.vmdk reports how much space the target needs before you write it.

Choosing raw or qcow2 depends on your storage more than on the guest. Raw has no format layer, so it fits block backends (LVM, Ceph RBD, SAN LUNs) and gives the simplest I/O path. qcow2 gives you thin provisioning on plain filesystems, internal snapshots, backing files and compression. If the platform you are moving to manages snapshots in the storage layer, raw is usually the right choice. If you are landing on a directory of files, qcow2 is.

What qemu-img does not do#

qemu-img copies blocks. It does not open the filesystem inside the image, so after a plain conversion:

  • No virtio drivers are guaranteed to load at boot. A Linux initramfs built on VMware may not include virtio_blk or virtio_scsi. Windows has no virtio drivers at all unless you installed them beforehand. If you attach the disk as virtio, the result is a kernel panic or INACCESSIBLE_BOOT_DEVICE.
  • VMware Tools stays installed and keeps looking for a VMware hypervisor.
  • The boot configuration is unchanged. Root device names, fstab entries and the bootloader still describe the old hardware.
  • Firmware metadata is gone. The VMX said whether the VM used BIOS or EFI. A bare disk image doesn’t, and booting a UEFI disk with SeaBIOS fails.

You can still migrate this way. You fix the guest yourself, before or after the copy, or you attach the disk as SATA first and install drivers. That works for a handful of VMs with known contents, and it is roughly what a pure-qemu-img pipeline has to automate.

What virt-v2v does#

virt-v2v copies the disks and converts the guest on the way. It uses libguestfs to inspect the image offline, finds the operating system, and then edits it. According to its manual it installs virtio drivers, removes VMware Tools, updates the bootloader and initramfs where needed, and relabels SELinux guests. It writes a new guest definition for the target. In copying mode the source is never modified.

The input side has five ways to reach VMware:

# 1. A VMX on local or NFS-mounted storage
virt-v2v -i vmx /mnt/datastore1/web01/web01.vmx -o local -os /var/tmp

# 2. A VMX read over SSH from ESXi (SSH must be enabled; no snapshots allowed)
virt-v2v -i vmx -it ssh \
  "ssh://root@esxi.example.com/vmfs/volumes/datastore1/web01/web01.vmx" \
  -o local -os /var/tmp

# 3. vCenter through VDDK (VMware's proprietary library; usually fastest)
virt-v2v \
  -ic 'vpx://root@vcenter.example.com/Datacenter/esxi01?no_verify=1' \
  -ip /root/vcenter-password \
  -it vddk -io vddk-libdir=/opt/vmware-vix-disklib-distrib \
  -io vddk-thumbprint=xx:xx:xx:... \
  web01 -o local -os /var/tmp

# 4. An OVA exported from vSphere (OVAs from other products will not work)
virt-v2v -i ova web01.ova -o local -os /var/tmp

# 5. vCenter over libvirt's vpx driver with no transport option (slowest)
virt-v2v -ic 'vpx://root@vcenter.example.com/Datacenter/esxi01' -ip /root/vcenter-password web01

The vCenter forms use -i libvirt, which is the default and can be left out. There is also an -it nfc transport that uses nbdkit’s NFC plugin on port 902. The -ip option takes a file containing the password, never the password itself. ?no_verify=1 skips TLS verification. It is convenient in a lab, but in production you should import the vCenter certificate instead.

On the output side, -o libvirt (the default) defines the guest in local libvirt with -os naming a storage pool. -o local writes disks plus libvirt XML to a directory. -o qemu writes a boot script, and -o openstack and -o ovirt-upload push to those platforms. A -o kubevirt mode also exists and is marked experimental. -of raw|qcow2 sets the disk format (the input format by default) and -oa sparse|preallocated sets allocation (sparse by default). Networks are mapped with --network or --bridge, or per NIC with --mac:

virt-v2v -i ova web01.ova -o libvirt -os default -of qcow2 \
  --mac 00:50:56:aa:bb:01:bridge:br-prod \
  --mac 00:50:56:aa:bb:02:bridge:br-backup

Before any of this, shut the guest down cleanly. The manual is explicit: if the guest runs during conversion, the converted copy will have a corrupted filesystem. Don’t start the source again unless the conversion fails.

Linux pitfalls#

virtio modules missing from the initramfs. Current virt-v2v does not install Linux kernels or drivers. It relies on the virtio support already in the guest’s kernel, warns if there is none, and rebuilds the initramfs for some Enterprise Linux guests (which is why it wants 50 MB free in /boot). Every mainstream kernel from the last decade has virtio built in or as modules. The risk is a host-only initramfs (dracut’s hostonly mode, for example) that only includes VMware’s storage drivers. With plain qemu-img, rebuild it before shutdown:

# RHEL family, before the final shutdown on VMware
dracut -f --add-drivers "virtio_blk virtio_scsi virtio_net virtio_pci" \
  /boot/initramfs-$(uname -r).img $(uname -r)

Device names change. A disk on PVSCSI is /dev/sda. On virtio-blk it becomes /dev/vda, though virtio-scsi keeps sdX. Any fstab line, GRUB root= or LVM filter that names /dev/sdX directly breaks. Switch to UUID= or LABEL= first.

NIC names change. systemd derives predictable names from the device’s bus position, for example from the PCI slot (ens192 on many VMware VMs) or the path (enp1s0). The virtio NIC sits at a different PCI address, so the name changes, and a NetworkManager profile, netplan file or ifcfg-ens192 bound to the old name no longer applies. Match configuration on MAC address where your tooling allows it, or plan a first-boot fix. virt-v2v’s manual says plainly that it cannot reconfigure the guest’s network configuration.

Debian and Ubuntu with GRUB 2. virt-v2v can’t change the default kernel there. Make sure the default boot entry is the kernel you want before converting.

Windows pitfalls#

virtio-win drivers. virt-v2v installs Windows drivers from the virtio-win collection. It looks in /usr/share/virtio-win/virtio-win.iso or /usr/share/virtio-win, or wherever the VIRTIO_WIN environment variable points. Without them the conversion can’t make Windows boot on virtio. --block-driver virtio-blk (the default) or virtio-scsi chooses which storage driver is preinstalled.

First boot does real work. Conversion runs in two stages, offline and then first boot. The manual warns that Windows may reboot four or more times while it installs drivers and agents, removes VMware Tools and configures the network. Don’t interrupt it. virt-v2v runs unsigned PowerShell scripts at first boot, so an AllSigned ExecutionPolicy breaks them. Antivirus and Group Policy can also block driver installs. Relax both for the conversion window.

VMware Tools removal often fails. virt-v2v uses VMware’s own uninstaller, and its documentation says this usually fails with error 1603 because the uninstaller expects to run on VMware. Plan a manual removal step using Broadcom’s documented procedure.

Static IPs. Windows binds IP settings to the adapter, and the virtio NIC is a new adapter, so a static address does not follow it. virt-v2v can set one at first boot:

--mac 00:50:56:aa:bb:01:ip:10.20.30.41,10.20.30.1,24,10.20.0.10,10.20.0.11

The fields are address, gateway, prefix length, then DNS servers. You don’t need this for DHCP guests.

Activation. Windows can treat a move to new virtual hardware as a significant hardware change and ask to be reactivated. Know how each VM is licensed (KMS, Active Directory-based, MAK or retail) before cutover. Check this when you test the converted copy, not on the night of the migration.

Firmware: BIOS, UEFI and Secure Boot#

Match the source firmware. A guest installed under BIOS has an MBR bootloader, and a UEFI guest has an EFI System Partition and boot entries in NVRAM. virt-v2v detects UEFI guests and writes libvirt XML that uses OVMF. Its support page notes that the OVMF build on the conversion host should match the target’s, or you will have to fix firmware paths.

If you write the XML yourself, libvirt’s firmware autoselection is the simplest route:

<os firmware='efi'>
  <type arch='x86_64' machine='q35'>hvm</type>
  <firmware>
    <feature enabled='yes' name='secure-boot'/>
    <feature enabled='yes' name='enrolled-keys'/>
  </firmware>
  <boot dev='hd'/>
</os>

enrolled-keys matters. Firmware with Secure Boot but no enrolled keys boots unsigned binaries too, so the guest isn’t actually protected. A Windows guest that ran with Secure Boot on VMware, or uses BitLocker or a vTPM, needs the target to provide Secure Boot with Microsoft’s keys and, where used, an emulated TPM. Have BitLocker recovery keys available before the first boot.

Pre-flight checklist#

  1. Inventory each VM’s firmware (BIOS or EFI, Secure Boot, vTPM), disk controllers, NICs with MACs and networks, and snapshots.
  2. Consolidate snapshots. The SSH transport refuses guests with them, and chains slow every method.
  3. Check free space. virt-v2v wants 100 MB on the Linux root, 50 MB on /boot and 100 MB on C:. The conversion host needs at least 1 GB in its temp directory ($VIRT_V2V_TMPDIR), plus room for the output.
  4. Linux: use UUID= in fstab and the bootloader, confirm virtio support in the running kernel’s initramfs, and know how the network configuration will find the renamed NIC.
  5. Windows: record static IPs, stage virtio-win on the conversion host, relax ExecutionPolicy, AV and GPO for the window, have BitLocker recovery keys at hand, and know the activation method.
  6. Shut down cleanly and keep the source powered off but intact until the converted VM passes its checks.
  7. Test-boot the copy on an isolated network before cutover. Check disks, services, NICs, time sync and the guest agent.
  8. Keep a rollback path. The source VM is the rollback until you decommission it.

Where Victor fits#

Victor’s VMware migration is cold migration today. Victor Community assesses the estate, builds plans and cold-migrates single Linux VMs, converting disks with qemu-img and keeping the guest’s network identity on the renamed virtio NIC. Victor Enterprise adds waves, virt-v2v conversion (virtio drivers in, VMware Tools out, Windows static addresses reapplied), isolated test boots, cutover and rollback. Windows conversion has so far been tested with stand-ins, and migration has been validated against a vCenter simulator rather than a production vCenter. Migrated VMs keep their BIOS UUID, instance UUID and MAC addresses, and on the classic KVM runtime their VM Generation ID.

See Victor's migration path