KubeVirt is a Kubernetes add-on that runs virtual machines as Kubernetes resources. Under the hood it is still KVM, QEMU and libvirt; what changes is who decides where a VM runs, how its disks and networks are attached, and which API you use to drive it. This post walks through the moving parts in the order a VM actually meets them.

The components#

KubeVirt is installed by an operator. You apply kubevirt-operator.yaml, which deploys virt-operator, then kubevirt-cr.yaml, which creates a KubeVirt custom resource. The operator reads that resource and deploys and upgrades the rest of KubeVirt, by default in the kubevirt namespace.

Component Runs as Job
virt-operator Deployment Installs, configures and upgrades the other components from the KubeVirt CR
virt-api Deployment The entry point for virtualization API flows: defaulting and validation of KubeVirt resources, plus subresources such as console, VNC, start and stop
virt-controller Deployment Cluster-wide logic: watches VMs and VMIs, creates and manages the pod for each VMI, drives migrations
virt-handler DaemonSet (one per node) Node agent: watches VMIs assigned to its node and reconciles the libvirt domain until it matches the spec
virt-launcher One pod per running VMI Provides the cgroups and namespaces for the VM; runs a private libvirtd that starts the QEMU process

Two details matter operationally. First, there is no shared libvirtd on the host: each VM gets its own libvirtd inside its own virt-launcher pod. Second, the nodes need hardware virtualization (/dev/kvm). Software emulation exists (spec.configuration.developerConfiguration.useEmulation: true in the KubeVirt CR) but is meant for development and nested test clusters, not production.

The CRDs: VM, VMI and friends#

  • VirtualMachineInstance (VMI) is a running VM. It is the ephemeral building block, closer to a Pod than to a Deployment. Delete it and the VM is gone.
  • VirtualMachine (VM) is the stateful, long-lived object you normally create. It holds a VMI template and a run strategy, and its controller creates or deletes the VMI to match. Start, stop and restart are operations on the VM.
  • VirtualMachineInstanceReplicaSet keeps N identical VMIs running, like a ReplicaSet does for pods. It suits stateless VMs whose disks need not survive termination (container disks, ephemeral or read-only storage).
  • VirtualMachineInstanceMigration requests a live migration of a VMI to another node.

All of these are served as kubevirt.io/v1. Storage objects (DataVolumes, DataSources) come from the separate Containerized Data Importer project and are covered in the storage post.

Run strategies#

A VM’s spec.runStrategy says whether a VMI should exist:

runStrategy Behaviour
Always A VMI always exists; if it stops for any reason (guest shutdown or crash) a new one is created
RerunOnFailure Restarted after a failure such as a node crash or OOM kill, but a shutdown from inside the guest is respected
Once Runs once and is not restarted when it finishes, whatever the outcome
Manual The controller does nothing on its own; you start, stop and restart explicitly
Halted No VMI; a running one is stopped

The older boolean spec.running was deprecated in KubeVirt v1.4.0, and the API server rejects a VM that sets both fields. Use runStrategy in new manifests. Note that virtctl changes the run strategy: virtctl stop on an Always VM sets it to Halted, and virtctl start on a Halted VM sets it to Always.

A minimal VirtualMachine#

This VM boots Fedora from a container disk, a disk image shipped as an OCI image, and injects a password through cloud-init. It is valid for the kubevirt.io/v1 API:

apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: fedora-demo
spec:
  runStrategy: Halted
  template:
    metadata:
      labels:
        kubevirt.io/vm: fedora-demo
    spec:
      domain:
        cpu:
          cores: 2
        resources:
          requests:
            memory: 2Gi
        devices:
          disks:
            - name: containerdisk
              disk:
                bus: virtio
            - name: cloudinitdisk
              disk:
                bus: virtio
          interfaces:
            - name: default
              masquerade: {}
      networks:
        - name: default
          pod: {}
      volumes:
        - name: containerdisk
          containerDisk:
            image: quay.io/containerdisks/fedora:latest
        - name: cloudinitdisk
          cloudInitNoCloud:
            userData: |-
              #cloud-config
              password: fedora
              chpasswd: { expire: False }

Every disk under domain.devices.disks must name a volume under volumes, and every interface must name a network. The masquerade binding on the pod network NATs the VM behind the pod IP, which works with Services and network policies and is compatible with live migration. A container disk is ephemeral: writes go to a scratch layer that disappears when the VMI stops. That is right for demos, CI and stateless replicas. For anything that must keep its data, use a PVC or DataVolume instead.

How a VM becomes a pod#

Apply the manifest and run virtctl start fedora-demo. Then:

  1. The VM controller in virt-controller sees that the run strategy now wants a VMI and creates one from the template. virt-api validates and defaults it on the way in.
  2. virt-controller creates a virt-launcher pod for the VMI (virt-launcher-fedora-demo-<suffix>). The VM’s CPU and memory requests, node selector, affinity and tolerations are translated into the pod spec, plus overhead for QEMU itself.
  3. The ordinary Kubernetes scheduler places the pod. KubeVirt does not ship its own scheduler.
  4. Once the pod is running, virt-controller records the node on the VMI. virt-handler on that node picks it up and tells virt-launcher to define and start the libvirt domain.
  5. QEMU runs inside the pod’s compute container. virt-handler keeps reporting status back into the VMI.
kubectl get vm,vmi
kubectl get pods -l kubevirt.io=virt-launcher

Because the VM is a pod, the usual Kubernetes machinery applies: resource quotas, priority classes, kubectl describe events and the node’s cgroup accounting.

Scheduling: node selectors, affinity, tolerations#

Placement fields go in the VMI template, at spec.template.spec of the VM, and mean exactly what they mean for a pod:

spec:
  template:
    spec:
      nodeSelector:
        storage: fast
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - weight: 100
              podAffinityTerm:
                topologyKey: kubernetes.io/hostname
                labelSelector:
                  matchLabels:
                    app: db
      tolerations:
        - key: dedicated
          operator: Equal
          value: vms
          effect: NoSchedule

Anti-affinity on the virt-launcher pods is how you spread a database cluster’s VMs across hosts, the equivalent of a “separate VMs” rule in other platforms. With the LiveUpdate VM rollout strategy, a change to a running VM’s node selector or affinity propagates to the VMI but only takes effect at its next migration.

virtctl basics#

virtctl is KubeVirt’s client. Download it from the KubeVirt releases page, or install it as a kubectl plugin with kubectl krew install virt and call it as kubectl virt.

virtctl start fedora-demo
virtctl stop fedora-demo
virtctl stop fedora-demo --grace-period 0 --force   # pull the power cord
virtctl restart fedora-demo
virtctl pause vm fedora-demo
virtctl unpause vm fedora-demo

virtctl console fedora-demo        # serial console; Ctrl+] to leave
virtctl vnc fedora-demo            # starts a local VNC viewer
virtctl vnc --proxy-only fedora-demo   # prints a port for your own viewer

virtctl ssh -i ~/.ssh/id_ed25519 fedora@vm/fedora-demo/default
virtctl migrate fedora-demo

Console and VNC traffic goes through the Kubernetes API server to virt-api and then to the node, so you need neither node access nor a routable VM IP. Add -v 4 to any command when a connection fails.

Live migration at a high level#

Live migration is enabled by default in current KubeVirt (versions before v0.56 needed the LiveMigration feature gate). Trigger it with virtctl migrate <vm> or with an object:

apiVersion: kubevirt.io/v1
kind: VirtualMachineInstanceMigration
metadata:
  name: migrate-fedora-demo
spec:
  vmiName: fedora-demo

KubeVirt creates a second virt-launcher pod on the target node, and QEMU streams memory into it. The upstream requirements to keep in mind:

  • Shared storage. Disks on PVCs must be ReadWriteMany. A VM with a ReadWriteOnce PVC is not live-migratable.
  • Network binding. The bridge binding on the pod network blocks migration; masquerade works. Ports 49152 and 49153 must be free in the virt-launcher pod.
  • Compatible CPUs. The target node must be able to provide the guest’s CPU model.
  • Eviction. Setting evictionStrategy: LiveMigrate on the VM template makes KubeVirt add a PodDisruptionBudget, so a kubectl drain migrates the VM instead of killing it.

GPU passthrough and some host devices restrict or prevent migration. Moving a disk between storage classes while the VM runs is a separate feature, volume migration, described in the user guide.

KubeVirt or a classic KVM cluster?#

KubeVirt fits when:

  • You already run Kubernetes and want VMs and containers under one scheduler, one RBAC model, one GitOps workflow and one network policy model.
  • The VMs are being modernized: they sit next to containerized services, or are expected to become containers over time.
  • Your storage team already provides CSI drivers with snapshots and RWX block, and your platform team already operates Kubernetes upgrades.

A classic KVM cluster (libvirt or Incus on hosts, with a management layer above) fits better when:

  • Most workloads are traditional VMs and nobody wants to run Kubernetes just to host them.
  • You need things that are harder in a pod: wide host-device passthrough, very large VMs, or unusual host networking.
  • The team’s skills and runbooks are host-centric: virsh, storage pools, bridges and VLANs.

Both run the same hypervisor. The difference is the control plane and the operational model around it, and the cost of operating Kubernetes itself has to be counted.

Where Victor fits#

Victor is a control plane and console that shows KVM VMs (on its Classic runtime) and KubeVirt VMs in one inventory, so the choice above does not have to be all or nothing. It drives KubeVirt through its normal API rather than replacing any of the components described here.

See how Victor manages both engines