VF (Hardware I/O virtualization mechanism)

VF (Virtual Function) in the context of SR-IOV is a hardware-isolated slice of a physical PCIe device that the guest OS sees as an independent PCIe adapter. Traffic routing and resource access decisions are made not by software but directly by the device hardware and the IOMMU, which removes the hypervisor from the critical data path.

SR-IOV (Hardware-level input-output device virtualization)IOMMU (Isolation of direct memory access addresses)

SR-IOV is used where latency and throughput are critical: high-load cloud platforms, HPC clusters, NFV infrastructure for telecom operators, and NVMe-oF storage systems. The physical adapter exposes several VFs, each assigned to a separate virtual machine or container, delivering performance close to bare metal.

Typical issues

Assigning a VF to a virtual machine tightly couples it to a specific physical host. Live migration without specialized solutions is impossible because the VF state is stored in the hardware of the PCIe slot. Additionally, exceeding the hardware limit of VFs per port requires either replacing the adapter or switching to software approaches like virtio, with some loss of performance.

How SR-IOV works

The PCI-SIG SR-IOV specification introduces two types of functions on a single physical device.

Physical Function (PF) is a full-featured PCIe interface with access to all device configuration registers. The PF driver loads in the host OS or hypervisor and acts as the sole manager of hardware resources: it activates SR-IOV via an extended PCIe capability block, sets the number of VFs, and allocates queues, buffers, and interrupts.

PF (Hardware virtualization of Input-Output devices)

Virtual Function (VF) is a lightweight PCIe interface with a limited set of registers. A VF gets its own PCIe BAR space, separate receive and transmit queues, and an isolated set of MSI-X interrupts. The guest OS interacts with the VF directly, bypassing the hypervisor: DMA operations go through the IOMMU, which translates guest physical addresses to host memory addresses without software layer involvement.

When SR-IOV is activated, the PF driver writes the desired number of VFs to the NumVFs register, after which the device registers the corresponding number of new PCIe functions. The IOMMU receives translation tables for each VF, isolating memory regions between virtual machines.

VF functionality

  1. VF as a hardware polymorphism mechanism. One physical device appears to multiple guests as independent adapters. Each VF has its own identifier (BDF – Bus:Device:Function) in the guest PCIe topology.
  2. Hardware isolation via IOMMU. Each VF is assigned a separate IOMMU domain. DMA requests from a VF are translated strictly to an allocated host memory region; any attempt to access memory of another VF or the host is blocked by hardware and logged as an error.
  3. Independent queues and interrupts. A VF receives dedicated hardware Tx/Rx queues and MSI-X vectors. The guest driver manages these directly through memory-mapped registers, without a context switch to the hypervisor.
  4. PF driver management. Operations affecting the global device state (MAC address change, VLAN tagging, QoS, firmware update) are performed exclusively via the PF. The VF driver has no access to the physical device’s management structures.
  5. Passthrough via VFIO. In Linux, a VF is assigned to a virtual machine through the VFIO subsystem: the vfio-pci module unbinds the VF from its standard driver and hands its memory space to QEMU/KVM or another hypervisor. VFIO provides safe user-space access to the PCIe space via /dev/vfio/.
  6. VFIO-PCI (Isolating PCI devices for virtualization)VFIO (Direct device Input-Output virtualization)
  7. VF count limit. The maximum number of VFs is set by hardware in the TotalVFs field of the PCIe capability block. Typical values: 32–64 VFs per port for network adapters such as Intel X710 and Mellanox ConnectX, up to 512+ for specialized accelerators and storage controllers.
  8. Traffic between VFs on one device. Communication between VFs of the same physical device can be routed by the hardware switch inside the NIC (eSwitch), bypassing the physical network. The PF driver configures eSwitch rules to manage this traffic.
  9. QoS and rate limiting. The PF driver can set guaranteed and maximum bandwidth for each VF via hardware mechanisms, providing performance isolation between tenants.
  10. Status and monitoring. Statistics for each VF (packet counters, errors, dropped frames) are available through the PF driver. The guest driver only sees the local counters of its own VF.
  11. VF reset. The PF driver can perform a Function Level Reset (FLR) on an individual VF without affecting others. This is used when changing tenants or recovering after a guest driver error.
  12. Migration: SR-IOV limitation. Classic SR-IOV does not support live migration: the hardware queue state of a VF cannot be saved and restored using standard methods. Migration is addressed either by proprietary solutions or by dynamically switching from VF to virtio.
  13. Scalable IOV (Intel, 2019). An evolution of SR-IOV: VFs are grouped into scalable sets using PASID, resources are allocated dynamically without reloading, and the number of virtual interfaces is not limited by the hardware TotalVFs field.

Comparisons

  • VF (SR-IOV) vs virtio. SR-IOV delivers bare-metal performance through direct hardware access but ties the VM to a physical host and requires SR-IOV support in the adapter. Virtio is a paravirtualized standard interface that works on any hardware and supports live migration, but has higher data path overhead through the hypervisor.
  • VF vs DPDK with virtio. DPDK bypasses the OS kernel when working with either VF or virtio interfaces. VF plus DPDK in the guest OS provides minimal latency and maximum PPS; virtio plus DPDK offers portability with acceptable performance.
  • VF vs vGPU / vDPA. vGPU (NVIDIA) and vDPA (virtio Data Path Acceleration) are related device virtualization technologies. vGPU divides GPU resources using a hardware-software approach with migration support; vDPA moves the virtio data path into hardware, combining virtio portability with SR-IOV performance.
  • vGPU (Splitting a GPU into virtual devices)
  • SR-IOV vs MR-IOV. Multi-Root IOV (MR-IOV) extends SR-IOV to multiple PCIe root complexes, allowing one device to be shared among several physical hosts. The standard exists but has not seen wide adoption due to implementation complexity.

OS and hypervisor support

The Linux kernel supports SR-IOV via the vfio-pci module (since version 3.6) and PCI passthrough in KVM. To assign a VF to a virtual machine in QEMU/KVM, you unbind the VF from its driver by writing vfio-pci to /sys/bus/pci/devices/.../driver_override and then specify the VF in the VM configuration.

Windows Server with Hyper-V has supported SR-IOV since version 2012 R2. VFs are assigned to virtual machines through the Hyper-V virtual switch with SR-IOV enabled; if guest support is missing, the VM automatically falls back to a synthetic network adapter.

VMware ESXi implements SR-IOV via DirectPath I/O: a VF is bound to a specific VM in the settings, bypassing the vSwitch. vMotion support with SR-IOV is limited and requires special configuration.

Security

Isolation between VFs is implemented at two levels. At the IOMMU level, each VF is assigned a separate translation group: a DMA request that goes outside its allocated memory region is blocked by hardware, triggering an IOMMU fault interrupt. At the PCIe Access Control Services (ACS) level, peer-to-peer DMA between VFs of different domains is forbidden without going through the root complex, preventing attacks on adjacent VFs via the bus.

The PF driver controls privileged operations: writes to the VF configuration space are filtered, and attempts to access PF control registers from guest space are blocked.

Compromising a guest VF driver should not affect the PF or other VFs if the PF driver implementation is correct.

Logging

The primary source of logs is the host PF driver. It records VF allocation and deallocation, attempts at invalid configuration operations from guest space, IOMMU fault events (DMA violations), FLR events, and timeouts or mailbox communication errors between VFs and the PF.

The guest VF driver maintains a local operation log in the guest OS space. Critical failures (operation timeout, loss of device communication) are relayed via the mailbox mechanism to the PF driver and appear in the host system log (dmesg, Event Log).

Limitations

The number of VFs is set by hardware in the TotalVFs field and cannot be increased via software. Typical values range from 16 to 512+ depending on the adapter model and generation.

A VF cannot change global device parameters (MAC address without PF permission, MTU, firmware) on its own. Any such operations go through the mailbox interface and must be approved by the PF driver.

Live migration of a virtual machine with an active VF is impossible in standard SR-IOV without completely detaching the VF. This is a fundamental architectural limitation: the hardware queue state cannot be serialized.

SR-IOV support is required simultaneously from three components: the physical adapter, the chipset/BIOS (enabling IOMMU: intel_iommu=on or amd_iommu=on), and the OS. Lack of any one of these prevents the technology from being used.

History and evolution

The PCI-SIG SR-IOV specification was published in 2007 in response to increasing workloads in virtualized data centers. Before its appearance, network I/O virtualization was implemented entirely in software: emulation of e1000/rtl8139 in QEMU or paravirtualized virtio (2008). Software approaches introduced significant overhead on the hypervisor data path.

The first production adapters with SR-IOV appeared around 2009–2010: Intel 82576 (igb), Chelsio T4, Mellanox ConnectX-2. By 2012–2013, the technology became standard for cloud providers.

In 2019, Intel introduced Scalable IOV, an architectural evolution of SR-IOV with dynamic resource allocation via PASID (Process Address Space ID) and support at the ADI (Assignable Device Interface) level. Scalable IOV removes the fixed VF count limitation and simplifies live migration by using a more flexible device state model.

Parallel development includes vDPA (virtio Data Path Acceleration): the virtio data path is moved into hardware or a SmartNIC, while preserving the standard virtio interface for the guest OS. This allows mixing portability and live migration support with performance close to SR-IOV.

Modern SmartNICs and DPUs (Data Processing Units) represent the next stage: the entire SR-IOV control path runs on the smart card’s embedded ARM processor, offloading the host CPU and adding programmability for traffic processing at the VF level.