AMD-Vi (Hardware I/O virtualization)

AMD-Vi technology that allows virtual machines to directly and securely work with physical computer devices such as graphics cards or network adapters, bypassing the hypervisors software emulator. The result is near-native performance of real hardware without compromising guest system isolation.

This technology is in demand for server consolidation, where a single physical host serves dozens of isolated systems, and for client configurations with discrete graphics card passthrough to a guest operating system for gaming or graphics workloads. AMD-Vi is also critically important in Network Functions Virtualization (NFV) environments, where network cards require direct access to virtual machine memory to process packets at line rate without the latency of a software switch.

The most common problem is incorrect grouping of devices in IOMMU groups, where several different devices end up in the same group, making it impossible to pass them separately to different virtual machines. Another challenge is poor Function Level Reset support by some consumer devices, causing the device to remain in the D3 state after a guest system reboot and refuse to reinitialize. Performance degradation can also occur when ACPI IVRS tables are missing or contain errors in the firmware, forcing the operating system to use software emulation instead of hardware address translation.

IOMMU (Isolation of direct memory access addresses)

How AMD-Vi works

AMD-Vi operates based on the IOMMU (Input/Output Memory Management Unit) block built into the northbridge or integrated into the processor. When a peripheral device initiates a direct memory access, the IOMMU intercepts the request and checks its legitimacy. Each device is identified by a unique PCIe identifier — a pair of BDF (Bus, Device, Function). In shadow page tables allocated for a specific virtual machine, the IOMMU looks up the mapping between the physical address the device is trying to access and the actual page address in the host physical memory. If the address requested by the device is outside the allowed region or access rights (read, write) are violated, the IOMMU generates an interrupt and blocks the transaction, preventing the device from reading or corrupting the memory of another virtual machine or the hypervisor. Simultaneously, interrupt remapping is performed: the IOMMU block intercepts MSI/MSI-X messages from devices, checks that they have not compromised the interrupt vector, and redirects it to a strictly assigned CPU core of the guest system. Thus, the hypervisor configures the IOMMU hardware tables, defining an access map for each passed-through PCIe device, while the entire dynamic address translation and memory protection is performed exclusively by the hardware, offloading the central processor and eliminating the overhead of software emulation for I/O operations.

Shadow Page Tables (Isolation of guest OS page tables)

AMD-Vi features

  1. Device Table. The Device Table is a basic structure located in system memory. It contains a set of entries (Device Table Entry, DTE), each corresponding to a specific peripheral device. A DTE stores a pointer to the root of the I/O page tables, as well as control bits defining the address translation mode for that particular device.
  2. Device Table Entry Format. Each DTE occupies 256 bits and contains several 64-bit fields (Quad 0 through 3). The fields include the base address of the device page table, domain ID, access rights, and control flags. The hardware checks reserved bits in each Quad, and if invalid values are detected, a hardware error (DEV_TAB_HARDWARE_ERROR) is generated.
  3. I/O Page Tables. AMD-Vi uses a multi-level page table structure. The devices virtual address is split into 9-bit groups, each serving as an index at the corresponding page directory level. An entry can be either a Page Directory Entry (PDE) pointing to the next level table, or a Page Translation Entry (PTE) containing the physical page address.
  4. IOMMU Control Registers. AMD-Vi block management is done through registers mapped into Memory Mapped I/O (MMIO) space. The register set includes the Device Table Base Address Register, Command Buffer Base Address Register, and Event Log Base Address Register. Command and event queue pointers are also configured via MMIO.
  5. Command Buffer. The driver sends commands to the AMD-Vi device through a shared ring buffer. The driver writes a command into the buffer at the tail pointer and updates its value. The IOMMU reads commands at the head pointer and executes them asynchronously. The ring buffer mechanism enables efficient batch processing of management operations.
  6. Cache Invalidation Commands. To maintain translation coherence, invalidation commands are used. The INVALIDATE_DEVTAB_ENTRY command flushes a cached DTE entry. The INVALIDATE_IOMMU_PAGES command selectively invalidates TLB entries for a specified domain. INVALIDATE_IOTLB_PAGES sends a request to the device to flush its local translation cache (IOTLB).
  7. COMPLETION_WAIT Synchronization Command. This command provides a synchronization barrier. It is placed into the command buffer and does not complete until all previous commands have been executed by the device. This ensures that cache invalidation operations are finished before the driver continues working with device memory.
  8. Event Log. To notify the driver of errors, the IOMMU writes entries into the event log ring buffer. The driver configures the buffers base address and its size in MMIO registers. If the buffer overflows and a new error occurs, the EventOverflow flag is set, signaling loss of events.
  9. Page Fault Handling. When a device attempts to perform DMA to an address for which there is no valid translation, the IOMMU generates an IO_PAGE_FAULT event. The entry contains the device ID and the virtual address that caused the fault. The driver can use this information to allocate memory on demand or to forcefully disable the device.
  10. Illegal Device Detection. If the IOMMU receives a request from a device that is not allowed to perform DMA according to the configuration, an INVALID_DEVICE_REQUEST event is generated. This can happen when the device has no corresponding entry in the Device Table or the Valid bit in its DTE is cleared.
  11. Hardware Transaction Errors. AMD-Vi monitors critical bus errors. When Master Abort or Target Abort errors are detected during access to page tables or the Device Table, PAGE_TAB_HARDWARE_ERROR or DEV_TAB_HARDWARE_ERROR events are generated. Similarly, an error during command buffer reading triggers COMMAND_HARDWARE_ERROR.
  12. Interrupt Remapping. AMD-Vi enables interrupt virtualization by translating interrupt requests from devices into specific vectors. This uses a separate Interrupt Remapping Table (IRT). IRTE entries associate a device ID with a vector and interrupt delivery mode, isolating guest system interrupts.
  13. Interrupt Remapping Table. Each IRTE entry describes the remapping parameters for a specific device interrupt vector. Entry fields include a remapping enable flag (RemapEn), target processor ID, interrupt vector, and delivery mode. When an entry is modified, the driver must invalidate the interrupt table cache.
  14. INVALIDATE_INTERRUPT_TABLE Command. A dedicated command is used to flush cached IRT entries. It instructs the IOMMU to invalidate all internal caches associated with the interrupt remapping table. Using this command is necessary after any change to the IRT to avoid using stale data.
  15. Hardware Support for Guest Systems (vIOMMU). AMD-Vi provides hardware support for virtualizing its own functions via Virtual Function MMIO (VF MMIO). Each guest virtual machine receives its own set of control registers, offset by a base address computed using a unique GuestID.
  16. vIOMMU (Translation of DMA request handling for VM)VF (Hardware I/O virtualization mechanism)
  17. Guest Command and Event Buffers. Through VF MMIO, the guest system configures its own command ring buffers and event logs. Guest drivers specify guest physical addresses (GPA) for these buffers. The hardware IOMMU translates these GPAs into system physical addresses (SPA) using the hosts nested page tables.
  18. Guest Control and Management. Guest control registers allow independent management of functions for each virtual machine. The guest OS can enable event logging and command processing while the hypervisor retains full control. Flags in the guest control registers mirror many fields of the main IOMMU control register.
  19. Validation of Guest Requests. All operations initiated by guest drivers through the vIOMMU undergo hardware validation. The hypervisor can intercept and modify guest commands. The nested page translation mechanism ensures that the guest system cannot use fake physical addresses to bypass memory access restrictions.
  20. IVRS Interface (I/O Virtualization Reporting Structure). The AMD-Vi configuration is described in the ACPI IVRS table. This table contains I/O Virtualization Hardware Definition (IVHD) blocks, which list IOMMU parameters and the devices belonging to it. Parsing the IVRS allows the driver to determine the system topology and map each IOMMU to a specific PCI segment.
  21. Device Identification in IVRS. The IVRS table can contain entries of various types to identify devices. For example, a DEV_SPECIAL type entry with an IOAPIC subtype indicates an I/O APIC device requiring special handling. Fields in the IVHD structure contain flags determining whether all devices are enabled by default or only those explicitly listed.
  22. Peripheral Page Request Interface (PRI). PRI is an extension of AMD-Vi for working with dynamically paged memory. When an ATS-capable device receives a fault during a translation request, it can generate a Page Page Request (PPR). This request is placed into a dedicated PPR log, and software must allocate a physical page in response.

Comparisons

  • AMD-Vi vs Intel VT-d. Both technologies represent hardware implementations of an IOMMU for passing through PCIe devices to virtual machines. AMD-Vi and VT-d perform identical functions: remapping DMA requests and routing interrupts, providing isolation and direct assignment of storage devices or graphics cards to guest OS without hypervisor involvement in data transfer. The difference lies only in implementation on platforms from different vendors.
  • Intel VT-d (Hardware isolation of direct device access)
  • AMD-Vi vs ARM SMMU. The System Memory Management Unit architecture from ARM is a functional equivalent of AMD-Vi for non-x86 processors. The SMMU also translates virtual addresses to physical addresses, ensuring secure device access to machine memory. However, the context lookup method differs: AMD-Vi relies on a Device Table, while ARM SMMUv3 uses a Stream Table, reflecting different approaches to identifying devices on the bus.
  • AMD-Vi vs SR-IOV. Single Root I/O Virtualization is not an alternative but a technology that works on top of AMD-Vi. While AMD-Vi allows passing an entire physical device to a single virtual machine, SR-IOV enables hardware-level partitioning of that device into multiple Virtual Functions (VF) for simultaneous distribution to dozens of VMs. Without an IOMMU (AMD-Vi), SR-IOV operation would be unsafe due to uncontrolled direct memory accesses.
  • SR-IOV (Hardware-level input-output device virtualization)
  • AMD-Vi vs Software I/O Virtualization. Software I/O emulation, such as virtio in QEMU, does not require specific hardware but heavily loads the host CPU due to copying data packets through the hypervisor. AMD-Vi, in contrast, offloads the central processor by allowing the device, via the IOMMU, to directly read and write to virtual machine memory. This delivers performance close to non-virtualized operation and is necessary for paravirtualization with hardware support.
  • QEMU (Emulator and hardware virtualizer of a computer)
  • AMD-Vi vs AMD-V. Despite the similar names, these are completely different processor virtualization components. AMD-V (SVM) is responsible for accelerating guest OS instruction execution on the CPU, using Nested Page Tables for VM kernel memory translation. AMD-Vi, on the other hand, works with memory from the peripheral side via the IOMMU, isolating direct memory access traffic from external PCI Express devices.
  • AMD-V (Hardware virtualization using the processor)SVM (Full hardware isolation of virtual machines)

OS and driver support

AMD-Vi support is implemented through the operating system software stack and drivers, where at the Linux kernel level, for example, the amd_iommu=on parameter is used, activating the iommu/amd driver to manage address translation and interrupts. In virtualization environments such as Xen, the iommu_interrupt_handler specialized interrupt handler is responsible for IOMMU event processing, masking interrupts in the IOMMU_STATUS_MMIO_OFFSET status register and delegating further work to tasklets for asynchronous checking of event logs and PPR (Peripheral Page Request). For guest systems, virtio drivers enable interaction with paravirtualized devices without emulation, reducing I/O overhead.

Security

AMD-Vi security is based on hardware isolation of Direct Memory Access (DMA) through device address translation using I/O page tables, allowing the hypervisor to restrict the memory region visible to a specific virtual machine. AMD processors implement Device Exclusion Vector (DEV) support, which can completely block specific hardware from accessing RAM, and AMD-Vi technology complements these measures by replacing legacy GART and DEV with a modern mechanism that prevents TOCTOU attacks during integrity measurement.

Logging

Logging in AMD-Vi is implemented through three hardware ring buffers — the Event Log, the Peripheral Page Request Log (PPR Log), and the Guest Address Log (GA Log) — each of which can generate a separate interrupt when filled with entries. In modern implementations, such as AMD IOMMU x2apic mode, three independent MSI interrupt vectors with corresponding processing threads are used, visible in /proc/interrupts as AMD-Vi<x>-[Evt/PPR/GA], avoiding performance bottlenecks under intensive logging. When errors occur, such as I/O page faults, detailed information about the device, address, and operation flags (read/write) is printed to the kernel system log, and the softirq processing mechanism ensures that log overflows do not overwrite existing entries.

Limitations

A fundamental limitation of AMD-Vi is its dependency on correct firmware support from the manufacturer and the integrity of ACPI tables (IVRS, I/O Virtualization Reporting Structure). When these are missing or contain errors, devices may end up in incorrect isolation groups, requiring the use of an unsafe workaround — the ACS override patch. Another serious bottleneck is error handling under memory pressure: when an iommu_map_page call fails to allocate a page via __alloc_pages_nodemask with the GFP_ATOMIC flag, cascading warnings are generated that can completely overload the CPU due to massive output to the serial console. Additionally, older systems that use legacy interrupt mode instead of x2apic continue to share a single MSI vector for all three logs, creating a potential bottleneck for event processing.

History and development

AMD-Vi technology was introduced by AMD in 2009 alongside HyperTransport 3.0 processors as an implementation of the IOMMU concept for the x86-64 architecture, evolving from earlier Sun developments for UltraSPARC and complementing the AMD-V hardware virtualization technology. Originally intended for secure device passthrough (PCI passthrough) to virtual machines, the technology later became a key component for reorganizing driver architectures even in non-virtualized environments. With the advent of the PCI Express SR-IOV extension, it gained the ability to provide multiple guest systems with direct access to shared hardware resources with hardware-level isolation. In modern implementations, AMD-Vi support is integrated directly into the Linux kernel as a configurable AMD_IOMMU option, mandatory for systems with more than 254 CPU cores, and continues to evolve in terms of interrupt handling and support for new technologies such as SVA (Shared Virtual Addressing) via the PPR log.

Latest publications