IOMMU (Isolation of direct memory access addresses)

IOMMU is a hardware manager that remaps device requests to physical memory, allowing virtual machines and drivers to safely use different address spaces.

IOMMU is used in servers for virtualization, where guest systems gain direct access to hardware without the risk of memory page swapping. In personal computers, it ensures driver security, allows 64-bit devices to work with memory over 4 GB, and is also used in operating systems to protect against DMA attacks.

Typical issues include a drop in I/O performance due to overhead from address translation, compatibility errors with legacy drivers that do not support remapping, and boot failures if the BIOS or UEFI has not configured the translation tables correctly.

How it works

The IOMMU connects in parallel to the memory controller on the system bus. When a device initiates a DMA operation, it sends a virtual address, which is intercepted by the IOMMU. Inside the IOMMU are a translation lookaside buffer (IOTLB) and registers for nested page tables. The translation process proceeds as follows: first, the IOMMU checks if the translation is already in the IOTLB; if not, it performs up to two sequential read cycles from RAM – the first to fetch the root table, then for second and third level directory pages. The result is a physical address that the device can access. Additionally, the IOMMU verifies read and write access rights based on the page mask, blocking accesses outside the permitted range. Finally, the physical address is passed to the bus arbiter, and data reaches the intended memory region. This scheme ensures isolation without requiring modification of the device itself.

IOMMU functionality

  1. Address translation for devices. Devices initiate DMA requests using virtual addresses from their I/O space. The IOMMU intercepts these requests and dynamically translates the virtual address into a physical DRAM address.
  2. DRAM (Storage and Byte-addressing of Data)
  3. Memory isolation for devices. The main task of the IOMMU is isolation. The driver assigns the device a dedicated range of physical memory. Through the IOMMU, the device cannot access pages outside this range, blocking malicious DMA attacks.
  4. Mechanism for working with page tables. The IOMMU uses its own I/O page tables, structured similarly to CPU page tables. The OS kernel manages these tables, defining mappings from PCIe virtual addresses to physical pages.
  5. Support for long physical addresses. When operating in 32-bit mode, a device can only generate 32-bit addresses. The IOMMU maps these to any physical address above 4 GB, solving the DMA problem in systems with large amounts of RAM.
  6. Translation caching (IOTLB). To speed up translation, the IOMMU has a built-in IOTLB cache. It stores recently used virtual-to-physical page mappings, reducing latency when accessing page tables in memory.
  7. Cache coherency management. When CPU page tables are modified, corresponding entries in the IOTLB must be invalidated. The IOMMU provides hardware invalidation commands by context, domain, or individual page to ensure data consistency.
  8. Direct access virtualization (PCI Passthrough). In hypervisor environments, the IOMMU allows a physical PCIe device to be safely assigned to a guest OS. The guest accesses the device’s real registers, but DMA goes through the IOMMU with mapping to its physical memory.
  9. Interrupt remapping. An additional IOMMU function is interrupt remapping. It redirects MSI messages from devices to target vectors and virtual CPUs, preventing interrupt spoofing in guest systems.
  10. I/O domains. The IOMMU groups devices into isolated domains. Each domain has its own page table. Devices from different domains cannot access each other’s memory pages without explicit mapping through the kernel.
  11. Support for ACS groups. For reliable isolation, all devices in a PCIe group must support ACS. This ensures that DMA packets from one device inside a switch are not intercepted by a neighboring device before reaching the IOMMU.
  12. Driver compatibility issues. Some device drivers use DMA granularity larger than a page. When strict IOMMU is enabled, the driver must respect address alignment, otherwise failures may occur due to partial translation of a single buffer.
  13. Bypass mode. When enabled but improperly configured, many IOMMUs operate in bypass mode. In this mode, devices with unassigned tables access physical memory directly, disabling isolation until the driver loads.
  14. Static and dynamic mapping. Static mapping reserves fixed physical ranges for specific device virtual addresses. Dynamic mapping uses a fault mechanism: when a translation entry is missing, the IOMMU generates an interrupt, and the kernel creates a mapping on the fly.
  15. Translation overhead. Each DMA operation may require up to two memory accesses to fetch page table entries (on IOTLB miss). This increases access latency and reduces throughput for high-frequency devices such as NVMe.
  16. Optimization with large pages. The IOMMU supports huge pages (2 MB, 1 GB). Using large pages reduces the number of IOTLB entries, lowers miss rates, and mitigates the performance impact of translation on storage and network cards.
  17. Handling access faults (Page Faults). If a device attempts to access an unmapped address or violates access rights, the IOMMU records the error in status registers. The OS can terminate the process associated with the device and reset the DMA engine.
  18. DMA transaction debugging. The IOMMU can be placed into logging mode for all translations. When this option is enabled, every virtual access by a device is recorded in a ring buffer, allowing analysis of incorrect DMA patterns.
  19. Integration with ARM SMMU. In the ARM architecture, the IOMMU function is performed by the SMMU. It supports more complex contexts with multi-level tables and built-in protection against transaction spoofing over the ACE bus.
  20. Activation parameters in the Linux kernel. IOMMU activation is done with parameters like intel_iommu=on for Intel or amd_iommu=on for AMD. The additional flag strict=1 disables bypass mode, forcibly isolating all devices with immediate effect.

Comparison with similar functions

  • IOMMU vs MMU. The IOMMU manages address translations for I/O devices, whereas the MMU handles translations only for the CPU. The IOMMU allows devices to access virtual memory, enhancing security and supporting DMA remapping, while the MMU protects CPU memory access but does not control external buses.
  • IOMMU vs VT-d vs AMD-Vi. VT-d (Intel) and AMD-Vi are implementations of IOMMU technology for x86 platforms. Both provide DMA isolation, ATS, and virtualization support. Differences lie mainly in register models and interrupt handling specifics, but they are functionally equivalent. IOMMU here is the general architectural concept.
  • IOMMU vs SMMU. The SMMU is an implementation for ARM systems, analogous to the x86 IOMMU. It also supports device transactions using virtual addresses and domain isolation. The key difference is integration with ARM TrustZone and the Stage 1/Stage 2 translation architecture, which is convenient for virtualization without modifying the guest OS.
  • IOMMU vs MPU. The MPU is a simplified memory protection unit for embedded systems that cannot perform address translation. The IOMMU, in contrast, dynamically remaps addresses and works with external devices. The MPU only defines access regions for the CPU, whereas the IOMMU allows multiple virtual machines to safely share a common device via DMA.
  • IOMMU vs SWIOTLB. SWIOTLB is a software translation buffer used when an IOMMU is absent. It copies data into memory regions accessible to the device, reducing performance and providing no isolation. The IOMMU translates addresses in hardware, offering high speed and full protection against DMA attacks and errors.

OS and driver support

IOMMU is activated through BIOS/UEFI and requires OS kernel support. In Linux, the AMD IOMMU or Intel VT-d modules are used. In Windows, management is handled via DMA Remapping and drivers with the DMA_CTRL_REMAP flag. For proper operation, a device driver must allocate buffers through kernel APIs (e.g., dma_map_sg), receiving translated physical addresses from the IOMMU. Virtualized OSes (KVM, Xen) provide the guest with direct DMA mappings via the IOMMU, bypassing hypervisor mediation.

Security

The IOMMU prevents DMA attacks (e.g., over Thunderbolt or PCIe slots) by isolating memory regions: each device is assigned its own protection domain with a translation table that blocks accesses outside the allocated buffer. It also implements page-level access control (usually 4 KB) and DeviceID tag checking. In virtualized systems, the IOMMU blocks direct access from a guest device to hypervisor memory or other VMs via nested page tables.

Logging

When a DMA boundary violation occurs (attempt to write to a forbidden page), the IOMMU generates a hardware interrupt, recording in a ring buffer of error registers (e.g., Fault Event Log or Interrupt Remapping Table Fault) the cause – operation address, RID, transaction type. The OS reads these records via the IOMMU driver and passes them to system logs (dmesg, Event Viewer), often accompanied by a stack trace and device name, allowing detection of vulnerabilities or incorrectly written drivers.

Limitations

Technical limitations of the IOMMU include: performance degradation due to TLB misses during address translation (up to 5-10% for high-speed NVMe/GPU), the need for buffer alignment to page size (increasing memory overhead), incompatibility with some legacy devices (lack of ATS support), and lack of protection against bus cache attacks (e.g., DMA using spoofed RIDs when ACS is poorly configured). Also, on systems without an IOMMU or with it disabled in the BIOS, all protection is absent.

History and development

The emergence of the IOMMU dates back to the late 1990s (Sun IOMMU for SPARC). In the x86 world, it appeared with AMD Opteron (GART, 2003) and Intel VT-d (2006). Milestones include the AMD-Vi (IOMMUv1) and Intel VT-d 2.0 standards, which added support for PCIe Passthrough virtualization. Modern developments (VT-d 3.0/3.1, AMD IOMMUv3) include scalable page tables, RID spoofing protection (PASID filtering), built-in DMA encryption (TDISP), and integration with CXL devices, gradually transforming the IOMMU from an optional isolator into a mandatory component of secure server and embedded systems.

PCIe Passthrough (Direct virtual machine access to PCIe device)