VMLAUNCH (Launching a guest virtual machine)

VMLAUNCH is a specialized processor instruction that transitions a physical core from host mode to guest mode. The processor hardware loads the virtual machine state from a dedicated structure in memory and begins executing guest code without hypervisor involvement.

This instruction is the foundation of Intel VT-x and AMD-V hardware virtualization. It is used in products such as VMware ESXi, Microsoft Hyper-V, KVM, and VirtualBox. VMLAUNCH is used only for the first launch of a virtual machine, when its virtual machine control structure has not yet undergone initial initialization during previous exits. For subsequent entries into the guest environment, the VMRESUME instruction is used.

The main problem is VMCS field validation errors. The processor strictly checks the correctness of the control structures, and if allowed bit combinations do not match, VMLAUNCH fails with an error code. Other difficulties include an unaligned physical address for the VMCS, access rights conflicts with shadow memory, and an incorrectly configured guest instruction pointer, which leads to immediate hypervisor interception right after launch.

How VMLAUNCH works

The operating principle begins with the hypervisor preparing a virtual machine control structure in non-contiguous physical memory, which rigidly defines guest behavior: segments, control registers, I/O permissions, and exception masks. Then a pointer to this structure is loaded into a special processor register via the VMPTRLD instruction. When VMLAUNCH executes, the processor hardware reads the entire guest configuration from the VMCS and atomically switches the execution context. The CPU transitions from root mode ring 0 to non-root mode ring 0, loading CR3, CR0, CR4, segment registers, and the guest instruction pointer directly from the prepared structure. The hardware shadow page table mechanism activates second-level address translation, separating host physical memory from guest physical memory. Simultaneously, the processor begins monitoring events according to the bitmaps defined in the VMCS: a VM exit will occur hardware-wise when privileged instructions are executed, configured exceptions trigger, or I/O ports are accessed, returning control to the hypervisor for emulation. Unlike VMRESUME, which uses an already filled guest state from a previous save, VMLAUNCH initializes the machine from scratch, activating the guest environment control block and setting the launch indicator in the VMCS to distinguish the initial entry from subsequent ones.

VMLAUNCH functionality

1. Execution conditions. The instruction is allowed only in VMX monitor mode with privilege level CPL=0. If the logical processor is not in root operation mode, attempting to execute it causes an invalid opcode exception, interrupting the control flow.
2. VMCS checking. Before launching the guest, the processor checks the correctness of VMCS fields, including the guest state, control structures, and MSR bitmaps. Non-compliance with validation requirements causes a VM exit with a reason code indicating a launch failure and passes control to the monitor.
3. Guest state loading. The hardware atomically loads the guest register context from the VMCS area. Segment registers, GDTR, IDTR, control registers CR0, CR3, CR4, general-purpose registers, RFLAGS, and the instruction pointer RIP are loaded, after which execution begins from the guest environment.
4. Operation atomicity. Despite the complexity of internal steps, VMLAUNCH is architecturally an atomic instruction. This means that all processor state changes, checks, and register loads happen as a single unit without intermediate interrupts or exceptions after the state commit begins.
5. Initial launch. VMLAUNCH is used strictly for the first launch of a virtual machine, when the VMCS launch state field is in a clean start state. Calling VMLAUNCH again for an already running or active VMCS results in failure, generating a VM exit with an error signal.
6. Difference from VMRESUME. Unlike VMRESUME, which resumes a previously interrupted guest environment, VMLAUNCH initializes a guest session from scratch. VMLAUNCH is not allowed to use data from a previous exit and requires explicit initialization of all mandatory guest state fields before execution.
7. VMCS link initialization. Before calling VMLAUNCH, the current VMCS must be activated via VMPTRLD. The pointer to the active VMCS is stored in a non-architectural processor register, and the load-check-launch operation proceeds through this hidden addressing mechanism for hardware state.
8. Guest MSR area handling. During launch, the processor applies access filters to model-specific registers. The VMCS stores read and write permission bitmaps for MSRs; if access rights do not match when the guest executes RDMSR or WRMSR, execution is immediately interrupted via a VM exit mechanism.
9. Interrupt and exception control. The VMCS fields specify exception vectors that force an exit from the guest. VMLAUNCH initializes this logic, allowing the monitor to intercept critical events such as page faults and external interrupts without interfering with the flow of normal guest code.
10. EPT and MMU handling. If EPT technology is enabled, VMLAUNCH activates nested address translation. The processor begins using the EPT pointer from the VMCS to translate guest physical addresses to machine addresses, isolating the guest from the host system’s real physical memory map.
11. EPT (Hardware second-level memory address translation)
12. Guest MSR loading. The processor loads the list of guest model-specific registers defined in the VMCS MSR load area. Values of MSRs such as EFER and PAT are atomically restored at startup, which is critical for correctly switching addressing modes and the guest memory architecture.
13. Host state saving. In parallel with loading the guest, VMLAUNCH does not touch the host state area. Saving host registers and MSRs is performed during the subsequent VM exit, not at entry. Thus, a launch requires the monitor to have previously saved its context programmatically before calling the instruction.
14. Execution control validation. The processor verifies secondary control elements based on capabilities advertised in the IA32_VMX_PROCBASED_CTLS2 MSR. If configuration bits are set incorrectly relative to the allowed hardware masks, VMLAUNCH fails with an exit reason code indicating an invalid virtualization control setting.
15. APIC virtualization activation. When the virtualized APIC usage field is configured, the instruction initializes the self-interrupt queue and virtual-APIC page access. This allows the guest to interact directly with its local interrupt controller, minimizing the number of exits to the monitor.
16. Shadow VMCS setup. If Intel PT or nested virtualization technology is active, VMLAUNCH may engage a shadow control structure. The hardware automatically switches links during asynchronous events, linking multiple VMCSs without additional instructions from the software monitor.
17. Monitor mode limitations. Debug modes that engage SMM interrupts are blocked at the moment of VMLAUNCH if the VMCS is not configured for dualistic monitoring. A standard fault in system management mode causes a chipset lock until explicit handling is permitted via the SMM monitor mechanism.
18. Launch failure reasons. A VMLAUNCH failure does not modify the guest context. The processor sets the launch success flag to zero in the VMCS instruction state field. The error code details the failure stage: invalid guest state, incompatible control bits, or exceeding allowed physical address limits.
19. RFLAGS.TF flag behavior. The trace flag from the host RFLAGS is not passed to the guest during VMLAUNCH. The processor saves the previous trap state in hidden fields, preventing unintended entry into a debugger inside the isolated environment. The behavior is regulated by the Monitor Trap Flag setting.
20. Translation caching. VMLAUNCH does not flush global TLB caches unless the forced flush bit is set in the control element. This allows reusing hardware tables for guests with identical address spaces, speeding up context switches in scenarios without CR3 changes.
21. Software security model. The instruction checks the VMCS address for alignment and membership in physical memory. An invalid pointer or a corrupted structure checksum (VMCS revision ID) causes an immediate fatal VM exit, blocking guest launch with a corrupted control state.

Comparisons

• VMLAUNCH vs VMRESUME. VMLAUNCH initiates a virtual machine launch in VMX root operation, performing full guest state initialization from the VMCS and carrying out the initial guest boot. VMRESUME, in contrast, resumes execution of a previously interrupted guest environment after a VM exit, not reinitializing the state but restoring the context from saved fields. The key difference is that VMLAUNCH is used once, while VMRESUME is used cyclically.
• VMLAUNCH vs VMRUN (AMD SVM). VMLAUNCH is an Intel VMX instruction for the initial entry into a guest, requiring a preconfigured VMCS and performing complex validation of control fields before loading state. AMD SVM uses a single VMRUN instruction for any transition into a guest environment, which is architecturally simpler. VMLAUNCH checks VMCS correctness atomically, whereas VMRUN relies on prior validation by the hypervisor, shifting the burden to software.
• SVM (Full hardware isolation of virtual machines)
• VMLAUNCH vs SYSENTER. VMLAUNCH provides a transition to the hardware virtualization level with VMX mode switching, saving and loading the full virtual machine context via the processor-controlled VMCS. SYSENTER implements a fast system call without hardware virtualization, simply switching the protection ring from CPL to a predefined handler. VMLAUNCH introduces strict guest isolation with shadow structures, while SYSENTER operates within a single kernel address space without resource emulation.
• VMLAUNCH vs VMPTRLD. VMPTRLD loads the physical address of the current VMCS into a special processor register, making it active and available for subsequent VMX operations, but it does not transition into the guest environment. VMLAUNCH uses the active VMCS to directly execute guest code, checking state integrity and initiating hardware saving of the host area. The VMPTRLD then VMLAUNCH sequence forms an atomic preparation and execution pipeline, where the first instruction specifies the target and the second performs the transition.
• VMLAUNCH vs IRET. IRET performs a classic return from interrupt, restoring a truncated user context from the kernel stack within the traditional ring protection model. VMLAUNCH performs a fundamentally deeper switch into the VMX non-root context, managing specialized hardware such as EPT and VPID. VMLAUNCH induces a transition between full processor modes with strict physical memory isolation, whereas IRET operates within a linear segmented protection model without hardware virtualization.

OS and driver support

The VMLAUNCH instruction is implemented at the hypervisor and processor levels, so the guest operating system does not require kernel modifications to run a virtual machine; however, for the guest OS to function correctly after control is transferred, a set of paravirtualized enlightenment drivers is needed. These drivers interact with the parent partition via synthetic interrupts and VMBus channels, replacing direct physical hardware accesses with high-performance virtual device operations (network adapters, storage, graphics) without emulating legacy buses.

Security

The security function of VMLAUNCH is based on hardware isolation of the guest machine state within the VMCS (Virtual Machine Control Structure), which is protected from modification by unprivileged code via MMU extensions (EPT — Extended Page Tables), as well as on environment authentication using launch keys (Launch Enclave within Intel SGX or AMD SEV-SNP), where before instruction execution the processor checks the integrity of the guest firmware image and initial vCPU state, cryptographically verifying the absence of memory attacks or code substitution by an attacker.

Logging

Logging of events related to VMLAUNCH is implemented by recording detailed information about launch errors in VMCS fault indication registers and hypervisor system logs, where upon failure (for example, due to invalid execution control field content or checksum violations) the processor automatically rejects the launch with an error code, and the virtual machine monitor records the failed logical processor ID, the error field offset, and launch attempt control flags for subsequent diagnostics by engineers.

Limitations

The main limitations of VMLAUNCH stem from the architectural requirement that the logical processor be in VMX root operation with an environment previously activated via VMXON, the impossibility of recursive nesting without hardware support for nested virtualization, strict validation of guest physical memory pointers (bits beyond the physical address width cause launch failure), and the requirement that the VMCS be aligned on a 4 KB boundary, which imposes strict constraints on the microarchitectural implementation and requires precise conformance to the documented memory model.

History and evolution

Historically, VMLAUNCH was introduced in the Intel VT-x architecture (codename Vanderpool) in 2005 as a means for a one-time transition from the hypervisor to the guest environment with automatic loading of the guest state from an activated VMCS area, and with technology development it gained support for address mode switching and launching 16-bit, 32-bit, and 64-bit guests within a single vCPU lifecycle; subsequently, in AMD-V, the VMRUN instruction with similar semantics was implemented for symmetry, and currently evolution has shifted toward launch time optimization through hardware preloading of shadow structures and integration with trusted execution environments (TEEs) for confidential computing.