XenStore (Shared Xen database)

XenStore is a tree-structured key-value store inside the Xen hypervisor. It provides primary communication and configuration exchange between the host system (Dom0) and guest virtual machines (DomU).

DomU (Automatic network port binding)Dom0 (Xen Virtual Machine management)

This mechanism is used exclusively in the Xen virtualization environment. It transmits paths to disk devices, virtual network interface (VIF) parameters, guest operating system states, and events related to hotplug device connection or disconnection.

The main problem is the formation of a bottleneck when Dom0 fails or the xenstored service crashes, causing all guest systems to hang. Race conditions often occur when rebuilding the tree after a guest suddenly shuts down, as well as transaction log overflow when writing many small keys frequently.

How XenStore works

XenStore functions as a server process (usually xenstored) running in the privileged Dom0 domain. All other domains connect to it via shared memory ring buffers using event channels. Data is organized in a filesystem-like hierarchy, where the root directory contains subdirectories for each domain: /local/domain/0 for Dom0 and /local/domain/ID for guests. Reads and writes are atomic; for consistency across operations involving multiple keys, XenStore supports transactions. A client starts a transaction, performs a series of read or write requests on an isolated copy of the tree, then commits the transaction. If external data has not changed during the operation, the changes are applied; otherwise the operation is rejected and the client must retry. Change tracking is implemented via a watch mechanism: one domain subscribes to a path in the tree, and when the value at that path changes, xenstored sends a notification message through an event channel. For example, when a backend driver in Dom0 creates a node with virtual disk parameters, the frontend driver in the guest receives a signal and initializes its device without active polling. A key feature is asynchronicity: all operations are non-blocking but are processed strictly sequentially by the server, preserving data integrity under high load.

XenStore features

  1. Hierarchy and access rights. The root directory / stores global records. Nodes at /local/domain/0 contain Dom0 parameters, while /local/domain/ID holds guest domain data. Read/write permissions are controlled via domain-based access control lists.
  2. Kernel and userspace interface. The Linux kernel implements a xenstore driver providing /dev/xen/xenbus devices. Userspace processes use the libxenstore library for calls such as xs_open, xs_read, xs_write, xs_directory, xs_rm.
  3. XenBus (Paravirtual domain management channel)
  4. Transaction support. XenStore supports atomic transactions via xs_transaction_start, xs_transaction_commit, and xs_transaction_end. All reads and writes within a transaction are isolated until commit. This prevents race conditions when updating multiple keys.
  5. Data format and limits. Node values are null-free UTF-8 strings, maximum size 4096 bytes. Nesting depth is limited to 256 levels, total nodes in the system can reach hundreds of thousands. Paths are case-sensitive.
  6. XenBus: event channel. XenBus is an abstraction layer over XenStore that delivers event notifications. When a node changes, an event is generated and delivered via a Xen event channel. Guests subscribe to watches.
  7. Watch mechanism. xs_watch registers a callback on a path. When a node is written or deleted, a notification is generated. Event counts can accumulate if the handler is slow. xs_unwatch is required to unsubscribe.
  8. Cross-domain synchronization. All operations through the hypervisor are serialized on Dom0 side where the xenstored daemon runs. Hypercalls: XENSTORE_read, XENSTORE_write, XENSTORE_transaction_start. This ensures distributed state coherence.
  9. Rate limiting. xenstored has built-in rate limiting: no more than 20 operations per second per non-privileged domain. Exceeding this causes temporary request blocking to prevent DoS attacks via write flooding.
  10. xenstored files and sockets. The daemon accepts requests via the socket /var/run/xenstored/socket for Dom0 and via an event portal for guests. For fault tolerance, it supports state replication to two nodes via shared memory.
  11. Node types in XenStore. Standard nodes: /control/shutdown (shutdown trigger), /device/vif/ID/mac (interface MAC address), /backend/vbd/ID/params (block device parameters). A guest creates its own nodes in its own space.
  12. Device backend and frontend. The frontend in the guest creates a node /local/domain/GUEST/device/.../state = Init. The backend in Dom0 reads this node, initializes resources, and changes the state to Connected. Parameter exchange uses keys like ring-ref, event-channel.
  13. Event channel registration. When a domain starts, it receives grant references to the ring buffer and an event channel number. These parameters are published in XenStore under the node /local/domain/ID/device/.../event-channel. The guest is notified via watch.
  14. Power management. For ACPI events, the node /local/domain/0/acpi/state is used. The guest writes to /local/domain/ID/control/shutdown: S5 for power off, S3 for sleep. xenstored forwards to Dom0, which runs the appropriate xend script.
  15. Error handling and logging. On write error, xenstored returns EINVAL or ENOSPC. The daemon logs to /var/log/xen/xenstored.log. State is dumped to /var/lib/xenstored/tdb upon SIGUSR1.
  16. Virtual machine migration. During live migration, xenstored copies the guest node tree to the target host. All transactions are atomically transferred while preserving watch IDs. After migration, watches are automatically re-registered.
  17. Isolation security. A guest cannot see nodes belonging to other domains except its own prefix /local/domain/ID and /local/domain/0/backend where only its own backend nodes are exported. Traversing outside permitted paths is blocked by checks in xenstored.
  18. Performance and latency. Average XenStore operation latency in a typical system is about 100-200 microseconds. 95% of operations complete in under 500 µs. With 1000 simultaneous watches, Dom0 CPU load increases to one core.
  19. Alternative implementations. In place of the traditional Python-based xenstored, oxenstored (OCaml) is used for better scalability. There is also a mini-os xenstored for embedded systems. All implementations are protocol-compatible with xenbus.
  20. Diagnostic tools. Commands: xenstore-ls -f (recursive output), xenstore-read (read), xenstore-write (write), xenstore-watch (monitoring). The xenstore-chmod utility changes node permissions. strace -p pid_xenstored shows system calls.
  21. Extension via permission mechanism. Each node has an owner_domid field. Permissions: r for read, w for write, b for both. Example: xenstore-chmod /local/domain/1/mykey r0 b1 gives domain 0 read, domain 1 read and write.

Comparisons

  • XenStore vs XenBus. XenStore provides a hierarchical key-value store for exchanging configuration and state between domains, whereas XenBus is an event-oriented messaging channel. XenStore is suitable for long-lived data (device paths, kernel parameters), while XenBus is for short notifications and hotplug device management.
  • XenStore vs virtio-vsock. XenStore operates via shared memory with peer-to-peer synchronization built into the Xen hypervisor, requiring a backend driver in dom0, whereas vsock uses virtual sockets over PCI/MMIO transport layer. XenStore offers simple storage without streaming, while vsock is oriented toward IPC with stream and datagram protocol support.
  • XenStore vs 9p (Plan 9). 9p provides a userspace filesystem with full access control and directory hierarchy, more suitable for file sharing between VMs. XenStore, in contrast, is not a filesystem — it is simply a node tree simulation, not supporting open/read/write operations in the standard POSIX sense, but it is more lightweight and low-latency.
  • XenStore vs QEMU QMP (QEMU Monitor Protocol). QMP is a JSON protocol for host-side device emulation control, asynchronous and requiring explicit request-response. XenStore is synchronous for read/write within the guest and does not involve complex commands. XenStore is integrated with the Xen boot process, while QMP only applies to scenarios with QEMU as a device emulator.
  • XenStore vs ACPI shared memory. ACPI SHM is implemented via tables and OperationRegions, used mainly during boot and power management, hardware-dependent. XenStore works exclusively in paravirtualization (PV/HVM with PV drivers), providing dynamic live reconfiguration of a running system, which is fundamentally unavailable with the static ACPI method.
  • HVM (Hardware isolation and virtualization acceleration)PV (Virtual machine I/O acceleration)

OS and driver support

XenStore is implemented as an in-memory key-value tree in domain 0 (Dom0), accessible to guest systems via a shared memory page and event channels. Paravirtual xenbus drivers exist for Linux, FreeBSD, and NetBSD providing APIs for reading/writing nodes. For Windows, an interface through xeniface is used, though full support is limited to Xen versions with official PV drivers.

Security

Access to XenStore is enforced by path-based ACLs and domain privileges: each guest domain sees only its own allocated branch /local/domain/<ID> and cannot modify other domains data. Operations are validated by the hypervisor via transactions and permission checks on nodes (watches/permissions), which prevents inter-VM attacks, but requires DoS protection via operation rate limiting.

Logging

XenStore does not have built-in detailed change logging. Debugging is done by enabling tracing in xenstored code (option --verbose or XENSTORED_TRACE=1), as well as via command-line tools (xenstore-changes, xenstore-ls -p) and gathering Dom0 logs in the system journal (syslog/journald), where access errors, transaction failures, and bad path mount attempts are recorded.

Limitations

Key XenStore limitations: maximum message size 4096 bytes (kernel limit), tree depth up to 20 levels, total node count limited by Dom0 memory (typically tens of thousands), and no atomic updates for multiple nodes without transactions, which if long-running can block all other operations on the same connection.

History and Evolution

XenStore first appeared in Xen 2.0 (2004) as a simple text database for passing boot parameters and device state between Dom0 and guests. Transactions were added in Xen 3.0, and in versions 4.x support for per-domain quotas, refactoring of xenstored into a server with sockets, and migration to libxenstore for unified client API were introduced. However, the fundamental structure (tree, watches, event channels) remains unchanged up to modern releases.