UDF (File system for optical media)

UDF (Universal Disk Format) is a way of organizing data on discs, replacing the old ISO 9660 standard. UDF allows you to write files to CD, DVD and Blu-ray in a familiar way, like a flash drive, supporting long names, large volumes and appending data without creating complex session structures. It was created for universality across different operating systems.

The main application of UDF is found in optical media: recordable and rewritable compact discs, DVDs and Blu-ray discs. Packet writing technology makes it the standard for devices that work with media like flexible magnetic disks. The file system is also built into consumer video cameras that record video directly onto optical discs, and is used in some implementations of boot partitions on DVDs.

Typical problems

The main difficulty is related to metadata fragmentation during multiple re-writes, which slows down reading on old drives. Compatibility problems arise with old players and UDF versions above 1.02. Damage to the last session due to a write interruption makes data invisible without recovery. There is also a driver conflict when reading discs formatted in incompatible packet writing implementations between Windows and macOS.

How UDF works

UDF is based on a flexible addressing structure that allows updating the file system in small portions, adding new data to free sectors without recalculating the entire table of contents. This capability is called a virtual allocation table and works on top of the physical track structure. The disc is divided into logical volumes and partitions, and file location information is stored in descriptors with unique tags. The key element is the Anchor Volume Descriptor Pointer, which serves as the entry point and contains references to the main structure — the Volume Descriptor Sequence. Unlike a master file table, UDF operates with chains of extents describing continuous blocks of sectors for each file. During mounting, the system reads the anchor point, then finds the partition map and the file descriptor table, where each object has a unique 64-bit identifier.

When appending, new data is placed in unused space, and metadata is supplemented with new descriptors, forming a new set of closing integrity structures at the end of the session, including an updated Anchor. The sparing tables mechanism allows bypassing defective sectors on rewritable media, redirecting operations to a reserve area, which is critically important for the reliability of RW discs. To implement direct file deletion on rewritable media, a Space Bitmap unused space tracker is used, which marks freed sectors as available without physical erasure. Journaling in some implementations, known as Metadata Partition, places service information in a separate area to speed up metadata operations on expensive professional-grade media.

UDF functionality

  1. File structure and logical volumes. UDF organizes data into logical volumes, which can consist of one or more physical partitions. Each partition is described in descriptors, allowing the creation of complex storage topologies. The logical volume hides physical fragmentation and damaged sectors, providing the operating system with a continuous linear address space for working with data.
  2. Allocation strategy and spared areas. The defect management function is implemented through a system of spared tables. UDF reserves special areas on the disc to replace failed blocks without performance loss. When a write or erase error is detected, data is transparently redirected to a spare zone, which is critically important for rewritable media with a limited rewrite cycle resource.
  3. Descriptor system and anchor points. Metadata in UDF is distributed rather than concentrated in one superstructure. The main reference points are anchor descriptors (Anchor Volume Descriptor Pointer), written in fixed logical sectors. They point to the main and backup chains of the volume descriptor, ensuring the ability to mount even with serious damage to the boot area of the media.
  4. Information control block. The logical volume is described by the Implementation Use Volume Descriptor and the File Set Descriptor structure. The latter contains a reference to the root directory. The uniqueness of the approach is that UDF supports multiple independent file sets within a single volume, allowing multi-session recording and data growth without re-burning the entire disc.
  5. Addressing via virtual blocks. Instead of physical or logical sectors, UDF operates with virtual address blocks of a fixed size. A special partition table (Virtual-to-Logical Mapper) translates virtual offsets into actual physical coordinates on the media. This mechanism abstracts the specifics of disc geometry and provides support for different media sector types.
  6. Extent allocation method. For storing files, UDF uses extents — continuous sequences of blocks described by a start address — length pair. Information about extents is stored either in a short file descriptor or in a more complex long allocation structure. This approach minimizes metadata fragmentation and overhead for describing large multimedia objects.
  7. File descriptor classification. File metadata is stored in a File Entry (FE), which contains attributes and extents. To save space for small files, an Extended File Entry (EFE) is used, adding support for extended attributes and checksums. UDF strictly types descriptors through Tag Identifier fields, which eliminates ambiguity in structure interpretation by the file system driver.
  8. Data stream implementation. UDF fundamentally supports multiple named streams (Alternate Data Streams) for a single file. In addition to the main data body, there can be streams with access rights information, image thumbnails or acoustic data. This property is used when recording high-definition video formats, where an audio track or subtitles are stored in a separate stream.
  9. Distributed directory tree mechanisms. A directory in UDF can be physically stored as a linked list of sections with file identifier (FID) entries. If a directory does not fit into one block, its continuation is set through the Allocation Extent mechanism. The tree connectivity is ensured by back references to the parent element, allowing the structure to be holistically restored in the event of navigation table crashes.
  10. Character encoding and name comparison. File names in UDF are stored in compressed encoding (Compressed Unicode), which uses one or two bytes per character. For name matching during search, the specification requires the application of a strictly defined binary comparison algorithm (binary collation order), guaranteeing unambiguous results regardless of operating system regional settings.
  11. Structure integrity control. Each UDF descriptor structure is supplied with a tag containing a unique identifier and a header checksum. During mounting or reading, the driver is obliged to validate the tags. Detection of an identifier mismatch or a broken checksum interrupts the operation and triggers a search for a metadata duplicate in the backup copy of the virtual allocation table.
  12. Metadata journaling. Within the specification for optical and magneto-optical drives, a sparing table and Metadata Partition mechanism are implemented. This functionality emulates soft journaling: before making changes to the file tree, a duplicate metadata partition with mirrored operations is created. This protects the integrity of the volume during a sudden power loss during a write operation.
  13. Support for non-erasable media. UDF is adapted for Sequential Recording on media like CD-R and DVD-R, where sector overwriting is impossible. The specification introduces the concept of VAT (Virtual Allocation Table), written at the end of each session. VAT redefines the physical location of files, allowing emulation of rewritability on worm media by appending to the last track.
  14. Attribute model and timestamps. The file system stores high-precision timestamps with microsecond resolution in Timestamp format, covering dates from the year 1 AD to 9999. Attributes are supported for differentiating write and execute rights, as well as flags for hidden, system and archive files. The extensible model allows adding application-specific key-value pairs.
  15. Critical data duplication mechanism. For critically important boot structures, UDF applies redundant storage. Descriptors are duplicated at the beginning, middle and end of the volume. The mounting algorithm scans these anchor points and selects a consistent set of metadata, increasing survivability even with fatal destruction of sector zero or the first logical track of the media.
  16. Space distribution strategies. Data placement is regulated by policies embedded in the Sparing Table and Unallocated Space Entry. During formatting, strategies for saving flash memory resources (for SSD or BD-RE) or optimization for streaming reads can be set. The system maintains a bitwise account of free blocks through the Space Bitmap, allowing quick location of continuous extents for recording large video streams.
  17. Compatibility with consumer recorders. The standard strictly limits permissible combinations of UDF versions and physical sectors for consumer devices. The recorder drive looks for a mount point in sector 256, checking the domain for compliance with UDF Bridge specification requirements. This allows creating discs that are read equally correctly on a personal computer and in a stationary video player with an AVCHD interface.
  18. Working with highly sparse files. UDF allows marking uninitialized blocks within a file through special extents of the Unallocated Space type. When reading, the driver is obliged to return zero bytes instead of missing data. This technology eliminates actual space reservation for holes, which is indispensable when containerizing virtual machine images and working with torrent clients in space pre-allocation mode.
  19. Implementation identification and domains. Each logical volume contains an Implementation Identifier field and specifies the UDF domain version. Domains strictly separate operating modes: from random access with full overwrite to purely incremental. Changing the domain closes the session and makes previously written sectors immutable, finalizing the disc for compatibility with read-only devices.

Comparisons

  • UDF vs ISO 9660. Unlike the archaic ISO 9660, limited to 8.3 file names and directory depth, UDF was originally designed for packet writing and supports long names, Unicode and files over 4 GB. ISO 9660 is static and requires a full session for changes, whereas UDF allows adding data to a disc, similar to working with a hard disk, without re-burning the entire structure.
  • UDF vs FAT32. FAT32 cannot handle files larger than 4 GB and is critically prone to fragmentation, which is fatal for optical media. UDF lacks the 4 GB limit thanks to 64-bit addressing and uses a flexible metadata placement strategy that prevents wear on rewritable sectors. Unlike FAT32, UDF is a full-fledged journaled system, which increases fault tolerance during write failures.
  • UDF vs NTFS. Although both systems are journaled and support large volumes, NTFS is redundant for optical discs due to aggressive MFT overwriting in a fixed zone, which quickly kills RW media. UDF, on the contrary, distributes metadata updates evenly across the disc (sparing table), ensuring wear leveling. The key difference: UDF is cross-platform, whereas writing to NTFS in other OSes is often limited or requires third-party drivers.
  • NTFS (File system with journaling and access control)
  • UDF vs exFAT. exFAT, as a lightweight system for flash memory, lacks defect management mechanisms critical for unreliable optical surfaces. UDF wins due to the built-in sparing mechanism, remapping failed sectors without data loss. Also, exFAT loses in metadata: UDF stores redundant copies of key structures (VAT, descriptor space), guaranteeing mounting even with partial media damage.
  • exFAT (File system for flash drives without FAT32 limitations)
  • UDF vs Rock Ridge (ISO extension). Rock Ridge adds POSIX attributes and long names on top of ISO 9660, but remains a hybrid add-on without its own write file structure. UDF natively supports symlinks and extended attributes, requiring no emulation. During incremental recording, Rock Ridge creates inconvenient sessions, whereas UDF uses a virtual allocation table (VAT) for dynamic directory tree restructuring without data duplication.

OS and driver support

The native implementation of UDF in operating systems is built on a modular architecture of file system drivers that interact with the kernel I/O manager. In Windows, the udfs.sys driver mounts disc volumes, recognizing NSR format descriptors (versions 1.02 to 2.60 are supported), and translates stream requests through the IRP (I/O Request Packet) mechanism, providing emulation of POSIX semantics on top of a non-POSIX storage. In the Linux kernel, the udf.ko module is integrated into the VFS (Virtual File System) subsystem, where inode index descriptors are dynamically generated from FID file descriptors on the fly, and metadata read operations are cached in the page cache to speed up work with optical and flash media. In macOS, the driver is based on the I/O Kit subsystem, inheriting from a common abstract IOService class and implementing a locking strategy in a multi-threaded environment through an IOLock object, which allows processing MMC (MultiMedia Commands) commands during packet writing burning. The write functionality in all systems is often moved to user space (for example, via libburn and libisofs), since kernel drivers are mainly read-oriented and do not contain complex space allocators for Sequential and VAT (Virtual Allocation Table) sessions.

Security

The UDF security model lacks built-in access control lists (ACLs) and permission bits characteristic of NTFS or ext4, so access control is implemented exclusively at the operating system level through volume owner identifiers (UID/GID in POSIX or SID in Windows), which the mounting process passes as session parameters when calling mount() or DefineDosDevice(). File structure integrity is guaranteed by duplication of critical metadata, including mirroring of the Anchor Volume Descriptor Pointer in sectors 256, N–256 and N, using descriptor tag checksums (Tag Checksum) calculated by the algorithm of summing the descriptor identifier and version bytes, which allows the driver to automatically select an undamaged copy upon mismatch during mounting. To protect against damage during emergency ejection in implementations with rewritable media, a Logical Volume Integrity Descriptor is used, recording the number of open files and allowing rollback of incomplete transactions in the VAT structure. When writing to blank discs, drivers use a strict write boundary checking strategy, preventing buffer overflow beyond the track size, and in packet writing mode, the WRITE12 SCSI interface command atomically writes data blocks with CRC control, excluding partial corruption of the file system.

EXT4 (Journaling file system for Linux)

Logging

Unlike journaled ext4, UDF does not support a classic metadata journal with a cyclic transaction buffer; instead, to track changes in rewritable volumes, the Sparing Tables and Sparable Partition mechanism is used, which redirects damaged or changed blocks to free areas of the disc, registering the mapping in the sparing table. The historical evolution of logging in the context of UDF peaked in the implementation of VAT (Virtual Allocation Table), where the virtual allocation table is written atomically to the last sectors of the session, creating a snapshot of the entire file system at the time of packet closure, which allows linear structure recovery when reading discs with incremental recording. Internal error logging is performed by the driver through system calls for debug message output: in Linux, the udf_debug macro is used, activated by a kernel parameter, which outputs information about tag checksum mismatches or incorrect descriptors to the kernel ring buffer (dmesg); in Windows, the driver sends an event to the system log via ZwCreateEvent and notifies the Plug and Play manager of media problems, forming an entry with the IoWriteErrorLogEntry error code.

Limitations

A fundamental technical limitation of UDF is the file size limit of 16 exabytes, resulting from 64-bit extent addressing in the Long Allocation Descriptor; however, when using 32-bit block pointers in old versions or incorrect layout during formatting, a developer may encounter a logical block addressing limit of 2^32, which narrows the practical volume size to 8 terabytes with a 2048-byte sector. Performance degrades on fragmented media due to the allocation strategy architecture in the Allocation Extent Descriptor, since the virtual inode analogue (FID) contains up to several hundred extent entries without the ability to build a balanced B+ tree, forcing the disc head to perform intensive seeks when reading large sparse files. A limitation of the VAT variant of UDF is the impossibility of true data deletion without full reformatting, as new VAT versions only mask old sectors by overwriting pointers, which leads to progressive space exhaustion on RW media. Drivers universally suffer from a limitation on symbolic link nesting depth allowed during recursive path traversal, where a static counter is introduced to prevent looping (up to 65535 redirects according to the ECMA-167 standard), exceeding which generates an ELOOP error.

History and development

The format was born as a response to the need for a unified file system for optical media, replacing the outdated ISO 9660, when in 1995 the OSTA (Optical Storage Technology Association) consortium developed the UDF 1.02 specification specifically for DVD video, rigidly fixing the minimum requirements for data placement in player firmware through a mandatory Implementation Use descriptor with the name "*OSTA Compressed Unicode". Technical evolution passed through version 1.50, which introduced support for rewritable CD-RWs and packet writing using the VAT mechanism, to version 2.0, which adapted metadata for streaming high-resolution video and large files using extended file descriptors with 64-bit addresses. Version 2.50 marked a fundamental shift towards enterprise media, adding Metadata Partition sections with duplication of file records in a separate region, which allowed speeding up search and copy operations by grouping metadata, similar to the MFT concept in NTFS. The modern version 2.60 is aimed at working with non-volatile memory and Blu-ray, introducing a pseudo-overwrite management method through the Pseudo-Overwrite Method for solid-state drives, where cell wear is minimized by remapping blocks without physical overwrite. The development of driver implementations in recent years has focused on improving compatibility through strict handling of Unicode NFD (Normalization Form D) in file names during cross-platform exchange, as well as on implementing a Send/Receive mechanism for cryptographic signing of UDF images in closed embedded systems.

Latest publications