AVC (Advanced Video Coding, or H.264) is an intelligent algorithm that turns a raw data stream into a small file without visible quality loss. The codec analyzes frames, finds repeating areas, and saves only the differences between them, discarding redundant visual information imperceptible to the eye.
The standard is used in Blu-ray discs, terrestrial and satellite television broadcasting, as well as on streaming video platforms. It is used by video surveillance systems and video conferencing. It is H.264 that ensures compatibility: files encoded with this algorithm are guaranteed to play on any modern TV, smartphone, or tablet without installing additional software.
At an excessively low bitrate, the image disintegrates into square blocks — the macroblocking artifact appears. In dynamic scenes, noticeable dirt and ringing noise emerge around objects. Also, encoding high-resolution video (especially 4K) requires significant computational resources, which overheats mobile processors and slows down file export time in video editors.
How AVC works
The working principle is based on hybrid coding. Video is divided into 16x16 pixel macroblocks. Unlike its predecessor MPEG-2, where prediction was coarser, H.264 introduces intra-frame compression: spatial redundancy (textures of sky, walls) is packed through Fourier transform and quantization. Inter-frame compression analyzes motion vectors with quarter-pixel accuracy — the camera pans, and the codec shifts the picture from the previous frame, transmitting only the difference. Then the entropy encoder CABAC or CAVLC kicks in, packing the remaining data mathematically more efficiently than the classic Huffman algorithm. Compared to MPEG-4 (Part 2), multi-reference frames and an adaptive deblocking filter were added, smoothing block boundaries in real time and making the picture visually cleaner even at low bitrates.
AVC functionality
- Block coding with a tree structure. Each frame is divided into
16x16pixel macroblocks, which are the basic processing unit. To improve prediction accuracy, a macroblock can be recursively split into sub-blocks down to4x4size, forming an adaptive quadtree structure. - 4×4 luma intra prediction. Restoration of the current block pixels occurs by extrapolating the boundary values of previously decoded neighboring blocks. Nine directional prediction modes are defined, including vertical, horizontal, diagonal, and a planar DC mode that calculates the arithmetic mean of available reference samples.
- 16×16 luma intra prediction. For uniformly colored areas, whole-macroblock prediction is applied. Four modes are available: vertical, horizontal, DC, and planar. The planar mode approximates the block surface with a two-dimensional linear function, effectively encoding smooth brightness gradients without blocking artifacts.
- Chroma component prediction. The color-difference planes U and V are predicted in
8x8blocks using the same four modes as for16x16luma macroblocks. The prediction order is always fixed: U is computed first, then V; the mode is coded once for both components to minimize overhead information. - Motion compensation with variable block size. For inter-frame compression, a
16x16macroblock is segmented into movable partitions. The standard allows horizontal or vertical splitting into16x8,8x16blocks, as well as into8x8square sub-blocks, which, in turn, can be broken down to8x4,4x8, and4x4pixels for precise approximation of complex motion. - Sub-pixel interpolation of motion vectors. The displacement accuracy reaches a quarter-pixel for luma and one-eighth for chroma. Half-pixel positions are computed by a 6-tap FIR filter with coefficients
(1, -5, 20, 20, -5, 1), quarter-pixels by bilinear interpolation of intermediate values, minimizing the residual error. - Multi-vector prediction for P-type macroblocks. A P-macroblock allows using up to four motion vectors for
8x8partitions. Each vector independently indicates the position of the reference block in a previously decoded frame of the List0 list, providing compensation for non-translational object motion and deformations in the frame. - Bi-predictive compensation of B-slices. In B-frames, averaging of two reference blocks from different temporal directions is supported. Samples can be taken from past and future frames or twice from the same direction. The predictor is formed by weighted summation of samples from List0 and List1, significantly reducing residual variance.
- Weighted prediction. The function applies a linear transformation to reference blocks: multiplication by a weight coefficient and addition of an offset. This is critical for smooth fade-ins or scene dissolves, where simple motion compensation without accounting for global brightness change generates a high-energy residual.
- Residual signal transformation. The prediction residual undergoes an integer approximation of the discrete cosine transform (DCT). An integer
4x4kernel is used, free from direct and inverse transform mismatch errors. For16x16intra-predicted blocks, a secondary4x4Hadamard transform over the DC coefficients is additionally introduced. - Quantization with logarithmic step size. The standard defines 52 quantizer levels, controlled by the QP parameter. The quantization step doubles with every increase of QP by 6. Independent scaling tables are used for luma and chroma transform coefficients, allowing flexible balancing of bitrate and visual texture preservation.
- Zigzag and field scanning of coefficients. The quantized coefficients of a
4x4block are ordered into a sequence by zigzag scan for progressive scanning. For blocks derived from interlaced content, an alternative field scan is applied, grouping coefficients by vertical frequencies earlier than horizontal ones to increase entropy efficiency. - Context-adaptive variable length coding (CAVLC). The entropy encoder adapts VLC table selection based on the statistics of already transmitted neighboring blocks. Separately encoded are the number of non-zero coefficients, trailing ones, levels of remaining coefficients, total zeros, and the length of zero runs before each non-zero value.
- Context-adaptive binary arithmetic coding (CABAC). A highly efficient entropy method providing 10–15% better compression than CAVLC. The probabilistic model of binary decisions is dynamically updated considering the context of the syntax element. Non-binary symbols are preliminarily converted into a binary stream by unary or exponential Golomb coding.
- In-loop deblocking filter. To eliminate visual blockiness at transform boundaries, an adaptive spatial filter is applied. It analyzes the boundary pixel difference of neighboring
4x4blocks and the quantization strength, choosing one of five smoothing levels or fully disabling processing to preserve real sharp scene transitions. - Multi-reference prediction frames. The mechanism allows storing up to 16 reconstructed images in the DPB buffers. P- and B-slices can refer to any of these long-term or short-term reference frames, providing effective compensation for periodic motion and object occlusions without increasing the vector bit cost.
- Arbitrary slice ordering (ASO). The stream is organized into independently decodable slices whose boundaries do not coincide with the raster scan. Slices can follow in any order, which is critically important for error resilience during packet network transmission and for organizing parallel processing on multi-processor decoders.
- Redundant slices. A fault-tolerance mechanism: the encoder generates a duplicate slice with the same spatial coverage but with coarser quantization. If the primary slice is lost during transmission, the decoder restores an image of acceptable quality from the backup copy, avoiding the complete loss of a frame fragment.
- Flexible macroblock ordering (FMO). The distribution of macroblocks into non-contiguous regions (slice groups) is defined by a mapping matrix. The technique allows interleaving blocks in a checkerboard pattern or isolating islands of interest, grouping spatially distant macroblocks in one slice to mask the consequences of packet losses.
- SP and SI slice switching. Synchronizing slices for switching between bitstreams at different rates or during trick play. An SP slice allows switching to another stream without reference drift artifacts, using secondary quantization, even if the reference frames differ. An SI slice implements switching through fully intra-frame recovery.
Comparisons
- AVC/H.264 vs HEVC/H.265. At equal visual quality, HEVC provides bitrate savings of up to 50% thanks to the use of
64x64coding blocks and improved intra prediction. However, the computational complexity of H.265 decoding is significantly higher, requiring more powerful hardware. The H.264 standard retains an advantage in compatibility with legacy devices and in low-latency broadcasting scenarios. - HEVC (Effective video compression with minimal losses)
- AVC/H.264 vs AV1. The AV1 codec, developed by the Alliance for Open Media, demonstrates superiority over H.264 in compression ratio by 30-40% due to more flexible block partitioning and improved post-processing filters. The main drawback of AV1 remains the extremely high encoding resource intensity, making it unsuitable for real-time tasks without specialized hardware encoders, whereas AVC runs efficiently on any processors.
- AV1 (Video codec for streaming)
- AVC/H.264 vs VP9. The VP9 format from Google occupies an intermediate position, surpassing AVC in compression efficiency by about 25% while maintaining acceptable computational complexity. Unlike the problematic licensing of H.264, VP9 is an open and royalty-free standard, which stimulated its adoption on the YouTube platform. Despite this, VP9’s prevalence in the professional video production and television broadcasting industry is significantly inferior to the ubiquitous support for AVC.
- AVC/H.264 vs MPEG-2. Being the direct predecessor, MPEG-2 is inferior to H.264 in coding efficiency by a factor of two or more, requiring double the bandwidth for standard definition video transmission. The transition to AVC allowed broadcasting HD content in the same frequency spectrum where previously only an SD signal fit. Key improvements include quarter-pixel motion compensation and intra-frame prediction, absent in MPEG-2.
- AVC/H.264 vs VVC/H.266. The Versatile Video Codec VVC represents the technological maximum, reducing the bitrate by 40-50% relative to H.264 when encoding 8K ultra-high definition content. This is achieved at the cost of colossal algorithm complexity, including a quadtree with multiple split types. H.264 is characterized by instant software decodability, whereas H.266 will critically require hardware accelerators in the coming years, which postpones its mass adoption.
OS and driver support
AVC implementation in operating systems is built on two levels: software decoding through system codecs and hardware acceleration through graphics processor drivers. The Windows kernel uses Media Foundation and DirectX Video Acceleration (DXVA), passing bitstreams directly to AMD, NVIDIA, or Intel drivers, which utilize specialized ASIC blocks for entropy decoding and motion compensation. Linux systems use Video4Linux (V4L2) and VA-API, where the Mesa driver for Intel and AMDGPU provides access to the Unified Video Decoder without data copying via DMA buffers. macOS and iOS use the VideoToolbox framework, interacting with the Apple Paravirtualized Graphics driver to switch between the x264 software decoder and the fixed hardware pipeline on M-series chips. In the absence of hardware support for Hi10P or 4:4:4 profiles, systems forcibly fall back to software implementation via libavcodec within FFmpeg.
Security
Attack vectors on AVC implementations are concentrated in bitstream parsing and reference frame memory management, where attackers exploit integer overflows when reading exponential Golomb codes or index mismatches in reference picture lists. Mitigation mechanisms include isolating the decoder process in a sandbox via seccomp-bpf with prohibition of system calls not related to memory allocation, and validation of all NAL unit headers for compliance with Annex B standard levels before passing to the hardware decoder. Modern Intel and NVIDIA drivers use IOMMU protection and shadow memory pages so that corrupted motion compensation structures cannot initiate writes beyond the allocated frame buffer. Integrity checks of SEI messages and discarding unsupported NAL unit types prevent information leaks through timing attacks during multi-threaded macroblock recovery.
Limitations
AVC technical limitations manifest in coding granularity: the maximum macroblock size is 16x16 pixels, and motion vectors are limited to a search range from -512 to +511.75 pixels relative to the current position, leading to signaling information redundancy at resolutions above 4K. The High 10 profile is limited to 10-bit transform precision with exclusively integer discrete cosine transform on 4x4 and 8x8 blocks, not allowing the use of more efficient 32x32 or 64x64 block sizes without splitting into macroblocks. The decoded picture buffer is strictly limited by the max_dec_frame_buffering syntax element, which, upon bitstream errors, causes irreversible reference frame state desynchronization until an IDR frame is received. The lack of built-in support for Scalable Video Coding (SVC) extensions in most hardware implementations forces the use of several independent streams, doubling the bandwidth load during adaptive streaming.
Logging
AVC encoding and decoding systems implement structured logging through NAL unit boundary detection and metadata extraction without analyzing copyrighted payload data. Media frameworks log intra-coded and predicted frame numbers, slice types, and slice-level QP quantization values in journals for compression artifact diagnostics without saving the images themselves. GPU drivers, when debug mode is enabled, register hardware decoder occupancy counters, the number of Syntax Errors, and the timings of each pipeline stage via event tracing, aggregating data into system channels like Linux tracefs or Windows ETW without affecting rendering latency. Application libraries such as x264 maintain separate CSV files with per-macroblock bit cost statistics and macroblock type distribution, allowing post-analysis of mode selection algorithm efficiency without re-encoding.
History and development
AVC standardization began in 2001 when the joint team of ITU-T VCEG and ISO/IEC MPEG formed the Joint Video Team with the goal of doubling compression efficiency compared to MPEG-4 Part 2 through the introduction of context-adaptive binary arithmetic coding and multiple reference frames. The first version of the specification was approved in May 2003, defining three profiles — Baseline, Main, and Extended — and key patents were licensed through the MPEG LA pool, which ensured adoption in Blu-ray, DVB-T2 digital television, and streaming without mandatory royalties for free internet broadcasting up to a certain threshold. Technological evolution included the Fidelity Range Extensions addition in 2004 for professional applications with 4:4:4 chroma subsampling and bit depths up to 14 bits, as well as the development of Scalable Video Coding in 2007, allowing reduced-quality streams to be extracted from a single bitstream file without transcoding. Despite the emergence of H.265 in 2013, AVC continues to evolve through optimizations of software encoders like x264 with macro-tree algorithms and psycho-visual distortion weighting, maintaining full compatibility with billions of installed decoders.