LLC (Shared cache memory for all processor cores)

LLC (Last Level Cache) is a large, relatively slow buffer located before RAM. It collects data evicted from the fast private caches of individual cores so that the processor does not stall waiting for slow exchange with RAM.

LLC is used in multi-core central processors (Intel Core, AMD Ryzen) and high-performance GPUs, as well as in server systems on a chip (SoC) to accelerate access to shared data. It is critically important in tasks with intensive information exchange between cores, such as scientific computing, rendering, processing large data arrays, and virtualization, where it reduces the load on the memory controller.

SoC (Integration of all computer components on a single chip)

The main difficulty is maintaining data coherence when a line is modified by one core, which forces the invalidation of copies in the LLC of other cores. False sharing, when cores change different variables in the same cache line, causes unnecessary data flushes. There is also displacement of useful information by streaming data when scanning large memory volumes, which temporarily destroys performance.

How LLC works

Physically LLC is static memory (SRAM) divided into banks for parallel request processing. Unlike L1 and L2, which usually operate on an inclusiveness or exclusion scheme, modern LLCs often use a non-inclusive policy, meaning data is not rigidly duplicated from higher-level caches but allocated flexibly, preventing capacity loss.

SRAM (Fast volatile random storage of bits)

The key difference from L1/L2 is the command flow. If L1 caches instructions and data with a latency of 3-5 cycles, and L2 acts as an intermediary with a latency of 10-15 cycles, then LLC takes on the role of a DRAM request filter, masking the colossal memory latency of hundreds of cycles. The victim selection algorithm (usually pseudo-LRU) analyzes the access history: when the LLC controller receives a miss from any core, it checks its tags. On a hit, the line is returned and its age is reset. On a miss, a request is sent to the system agent for reading from RAM. LLC also contains a hardware prefetch mechanism that spectrally analyzes miss patterns and loads data before a direct core request, which fundamentally distinguishes it from static buffers of lower levels that respond only to direct requests.

DRAM (Storage and Byte-addressing of Data)

LLC functionality

  1. Physical topology and distribution. In modern multi-core processors, LLC is physically distributed among cores or clusters (chunks), but logically unified. Such non-uniform cache architecture (NUCA) leads to differences in access latencies depending on the location of the requesting core and the LLC bank controller.
  2. Inclusiveness and placement policy. LLC can be inclusive, exclusive, or non-inclusive. In inclusive schemes, LLC stores copies of all lines from upper caches, which simplifies inter-core snooping but leads to inefficient area usage due to duplication. Exclusiveness maximizes useful capacity.
  3. Search mechanism and tagging. Addressing within LLC is performed by physical address. The tag directory stores coherence state bits (MESI/MOESI), allowing LLC to serve as an arbiter. Data search in distributed banks proceeds in parallel or via a hash function for request balancing.
  4. Coherent request filtering. LLC is equipped with snoop filters (usually based on JETTY-like structures or implemented in the directory), preventing the need to poll all cores. If a line is not found in the LLC, it is considered not locally modified, which cuts off unnecessary broadcast requests.
  5. Line replacement schemes. Adaptive schemes that simulate Belady's algorithm without full knowledge of the future are applied to manage data eviction. Modern controllers use RRIP with dynamic priority boosting for lines that have been re-accessed, which is more efficient than classic LRU for scan-miss workloads.
  6. Multi-level parity and ECC. Although LLC SRAM cells are more stable than DRAM, the volume of accumulated charge creates a risk of soft errors. In server processors, ECC or SECDED schemes are implemented directly into LLC tag and data arrays to ensure fault tolerance without resorting to slower memory.
  7. ECC (Memory Error Detection and Correction)
  8. Power management. LLC generates significant static leakage current. Voltage-based power management is applied for unused banks. Sleep transistor technology cuts power to entire SRAM blocks, preserving only the tag state needed for coherence tracking.
  9. Interaction with hardware prefetch. LLC controllers interact with prefetch engines that analyze miss patterns. If the prefetcher detects a regular address stride, data is preloaded into the LLC, hiding DRAM latencies while protecting against cache pollution with useless data.
  10. Write policy and non-blocking behavior. LLC operates as a write-back cache, accumulating modified lines for deferred writing. At the same time, it must be non-blocking, meaning it serves other hits or misses while waiting for a line fill from memory due to a previous miss. MSHR structures are used to track outstanding transactions.
  11. DMA and I/O support. In shared memory architectures, LLC is the collection and coordination point for I/O streams. Without DMA caching, devices would have to work directly with slow DRAM, while LLC enables protocols like DDIO, significantly accelerating network packet operations.
  12. Instruction and data caching. LLC is unified, so it accepts both commands and data. On an L1-I miss, the instruction is requested from the LLC. Technically important is support for large page caching: the unified TLB and eviction policies account for macro-block boundaries to prevent miss conflicts.
  13. TLB (Translation Lookaside Buffer)
  14. Quality of service and isolation. Cache Allocation Technology allows the hypervisor or OS to reserve portions of LLC for specific service classes, assigning them a bitmask of allowed associativity ways. This prevents a situation where one noisy neighbor completely displaces the useful data of a latency-sensitive application.
  15. Dead block technology. To increase effective LLC capacity, the moment of last access to a line is predicted. Blocks with zero reuse probability are marked as candidates for early eviction, bypassing the standard replacement algorithm and not occupying space in the save queue.
  16. Bandwidth compression. Some microarchitectures implement LLC line compression. When a line is filled from memory, data is compressed by a hardware compressor, allowing multiple effective lines to be stored in one physical slot, which increases effective capacity without increasing physical SRAM.
  17. DRAM request scheduling. The LLC controller reorders back-invalidations and write streams to the memory controller. Reordering dirty evictions considering open DRAM banks reduces latencies by maximizing page-hit bursts, transforming a chaotic request stream into rhythmic traffic.
  18. Cryptographic data protection. In trusted execution environments, LLC buffers data from enclaves in encrypted form. Memory controllers integrate with the MKTME encryption engine, and LLC ensures that lines belonging to protected domains never leave the processor die in plain form.
  19. Miss diagnostics and telemetry. CBo and LLC controller monitoring blocks provide occupancy counters for associativity ways, hit and miss latencies. These PMU events are streamed to profiling tools to detect false line sharing or suboptimal memory access patterns.
  20. Patch update granularity. To fix incorrect operation of eviction schemes or adaptive prefetch thresholds, manufacturers implement the ability to microcode patch the LLC controller directly during operation, changing behavior without stopping the system by writing to special model-specific registers.
  21. Atomic operations at the cache level. LLC contains logic to execute atomic instructions (compare-and-swap, atomic addition) directly within the controller, without forwarding to the core. This reduces bus load and synchronization latencies by holding the line in an exclusive state during modification.
  22. Row-hammer and memory error mitigation. LLC acts as a filter for repeated DRAM row activations. By tracking miss history, the controller can aggressively close rows receiving suspiciously frequent accesses, preventing charge leakage and bit corruption in neighboring physical memory rows.

Comparisons

  • LLC vs Upper-level cache memory (L1/L2). LLC functions as a shared data reservoir for all cores, while L1 and L2 caches are strictly isolated or distributed among specific cores. Data evicted from private caches migrates exactly to the LLC, which minimizes costly RAM accesses and ensures data coherence in a multi-core architecture without duplication.
  • LLC vs RAM (DRAM). LLC acts as a high-speed buffer leveling the colossal latency difference between the core and DRAM. If DRAM is measured in tens of nanoseconds, LLC access takes just a few cycles. LLC contains the latest copies of data before their evacuation to memory, significantly increasing the effective throughput of the memory subsystem.
  • LLC vs Translation Lookaside Buffer (TLB). LLC operates by caching data and instructions based on physical addresses, while TLB caches virtual addresses to accelerate page translation. LLC is physically indexed and tagged, which requires the address translation process to be completed before access. Their interaction is critical: a TLB miss can delay access even to data already present in the LLC.
  • LLC vs Non-coherent I/O cache memory. Unlike LLC, which guarantees hardware coherence for processor cores, I/O devices often use non-coherent buffers. During DMA operations, the controller must manually check the LLC for data relevance, otherwise inconsistency will arise. LLC requires bus snooping mechanisms or filters to forcefully synchronize data streams with peripherals bypassing the classic MESI protocol.
  • LLC vs High Bandwidth Memory (HBM). Although HBM can be used as LLC in specialized accelerators, architecturally it is a separate on-interposer level, distinguished by much larger capacity but also greater latency than traditional on-die LLC. HBM integration does not replace but rather extends the memory hierarchy, creating an additional buffer between the on-die SRAM cache and classic external DRAM.
  • HBM (3D stacked memory with silicon vias)

OS and driver support

The operating system interacts with LLC indirectly, through cache coherence mechanisms and the memory management subsystem, without directly managing line replacement, since eviction algorithms are implemented in hardware in the cache controller. Device drivers using direct memory access interact with LLC through write-through or write-back, and in architectures with non-coherent DMA perform manual invalidation or flushing of address ranges using instructions like CLFLUSH. Hardware-accelerated I/O employs Intel Data Direct I/O technology, allowing network adapters and NVMe drives to read and write data directly to the LLC without accessing main memory, which reduces latencies and power consumption, while system cache state monitoring is carried out through model-specific registers, providing hit, miss, and replacement event counters.

Security

LLC is a vector for side-channel attacks, so protection is implemented through hardware cache way isolation using Cache Allocation Technology, which assigns limited subsets of ways to specific service classes or virtual machines, preventing the displacement of their data by untrusted code. Spectre and Meltdown class attacks required the introduction of indirect speculative execution barriers at the microarchitecture level, which force the processor to flush predictions when crossing privilege boundaries and clear the LLC of traces of speculatively loaded data. Full LLC encryption with one key per core or per virtual machine (AMD SME technology) ensures data confidentiality at the moments of its eviction to RAM, while the SEV-ES extension additionally protects register state when exiting to the hypervisor level. Timing-based attacks are mitigated by randomizing the mapping of physical addresses to LLC slices and ways, complicating the attacker’s construction of eviction sets and forced eviction of specific cache lines.

Logging and diagnostics

Hardware logging of LLC events is conducted through built-in programmable performance counters, which are capable of recording the number of hits, misses, write-through operations, and hidden prefetch events with precise binding to a hardware thread or core without generating interrupts. To reconstruct request traces in real time, Intel Processor Trace is used, recording packets about cache access results, including precise timestamps, which allows the hypervisor and debugger to analyze abnormal miss spikes with instruction-level accuracy. The extended Resource Monitoring IDentification subsystem exports LLC occupancy metrics to the OS level broken down by resource monitoring identifiers, enabling the task scheduler to dynamically detect cache-polluting threads and migrate them to a less loaded compute domain. Internal controller error events (correction of soft tag or data failures) are logged in machine check banks with severity classification and are often transparently corrected by error correction code without stopping the computational process, leaving only an informational record in the system log.

Limitations

The fundamental limitation of LLC lies in its finite size and fixed associativity, due to which, when the working set exceeds the cache volume, a thrashing effect occurs with continuous eviction of actively used lines, and no software management can compensate for this shortage without restructuring the algorithm to improve spatial-temporal locality. Determinism of access time is absent in the mode of LLC sharing between hardware threads, since one processor can monopolize shared ways, displacing the data of a critical real-time application, and to prevent such collisions, mandatory application of Quality of Service way partitioning is required, which reduces the effective cache volume for each side. LLC is physically unable to service non-cacheable I/O transactions coming from strictly ordered ranges, which forces the placement of DMA buffers in non-cacheable memory and completely excludes such data from the fast track. With deep scaling of the number of cores, increasing a unified LLC encounters a quadratic growth in latencies on coherent buses and a huge die area occupied by tags and comparators while maintaining low latency, which forces a transition to segmented non-uniform architectures with partitioned last-level clusters.

History and development

Early systems used a motherboard with external cache memory on static memory operating at bus frequency, and only with the release of the Pentium Pro was the LLC first moved onto the same die with the core, implementing full-speed block exchange over a dedicated back-side bus. The next stage was the integration of the LLC into a common ring or mesh on-die interconnect, turning the last-level cache into a distributed bank, physically distributed across tile sections but logically addressable as a single whole with non-uniform access time depending on the core’s distance to the data slice. The introduction of a non-inclusive storage policy and victim cache technology eliminated the mandatory duplication of lines from L1 and L2 into the LLC, freeing significant capacity for truly evicted data and radically increasing effective capacity without increasing the physical transistor count. Modern development is aimed at three-dimensional vertical stacking with LLC arrays placed on separate dies connected by through-silicon vias, which provides extremely high throughput and reduces energy costs for petabyte-scale transfers in artificial intelligence accelerators, where LLC transforms into a hierarchical high-speed tensor buffer directly managed by the computational graph compiler.