LSU (Execution of load and store operations)

LSU (Load/Store Unit) is a specialized processor block responsible for reading data from memory and writing data to memory. In simple terms, when a core needs to get a number for a calculation or save a result, it sends a request precisely to this module, bypassing the arithmetic execution pipelines.

LSU is indispensable in superscalar architectures (x86, ARM) and graphics accelerators, where dozens of execution units work in parallel. It services operations with RAM and L1 data cache, handling misaligned addresses in a single pass. In digital signal processors, LSU is often divided into independent read and write channels to simultaneously feed operands to the multiplier-accumulator.

The main challenge is cache fragmentation due to block size mismatches, which reduces throughput. Memory barriers and dependency resolution require a strict data visibility order, otherwise hard-to-catch races occur. Register window stalls due to long DRAM latency cause pipeline bubbles. Alignment errors on some platforms can generate exceptions, destroying the execution flow without hardware correction.

How LSU works

The operating principle is based on the interaction of an asynchronous pipeline with buffer queues. LSU decodes the address, computing the base register plus offset, and simultaneously checks cache memory tags. Unlike a simple DMA controller, which copies large blocks autonomously, LSU executes single instructions within the general stream, maintaining a strict consistency model. On an L1 cache miss, the block sends a request to the ring bus or inter-node switch, temporarily saving the operation state in the miss buffer, which allows the pipeline to continue speculative execution of subsequent instructions. Compared to the classic request-response model through the northbridge, a modern LSU integrates address translation tables and access pattern predictors, providing data prefetch before the program explicitly requests it.

LSU functionality

1. Organization of load and store queues. LSU contains separate hardware queues for read and write operations, buffering micro-operations until their execution and afterwards. The store queue holds data and address until flushed to cache, and the load queue aggregates requests for out-of-order execution, minimizing pipeline blockages on misses.
2. Static and dynamic scheduling. The LSU scheduler analyzes address and data operand readiness, managing instruction dispatch to execution ports. In out-of-order processors, the block dynamically selects the oldest ready operation while tracking register and flag dependencies, eliminating conflicts when accessing data cache ports.
3. Virtual address generation. The arithmetic-logic unit within LSU computes the effective address by summing the contents of the base and index registers with the immediate offset according to the addressing mode. The resulting virtual address is passed to the translation subsystem and simultaneously directed to the tagged cache for speculative indexing at early pipeline stages.
4. Address translation block (TLB). The integrated associative translation micro-buffer performs accelerated conversion of virtual address to physical, caching recent page table entries. LSU initiates a hardware page table walk on a TLB miss, saving the operation context in the queue until the page fault is resolved.
5. TLB (Translation Lookaside Buffer)
6. Speculative execution of loads. LSU implements a value prediction mechanism to bypass memory dependencies, allowing loads to execute before conflicting older stores are resolved. The predictor block analyzes instruction history, forming a hypothesis about reordering safety, and the result undergoes verification at the reorder buffer stage.
7. Memory conflict detection. Store queue comparators continuously check address range overlap of speculative loads with addresses of incomplete stores. When a match is detected, the younger load is marked as non-executable until the older data is flushed, and the pipeline initiates a restart with recovery of speculative state.
8. Store data forwarding. To minimize latency, LSU forwards bytes from incomplete write operations directly to the load buffer, bypassing the first-level cache. The alignment and shift scheme assembles the resulting word from intersecting store blocks, supporting misaligned access without penalties on a forwarding buffer hit.
9. Level-one data cache interface. The block interacts with L1D cache banks through multiplexed read and write ports, supporting non-blocking transactions on cache misses. The line fill controller receives data from the lower memory hierarchy, restarting the waiting load immediately upon critical word arrival.
10. Misaligned access handling. LSU hardware splits requests crossing a cache line or natural alignment boundary into a series of atomic micro-sequences. A dedicated finite state machine generates paired accesses to adjacent lines, merging the result in the register file without generating an exception and with minimal throughput loss.
11. Atomic operation support. The module manages execution of Read-Modify-Write family instructions, holding exclusive cache line monitoring throughout the modification cycle. The arbiter blocks invalidating snoop requests between read and write, guaranteeing transaction atomicity in the memory subsystem with coherence.
12. Memory barrier management. LSU decodes and dispatches ordering operations (fence), blocking the issue of new instructions until queues are fully drained. For load or store barriers, selective blocking is applied: the block waits for global visibility confirmation for stores or completion of all preceding reads.
13. Multithreaded buffering (SMT). In simultaneous multithreading processors, LSU queues are statically or dynamically partitioned between logical cores. The arbiter prioritizes command issue from threads with critical latencies, preventing resource exhaustion by one thread through quota allocation of entries in the senior store buffer.
14. SMT (Hardware emulation of two logical processors)
15. Data prefetching. The LSU hardware prefetcher analyzes load miss patterns, generating anticipatory requests for cache fill without blocking the destination register. The learning block detects striding address sequences, and the predicted line is requested at low priority so as not to clog bandwidth for real demand.
16. Memory type validation. The controller checks physical address page attributes for operation type compliance: prohibition of speculative reads from Device memory, detection of writes to execute-only regions. Upon policy violation, LSU throws a fatal alignment or access exception directly at the instruction commit point.
17. Register file and result writeback. The writeback interface translates loaded data to the destination physical register file port with byte shift correction and sign or zero extension. The data passes a recovery stage after speculative states before being committed to architectural state at retire.
18. Resource and power monitoring. LSU is equipped with performance counters recording the number of full queues, pipeline idle cycles, and forwarding triggers. The block local clock generator can be gated independently of the core, cutting off clock supply during prolonged idle time, which reduces dynamic power consumption.
19. Exception and interrupt handling. When an emergency situation occurs (cache parity error, TLB fault), LSU raises an abort flag for the corresponding execution branch. The pipeline flushes the opcode, and the faulting instruction address and error status are captured in dedicated block registers for the interrupt handler.
20. Interaction with the coherence subsystem. The block participates in the snoop request protocol, listening on the coherence bus for shared line invalidations. If a modified line is requested by another agent, LSU initiates a deferred write to the lower-level cache, temporarily delaying acknowledgment to maintain memory ordering.
21. Decompression and word assembly. When reading compressed data from memory, LSU performs on-the-fly unpacking, placing variable-length objects in the register file. The alignment block shifts byte streams, masking unused bits, and also merges sub-words during partial-word write operations, implementing atomic memory updates.

Comparisons

• LSU vs AGU (Address Generation Unit). LSU is responsible for the complete load/store command processing cycle, including cache access, virtual-to-physical address translation, and alignment checking. AGU performs a narrower task, computing the effective address by summing the base and index registers, passing the ready result to LSU for further memory operations.
• AGU (Computing effective memory addresses)
• LSU vs MOB (Memory Ordering Buffer). LSU in modern out-of-order cores contains several load and store buffers, but it is the MOB that guarantees architectural memory access correctness. MOB tracks dependencies between instructions, detects ordering violations, and initiates pipeline restarts, while LSU directly executes reads after all conflicts are resolved.
• LSU vs Store Buffer. The store buffer is a logical part of LSU and is intended for deferred result writing. While LSU manages entire memory pipelining, the Store Buffer accumulates data and addresses of store instructions, allowing the core to continue execution without waiting for the write to first-level cache memory to complete.
• LSU vs L1 Data Cache Controller. The data cache controller performs low-level access to tag and data arrays of static memory, checking hits and misses, implementing replacement policy, and servicing misses through the interface with the next memory level. LSU coordinates the operation of this controller, managing address translation and priorities of multiple requests.
• LSU vs MAU (Memory Access Unit) in GPU. In graphics architectures, the MAU implements a massively parallel access model, aggregating requests from dozens of threads to form optimal transactions to global memory. Unlike a central processor LSU, which is oriented towards minimizing latency for a single thread, the MAU sacrifices latency for achieving maximum throughput through request coalescing.

OS and driver support

The operating system kernel and drivers interact with LSU through instructions with explicit memory barrier semantics (e.g., sfence, dmb), guaranteeing write ordering to device control registers; the LSU implementation includes a hardware Store Buffer queue, which, upon receiving a barrier command, forcibly stalls the pipeline until all pending writes are fully flushed to cache memory, preventing I/O operation reordering critical for correct hardware management.

Security (process isolation)

To prevent data leakage between processes, LSU checks virtual addresses at the instruction execution stage: the Address Generation Unit (AGU) computes the linear address, while the parallel protection module verifies segment access rights, protection keys, and current protection ring privileges, using high-speed associative memory of the Translation Lookaside Buffer (TLB), which guarantees a hardware page fault exception is raised before a speculative load can alter the microarchitectural cache state.

Logging (error tracing)

The integrated logging system writes packets containing exact virtual and physical addresses of missed requests, bus error code, and thread identifier into a dedicated trace bus ring buffer (e.g., Intel PT or ARM CoreSight), allowing post-factum data analysis without pipeline stalling; upon detection of an uncorrectable parity error on the data bus, LSU immediately generates a machine check exception, capturing a snapshot of the architectural state in Model-Specific Registers (MSR).

Limitations (port throughput)

The main microarchitectural limitation of LSU lies in the finite number of read and write ports, which strictly limits throughput: even with zero first-level cache latency, a superscalar core cannot issue more than two load operations per cycle if the hardware contains only two AGUs, so the scheduler dynamically analyzes address dependencies and delays the dispatch of instructions whose base registers have not yet been computed, blocking the reservation station.

Evolution (speculative execution)

LSU evolution has led to the introduction of speculative loads that bypass strict program order to hide RAM access latency: the scheduler dispatches a load instruction without confirming the readiness of preceding store addresses, and the memory ordering check block in the retirement phase compares overlapping ranges, rolling back the pipeline with a pipeline flush upon detecting a data forwarding conflict, which is technically implemented through an associative search in the Load Queue by the lower bits of the physical address.