What is INT8 (Quantization of neural network weights and activations)

INT8 is a computation compression method where 32-bit floating-point numbers are replaced with 8-bit integers. The neural network begins to consume less memory and runs faster, while prediction accuracy decreases only slightly. This is a compromise between inference speed and model quality.

The technology is applied for inference on resource-constrained devices. These include data center accelerators, smartphone processors, IoT devices, and automotive driver assistance systems. INT8 is critically important for running large language models in real time with low latency and deploying computer vision on surveillance cameras.

The main engineering challenge is information loss due to weight rounding and activation outputs exceeding dynamic range boundaries. Deep or noise-sensitive layers (first and last) may produce incorrect results. Range calibration on a representative data sample is strictly mandatory, otherwise a significant shift in feature distribution occurs, destroying accuracy metrics.

How INT8 works

The working principle is based on an affine transformation between spaces: the real value equals scale_factor multiplied by ( integer_8bit minus zero_point ). Unlike FP16, where numbers are represented non-uniformly and precision drops only at extreme values, INT8 requires prior statistics collection to determine the scaling coefficient. Compared to FP32, where multiply-accumulate is performed in IEEE-754 format without mantissa significance loss, INT8 replaces expensive floating-point operations with fast integer tensor cores. Modern frameworks implement per-layer quantization: a unique quantization step is selected for each tensor, and accumulation calculations are performed in INT32 to prevent overflow, with subsequent dequantization of the result.

FP16 (Compact representation of Floating-Point numbers)

INT8 functionality

Activation quantization to 8 bits. The process converts 32-bit activation tensors into INT8 format through symmetric or asymmetric mapping. The main goal is a radical reduction of floating-point multiplication operations during inference without full network weight recalibration.
Dynamic range and quantization step. The FP32 value range is mapped onto 256 discrete levels. Computing the scale coefficient and zero_point determines the quantization step, critically affecting the preservation of low-power signals in distribution tails.
Integer multiplication arithmetic. INT8 inference replaces FP32 accumulation with multiply-accumulate operations on integer operands. Modern GPU and NPU compute units perform INT8 dot product, accumulating the result in a 32-bit register to prevent overflow and preserve gradient precision.
NPU (Specialized processor for parallel tensor computations)
Asymmetric quantization versus symmetric. The asymmetric scheme uses a non-zero reference point, precisely mapping the tensor zero to the physical signal zero. The symmetric scheme fixes zero_point strictly at zero, simplifying convolution logic but reducing accuracy for signals with shifted distribution (ReLU activations).
Post-training weight quantization. Model weights are calibrated on a small set of representative data without retraining. Entropy calibration collects activation histograms, minimizing Kullback-Leibler divergence between original and quantized distributions to select optimal saturation thresholds.
Quantization-aware training. The technique emulates quantization errors during the forward pass, applying a straight-through estimator for backpropagation. This allows weights to adapt to quantization noise, restoring accuracy lost through naive rounding, especially in models with low parameter redundancy.
Layer fusion scheme. Batch normalization operations are mathematically folded into the weights of the preceding convolutional layer before quantization. This eliminates a separate FP32 node from the computation graph, allowing convolution with bias to be quantized into a single INT8 block and reducing memory context switching delays.
GPU tensor cores. NVIDIA Tensor Cores architectures and analogues accelerate INT8 inference by 4-8 times compared to FP16. The hardware design supports 8x8 by 8x8 matrix multiplication with a 32-bit accumulator. Throughput reaches its peak with tensor structures aligned to 128 bytes.
Tensor (Multidimensional container for numerical data)
Range and percentile calibration. Beyond min-max, the 99.9th percentile of the distribution is chosen to clip outliers. Long activation tails caused by anomalous values are truncated, preventing resolution degradation of the main data array and sharply improving model accuracy metrics after quantization.
Bias correction mechanism. Weight quantization creates a systematic shift in layer output. The difference in mathematical expectations between the original and quantized filter is computed for per-channel bias compensation, eliminating static accumulation error and stabilizing the accuracy of deep residual networks.
Layer sensitivity to precision. Analysis determines that the initial and final layers of the network are more critical to 8-bit reduction than intermediate ones. A mixed quantization strategy leaves sensitive layers in FP16, converting the rest to INT8, balancing overall model compression with preservation of information capacity.
Hardware support for symmetry. Processor integer instructions (VNNI on x86, SME on ARM) require symmetric quantization with a zero point. The uint8 format is multiplied by int8 with accumulation in int32, minimizing the latency of vpdpbusd instructions, significantly outperforming naive scalar extensions on mobile architectures.
ARM (Energy efficient execution of processor instructions)x86 (Execution of instructions based on CISC architecture)
Embedding quantization. Lookup table operations in recommendation systems are quantized to 8 bits with controlled degradation of ranking quality. Stochastic rounding is preferable here to nearest rounding, as it preserves the mathematical expectation, preventing one-sided gradient shift in the lookup layer.
Stochastic rounding. Instead of deterministic bit dropping, a probabilistic function is applied where the rounding probability is proportional to the proximity to the upper quantization level. The method gives an unbiased estimate on average, which is vital for gradient descent and quantization of ultra-light mobile networks.
Inference graph optimization. Compilers (TensorRT, OpenVINO) aggressively search for Conv-Bias-ReLU patterns, fusing them into a single INT8 node. This eliminates extra memory cycles for writing intermediate results and allows zero-copy data transfer cycles when passing between compute cores.
Integer approximation of sigmoid. Non-linear functions like SWISH or Sigmoid have no native 8-bit instructions. LUT tables of 256 values or piecewise-linear second-order approximations are applied, computed entirely in integer arithmetic without converting back to float at activation operator boundaries.
Memory bus width utilization. The transition to INT8 reduces pressure on DRAM bandwidth by two to four times compared to FP32. For LLM models limited by HBM exchange speed, packing four 8-bit scalars into one data packet is the key driver of inference throughput growth.
DRAM (Storage and Byte-addressing of Data)HBM (3D stacked memory with silicon vias)
Per-channel quantization. Instead of a common scale for the entire tensor, the scale coefficient is computed for each output channel of the convolution. This accounts for significant dynamic range variation between filters, sharply reducing rounding error but increasing the computational load on the bias correction block.
Conversion for microcontrollers. TF Lite Micro quantizes the model in int8 from input tensor to output. A fully integer computation pipeline is implemented without floating-point emulation, allowing keyword detection to run on Cortex-M4 with power consumption under one milliwatt in real time.

Comparisons

INT8 vs FP32. Quantization to INT8 reduces the bit width of weights and activations from 32-bit floating-point numbers to 8-bit integers, cutting memory footprint by four times and significantly increasing throughput through the use of fast integer instructions. The cost is a minimal loss of model accuracy controlled by calibration, while FP32 remains the reference training format due to its wide dynamic range.
INT8 vs FP16. Compared to the 16-bit floating-point format, INT8 halves the model size and is suitable for deployment on processors without efficient half-precision computation support. However, FP16 retains a wider dynamic range, which often allows avoiding the complex calibration inherent to INT8 and demonstrates better quality on networks with non-uniform weight distributions.
INT8 vs BF16. BF16 retains an 8-bit mantissa like FP16 but inherits an 8-bit exponent from FP32, preventing overflow during training unlike pure integer INT8. Where BF16 provides simple model transfer without recalibration, INT8 offers a two-fold advantage in throughput and energy efficiency at inference, sacrificing dynamic range for compute density.
BF16 (Floating-point number format for machine learning)
INT8 vs INT4. Quantization to INT4 is twice as aggressive as INT8, which is critical for running large language models on edge devices with strict memory constraints. Despite this, INT8 retains significantly higher accuracy without requiring complex error correction and mixed-precision techniques, remaining the standard for server inference with stringent quality requirements.
INT8 vs FP8. The new 8-bit floating-point format (FP8) eliminates the main drawback of INT8 — the need for static calibration with per-tensor scaling coefficient determination. Unlike the fixed scale of INT8, the FP8 exponent dynamically adapts to distribution outliers, providing more predictable quality in generative models at the cost of slightly more complex hardware implementation compared to a classic integer accelerator.
FP8 (Compact representation of numbers for fast computations)

OS and driver support

INT8 computation support is implemented through low-level libraries such as Intel oneDNN and NVIDIA cuDNN, which directly interact with kernel-mode drivers to program specialized tensor cores or DL Boost instructions. The operating system is not required to have native support for the INT8 data type; instead, the driver translates high-level computation graphs into machine code, configuring power and frequency control registers to prevent throttling during sustained execution of saturated vector operations.

Inference security

Model security during the transition to integer precision is ensured by a calibration procedure that minimizes information entropy between the outputs of the reference FP32 network and the quantized version, as well as anomaly detection mechanisms at the execution stage that analyze layer activation distributions. To prevent attacks via malicious input tensors, the driver restricts value boundaries to the strict range [-128, 127], and the runtime environment forcibly zeroes out unused alignment bytes, eliminating leakage of residual data from GPU or neural processor memory into the output vector.

Deterministic logging

The logging system is built on API-layer inference call interception, where each quantization operation records scaling parameters and zero_point, saving them to an immutable binary log with a GPU monotonic clock timestamp. When numerical instability occurs, the tracing module automatically dumps input tensors in INT8 format along with the selected rounding mode and overflow flags written to the status control register, enabling layer-by-layer divergence comparison with the reference computation graph without stopping the production service.

Limitations

The fundamental limitation of the INT8 type lies in the absence of a native zero representation in the symmetric quantization scheme, requiring manual range offset, which leads to precision loss on sparse feature maps and incorrect operation of division operations. At the hardware level, inference speed is limited by PCIe bus bandwidth when transferring compressed weights to video memory, as well as by the architecture of execution units, which in most accelerators do not support atomic INT8 reduction operations, forcing partial sums to be conducted in INT32 accumulators with subsequent back-conversion and saturation risk.

Format history and development

The 8-bit inference technique evolved from early post-training weight compression methods, where the scale was determined exclusively by the global maximum of the tensor, to modern adaptive range training, which computes quantization boundaries via a moving average of extrema during the forward pass. The next stage of development became hardware-aware quantization, where simulation of saturation and noise effects is introduced through error backpropagation into fully connected and convolutional layers, and the final graph optimization replaces channel concatenation with explicit byte repacking in memory taking into account processor cache-line alignment.