Compute-in-Memory

The defining characteristic of CIM is the physical co-location of logic and memory. In a standard architecture, data shuttles between discrete DRAM and a CPU, consuming over 80% of the energy budget. CIM performs multiply-accumulate (MAC) operations directly at the data's resting place using the memory array's intrinsic analog properties, such as Kirchhoff's current law, to sum results instantaneously on the bitlines.

CIM architectures bifurcate into two primary physical implementations:

• Analog CIM (ACIM): Uses flash or resistive RAM (RRAM) cells as programmable resistors. Input voltages are applied to wordlines, and the resulting current on the bitline represents the dot-product output. This offers extreme density but requires careful management of process variation and noise.
• Digital CIM (DCIM): Embeds standard logic gates within the SRAM array to perform bit-wise operations. This provides deterministic, high-precision computation at the cost of lower density compared to analog approaches.

By avoiding the energy-intensive process of reading weights from SRAM and moving them to a MAC unit, CIM achieves energy efficiencies measured in tera-operations per second per watt (TOPS/W). For always-on medical wearables performing keyword spotting or arrhythmia detection, CIM chips can reduce the inference energy per operation to femtojoules, extending battery life from hours to weeks.

CIM arrays are naturally suited to exploit weight sparsity. When a weight value is zero, the corresponding memory cell simply does not contribute any current to the bitline summation. This zero-skipping happens automatically at the physical level without requiring complex decoding logic or clock cycles, providing a direct, linear improvement in energy consumption proportional to the model's sparsity ratio.

A single CIM macro can activate multiple rows simultaneously, performing hundreds or thousands of MAC operations in a single clock cycle. This massive parallelism is achieved because the memory array itself acts as a crossbar, directly mapping the matrix-vector multiplication of a neural network layer onto the silicon. The throughput is limited only by the array size and the speed of the analog-to-digital converters (ADCs) on the periphery.

Advanced CIM designs utilize non-volatile memory technologies like RRAM, MRAM, or Phase-Change Memory (PCM) . These elements retain weight data even when power is removed, enabling instant-on inference for medical devices. Crucially, the same element performs both storage and computation, eliminating the need to load weights from a separate flash chip and enabling true processing-in-memory.

A specialized hardware accelerator designed to execute the matrix multiplications and convolutions central to neural networks. Unlike general-purpose CPUs, NPUs employ a spatial dataflow architecture that maps directly to the computational graph of a model. When combined with Compute-in-Memory macros, an NPU can bypass external memory fetches entirely, keeping both weights and activations stationary within the analog or digital compute fabric. This synergy is critical for achieving sub-milliwatt power budgets in medical wearables performing continuous ECG classification.

A compression technique that reduces the numerical precision of a neural network's weights and activations from 32-bit floating point to low-bit integers, typically INT8 or INT4. This is a direct enabler for Compute-in-Memory, as the physical area and energy of a CIM multiply-and-accumulate operation scale quadratically with bit precision. Aggressive quantization to 4-bit allows a CIM macro to store a larger model entirely on-chip, eliminating the energy cost of accessing off-chip DRAM. Post-training quantization requires a calibration dataset to minimize accuracy loss.

A model design paradigm that incorporates the physical constraints of the target silicon directly into the neural architecture search (NAS) and training loop. For Compute-in-Memory, this means optimizing for the non-idealities of analog computation, such as conductance drift in resistive RAM or limited ADC resolution. The training process may simulate stuck-at-faults or thermal noise to produce a model that is inherently robust to the analog errors of the CIM substrate, ensuring that diagnostic accuracy on a medical device matches the simulated performance.

A graph optimization strategy that combines multiple discrete neural network operations—such as convolution, batch normalization, and ReLU activation—into a single computational kernel. For a CIM accelerator, this is essential to prevent the costly movement of intermediate feature map data between the memory array and an external vector processor. By fusing the non-linear activation function directly at the periphery of the CIM array, the data remains stationary, maximizing the throughput and energy efficiency of the analog compute fabric.

The principle of processing and storing data physically close to its source on the edge device or local gateway. Compute-in-Memory is the ultimate physical manifestation of data locality for AI workloads, as it collapses the spatial separation between memory and logic. For healthcare, this means a CIM-equipped wearable can process raw photoplethysmography (PPG) signals locally to detect atrial fibrillation without ever buffering raw patient data in off-chip DRAM, satisfying strict data residency regulations and eliminating a primary attack vector.

A model compression method that removes entire channels or layers from a neural network, creating a smaller, dense model that is readily accelerated by standard hardware. Unlike unstructured pruning, which creates sparse weight matrices that require specialized decoding logic, structured pruning produces a regular computational pattern that maps efficiently onto a systolic array or CIM crossbar. This allows a pruned model to fully utilize the parallel compute capacity of the CIM macro without the overhead of managing sparse data representations.