Large language models (LLMs) have achieved high accuracy in diverse NLP and computer vision tasks due to selfattention mechanisms relying on GEMM and GEMV operations. However, scaling LLMs poses significant computational and energy challenges, particularly for traditional Von-Neumann architectures (CPUs/GPUs), which incur high latency and energy consumption from frequent data movement. While high energy consumption is a crucial challenge for serves, these issues are exacerbated in edge applications with energy-constrained computational resources. DRAM-based near-memory architectures offer high energy efficiency and throughput, but their basic processing elements have limited operations due to stringent area, power, and timing constraints. This work introduces CIDAN-T3D, a novel Processing-in-Memory (PIM) architecture optimized for LLMs. It features an ultra-low-power Neuron Processing Element (NPE) with high compute density (#Operations/Area), enabling efficient in-situ execution of LLM operations by leveraging high parallelism within DRAM. CIDAN-T3D minimizes data transfers, enhances locality, and achieves significant gains in throughput and energy efficiency. Experiments show a 1.3× throughput and 21.9× improvement in energy efficiency compared to existing near-memory designs, making CIDAN-T3D a scalable and efficient solution for LLM-based Gen-AI applications.

This paper presents TULIP, a new architecture for a variable precision Quantized Neural Network (QNN) inference. It is designed with the goal of maximizing energy efficiency per classification. TULIP is constructed by arranging a collection of unique processing elements (TULIP-PEs) in a single instruction multiple data (SIMD) fashion. Each TULIP-PE contains binary neurons that are interconnected using multiplexers. Each neuron also has a small dedicated local register connected to it. The binary neurons are implemented as standard cells and used for implementing threshold functions, i.e., an inner-product and thresholding operation on its binary inputs. The neurons can be reconfigured with a single change in the control signals to implement all the standard operations used in a QNN. This paper presents novel algorithms for implementing the operations of a QNN on the TULIP-PEs in the form of a schedule of threshold functions. TULIP was implemented as an ASIC in TSMC 40nm- LP technology. A QNN accelerator that employs a conventional MAC-based arithmetic processor was also implemented in the same technology to provide a fair comparison. The results show that TULIP is 30-50X more energy-efficient than an equivalent design, without any penalty in performance, area, or accuracy. Furthermore, TULIP achieves these improvements without using traditional techniques such as voltage scaling or approximate computing. Finally, the paper also demonstrates how the run- time trade-off between accuracy and energy efficiency is done on the TULIP architecture.