What we know
8-trillion-parameter Kimi K3 model using only CPUs paired with second-generation Intel Optane Persistent Memory (PMem) modules. 85 tokens per second with 1-bit quantization applied. This demonstration highlights the use of Optane PMem to handle extremely large AI models without relying on GPUs. 8 trillion parameters and was run on CPUs alone. Second-generation Intel Optane PMem was used to enable this CPU-only inference. 85 tokens per second using 1-bit quantization.
Why it matters
Large AI models with trillions of parameters usually require powerful GPUs and extensive memory bandwidth, which can be costly and energy-intensive. Using Intel’s Optane PMem modules offers a high-capacity, persistent memory solution that can expand CPU capabilities for AI workloads.
What is still unknown
No independent tests have verified the accuracy or performance of the Kimi K3 model running on Optane PMem. Details about supported languages, broader availability, and practical deployment scenarios remain unclear. The overall efficiency and energy consumption of this CPU-plus-Optane approach have not been independently measured.