Local AI

Running language models locally on a laptop CPU, no GPU.

Articles

Lunaris: Install, Test and Optimization
An 8B model on a four-core laptop CPU: 6.6 tokens per second, about 4.6 GB of memory, under a 22 W power cap. Settings, thermals and how to reproduce it.

Llama 3 8B: Training Notes
Background on the base model: how Meta trained it and why it performs well for its size.