How To Use Llama Cpp

What llama.cpp is

Every other “easy” local-LLM tool eventually bottoms out here. Knowing llama.cpp directly means you can skip the wrappers when they get in your way.

Building from source

Clone the repo and build with CMake. The default build is CPU-only; pass flags for your accelerator:

Getting a GGUF model

Single-shot prompt from the CLI:

Running inference

When to reach past llama.cpp