DOALocal AI › llama.cpp
llama.cpp: what it is, how to install it and requirements
C/C++ library and command-line tools that many apps are built on and that define the GGUF format. The most technical option.
Checked on 13 Oct 2026. Versions change every week: check the official site before installing.
| Operating systems | Windows, macOS and Linux (source) |
|---|---|
| Licence and price | MIT · free |
| Ease of use | Technical |
| Current version | v0.6.0 (5 Oct 2026) |
| Official site | github.com/ggml-org/llama.cpp · GitHub |
How to install it
- Official installer (current README): macOS/Linux
curl -LsSf https://llama.app/install.sh | sh; Windows (PowerShell)irm https://llama.app/install.ps1 | iex. There are also binaries under Releases and a winget package. The README uses the unifiedllamacommand:llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF(chat in the terminal) orllama serve -hf ggml-org/Qwen3.5-0.8B-GGUF(server with a web interface);-hfdownloads the model from Hugging Face.
Install guides by system: Windows · Mac · Linux. Which model to download: Which local AI model to pick for your RAM: 8, 16, 24, 32 GB.
Other local AI tools
- Ollama: Engine and app to download and run language models with a single command. It includes a local API so other apps can connect to it.
- LM Studio: Desktop app with a graphical interface to find, download and chat with models in GGUF or MLX format. It also has a local server.
- Jan: Open-source desktop app to chat with models offline, ChatGPT-style.
- GPT4All: Nomic's desktop app to chat locally and ask questions about your own documents (LocalDocs).
- Open WebUI: Self-hosted, ChatGPT-style web interface that connects to Ollama or OpenAI-compatible APIs.
- ComfyUI: Node-based interface to generate images and video with open models on your own computer.