What is a local LLM?

A local LLM is a large language model that runs on your own computer instead of a provider’s servers, usually through Ollama, LM Studio, llama.cpp or vLLM. Your prompts and documents never leave the machine, there is no per-question cost, and it works offline. The trade-off is that models small enough for a laptop are less capable than the largest cloud models.

Models of about 7–8 billion parameters run on a laptop with 16 GB of memory; Apple silicon and modern GPUs make them faster.

For questions about one paper with the right passages supplied, a small local model is often enough.