Local LLM Deployment: Run Ollama on Mac and Linux in 2026
Run capable open-weight models on your own hardware with Ollama. This guide covers installation, model selection, RAM and GPU tradeoffs, and how to decide between local inference and cloud APIs.
Why run a model locally
Local inference solves three problems at once: latency, cost, and privacy. Requests do not leave your machine, there is no per-token bill, and you can tune models for private data without shipping logs to a third party. The tradeoff is hardware: model quality now scales directly with available RAM, GPU VRAM, and storage.
In 2026, the practical floor for a usable local LLM is 16 GB unified memory on Apple Silicon or 12 GB GPU RAM on NVIDIA. Below that, expect slow generation and small context windows. Above that, the gap versus cloud quality is much smaller than it was two years ago.
Installation baseline
Ollama remains the lowest-friction path because it handles model download, quantization, serving, and an OpenAI-compatible API in one binary. On Mac, install via Homebrew. On Linux, use the install script or package repo.
brew install ollama
# Linux
curl -fsSL https://ollama.com/install.sh | shAfter installation, verify the daemon and CLI are working:
ollama --version
ollama serve &
ollama listThe default API listens on http://localhost:11434. Most local tooling—editors, agents, eval harnesses—can point there directly.
Model selection by hardware
Start with model size, not brand. A 7B-quantized model fits most developer laptops; a 34B model needs workstation RAM or a 24 GB+ GPU. The table below is a practical 2026 starting point.
| Hardware | Practical model size | Example use |
|---|---|---|
| MacBook Air / 8 GB | 1B–3B quantized | Light extraction, classification, fast local helper |
| MacBook Pro 14 / 16 GB | 7B–13B quantized | General chat, coding, drafting, local agent loop |
| Mac Studio / 32–64 GB | 13B–34B quantized | Long context, agent reasoning, multi-step tasks |
| Linux workstation / 24 GB GPU | 7B–34B full or quantized | Sustained throughput for batch work or serving |
If you want one default recommendation: a 7B chat model on a 16 GB Mac. It is the point where quality, speed, and cost balance is still acceptable for daily use.
Local versus cloud
Local is not always better. Use local first when the workflow involves sensitive documents, offline environments, or high-volume repeated calls. Use cloud APIs when you need frontier reasoning, large context beyond local VRAM, or guaranteed uptime without tuning your own box.
A useful split is to keep the last mile local and the hard reasoning remote. Draft, format, classify, and validate locally; send only the genuinely hard subproblem to a cloud model. That keeps cost down and data small.
For selecting tools in this stack, see AI tool recommendations for 2026. For automating outputs from local runs, AI workflow automation for content teams shows how to connect local inference into repeatable pipelines.
Operational limits and notes
Quantization is not free. Every step down from FP16 to Q4 or Q3 costs some capability. Benchmark on your own workload rather than relying solely on leaderboard scores. Also keep an eye on prompt caching and context reuse; local models benefit more from compact prompts than cloud models because their throughput is more sensitive to token count.