Local LLM Deployment on Windows: A Practical WSL2 + Ollama Guide (2026)

Why WSL2, not the native Windows build

Ollama ships a native Windows app, and it works. But the WSL2 route — running the Linux Ollama server inside Windows Subsystem for Linux 2 — has three practical advantages that matter after day one:

• Faster access to models and fixes. The Linux build of Ollama gets new model support and bug fixes first; the Windows preview trails it. Running the Linux build inside WSL2 removes that lag.
• Full NVIDIA GPU passthrough. Since WSL2 got CUDA support, your NVIDIA GPU is visible inside the Linux environment with near-native speed. Quantized 7B models generate at full speed, exactly as on Linux.
• The whole Linux toolchain. Every companion tool — Open WebUI, llama.cpp, text-generation-webui, LangChain scripts — is written for Linux first. In WSL2 you install them with the same commands as everyone else's tutorials.

If you are on macOS or already run Linux, see our Ollama on Mac and Linux guide instead. The rest of this guide assumes a Windows 10 or 11 machine with an NVIDIA GPU.

Prerequisites: the 5-minute checklist

Before touching the terminal, verify these. Skipping this list is where most failures come from:

• Windows 10 21H2+ or Windows 11 — WSL2 requires a 64-bit build; Windows 11 is smoother.
• Virtualization enabled in BIOS — check Task Manager → Performance → CPU: "Virtualization: Enabled". If it says Disabled, reboot into BIOS and turn on Intel VT-x / AMD-V.
• An NVIDIA GPU with recent Game Ready / Studio drivers — download the latest driver from nvidia.com. This is the key detail: the driver installs on Windows only; the CUDA toolkit installs inside WSL2. Installing a GPU driver inside WSL2 will break passthrough.
• 8 GB+ of GPU VRAM (see the model table below) and ideally 16 GB+ of system RAM, since WSL2 shares memory with Windows.
• About 30 GB of free disk space — Ubuntu on WSL2 plus a couple of quantized models.

Step-by-step: WSL2 + CUDA + Ollama in one session

1. Install WSL2 with Ubuntu

Open PowerShell as administrator and run one command. This installs WSL2 and Ubuntu (default distro):

Reboot when prompted, then launch "Ubuntu" from the Start menu and create your Linux username and password. Confirm you are on WSL version 2:

Ubuntu should show VERSION 2. If it shows 1, run wsl --set-version Ubuntu 2 and wait for the conversion.

2. Verify GPU passthrough inside WSL2

Inside the Ubuntu terminal, check that the GPU is visible through the passthrough driver:

If you see your GPU name and driver version, passthrough works — no CUDA toolkit install needed for Ollama. (Only install the CUDA toolkit inside WSL2 if you plan to compile tools like llama.cpp from source.)

3. Install Ollama and pull your first model

If the answer streams back at usable speed, your Windows PC is now a working local LLM server. Check GPU offload with ollama ps — it should show the model loaded on GPU.

4. Expose the API to Windows apps

By default Ollama listens only on localhost inside WSL2, which Windows apps cannot reach. Tell it to listen on all interfaces, then restart:

From Windows PowerShell, test the connection to the WSL2 IP (or use localhost — WSL2 mirrors it since the 2023 networking update):

You now have an OpenAI-compatible API endpoint that any Windows AI app — LibreChat, AnythingLLM, Continue for VS Code — can point at. Set the app's base URL to http://localhost:11434/v1.

Which model fits your VRAM: the 2026 quick table

Quantized (Q4_K_M) models are the sweet spot: roughly 70% of full quality at a fraction of the memory. Match your card to a row and you will not run out of memory mid-generation:

Rule of thumb: a Q4_K_M model needs roughly 0.6 GB of VRAM per billion parameters, plus ~1 GB of overhead for context. For a deeper cost-and-hardware breakdown, see our budget hardware model comparison.

The three problems that catch everyone (and their fixes)

Problem 1: WSL2 eats all your RAM

By default WSL2 can consume up to half your system RAM, which slows Windows to a crawl when a model is loaded. Cap it in %USERPROFILE%\.wslconfig (create the file if missing):

Then run wsl --shutdown in PowerShell to apply. With this cap, Windows stays responsive while Ollama holds a model on the GPU.

Problem 2: Windows Firewall blocks LAN access

If you want other devices on your home network (a phone, another PC) to reach Ollama, exposing 0.0.0.0 is not enough — Windows Firewall blocks inbound 11434. Add an inbound rule for port 11434 scoped to your private network only. Never expose this port to the public internet without authentication in front of it.

Problem 3: Ollama does not start after reboot

WSL2 distros do not boot on their own. Two options: keep it simple and launch Ubuntu once after login, or automate it with a Windows Task Scheduler task that runs wsl -d Ubuntu -e sudo systemctl start ollama at logon. Pick the simple route until you know you need it running 24/7.

Related reading

• Local LLM Deployment: Run Ollama on Mac and Linux in 2026 — the same workflow on Apple Silicon and native Linux
• Local LLM OpenAI-Compatible API Endpoints — wire your WSL2 server into any AI app
• Local LLM Model Comparison for Budget Hardware — pick the right quantized model for your card

Limits and notes

This guide targets NVIDIA GPUs; AMD and Intel Arc cards can run local models via llama.cpp and ROCm-on-WSL builds, but driver support is rougher and outside this guide's scope. Context windows above 32k tokens on consumer cards get slow regardless of setup — split long documents before sending them. And remember: a local model is private by default, but the moment you open port 11434 to your network, treat it like any other service and firewall it.