How to Run a Private AI Assistant on Your Windows PC

How to Run a Private AI Assistant on Your Windows PC

You can run an AI assistant directly on a Windows computer instead of sending every prompt to a hosted service. Ollama provides a runtime for downloading and running open models locally, along with a local API that other programs can use. This guide covers the basic setup, hardware limits, and privacy trade-offs.

Choose the local runtime

Ollama is an application that manages AI models on your computer. A model is the file set that performs the language work. Ollama downloads models, starts them, and makes them available through a command-line interface or a local web service.

The official Windows documentation supports Windows 10 version 22H2 or newer, in Home or Pro editions. Ollama supports compatible NVIDIA and AMD Radeon graphics hardware for acceleration, but it can also run without a supported GPU. On systems without a strong GPU, larger models may respond slowly. Ollama says the program needs at least 4 GB of storage, while downloaded models may require tens or hundreds of gigabytes depending on the models you select (Ollama's Windows documentation).

Install Ollama on Windows

Download the installer from the official Windows download page. The Windows installation does not require administrator access and installs in the user's home directory by default. After installation, Ollama runs in the background and makes the ollama command available in Command Prompt, PowerShell, and other terminal applications.

Open PowerShell or Windows Terminal and confirm that the command is available:

ollama --version

If Windows displays a version number, the command-line tool is ready. Ollama's local API is served at http://localhost:11434. “Localhost” means the service is reachable from the same computer rather than from the public internet (Ollama's API documentation).

Download a right-sized model

With Ollama installed, choose a model from its model library. The library lists general-purpose, coding, vision, reasoning, and embedding models. Model names often include a size such as 3B, 8B, or 30B. The number identifies the model's approximate parameter count. Larger models can be more capable, but they normally need more memory and take longer to run.

For a first test, choose a smaller general-purpose model listed in the library. The exact model name can change, so copy the current name from the model's official page. Gemma 3 currently includes a 4B model, and its official Ollama page lists this command:

ollama run gemma3:4b

Ollama downloads the model and starts a local chat. You can ask it to summarize text, explain a programming concept, draft a short document, or transform data into a requested format. Local models are useful for private notes, offline experimentation, and development tasks, but their answers can still be wrong. They may also lack current information unless you provide that information in the prompt.

Call the local API

After the model works in a chat, you can connect it to software. A program can send a chat request to Ollama's local API and receive the model's response. Ollama's official API documentation provides local examples and lists official Python and JavaScript libraries (Ollama's API introduction).

This PowerShell example sends a prompt to the local model through the documented /api/chat endpoint:

$body = @{ model = "gemma3:4b"; messages = @(@{ role = "user"; content = "Explain DNS in three short paragraphs." }); stream = $false } | ConvertTo-Json
Invoke-RestMethod -Method Post -Uri "http://localhost:11434/api/chat" -ContentType "application/json" -Body $body

The response is JSON, so a script can extract the generated message and pass it to another part of an application. Ollama's official documentation lists Python and JavaScript libraries for integrations built with those languages.

Know the privacy limits

A locally downloaded model runs on the computer, so requests sent to that model do not need to use Ollama's cloud service. Cloud models work differently: they run on Ollama's servers and do not require a local model download. The official download page distinguishes local models from cloud models and identifies cloud model names with the :cloud suffix (Ollama's download documentation).

Local processing does not make every connected application private. A local model is handled by the local Ollama server, while a cloud model or remote application sends requests to an online service. Review which model and endpoint an application uses, avoid placing passwords or personal records in prompts, and keep downloaded models on a drive with enough free space. Start with a small model, a simple test prompt, and a clear understanding of whether each request is local or cloud-based.

References

Ollama. (n.d.). Windows. GitHub. https://github.com/ollama/ollama/blob/main/docs/windows.mdx

Ollama. (n.d.). Download Ollama on Windows. https://ollama.com/download/windows

Ollama. (n.d.). Gemma 3. https://ollama.com/library/gemma3

Ollama. (n.d.). Introduction. Ollama API documentation. https://docs.ollama.com/api/introduction

Ollama. (n.d.). Ollama model library. https://ollama.com/library