Bring Your Local Models to the Agents You Love

tl;dr

Run local models with your favorite coding agent. VibePod connects Pi, Claude Code and Codex to Ollama through saved providers, handling the configuration for you.

Running language models locally has become more accessible, and some models are performing surprisingly well.

This also makes them interesting for coding agents. You can use the agent with a model running on your own hardware, keeping your prompts and code local and avoiding a per-token bill. The model can respond without an internet connection, although the agent may still need one for external tools or update checks.

The configuration requires a bit of work, as each agent has its own way of connecting to a model server. To make this easier, VibePod, the CLI I maintain for running agents in containers, supports providers. You save the server's address and model, then select the provider with --provider when starting an agent.

In this article, I'll go through the setup with Ollama and Pi, then show what needs to change for Claude Code and Codex.

Step 1: Start Ollama

You'll need Ollama and VibePod installed, with Docker or Podman running. The VibePod README covers installation. If local models are new to you, my Getting Started with Local LLMs post covers hardware and model choice.

First, start Ollama with a larger context window and allow connections from your network:

OLLAMA_HOST=0.0.0.0 OLLAMA_CONTEXT_LENGTH=32768 ollama serve

Warning: 0.0.0.0 makes Ollama listen on all network interfaces. Its local API has no authentication, so anyone who can reach port 11434 can use it. Restrict access with your firewall and never expose this port to the internet.

Keep the server running and open a second terminal to download a model that supports tool calls:

ollama pull qwen3.8:27b

A good alternative is gemma4:12b if you're short on resources. You can browse more local models that support tool calls in Ollama's catalog.

If Ollama already runs as an app or service, configure those environment variables using the Ollama FAQ and restart it instead of starting a second server.

The context window determines how much text the model can work with at once. This includes the agent's instructions, your code and tool results, so it can fill up quickly. I'm using 32K tokens in this example, but Ollama recommends 64K or more for coding tools if your hardware allows it.

Step 2: Register the server

Inside a container, localhost means the container itself. Use your computer's LAN address so both VibePod and the agent can reach Ollama. I'll use 192.168.1.10 below; replace it with your own address.

To add the server as a provider, run:

vp provider add

Use these settings in the wizard:

When the wizard asks for model settings, set the context window to 32768 to match the server, and a max output size such as 4096. Pi will use these limits when making requests. You only need to save this provider once to use it in future sessions.

Step 3: Launch Pi

From the project you want to work on, run:

vp run pi --provider ollama

Pi will start with the model you selected as the default. A small task, such as explaining a function or adding a test, is a good way to check whether the model works with the agent's tools.

VibePod applies the provider configuration for this launch, and conversations persist in your profile. To inspect the agent's requests, run vp logs start and open the dashboard at http://localhost:8001.

What about Claude Code and Codex?

You can use the same Ollama server for Claude Code and Codex. The difference is the API format they expect, so you need to run vp provider add again and save a separate provider:

Select your local model as the saved default, then launch:

vp run claude --provider ollama-anthropic
# or
vp run codex --provider ollama-responses

One detail to watch out for: the Anthropic URL must not end with /v1, as the client adds it. Each provider uses one API format, so you register the server once for each format you need.

For LM Studio, other agents or alternative networking setups, see the provider documentation.

Three things to check if it doesn't work

• Can the model use tools? Appearing in the model list does not guarantee it works well with a coding agent. Start with a model built for coding and test it on a small task.
• Is the context window large enough? Agent instructions and tool results take space. If requests fail as the conversation grows, check the context settings in both the server and the provider.
• Does the API format match? Claude Code needs Anthropic Messages; Codex needs OpenAI Responses. A server offering only Chat Completions cannot serve either through these providers.

Conclusion

I wanted to make it easier to try local models without configuring each agent separately. Providers in VibePod take care of that part, but how well the setup works still depends on the model and your hardware. If you try it with your own projects, I'm curious which combinations work well for you.

• VibePod CLI on GitHub
• Provider documentation