Running Local LLMs in Zed via LM Studio and Ollama

Configuring Local AI Providers

Zed integrates with local model runners by connecting to their exposed API endpoints. Users can choose between graphical interfaces or terminal-based workflows to serve models locally.

• LM Studio: Download and load a model within the application, then navigate to the local server section to start the API. In Zed, open the agent settings via the command palette and ensure the provider is configured to point to the local server URL.
• Ollama: Install the tool and pull models using ollama pull <model-name>. Start the server with ollama serve. Zed typically auto-detects running Ollama instances on port 11434.
• llama.cpp: For advanced users, manually configure the connection to a llama.cpp server endpoint within the Zed agent settings to maintain low-level control over model execution.

Optimizing Local Model Performance

Local models are best suited for specific, scoped tasks rather than broad codebase reasoning. To achieve optimal results, users should balance model size with hardware constraints, specifically targeting quantized versions (e.g., Q4) to fit within available VRAM.

• Task Scoping: Use local models for code explanations, small refactors, boilerplate generation, and adding comments. Avoid sending entire large codebases, as local models often have smaller context windows than cloud-based alternatives.
• Model Selection: Utilize Mixture of Experts (MoE) models like Qwen 2.5 Coder or Qwen 3.6 35B (which uses only ~3B active parameters per token) to achieve faster inference speeds on consumer hardware.
• Workflow Integration: Leverage Zed's ability to switch between local and cloud providers (Anthropic, OpenAI, Gemini) on a per-task basis, reserving cloud models for complex debugging and local models for privacy-sensitive or routine edits.