Technical writing from the VoidLLM team — architecture decisions, performance deep-dives, and lessons learned building an open-source LLM gateway.
An LLM that ignores your tool description costs round-trips. We rewrote the Code Mode execute_code description with STRONG PREFERENCE labels, workflow steps, and concrete code patterns. Here's what changed and why.
MCP tools advertise inputs but not outputs. We taught Code Mode to learn return types from the first successful call and surface them as TypeScript on the next discovery.
Code Mode is great, but you shouldn't need an LLM proxy to use it. VoidMCP is a separate Go binary that brings Code Mode and the MCP gateway to anyone who just wants their IDE to talk to multiple MCP servers.
Step by step migration from LiteLLM to VoidLLM - config translation, key rollout, client switchover, and gotchas to know before you start.
A practical guide to tracking and allocating LLM spend across teams using org/team/user/key hierarchies. No more monolithic OpenAI bills.
An honest comparison between VoidLLM (self-hosted LLM proxy) and OpenRouter (hosted LLM aggregator) - when each makes sense.
Most LLM proxies log your prompts. The EU AI Act makes that a compliance problem. Here's how VoidLLM's architecture simplifies things.
An honest look at how VoidLLM and LiteLLM differ - where each one shines and where it doesn't.
Route LLM requests across multiple deployments with automatic failover, health-aware routing, and four balancing strategies.
VoidLLM's Code Mode lets AI agents orchestrate multiple MCP tool calls in a single WASM-sandboxed JavaScript execution. No round-trips, no latency penalty.
Real benchmark numbers for VoidLLM's LLM proxy, MCP gateway, and Code Mode. Sub-500 microsecond overhead at 1000 requests per second.
VoidLLM acts as an MCP gateway - proxy, manage, and control access to external MCP servers from one place.
How VoidLLM enforces privacy at the code level - no content logging, no opt-out, no exceptions.
Switch from direct OpenAI API calls to VoidLLM by changing one line. Route to any provider - Anthropic, Azure, vLLM - through the same SDK.
A quick walkthrough of VoidLLM - from docker run to your first proxied LLM request, with a look at the built-in UI.
Step-by-step setup for using VoidLLM as your LLM proxy in Cursor and Windsurf, and as an MCP server in Claude Code.
Existing LLM gateways are bloated, cloud-only, or charge per request. We built a self-hosted alternative that doesn't compromise on privacy or performance.