llmtee Launches Self-Hosted Proxy to Log LLM Cost and Latency

llmtee

See what your LLM calls cost. llmtee is a self-hosted proxy for the OpenAI and Anthropic APIs and for OpenAI-compatible providers such as OpenRouter, Ollama, vLLM and Groq. You point your client's base URL at it, and it logs every request with tokens, cost, latency and time to first token.

Like the Unix tee, it passes each response through to your client unchanged and keeps a copy.

• Any client that lets you set a base URL works without code changes, including the OpenAI and Anthropic SDKs, Claude Code, Codex and automation tools.
• It runs as one Go binary with a SQLite file, so a backup is a file copy. PostgreSQL works too.
• Each key can have a daily and monthly budget. Once a key reaches its limit, llmtee answers with HTTP 429 and does not forward the request.
• Local models (Ollama, vLLM) and hosted APIs show up in the same dashboard.
• You can read your data through the web UI, CSV export, Prometheus metrics or an MCP server.

llmtee is built for individuals and small teams who want to see and cap their LLM spend. It is not a gateway with routing, fallbacks or multi-tenancy. The current release is 0.x, so configuration may still change between minor versions.

Quickstart

docker run -d --name llmtee -p 8080:8080 -v llmtee-data:/data \
  -e LLMTEE_ADMIN_PASSWORD=change-me ghcr.io/garritfra/llmtee

Point your client at llmtee. It forwards your provider key unchanged:

export OPENAI_BASE_URL=http://localhost:8080/openai/v1
export ANTHROPIC_BASE_URL=http://localhost:8080/anthropic

Send any request, then open http://localhost:8080 and sign in with the admin password. The request appears in the list with its tokens and cost.

Without Docker, download a binary from the releases (Linux, macOS and Windows on amd64 and arm64) and run LLMTEE_ADMIN_PASSWORD=change-me ./llmtee. llmtee creates llmtee.db in the current directory.

Other setups: docker-compose.yml, PostgreSQL compose, systemd unit, Caddy with TLS. The Setup page in the web UI shows how to connect Claude Code, Codex, opencode and Pi, with your instance's base URL filled in. Guides for these agents, the SDKs, Ollama, OpenRouter, vLLM and n8n are in docs/integrations.md.

Endpoints

llmtee logs chat completions, responses, embeddings and Anthropic messages, streamed or not. It forwards all other paths without logging them. Details are in docs/headers.md.

Keys and budgets

llmtee works with two kinds of keys, and you can use both at once.

A provider key is your OpenAI, Anthropic or other API key. By default, clients send it as usual and llmtee forwards it. llmtee stores only its SHA-256 hash, as a key id starting with pt-. To give a provider key a name and a budget, label it in the UI or with llmtee keys label.

A llmtee key (llmtee-…) replaces the provider key on the client side. llmtee swaps in the real provider key from OPENAI_API_KEY, ANTHROPIC_API_KEY, OPENROUTER_API_KEY or LLMTEE_UPSTREAM_<NAME>_API_KEY, so teammates and agents never see it. Create one in the UI or on the command line:

llmtee keys create hermes-agent --budget-day 5 --budget-month 60
llmtee keys list

A client can also send a llmtee key next to its provider key, in a Llmtee-Key header, to attribute its requests to that key. To reject all requests without a valid llmtee key, set LLMTEE_REQUIRE_PROXY_KEY=true.

llmtee checks budgets in memory before it forwards a request. A key at or over its limit gets a 429 response in the provider's error format, with a Retry-After header. Limits reset at the start of the next UTC day or month. llmtee only knows the cost of a request after the response arrives. The request that crosses the limit therefore completes, and a key can overshoot its budget by that one request.

To tag requests with a user, session or custom property, or to skip storing a body, see docs/headers.md.

（注：更多技术实现细节与完整 API 文档请查阅原项目 README）

作者发帖说明（HN）：

Hi HN,

Llmtee is a lightweight llm proxy. I built this project because I needed a simple way to log llm calls across my agents.

Existing options require running Postgres, ClickHouse and Redis, or have me use an SDK. llmtree is just a single binary. You point your provider base url to llmtee, it will pass the traffic through chunk by chunk and log the usage to an SQLite database (or optionally Postgres).

The container image is ~25 MB. Memory footprint is around ~40 MB and added latency is ~0.2 ms.