Gauge CLI 开源发布 递归优化 Agent 文档体验

Gauge CLI

An open-source CLI for hillclimbing Agent Experience.

Make your Skills, CLIs, SDKs, MCPs and docs easier for coding agents to discover and use. Gauge runs real agents against your product, captures where they get stuck, and lets your own coding agent test fixes before you publish them.

The full loop runs locally with Docker and your existing agent subscriptions or API keys. No Gauge account required.

Website · Documentation · Hosted Gauge · npm · Changelog

Access Gauge Chat data

Sign in and choose your organization to access Chat data:

gauge auth login
gauge orgs use acme
gauge chat categories list
gauge chat query datasets

Chat commands cover topics, prompts, tags, brands, reports, answers, and citations. See Gauge Chat commands and organization selection and upgrade guidance. These commands use hosted Chat data and do not require Docker.

What does Gauge find?

Gauge runs and analyzes coding sessions to find the issues that coding agents run into.

Common findings include:

• An interactive command blocks a coding agent
• An agent reaches for deprecated commands or outdated API patterns
• A Skill goes unused even when it should help with the task
• An agent confidently claims a limitation that does not exist
• An agent cannot figure out the intended workflow across multiple commands

The right fix might change error messages, Skill content, help text, code, or documentation. The Gauge CLI lets you validate proposed fixes before release.

Quickstart

Gauge is designed to be used by your coding agent. Point it at this repository and ask it to get started, or follow the steps below in the project you want to improve.

You need Node.js 22.13+, npm, Docker running Linux containers, and credentials for the agent you want to test. Gauge supports Claude Code, Codex, Pi, and OpenCode. It can use native Claude/Codex logins or API keys; agent usage is billed through your provider or subscription.

1. Install Gauge and its runtime

npm install -g @withgauge/cli
gauge local runtime pull

Gauge downloads the published Docker image pinned to your CLI release. The image contains the agent harnesses and standard development tools.

2. Set up your project

From your project directory (Git is optional):

gauge init --local

This creates a private .gauge/ workspace, including a SQLite database and local backend preference. Plain init changes no tracked files. Add --cases when you want shared Markdown cases: it also writes a gauge.json manifest, gauge-cases/, and guidance in AGENTS.md. Commit those shared files. Keep .gauge/ private: it contains durable measurements and analyses.

3. Install the skill

The Gauge skill teaches your coding agent where to find the CLI's instructions:

gauge install-skill

How the improvement loop works

Hillclimbing means making a change, measuring whether it helped, and repeating.

1. Define realistic tasks. Your agent saves evals or preference prompts with stable identities, editable defaults, and observable outcomes. Use optional Markdown cases for fully specified measurements intended for sharing or CI.
2. Test your current product. In local mode, Gauge starts fresh Docker sessions and injects the working-tree version of your configured inputs. Each test agent sees the task and configured inputs within its sandbox. Local mode supports Skills, SDKs, CLIs, and files. Hosted compute adds support for docs, authenticated MCPs, and authenticated APIs.
3. Find friction. Gauge normalizes events from Claude Code, Codex, Pi, and OpenCode into consistent transcripts. In local mode, your agent reads the evidence, checks the outcome against the criteria, and identifies what slowed or blocked the task. It publishes structured Findings and judgments through gauge run analysis save. In hosted mode, this analysis happens automatically after each session. gauge run insight shows the evidence.
4. Change and retest. Your agent edits the product or its guidance and reruns the affected cases against the new artifacts.
5. Analyze the evidence. Check whether the original problem disappeared, whether the task succeeded, and whether the change introduced new friction.

Skills and file inputs are read from your working tree. For an npm CLI or SDK, build and pack a new tarball before rerunning. Gauge installs these pre-release artifacts in the test sessions, so you can validate a fix before publishing it.

The CLI handles execution and evidence collection. In local mode, your coding agent authors the changes and analyzes the local results.

（注：更多技术实现细节与完整 API 文档请查阅原项目 README）

作者发帖说明（HN）：

Hi HN, I’m Evan from Gauge. We’re open-sourcing our CLI to recursively improve your docs for coding agents. The CLI includes a local execution backend, so you don't need an account to try it. Explainer video here: https://youtu.be/zhLUlIKB1Mg

The CLI supports a hillclimbing loop:

• set up tasks for coding agents that would need your docs, like 'set up Gauge CI checks'

• run the tasks

• analyze session transcripts for friction or failures

• create variants of your docs to resolve the issues

• re-run the tasks with the doc variants injected (transparently, at the same hostname)

• repeat until you have a variant that resolves the issues

The same loop supports hillclimbing your Skills, CLI, and other agent-facing surfaces.

We started out 2 years ago building open-source developer tools (like Tach https://github.com/tach-org/tach), then pivoted to AI Search optimization, and are now focused on the intersection of the two spaces. We were already working with awesome devtool companies, and they were the first ones to ask us how to optimize for Claude Code and Codex. We realized this would be something that eventually every company needs to do. Our CLI provides primitives for docs/devrel/eng teams to measure, analyze, and improve how coding agents use their product.

The design boils down to two components: VMs and a network gateway.

The VMs are straightforward, we want the agents to be highly privileged wi...