Documentation
Which AI coding tools actually work on your code.
Teev Bench builds a private benchmark from your own merged pull requests, then runs model and harness combinations against those tasks in your own cloud. This site will document how to set that up and how to read the results.
Documentation is being written. One page is published below; the rest are listed so you can see what is coming.
Where your data lives
Teev Bench runs as two parts. A hosted control plane handles sign-in, accounts and scheduling. A runner you deploy into your own cloud account does the actual work: reading your repositories, building environments, running agents and verifying results.
Never leaves your account
- Source code and repository snapshots
- Your GitHub token
- Your model provider API keys
- Patches, diffs and agent transcripts
- Container logs and full run artifacts
Reported to the control plane
- Account and user identity, sign-in events
- Repository names and task identifiers
- Benchmark definitions — rules, targets, model selections
- Result summaries — pass rates, timings, token counts, cost
- Runner health and capability metadata
The claim is your source and your compute never leave your account — not that we see nothing. The runner reports outward over HTTPS; nothing reaches into your account, and you hold its source and control its egress rules.
Planned
Not yet written. Listed so the shape of the documentation is visible.
- Getting startedConnect a repository, screen candidate tasks, build your first benchmark.
- Deploying a runnerTemplates for AWS, Azure and plain Docker, and what each one provisions.
- Task selectionHow pull requests become tasks, and the rules that decide eligibility.
- Execution targetsHarness, provider, model, effort and trial count — how a comparison stays fair.
- Reading resultsPass rates, calibration, cost per task, and what a result does not tell you.
- Security and operationsThe full data-handling model, retention, and access control.