# agentcheck

> Drift and regression monitoring for production AI agents and MCP servers. Quiet by default: it contacts you only when something changed. Uptime-style checks are free (1 target, hourly, badge + status page + trust.json). Paid tiers re-run your suite when a model changes and tell you what broke, replay production traces nightly with cause grouping (prompt / model / tool / flake), run an adversarial pack, and alert by email/Slack/Discord. A free public model-drift index measures each frontier model daily.

Errors are JSON with code, cause, fix and retryable. Automated clients are explicitly permitted (https://agentwares-agentcheck.vercel.app/terms). Credentials: create an API key at https://agentwares-agentcheck.vercel.app/dashboard/keys after GitHub login, then send it as the header "Authorization: Bearer ak_live_…" on both the REST API and the MCP endpoint; verify with GET https://agentwares-agentcheck.vercel.app/api/v1/whoami, which answers 401 {code:"UNAUTHORIZED"} if the key is bad. agentcheck_get_status and agentcheck_get_pricing need no key, so an agent can use the service before it has one. Full recipe: https://agentwares-agentcheck.vercel.app/docs/api-keys. Every MCP tool has a REST mirror under https://agentwares-agentcheck.vercel.app/api/v1 (see the OpenAPI document).

## Links

- [Model drift index (JSON)](https://agentwares-agentcheck.vercel.app/drift.json): daily pass rate + changed answers per model, no login
- [Model drift index (page)](https://agentwares-agentcheck.vercel.app/drift)
- [Pricing (machine-readable)](https://agentwares-agentcheck.vercel.app/pricing.json)
- [MCP server (Streamable HTTP)](https://agentwares-agentcheck.vercel.app/api/mcp)
- [OpenAPI](https://agentwares-agentcheck.vercel.app/openapi.json)
- [How an agent authenticates](https://agentwares-agentcheck.vercel.app/docs/api-keys): no login needed to read
- [Terms (automated clients permitted)](https://agentwares-agentcheck.vercel.app/terms)
- [Privacy](https://agentwares-agentcheck.vercel.app/privacy)
- [Refunds](https://agentwares-agentcheck.vercel.app/refunds)
- [Health](https://agentwares-agentcheck.vercel.app/api/health)
- [Demo status page](https://agentwares-agentcheck.vercel.app/demo): agentcheck monitoring itself

## Public per-target endpoints

- [Status page](https://agentwares-agentcheck.vercel.app/<owner>/<slug>)
- [Badge SVG](https://agentwares-agentcheck.vercel.app/badge/<owner>/<slug>.svg): Cache-Control max-age=300
- [trust.json](https://agentwares-agentcheck.vercel.app/<owner>/<slug>/trust.json): uptime, last nightly, adversarial pass rate, models seen
- [Run page](https://agentwares-agentcheck.vercel.app/<owner>/<slug>/runs/<id>): shareable nightly / drift run with the diff

## REST (Bearer ak_live_…)

- [POST /api/v1/probe](https://agentwares-agentcheck.vercel.app/api/v1/probe): probe a would-be target, get 3 proposed checks
- [GET|POST /api/v1/targets](https://agentwares-agentcheck.vercel.app/api/v1/targets)
- [GET|PATCH|DELETE /api/v1/targets/{id}](https://agentwares-agentcheck.vercel.app/api/v1/targets/{id})
- [GET|POST /api/v1/targets/{id}/checks](https://agentwares-agentcheck.vercel.app/api/v1/targets/{id}/checks)
- [POST /api/v1/targets/{id}/run](https://agentwares-agentcheck.vercel.app/api/v1/targets/{id}/run)
- [GET|POST /api/v1/targets/{id}/traces](https://agentwares-agentcheck.vercel.app/api/v1/targets/{id}/traces): paste / Langfuse / LangSmith / Promptfoo / Braintrust
- [POST /api/v1/traces/{traceId}/promote](https://agentwares-agentcheck.vercel.app/api/v1/traces/{traceId}/promote)
- [GET /api/v1/incidents](https://agentwares-agentcheck.vercel.app/api/v1/incidents)

## MCP tools

- `agentcheck_get_status` (owner, slug): Current status of a public monitored target: overall state, uptime over 24h/7d/30d, last check time, last nightly scores, open incidents and the badge/status URLs. Use the owner (GitHub login) and target slug from the status page URL https://agentwares-agentcheck.vercel.app/<owner>/<slug>. No API key needed.
- `agentcheck_get_pricing`: Machine-readable pricing for agentcheck: tiers with monthly USD price, target limits, check interval and features, plus per-run add-ons. Same data as /pricing.json. No API key needed.
- `agentcheck_create_target` (name?, slug?, kind, url?, packageSpec?, agentCardUrl?, format?, headers?, auth?, modelVar?, modelHeader?, corpusUrl?, alerts?, isPublic?, checks?): Enroll something to monitor: an http endpoint (JSON or OpenAI-style chat), a remote MCP server (Streamable HTTP url) or an A2A agent (origin with /.well-known/agent-card.json). Pass `checks` to create checks in the same call (POST /api/v1/probe proposes three). Returns the target id, the public status page, the badge SVG URL and a README snippet. The first check runs on the next minute tick; call agentcheck_run_now to run immediately. Free tier: 1 target, hourly; Starter+: 5-minute checks. Requires an API key.
- `agentcheck_add_check` (name, kind, input?, golden, intervalSec?, targetId): Add a check to one of your targets. A check runs an input (http path/prompt, mcp tool call, a2a message) on a schedule and judges the answer with a golden: exact, contains, regex and json_schema cost nothing; rubric and baseline use the LLM judge (baseline = same outcome as the last known-good answer). Returns the check id. Requires an API key.
- `agentcheck_run_now` (targetId): Run every check of one of your targets immediately (outside the schedule) and return pass/fail per check with latency, judge cost and any incident opened or closed. Use it right after enrolling, or to confirm a fix. Requires an API key.
- `agentcheck_record` (targetId, name?, input, output, toolCalls?, model?): Record one production interaction with your agent (the prompt or messages, the final answer, the tools it called, optionally the model) as a trace on a target. Call it from your agent or from a proxy in front of it after each task; promote a good trace with agentcheck_promote_trace to replay it nightly and catch regressions. Requires an API key.
- `agentcheck_promote_trace` (traceId): Turn an imported or recorded trace into a replayable check whose golden is the recorded outcome (tool sequence + final-answer rubric). The check runs daily and in the nightly replay (Pro). Returns the check. Requires an API key.
- `agentcheck_list_incidents` (targetId?): Open and recent incidents across your targets (or one target): when they opened/closed, the failing check and the cause. Requires an API key.