# AI agent API evaluation checklist

Canonical URL: https://commerce.delx.ai/guides/ai-agent-api-evaluation-checklist

Published and reviewed: 2026-08-12

## The short answer

Do not put an AI agent API into production because its feature list looks
complete. Record the outcome, canonical route, live contract, data boundary,
execution limits, delivery evidence, bounded smoke, and recovery path first.

Payment authorizes a request; it is not identity authentication. A payment challenge is not delivery evidence. A directory entry, ranking, crawler result, or payment receipt is not delivery evidence by itself.

## Eight checks before production

| # | Check | Evidence to keep | Stop if |
| --- | --- | --- | --- |
| 1 | Outcome | Exact result, input and failure boundary | The route was selected only because its name sounds close |
| 2 | Canonical route | First-party catalog, OpenAPI operation or intent response | A reseller, redirect or search result is the only source of truth |
| 3 | Contract | Method, schemas, version, price, network, recipient, timeout and response | Terms come from a stale screenshot or generic marketing copy |
| 4 | Data boundary | Allowed fields, source rights, processing, retention and deletion | The route needs data the policy does not allow |
| 5 | Execution limits | Max amount, payload, latency, retries, concurrency and idempotency | A retry could duplicate a paid side effect or exceed the cap |
| 6 | Delivery proof | Expected fields, artifact URL/hash, receipt, status and settlement evidence | Only a 200, payment receipt or 402 challenge is available |
| 7 | Bounded smoke | Read-only preflight plus one authorized production smoke | There is no safe input, owner approval or refund path |
| 8 | Recovery | Timeout, duplicate, refund, revocation, rollback and incident owner | Nobody can reconcile or stop a partial delivery |

## Safe evaluation flow

1. **Discover.** Describe the outcome to a governed catalog or intent router.
   Discovery is free and should return a canonical route or an explicit gap.
2. **Preflight.** Read the live OpenAPI operation and HTTP 402 challenge. Check
   method, schema, amount, network, recipient, timeout and output contract.
3. **Authorize.** Apply the caller's identity, privacy, spend, retry and
   idempotency policy. Use one exact resource and keep source attribution.
4. **Verify.** Validate the returned fields or artifact, receipt/hash, status,
   settlement evidence and refund behavior before downstream automation.
5. **Recover.** Record what happens after timeout, duplicate, revocation or
   partial delivery. Do not scale a route with an unresolved stop condition.

## Independent field review

Would Pay Again paid for **10 independent paid calls** across Delx REST and MCP
surfaces and published a **“would pay again”** verdict on 2026-08-11. The
review tested price parity, independently checkable output and bad-input
behavior instead of treating a catalog page or a 402 challenge as proof.

- The sampled utility calls charged the same listed price and returned matching
  output over REST and MCP.
- The MCP 402 resource-host label and `util_uuid_generate` `count=999`
  behavior were corrected the same day and **retested independently** on
  2026-08-12. Issue 6 (2026-08-13) printed that live retest in public.
- Ten calls are evidence about a tested subset, **not a certification**, ranking
  claim, endorsement of every route, or guarantee of future behavior.

Source: https://wouldpayagain.com/issue/5 · confirmation: https://wouldpayagain.com/issue/6

## Evidence layers

- **Discovery** locates a candidate; it does not authorize payment.
- **Contract** binds method, input, price, recipient, network and output.
- **Delivery** proves what happened on this request through the returned result,
  receipt, artifact or hash and status.
- **Recovery** explains how the caller stops, refunds, reconciles and rolls back.

## Delx Commerce references

- Paid OpenAPI: https://api.delx.ai/openapi.x402.json
- x402 manifest: https://api.delx.ai/.well-known/x402
- Buyer decision center: https://commerce.delx.ai/discovery-center
- Agent Reliability Passport: https://commerce.delx.ai/agent-reliability-passport.json
- Integration readiness pack: https://commerce.delx.ai/packs/ship-an-agent-integration
- Website inspection guide: https://commerce.delx.ai/guides/website-inspection-apis-for-agents
- x402 vs MPP guide: https://commerce.delx.ai/guides/x402-vs-mpp-for-ai-agent-payments

This is a decision aid, not a security certification, legal opinion, SLA, or
guarantee of a provider's future behavior. Re-read the live contract before a
real call.

## FAQ

### What should the evaluation start with?

Write the exact outcome and failure boundary, then resolve one canonical route
and read the live contract.

### Does a 402 prove production readiness?

No. It declares payment terms. Production readiness also needs data limits,
delivery evidence, refunds and recovery.

### What does payment authorize?

Payment authorizes a request; it is not identity authentication. The caller
still owns identity, authorization, spend and audit policy.

### When is one production smoke enough?

After a non-payment preflight passes and the caller can verify the expected
result, receipt, timeout and refund path.