x402 pilot (agents)

Pilot, feature-flagged, testnet only. GET /x402/legal/documents/{id} is an experimental, keyless, pay-per-call mirror of GET /v1/legal/documents/{id} for autonomous agents that would rather pay a few tenths of a cent per call than hold a CrawlPipe API key. It is off by default in every environment (X402_ENABLED=false) and, when enabled, only ever settles against a simulated facilitator on a testnet label (base-sepolia) — no real crypto, no mainnet money, ever, in this pilot.

Why this exists

CrawlPipe is sold primarily via metered API key. But the emerging "agent economy" (autonomous LLM agents that discover and pay for tools on the fly, per the x402 protocol) doesn't want to sign up for an account first — it wants to hit an endpoint, get told the price, pay inline, and get the data back in the same request/response cycle. This pilot is CrawlPipe's cheap peg in that ground: one endpoint, one document type, $0.002/call, so the integration cost of supporting x402 stays near zero while the option exists for when agent-to-agent payments have real volume.

The flow

  1. Request without payment. The agent calls the endpoint with no X-PAYMENT header. The server replies 402 Payment Required with a JSON body describing exactly what payment it wants (the accepts array, x402's PaymentRequirements shape).
  2. Agent builds a payment. The agent's wallet/x402 client constructs a payment payload for one of the accepts entries, base64-encodes it as JSON, and retries the SAME request with an X-PAYMENT: <base64> header.
  3. Verify, settle, respond. The server decodes the header, asks its facilitator to verify the payment is well-formed, checks the document actually exists (a 404 never charges), settles the payment, and returns 200 with the SAME document body GET /v1/legal/documents/{id} returns, plus an X-PAYMENT-RESPONSE header carrying the settlement receipt.

Example: the 402 challenge

GET /x402/legal/documents/vn:tt:2017:31:btc HTTP/1.1
Host: api.crawlpipe.com
HTTP/1.1 402 Payment Required
Content-Type: application/json

{
  "x402Version": 1,
  "accepts": [
    {
      "scheme": "exact",
      "network": "base-sepolia",
      "maxAmountRequired": "2000",
      "resource": "https://api.crawlpipe.com/x402/legal/documents/vn:tt:2017:31:btc",
      "description": "CrawlPipe x402 pilot — GET /x402/legal/documents/{id} ($0.002/call)",
      "mimeType": "application/json",
      "payTo": "0x000000000000000000000000000000000000dEaD",
      "maxTimeoutSeconds": 60,
      "asset": "0x036CbD53842c5426634e7929541eC2318f3dCF7e",
      "extra": { "name": "USDC", "version": "2" }
    }
  ],
  "error": {
    "code": "payment_required",
    "message": "Payment required",
    "details": { "price_usd": 0.002 }
  }
}

maxAmountRequired is the price in atomic units of the asset (USDC, 6 decimals) — "2000" is $0.002. resource always echoes back the exact URL that was requested, so a generic x402 client doesn't need any CrawlPipe-specific logic to know what it's paying for.

Example: the paid request

GET /x402/legal/documents/vn:tt:2017:31:btc HTTP/1.1
Host: api.crawlpipe.com
X-PAYMENT: eyJmcm9tIjoiMHhBQkMuLi4iLCJub25jZSI6ImFiYzEyMyIsInNpZ25hdHVyZSI6Ii4uLiJ9
HTTP/1.1 200 OK
Content-Type: application/json
X-PAYMENT-RESPONSE: eyJzdWNjZXNzIjp0cnVlLCJ0eFJlZiI6InNpbS1hYmMxMjMiLCJuZXR3b3JrIjoiYmFzZS1zZXBvbGlhIn0=

{
  "success": true,
  "data": {
    "document": {
      "id": "019f6ec2-afa9-7baa-bc22-1b8741767e2b",
      "natural_key": "vn:tt:2017:31:btc",
      "doc_number": "31/2017/TT-BTC",
      "...": "— identical shape to GET /v1/legal/documents/{id}"
    }
  },
  "meta": { "request_id": "..." }
}

X-PAYMENT-RESPONSE is base64(JSON) too — decode it to get {success, txRef, network}. Replaying the exact same payment (same nonce) returns 200 again with replay: true in that header and charges nothing new — an agent that retries a request it already paid for is never double-billed.

Rate limit

10 requests/minute per payer. The payer is identified by the payment payload's from field once a payment is presented (or by client IP for pre-payment 402 challenges). Exceeding the limit returns 429 with error.code: "rate_limited" and a Retry-After header giving the number of seconds until the window resets.

Isolation from key-based metering

This endpoint takes no API key and is completely outside the metering system documented in Metering & errors: no usage_event/usage_counter row is ever written for x402 traffic, and GET /v1/usage is never affected by it. Payment records live in their own x402_payment table, entirely separate from plan quotas.

Known pilot-scale limitations

  • In-process only. The rate limiter and the payment-replay dedupe both live in server memory and reset on every process restart/deploy — acceptable for a low-traffic, flagged pilot, not yet a production guarantee.
  • Simulated settlement. By default (and in every environment this pilot has been tested in) a SimulatedFacilitator verifies payment shape and settles deterministically with a synthetic transaction reference — no real chain call is made, no real funds move. A real facilitator can be wired in via X402_FACILITATOR_URL, but that path is untested against a live facilitator.
  • Global daily spend cap. Since this endpoint is keyless, per-org spending caps don't apply — instead a single global daily cap (X402_DAILY_CAP_USD, default $10.00) protects against a runaway settlement loop; once the day's settled total would exceed it, further requests get the 402 challenge again with a "pilot cap reached" message until the next UTC day.
x402 pilot (agents) — CrawlPipe Docs