When a client times out, users double-click, or a worker retries a job, your server may see the same logical operation twice. Without idempotency, you create duplicate charges, rows, or side effects. This guide explains the concept and a standard fix: the Idempotency-Key header.
What "idempotent" means
An operation is idempotent if performing it multiple times has the same effect as performing it once. In HTTP terms, PUT replacing a resource with a fixed body and DELETE are naturally idempotent. POST that creates something is usually not idempotent — calling it twice creates two resources unless you add extra mechanics.
Why networks force you to care
- TCP/HTTP can lose responses while the server already committed work.
- Mobile clients and browsers retry aggressively.
- Message queues deliver at-least-once; consumers must tolerate duplicates.
So "exactly-once" end-to-end is a myth at the wire; you aim for exactly-once effect using design.
The Idempotency-Key pattern (Stripe-style)
- Client generates a unique key per logical attempt (UUID v4 is fine) and sends it on mutating requests, commonly:
Idempotency-Key: <uuid>. - Server uses the key + authenticated actor (e.g. user or API key id) as a compound idempotency scope so two users cannot collide.
- On first request: run the business transaction, store
(scope, key) → result fingerprint(status + response body hash or stored response). - On duplicate request with same key: return the stored response without re-running side effects (or return 409 if the first attempt is still in flight — pick one policy and document it).
What to store
Minimum: HTTP status code, response body (or safe subset), and created_at. Expire keys after a TTL (e.g. 24–72h) matching your support window. For financial APIs, regulatory needs may require longer retention — adjust accordingly.
Implementation sketch (pseudocode)
function handleCreatePayment(Request $req) {
$key = $req->header('Idempotency-Key');
if (!$key) {
return error(400, 'Missing Idempotency-Key');
}
$scope = $req->user()->id;
$cacheKey = "idem:{$scope}:{$key}";
if ($cached = cache()->get($cacheKey)) {
return response($cached['body'], $cached['status']);
}
return DB::transaction(function () use ($cacheKey, $req) {
if ($row = IdempotencyRecord::where(...)->lockForUpdate()->first()) {
return response($row->response_body, $row->response_status);
}
// perform payment + persist IdempotencyRecord with outcome
$out = chargeAndBuildResponse($req);
IdempotencyRecord::create([...]);
cache()->put($cacheKey, $out, 86400 * 3);
return $out;
});
}
The same idea maps to Node, Go, or Python: atomic claim of the key (unique constraint or Redis SET key NX) then process.
Design choices that bite teams
- Reusing keys across different payloads: reject or namespace; never silently apply a new body to an old key.
- Partial failure: if you charged a gateway but died before saving the idempotency row, retries might double-charge — use ledger + outbox or make the payment provider idempotent too (many expose idempotency).
- GET with side effects: forbidden; never encode mutations in GET.
- Long-running jobs: return
202with a job id; second call with same key returns the same job id.
When GET/PUT/DELETE are enough
For pure CRUD where clients address resources by stable ids (e.g. PUT /orders/7 with full replacement), you may rely on natural idempotency — but POST /orders for create still needs keys or natural unique constraints (e.g. client-supplied slug) enforced in the database.
Takeaways
- Treat retries as inevitable.
- Use Idempotency-Key on non-idempotent creates and critical POSTs.
- Scope keys per tenant/user; store and replay the same response.
- Align TTL and DB constraints with product and compliance requirements.
This pattern appears in payment APIs for a reason — it is one of the highest-leverage reliability upgrades for public HTTP APIs.