Idempotency and Deduplication

An operation is idempotent when repeating the same intended operation has the same intended effect as applying it once. Deduplication is one implementation technique: identify repeated delivery and reuse or reject the earlier outcome. Neither means that only one network request or execution attempt occurs.

Define logical identity

For an API command, a client-generated idempotency key should identify one logical operation. The server commonly stores:

• the key and caller or tenant scope;
• a fingerprint of semantically relevant request fields;
• processing state and the stable outcome;
• creation and expiry information.

A database uniqueness constraint must arbitrate concurrent first requests. Reusing a key with a different request should be rejected; silently returning an unrelated result hides a client defect.

CREATE TABLE idempotency_record (
    tenant_id       bigint       NOT NULL,
    operation_key   varchar(128) NOT NULL,
    request_hash    varchar(128) NOT NULL,
    status          varchar(24)  NOT NULL,
    response_body   text,
    created_at      timestamp    NOT NULL,
    PRIMARY KEY (tenant_id, operation_key)
);

The table is a sketch, not a complete protocol. Define ownership of records left in progress after a crash, response-size limits, retention, privacy, and how a waiting duplicate learns the final outcome.

Messages and side effects

Brokers frequently provide at-least-once delivery. A consumer can record a message identifier in the same local transaction as its business change; the unique record turns redelivery into a no-op. If it calls another system, that system needs its own idempotency boundary. A check followed by an unprotected write is racy.

Idempotency is defined at a boundary. Updating a row to a requested value is often idempotent; incrementing a balance is not unless the command itself has a durable identity. Sending an email, charging a card, and publishing a message require separate analysis.

Relationship to retries and outbox

Retries are safer when attempts share one logical key. A transactional outbox makes local state and an event record atomic, but its relay can publish duplicates; consumers still need idempotency or deduplication.

Test simultaneous duplicates, same key with different input, crashes before and after commit, expired keys, redelivery, and result reconstruction. Do not claim exactly-once effects without defining every boundary involved.