API Security

Process Webhook Deliveries Without Duplicate Side Effects

Design webhook consumers for at-least-once delivery, out-of-order events and retries without repeating business actions.

Webhook providers commonly retry when they do not receive an acknowledgement quickly. A delivery may therefore arrive more than once, and events may arrive out of order. Your endpoint must assume duplicate delivery rather than relying on the provider to send each event exactly once.

Establish a delivery identity

Use the provider's event or delivery ID as a deduplication key. Store it with a unique database constraint so concurrent requests cannot both claim the same event. If the provider has no stable event ID, derive a carefully scoped fingerprint from documented immutable fields; do not hash arbitrary JSON and assume semantically identical events will always serialize identically.

Separate acknowledgement from work

Verify and persist the event, then acknowledge it according to the provider's contract. Perform slow business processing in a worker when possible. The database record should track received, processing, completed and failed states, along with attempt count and timestamps. A worker should claim work safely so two workers do not execute the same event simultaneously.

Make the business operation idempotent

Deduplication at the HTTP layer is not enough if a worker crashes after making a business change but before marking the event complete. Use a transaction to commit the business change and completion state together where possible. For external side effects, use a downstream idempotency key or a durable outbox pattern so recovery does not send the same action twice.

Handle ordering explicitly

If event sequence numbers or resource versions exist, compare them before applying changes. A late “payment pending” event must not overwrite a newer “payment completed” state. When ordering cannot be established, retrieve the current resource state from the provider instead of trusting arrival order.

Operational visibility

Expose safe metrics for received, duplicate, delayed and failed events. Provide a controlled replay process that preserves idempotency and records who initiated the replay.

Further reading

Share this page Help other developers discover this resource.