The direct answer is to resend the same authenticated webhook more than once and verify two things separately: the endpoint may receive and acknowledge every delivery, but the customer-facing action must commit only once. Build that guarantee around a stable idempotency key, an atomic processed-event record, and a test that includes retries, concurrency, failures, and out-of-order events.
Start with the delivery facts
Webhook delivery is not the same as exactly-once business processing. Stripe says an endpoint can receive the same event more than once, recommends recording processed event IDs, and notes that two distinct Event objects can sometimes describe the same underlying object and event type. Stripe also does not guarantee delivery order. Its webhook documentation therefore supports testing duplicates and out-of-order arrivals as normal conditions rather than exceptional ones.
AWS gives the same architectural warning from a different system: event processing can be at least once, duplicate records can occur, and functions should be idempotent. The AWS Lambda best-practices guide explains that idempotent code handles a duplicate event with the same result instead of repeating its effects.
Choose the right uniqueness key
Store the provider event ID, but also consider the business action you are protecting. If two provider events can legitimately refer to the same action, define a second key such as `invoice-paid:invoice_123`, `welcome-email:customer_456`, or `shipment-create:order_789`. Stripe specifically recommends using the underlying object ID together with the event type when separate Event objects may represent a duplicate.
The deduplication record and the protected action should be claimed atomically. A common pattern is a database row with a unique key and a state such as `processing`, `completed`, or `failed`. Insert or claim the key before sending mail, creating a fulfillment, or changing a balance. If another worker cannot acquire the same key, it should return safely without repeating the action. Do not rely on an in-memory flag, because a restart or a second server will not share it.
Run a retry test matrix
Use a sandbox or test account and a customer record created only for the test. Verify the webhook signature before processing; Stripe requires the raw request body, signature header, and endpoint secret for signature verification. Then run these cases:
- Deliver the identical event twice in sequence.
- Deliver the identical event to two workers at nearly the same time.
- Fail after the idempotency record is claimed but before the customer action completes.
- Fail after the customer action completes but before the handler records completion.
- Deliver related events in reverse order.
- Replay an event after a process restart.
For each case, count provider deliveries, handler attempts, idempotency claims, and completed customer actions. The first three counts may exceed one; the last count must remain one. If the workflow sends email, inspect the test mailbox. If it writes a record, query by the business key. If it charges or fulfills, use only the provider sandbox and confirm a single test transaction.
Separate acknowledgment from slow work
Stripe recommends returning a successful response quickly before complex logic. A useful design verifies the request, records or queues the event, and acknowledges it; a worker then performs the slower action under the idempotency key. This reduces avoidable timeouts without pretending that acknowledgment proves the downstream action succeeded.
Keep a reconciliation job that compares received events with completed actions. Alert on a key left in `processing`, repeated failures, or a payload conflict for an existing key. Retain deduplication records for at least the period in which the provider or an operator can retry old events, based on the provider's current documentation and your own recovery policy.
The evidence that proves the fix
A passing test includes logs showing multiple attempts with the same event or business key, one successful claim, one durable customer action, and safe acknowledgments for later deliveries. Also save the test event IDs, timestamps, worker identifiers, and the query or screenshot that shows exactly one result. That evidence is stronger than a unit test of a helper function because it exercises the queue, database constraint, worker concurrency, and external side effect together.
Do not disable retries to hide duplication. Retries are a recovery mechanism. The goal is to make every retry safe, observable, and incapable of repeating the customer's action.
Sources
- Receive Stripe events in your webhook endpoint — Stripe
- Best practices for working with AWS Lambda functions — Amazon Web Services