The first version of a webhook is always the same four lines:
await fetch(customer.webhookUrl, {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify(event),
});
It works. You ship it. Then the tickets start.
A customer's endpoint was down during a deploy and they want the events from that window. Another integration processed the same order twice. Someone asks how they can prove the request came from you. Six hours after an incident, a customer wants to know exactly when you attempted event evt_8813 and what their server returned.
The HTTP call is the easy part. Delivery is the queue, retry policy, signature scheme, idempotency story, and attempt log around it.
Those failure paths are difficult to learn from a finished code sample, so we built an interactive Webhook Delivery Simulator. It lets you break a delivery on purpose, follow every attempt, inspect the signed bytes, and redeliver the same message to see what a safe receiver does.
The delivery outcomes are simulated. The HMAC-SHA256 signing and verification are real and run locally in your browser.
TL;DR
- A webhook sender is a durable state machine, not an HTTP client with a retry loop.
- Retry timeouts,
429, and5xxresponses. Most other4xxresponses should stop immediately. - A timeout means the outcome is unknown. The receiver may have completed the work before its response was lost.
- At-least-once delivery makes receiver-side deduplication mandatory.
- Sign the message ID, timestamp, and raw body. Verify before parsing.
- The simulator uses the published Svix retry schedule and Standard Webhooks signing model so the behavior maps to a concrete production system.
Start with the failure that retries are for
Open the simulator, leave the endpoint response set to Intermittent, and send invoice.paid.
The endpoint returns two temporary failures and then recovers:
Attempt 1: 503 -> retry in 5 seconds
Attempt 2: 503 -> retry in 5 minutes
Attempt 3: 200 -> delivered
This is the shape of a deploy, a brief database outage, or a process restarting at the wrong moment. Trying again helps because the endpoint is broken now, not permanently.
The simulator compresses the real waits into a few seconds. Its time-warp indicator still shows the actual gap being skipped, and every attempt keeps its own status, scheduled time, duration, response body, and explanation.
That attempt history is not optional operational polish. It is how you answer the customer who asks why an event arrived five minutes late.
Retry failures, not mistakes
The useful question is not "Did the request fail?" It is "Could the same request plausibly succeed later?"
| Result | Retry? | Reason |
|---|---|---|
| Timeout | Yes, carefully | There is no reliable outcome |
429 Too Many Requests |
Yes | Back off and reduce the endpoint's rate |
500, 502, 503
|
Yes | The server may recover |
408 Request Timeout |
Yes | The server explicitly asks for another attempt |
400, 422
|
No | The payload is still wrong on the next attempt |
401, 403
|
No | Retrying cannot repair credentials |
404, 410
|
No | The endpoint is gone |
Try the 400, 429, 500, and timeout modes. They all produce a non-successful first attempt, but they should not produce the same next state.
Hammering a malformed request for 27 hours does not make it valid. It turns a customer's configuration error into your outbound traffic problem and buries useful failures in the delivery log.
Why the retry gaps get large
Fixed retries are cheap to write and expensive to operate. Retrying every 30 seconds for an hour produces 120 attempts against an endpoint that may already be struggling to recover.
The simulator uses Svix's published retry schedule:
attempt 1 immediately
attempt 2 +5 seconds
attempt 3 +5 minutes
attempt 4 +30 minutes
attempt 5 +2 hours
attempt 6 +5 hours
attempt 7 +10 hours
attempt 8 +10 hours
The final attempt lands roughly 27 hours and 35 minutes after the first.
The exact numbers are less important than the shape. Try quickly enough to ride out a small network blip, then spread later attempts out so a long outage costs a handful of requests rather than thousands. A production sender also adds jitter and per-endpoint rate limits so thousands of delayed messages do not wake up together.
A timeout does not mean the work failed
Timeout is the case that makes webhook delivery genuinely awkward.
Consider this sequence:
- The receiver accepts the request.
- It updates its database.
- Its response is lost, or arrives after your timeout.
- Your sender schedules another attempt.
The receiver completed the work, but the sender cannot know that. If you retry, the receiver sees the message twice. If you do not retry, you might silently lose an event. There is no third choice that avoids both risks.
The normal answer is at-least-once delivery: retry the unknown outcome and make repeated processing safe.
A stable message ID must survive every retry and manual redelivery. The receiver stores that ID in the same transaction as the business change:
async function handleWebhook(messageId, event) {
await database.transaction(async (tx) => {
if (await tx.processedMessages.exists(messageId)) {
return; // Seen already. Return 200 so retries stop.
}
await applyBusinessChange(tx, event);
await tx.processedMessages.insert({ messageId });
});
}
Writing the dedup row before the work risks losing the event after a crash. Writing it after the work lets two concurrent deliveries pass the check. One transaction closes both gaps.
After a successful run in the simulator, click Redeliver same message. The message ID stays stable and the receiver acknowledges the duplicate without applying the event twice.
Verify bytes before you trust JSON
Webhook signatures prove that the sender knows a shared secret and that the signed content was not changed in transit.
The simulator follows the Standard Webhooks format used by Svix. The signature covers the message ID, timestamp, and raw request body:
const signedContent = `${messageId}.${timestamp}.${rawBody}`;
const signature = hmacSha256(secret, signedContent);
The important word is raw.
If your framework parses the JSON and you serialize it again, whitespace, escaping, or property order can change. The object may mean the same thing while its bytes are different, which correctly produces a different signature.
A receiver should:
- Read the raw body.
- Reject timestamps outside the accepted replay window.
- Verify the HMAC using constant-time comparison.
- Parse the JSON only after verification passes.
- Deduplicate using the stable message ID.
Open Verify signature in the simulator and try each failure mode:
- Edit body changes one value after the request was signed.
- Wrong secret calculates a valid HMAC with the wrong key.
- Replay later moves verification outside the timestamp tolerance.
- Untouched restores the valid request.
The simulator shows the signed content and the exact rejection reason rather than reducing every failure to "invalid signature."
Where Svix fits
The useful thing about Svix Dispatch is not that it sends an HTTP request. The useful thing is that it packages the operational surface around the request: durable delivery, automatic retries, signing and secret rotation, rate limits, event filtering, searchable attempt logs, manual replay, and a customer-facing endpoint portal.
Building can still be reasonable when you control every consumer, event volume is low, or you already run a capable job system such as Temporal, Sidekiq, or River.
The boundary changes when the endpoints belong to customers. At that point, retries are only one part of the product. Customers also need to configure endpoints, rotate secrets, inspect failed attempts, and replay messages without asking an engineer to search production logs.
A useful test is to write down who answers this question:
"Did you send event
evt_8813, and what did our endpoint return?"
If the answer is "an engineer with log access," include that recurring support cost in the build-versus-buy calculation.
A five-minute simulator walkthrough
- Send
invoice.paidto the Intermittent endpoint. - Watch the delivery path and compressed retry delays.
- Select each attempt and compare its response and scheduled time.
- Inspect the request headers and raw body.
- Open Verify signature and edit the body.
- Restore the untouched request and verify it successfully.
- Redeliver the same message and confirm that the receiver ignores it.
- Repeat with Timeout, then with 400 bad request.
The final comparison is the useful one. A timeout and a 400 both lack a successful response, but they require opposite decisions: retry the unknown outcome and drop the invalid payload.
What to carry into production
Before calling a webhook sender reliable, check that it has:
- Durable storage before the first attempt
- Explicit response classification
- Backoff with jitter and a defined give-up point
- Stable message IDs across retries
- HMAC signatures over the raw body
- Timestamp-based replay protection
- Receiver-side deduplication
- Per-endpoint rate limits
- Searchable attempt history
- Manual replay with an audit trail
The POST is the smallest part of the system. Reliability comes from the state and operational controls around it.
Use the Webhook Delivery Simulator to exercise the failure paths, then read the full production webhook delivery guide for a typed Svix sender and Express receiver.