Webhook Dead Letter Queues: What to Do When Delivery Permanently Fails

Most webhook reliability advice stops at "add retries with exponential backoff." That's necessary, but it only solves transient failures — a brief timeout, a deploy that took the endpoint down for thirty seconds. Retries assume the problem goes away on its own.

But what about the events where it doesn't? Your endpoint had a bug for two hours. A database migration locked a table. A bad deploy returned 500s until someone noticed. The retries fire, back off, fire again, and eventually run out. What happens to that event then?

If the answer is "nothing, it's gone," you have a data-loss bug waiting to happen. This is what a dead letter queue (DLQ) is for.

What a Dead Letter Queue Actually Is

A dead letter queue is a holding area for messages that could not be processed after all delivery attempts were exhausted. Instead of dropping the event, you move it somewhere durable where a human (or an automated job) can inspect it, fix the underlying problem, and re-process it later.

The concept comes from message brokers — RabbitMQ, SQS, and Azure Service Bus all have native DLQ support — but the same pattern applies to webhooks. When a webhook can't be delivered after N retries, it goes to the DLQ rather than into the void.

Designing One Yourself

A minimal DLQ for webhooks needs four things:

1. Durable storage of the full event. Not just the ID — the entire raw payload, headers, the destination URL, and the timestamp. You need enough to reconstruct and re-send the exact original request.

2. Failure metadata. Why did it fail? Store the last HTTP status code, the response body, and the number of attempts. This is what turns a DLQ from a graveyard into a debugging tool.

3. A replay mechanism. The whole point is to re-deliver once the problem is fixed. You need a way to take a dead-lettered event and push it back through delivery — ideally in bulk, filtered by date range or destination.

4. Visibility and alerting. A DLQ that nobody looks at is just a slower way to lose data. You need to know when events land in it.

Here's the shape of the storage:

public class DeadLetter
{
    public Guid Id { get; set; }
    public string DestinationUrl { get; set; }
    public string RawPayload { get; set; }   // exact original bytes
    public string Headers { get; set; }       // JSON
    public int AttemptCount { get; set; }
    public int? LastStatusCode { get; set; }
    public string LastResponseBody { get; set; }
    public DateTime FailedAt { get; set; }
    public bool Replayed { get; set; }
}

The tricky parts aren't the storage — they're the operational pieces. Replaying needs idempotency so you don't double-process events that actually succeeded. Bulk replay needs throttling so you don't hammer a just-recovered endpoint. And you need retention policy: how long do you keep dead letters before purging them?

The Replay Problem

Replay is where naive implementations fall down. Say your endpoint was broken for two hours and 400 events dead-lettered. You fix the bug and replay all 400. Two things can go wrong:

• You re-deliver events that actually succeeded. If an event returned 500 after your handler committed its work, replaying it processes it twice. This is why idempotency keys matter even with a DLQ.
• You overwhelm the freshly-recovered endpoint. Dumping 400 requests in one second onto an endpoint that just came back up can knock it over again. Replay needs the same rate limiting and backoff as normal delivery.

Skipping the Build

A dead letter queue is one of those features that's straightforward to describe and surprisingly involved to build well — durable storage, failure metadata, throttled bulk replay, retention, and alerting all have to work together.

Bluejay Relay includes a dead letter queue out of the box. When a destination exhausts its retry policy, the event is moved to the dead letter queue with the full payload, headers, and the failure details — last status code and response body included. From the dashboard you can inspect exactly why each delivery failed, fix your endpoint, and replay events individually or in bulk filtered by date range. Replays go through the same throttled, retried delivery path as live traffic, so you won't knock over an endpoint that just recovered.

The result: a two-hour outage on your side becomes a non-event. The webhooks waited safely in the queue, and one click sends them on their way.

---

Don't lose webhooks to permanent failures. Try Bluejay Relay free — the dead letter queue is included on every plan, with one-click replay on the paid plans.