WordPress Webhook Retry, Replay & Dead-Letter Queue

/ The Problem

The Webhook Problem

WordPress fires a webhook, the receiving endpoint is down for 30 seconds, and the event is gone forever. No retry. No alert. The order completed, the form submitted, the customer updated — but the downstream system never heard about it.

This is the default behavior of every fire-and-forget webhook implementation. The HTTP call is made once, inline, during the PHP request. If it fails, nothing records the failure. If the endpoint returns a 500, nothing reschedules delivery. If PHP crashes mid-request, the attempt disappears entirely.

The problem compounds over time. Integration drift accumulates silently — your CRM, ERP, or automation platform slowly diverges from WordPress reality. Users report inconsistencies days later, when there is no forensic record of what happened and no mechanism to recover the lost events.

The reliability gap is well-documented in production systems. Stripe's webhook best practices documentation notes that endpoints can be unavailable for a range of reasons — server errors, network issues, timeouts — and recommends building retry logic with exponential backoff as a baseline expectation, not an edge case.

Reliable webhook delivery requires two distinct mechanisms: automatic retry for failed deliveries, and manual replay for events that delivered successfully but need to be resent. Most implementations implement one or neither. This article explains how to build both — and why both are necessary for a production-grade system.

See also: why WordPress webhooks silently fail in production — a detailed breakdown of every structural failure mode in the default WordPress webhook model.

/ Reliability

Why do Retries Alone fail?

Automatic retry covers the most common failure scenario: a transient network error, a temporary endpoint outage, a brief API rate limit. You send the webhook, it fails, you wait and try again. After a few attempts it succeeds and the problem resolves itself.

But retry logic cannot solve every delivery problem. Consider what happens after a successfully delivered event:

Your receiving system had a bug. It accepted the webhook and returned 200, but processed the payload incorrectly — wrote wrong data to the database, triggered the wrong automation, or silently dropped the record due to a validation error. By the time you discover the problem, the retry window has long passed. The event has status "delivered." No retry will ever fire.

Or your integration was rebuilt. You migrated to a new CRM, redeployed your automation platform, rewrote the receiving endpoint. Now you need to resend six months of order events to populate the new system. Retries only cover the recent past — and only failures at that.

Or a downstream service had an outage. Events delivered successfully, but the service did not process them correctly during the outage window. Their support team tells you to resend the events from that period. You have no mechanism to do that.

Replay is the mechanism that covers these cases. It lets you resend any previously delivered event — on demand, to the original endpoint or a new one — regardless of how long ago it was delivered. Retry handles failures automatically. Replay handles the rest manually.

/ Mechanisms

Retry vs Replay

The two mechanisms are distinct in trigger, process, and use case. Conflating them leads to systems that do one poorly while lacking the other entirely.

Retry is automatic and reactive — it fires when something goes wrong, without human intervention. Replay is manual and intentional — a developer or operator decides to resend a specific event or batch of events.

Both mechanisms require the same prerequisite: the original payload must be stored. You cannot retry or replay an event you have not persisted. This is the foundational design decision that everything else depends on.

/ Architecture

Architecture of the System

The system has three components: payload storage, a retry queue, and a replay mechanism. Every webhook event passes through all three in order.

When a WordPress event fires — an order completed, a form submitted, a post published — the first action is always to store the payload. Not to send it. Storage happens synchronously, before any HTTP call is made. Once persisted, the event cannot be lost, regardless of what happens next.

The delivery attempt is then made asynchronously, outside the WordPress request cycle. Success writes to the event log and marks the event as replay-eligible. Failure enqueues the event for automatic retry with exponential backoff. After exhausting retries, the event reaches dead-letter status and waits for manual intervention.

Replay-eligible events remain in the log indefinitely. Any of them can be manually triggered for resend — individually or in bulk — without creating a new event or modifying the original payload.

/ Storage

How does Payload Storage work?

Every webhook event is stored before any delivery is attempted. The stored record includes everything needed to reconstruct the original delivery and any subsequent retry or replay:

• payload_body — The complete JSON payload as it will be sent. Stored verbatim — no transformation, no reference to the original WordPress object that may have since changed.
• headers — All request headers including Content-Type, X-Event-ID, X-Event-Timestamp, and any signature or authentication headers. Stored with the event so retries and replays send identical headers.
• target_url — The endpoint URL at the time of the original event. Replay uses this by default but allows override — useful for sending to a new endpoint after a migration.
• timestamp — Unix epoch of when the WordPress event fired, not when delivery was attempted. X-Event-Timestamp always carries the original event time, even on retry attempt 5 or a replay six months later.
• response_status — The HTTP status code of each delivery attempt. A complete attempt history is stored — not just the final outcome — so you can see exactly what happened on attempt 1, 2, 3.

The stored payload is immutable after creation. Retries and replays read from it; they do not modify it. This guarantees that every delivery attempt — across all retries and replays — sends exactly the same payload the original event generated.

Storage happens in the same database transaction as the WordPress event. If storage fails, the event is not lost — the WordPress transaction rolls back cleanly. If storage succeeds and the subsequent delivery attempt fails, the event is already in the retry queue.

/ Retry

Retry Logic

Retry fires automatically on any delivery failure that has a realistic chance of succeeding on a subsequent attempt. The key distinction is between transient failures — where retry makes sense — and permanent failures, where it does not.

Retry on: 5xx responses (server errors), connection timeouts, network errors, 429 rate limit responses.

Do not retry on: 4xx responses (except 429). A 400 means the payload is malformed. A 401 means authentication failed. A 404 means the URL no longer exists. None of these resolve themselves through retrying — they require a code fix, a configuration change, or human review. Retrying 4xx responses burns the entire attempt budget on an unrecoverable failure and masks the real problem.

Retry attempts use exponential backoff to avoid hammering a struggling endpoint:

Exponential backoff is the industry-standard approach for retryable HTTP operations. Google Cloud's retry-strategy documentation describes truncated exponential backoff, where spacing retries progressively further apart — rather than at fixed intervals — reduces load on a recovering service and improves overall delivery success rates for transient failures.

Each retry reads the stored payload and sends an identical request. The X-Event-ID header carries the same UUID as the original attempt, allowing the receiving endpoint to deduplicate if needed. The X-Event-Timestamp always carries the original event time, not the retry time.

After exhausting all retry attempts, the event moves to dead-letter status. It remains in the log, visible, with a complete attempt history. From dead-letter status it can be manually replayed once the underlying problem — a misconfigured endpoint, a broken authentication key, a malformed payload — has been fixed. For the design rationale behind these numbers — what to retry, jitter, caps, and the tracking schema — see webhook retry policy: exponential backoff & schema.

See also: async webhooks in WordPress — the architectural approach to moving webhook dispatch outside the PHP request cycle entirely.

/ Replay

Replay Logic

Replay resends a previously stored event. It works on any event in the log — delivered, failed, dead-letter — regardless of age. The trigger is always manual: a developer or operator selects an event and initiates replay.

The replay process reads the stored payload verbatim and sends it as a new HTTP request. By default it uses the original target URL, but allows override to redirect delivery to a new endpoint. The X-Event-ID header carries the same UUID as the original event — this is intentional. The receiving endpoint should recognize the UUID and treat the replay as a duplicate, updating its records rather than creating a new entry.

Replay does not modify the original stored event. It creates a new delivery attempt record linked to the same event. The event log shows the full history: original delivery, any retry attempts, and all replay attempts — each with timestamp, HTTP status, and response body.

Bulk Replay

Individual replay handles one-off cases. Bulk replay handles migrations, outages, and integration resyncs. You select a date range, an event type, or a set of target endpoints, and the system re-queues all matching events for delivery.

Bulk replay runs through the same delivery queue as normal dispatches and retries — it does not bypass the rate limiting or concurrency controls. This prevents a large replay batch from overwhelming a receiving endpoint that is processing at its normal capacity.

When to Use Replay

Replay is the right tool when retry has already completed — either successfully or by exhausting attempts — and you need to resend. Common scenarios:

• Bug in receiving code — Events delivered successfully but processed incorrectly. Fix the bug, replay the affected window.
• Integration migration — Moved to a new CRM or automation platform. Replay historical events to populate the new system.
• Downstream outage — Receiving service was down or malfunctioning during a window. Replay the events from that window once service is restored.
• Dead-letter recovery — Events exhausted retry attempts due to a configuration problem. Fix the configuration, replay from dead-letter status.

/ Lessons

Lessons Learned

Four principles that shaped the design of this system — and that apply to any webhook reliability implementation:

• Assume failures — Design for the failure case, not the success case. Every webhook will eventually fail to deliver. The question is whether your system notices, records it, and recovers automatically — or silently drops it.
• Store payloads before sending — The original payload is the source of truth for all future retries and replays. If you do not store it before the first delivery attempt, you have no recovery path. Store first, send second, always.
• Separate retry and replay concerns — Retry is automatic and covers transient failures. Replay is manual and covers everything else. Building one does not give you the other. Both are necessary for a complete system.
• Observability is not optional — A webhook system without visibility into delivery status, attempt history, and failure reasons is not a reliable system — it is a less-visible version of the original fire-and-forget. Every event should be queryable, every attempt logged, every failure surfaced.

The dead-letter pattern — preserving exhausted-retry events for later inspection and replay — is now standard across major cloud messaging systems. AWS SQS dead-letter queues, Azure Service Bus, and Google Cloud Pub/Sub all implement the same concept: when a message exhausts its retry budget, it moves to a dead-letter store rather than being discarded. The webhook equivalent is essential for the same reason.

/ Final Thoughts

Final Thoughts

WordPress ships with no webhook persistence, no retry logic, and no replay mechanism. Every one of these capabilities has to be built — or used from a plugin that provides them. The gap between the default behavior and production-grade reliability is not small.

The architecture described here — store before send, automatic retry with exponential backoff, manual replay on demand — covers the full range of delivery failure scenarios. Transient failures recover automatically. Permanent failures surface for human review. Successful deliveries remain replayable indefinitely.

The observability layer is what makes the rest of it trustworthy. Without a complete log of every delivery attempt, you cannot verify that the system is working — you are back to hoping the events arrived.