Webhook failures are normal. Networks time out, destination APIs throttle requests, credentials expire, and field validation changes. A production workflow is not one that never fails; it is one that makes each failure visible, retries only when useful, and lets an operator recover without creating duplicates.
Define a stable delivery contract
Every outbound event should include:
- A stable event name such as `lead.created`.
- A unique delivery ID for tracing a single attempt.
- A stable lead or event ID for idempotency.
- A timestamp and schema version.
- The normalized contact, source, campaign, routing, and consent fields the destination needs.
- A cryptographic signature when a shared signing secret is configured.
Keep the event envelope stable even when optional lead fields vary. This reduces brittle destination mappings and makes version changes deliberate.
Sign and verify the payload
InstaChime can sign CRM webhook requests with the `x-instachime-signature` header. The receiver should compute an HMAC from the exact raw request body and compare it with a constant-time function before parsing or processing the event.
Reject missing or invalid signatures with a clear 401 or 403 response. Do not log the shared secret, full authorization headers, or unnecessary personal data while troubleshooting.
Make the receiver idempotent
Retries can deliver the same logical event more than once. The destination should store the event or lead ID and return success when an already-processed event is received again. Upsert operations are usually safer than blind inserts for CRM contacts.
A common pattern is:
1. Check whether the event ID was processed.
2. If yes, return a successful response without repeating side effects.
3. If no, start a transaction, create or update the destination record, store the event ID, and commit.
Idempotency is especially important when a request succeeds downstream but the response is lost before InstaChime receives it.
Classify responses before retrying
Retry transient failures:
- Connection resets and network timeouts.
- HTTP 408, 425, 429, and selected 5xx responses.
- Temporary destination maintenance or capacity errors.
Do not repeatedly retry permanent failures:
- Invalid credentials until the credential is corrected.
- Validation errors caused by missing required fields.
- Unsupported methods or malformed endpoint URLs.
- Explicit permission denial.
The delivery record should retain the status code, a redacted response excerpt, attempt count, and next retry time. This gives operators enough evidence without exposing secrets.
Use bounded exponential backoff
Immediate repeated retries can amplify a destination outage. Space attempts with increasing delays and a small random jitter. Cap both the maximum delay and total number of attempts. After the final attempt, move the delivery to a dead-letter state that is visible in the integration dashboard.
InstaChime queues outbound CRM deliveries, records each attempt, and stops retrying after the configured maximum. Operators can inspect failures and manually replay a delivery after fixing the endpoint, credential, or field mapping.
Run a safe replay
Before replaying:
1. Confirm the destination is healthy.
2. Correct the specific credential, permission, URL, or validation problem.
3. Verify the receiver is idempotent.
4. Replay one failed delivery.
5. Confirm the CRM record and delivery response.
6. Replay the remaining matching failures in controlled batches.
Do not bulk-replay an unknown failure class. One incorrect mapping can turn a recoverable queue into duplicate or corrupted CRM data.
Monitor the right signals
Track delivery success rate, retry rate, dead-letter count, age of the oldest failed event, destination latency, and failures by status code. Alert on sustained patterns rather than a single transient timeout.
For daily operations, review newly dead-lettered deliveries and any destination with a rising retry rate. For weekly operations, compare failures with recent credential rotations, CRM field changes, automation edits, and provider incidents.
Test before relying on live leads
Follow the CRM webhook overview to configure the destination and signing secret. Send a controlled test lead, confirm the signature, force one temporary failure, verify the retry, and confirm that replay does not create a duplicate. Then use the integration launch checklist before enabling production traffic.
The objective is not merely to send a webhook. It is to preserve a traceable lead handoff when networks and downstream systems behave imperfectly.