
Event flow, payload design, authentication, retries and idempotency, human handoff, and observability — with a worked example you can adapt.
An AI agent that lives in a chat window is a demo. An AI agent wired into your CRM — triggered by real events, writing back structured results, escalating to a human when it should — is a system. The difference is almost entirely integration engineering, and webhooks are where that engineering lives.
This guide walks the full path: which events should fire, what the payload should contain, how to authenticate it, how to survive duplicates and outages, when to hand off to a person, and what to log so you can debug it later.
The event flow
Keep the shape simple and one-directional at each hop. Complexity in the topology is what makes these systems impossible to reason about at 2am.
CRM event (stage change, new lead, inbound reply)
|
v
Webhook receiver --- verify signature, dedupe on event_id
|
v
Enqueue job (return 200 immediately)
|
v
Agent runner --- build context -> call model -> tools -> validate output
| |
| +--> low confidence / risky intent
v |
Write back to CRM (note, field, task) v
| Human review queue
v
Observability: log event_id, prompt, tools, output, latency, costThe critical rule: the webhook receiver acknowledges within a couple of seconds and does nothing slow. Model calls take seconds and sometimes fail. If you call the model inside the webhook handler, the CRM's delivery timeout becomes your reliability ceiling and it will retry, giving you duplicate work.
Payload design
Design the payload as a contract, not as whatever the CRM happens to emit. Three properties matter: it must be versioned, it must carry a stable event identifier, and it must be minimal.
{
"schema_version": "1.0",
"event_id": "evt_01HZX9K2QW",
"event_type": "deal.stage_changed",
"occurred_at": "2026-08-07T14:02:11Z",
"source": "crm",
"object": {
"type": "deal",
"id": "deal_4821",
"url": "https://crm.example.com/deals/4821"
},
"changes": { "stage": { "from": "qualified", "to": "proposal_sent" } },
"actor": { "type": "user", "id": "user_17" }
}- Send identifiers and a URL, not a full record dump. Fetch the current record when you process it — the payload is a notification, and by the time you read it the record may have changed.
- Never put PII you do not need into the payload, and never put secrets in it at all.
- Include schema_version from day one so you can evolve without breaking consumers.
- event_id must be stable across retries of the same event. If your CRM does not provide one, derive it deterministically from object id + change + timestamp.
Authentication
A webhook URL is a public write endpoint. Verify a signature rather than relying on a secret in the query string, and compare with a timing-safe function. Reject stale timestamps so a captured request cannot be replayed indefinitely.
import { createHmac, timingSafeEqual } from "crypto";
const MAX_SKEW_MS = 5 * 60 * 1000;
export function verifyWebhook(
rawBody: string,
signatureHeader: string | null,
timestampHeader: string | null,
secret: string,
): boolean {
if (!signatureHeader || !timestampHeader) return false;
const ts = Number(timestampHeader);
if (!Number.isFinite(ts)) return false;
if (Math.abs(Date.now() - ts) > MAX_SKEW_MS) return false; // replay window
const expected = createHmac("sha256", secret)
.update(timestampHeader + "." + rawBody)
.digest("hex");
const a = Buffer.from(signatureHeader);
const b = Buffer.from(expected);
return a.length === b.length && timingSafeEqual(a, b);
}Verify against the raw request body, before any JSON parsing or re-serialization — reserializing changes bytes and breaks the signature. Also allow two active secrets at once so you can rotate without downtime.
Retries and idempotency
Assume every event will arrive more than once. Delivery systems retry on timeouts, and a timeout does not tell the sender whether you processed the event. Idempotency is what makes that safe.
create table webhook_events (
event_id text primary key,
event_type text not null,
status text not null default 'processing',
result_ref text,
received_at timestamptz not null default now(),
completed_at timestamptz
);
-- Claim the event. If a row already exists, this returns nothing
-- and the handler acknowledges without doing the work twice.
insert into webhook_events (event_id, event_type)
values ($1, $2)
on conflict (event_id) do nothing
returning event_id;- Return 2xx as soon as the event is durably recorded; do the work after.
- Return 4xx only for payloads that will never succeed (bad signature, unknown schema). A 4xx tells the sender to stop retrying.
- Return 5xx for transient problems so the sender retries. Make sure your own retries use exponential backoff with jitter.
- Cap retries and move exhausted events to a dead-letter table with the original payload, so nothing disappears silently.
- Make CRM writes idempotent too: upsert on a key, or record the write reference in result_ref so a replay can detect it already happened.
Free 30-min audit
Want us to build this for your business?
We build n8n automations and AI agents for SMBs. Book a free 30-minute audit — we'll map your highest-ROI workflow live, no pitch.
Book a free callThe agent step
The agent is the least reliable component in the chain, so constrain it. Give it a narrow job, structured output, and explicit tools rather than free-form authority over your CRM.
const OutputSchema = z.object({
intent: z.enum(["follow_up", "schedule", "answer_question", "escalate"]),
confidence: z.number().min(0).max(1),
crm_note: z.string().max(1200),
suggested_reply: z.string().max(1200).optional(),
escalation_reason: z.string().optional(),
});
const result = OutputSchema.safeParse(JSON.parse(modelOutput));
// Anything unparseable, low-confidence, or explicitly escalated
// goes to a person. The agent never decides to skip that path.
if (!result.success) return toReview(event, "unparseable_output");
if (result.data.intent === "escalate") return toReview(event, result.data.escalation_reason);
if (result.data.confidence < 0.7) return toReview(event, "low_confidence");
await crm.addNote(event.object.id, result.data.crm_note);
if (result.data.suggested_reply) {
await crm.createDraft(event.object.id, result.data.suggested_reply);
}Note what the agent is not allowed to do here: it drafts a reply, it does not send one. Promoting a step from draft to autonomous send should be a deliberate decision made after you have evidence, not the default. Our guide on evaluating AI agent reliability before production covers how to gather that evidence.
Human handoff
Handoff is a product surface, not an error path. Design it properly and your team will trust the system; bolt it on and they will quietly stop using it.
- Every escalation lands somewhere a specific person owns — a CRM task queue, a shared inbox, or a Slack channel with rotation.
- Include the reason, the source event link, and the agent's draft so the human starts from something instead of nothing.
- One-click accept, edit, or reject. Store which one they chose — that is your highest-quality eval data.
- Set a time budget: if nothing is actioned within N hours, escalate again rather than letting it age.
- Always allow a human to disable the automation for a single record without disabling the whole workflow.
Observability
Log an event-scoped record you can query later. The single most useful field is the correlation id threaded through every hop, so one search reconstructs the whole story.
{
"event_id": "evt_01HZX9K2QW",
"correlation_id": "evt_01HZX9K2QW",
"stage": "agent_completed",
"model": "gemini-2.5-flash",
"input_tokens": 1840,
"output_tokens": 260,
"latency_ms": 2310,
"intent": "follow_up",
"confidence": 0.82,
"action": "crm_note_created",
"human_review": false
}- Track delivery success rate, duplicate rate, and dead-letter count on the webhook side.
- Track escalation rate and human accept/edit/reject ratios on the agent side — a rising edit rate is an early warning.
- Track cost per event; see our cost-modelling guide for how to turn those token counts into a monthly number.
- Alert on failure rate and dead-letter growth, not on individual failures, or the alert gets muted.
Rollout sequence
- Shadow mode — the agent runs on real events and logs its output, but writes nothing to the CRM.
- Draft mode — output appears as CRM notes and drafts for humans to send.
- Assisted mode — low-risk intents write automatically; anything else escalates.
- Reviewed autonomy — a defined, narrow set of intents runs unattended with sampled review.
Every integration failure we have debugged came down to one of three things: no idempotency, no signature, or no place for a human to intervene.
Related reading: our sales automation and AI lead generation pages cover the workflow patterns these integrations usually serve, and the case studies show how the pieces fit together in delivered builds.
Need this built for you? We scope, build, and hand over automation systems as a done-for-you engagement — see what an AI automation agency actually delivers, the published pricing ranges, or book a 30-minute call.
Frequently asked questions
What's your AI Readiness Score?
10 questions. A personalized score, profile, and a 90-day automation roadmap built for your business. No email required.
30-minute working session
Find the highest-ROI automation in your business
Bring one workflow that is slow, repetitive, or leaking opportunities. We will map the bottleneck, the systems involved, and whether automation is actually worth implementing.
No obligation. If automation is not the right answer, we will say so.



0 Comments
Be the first to comment. Start the conversation below.