Deduplication & idempotency

Two distinct mechanisms prevent duplicate notifications. They solve different problems and can be used together.

Which do I need? If two code paths could trigger the same logical event → use deduplication. If your caller might retry the same API call → use idempotency. If both are possible → use both.

Match your scenario

Find your situation below. Each card tells you which mechanism to use and why.

Webhook handler fires twice

Stripe/GitHub retries the same webhook because your endpoint was slow to respond. The same event arrives again.

Use: idempotencyKey

Key: webhook:{eventId} — the provider gives you a stable event ID. Second delivery returns the original result.

Two code paths trigger same notification

An async worker and a realtime handler both fire "mentioned you" for the same comment edit.

Use: dedupeKey

Key: mention:{postId}:{actorId} — scoped to the logical event. Second call is skipped within the window.

Background job crashes mid-send

Your worker starts processing, the notification sends, then the job crashes before marking complete. The queue retries the job.

Use: idempotencyKey

Key: job:{jobId} — tied to the job run, not the event. Replay returns the cached result without re-delivering.

User spams "save" on a form

Each save triggers a notification. Rapid clicks could send 5 identical emails about the same edit within seconds.

Use: dedupeKey + short window

Key: edit:{docId}:{userId} with a 2-minute window. Only the first save notifies; the rest are suppressed.

Still unsure? Ask: "is it the same call arriving twice (retry) or the same event triggering from multiple places?" Same call → idempotency. Same event → dedup. Both → use both.

Deduplication (semantic)

Prevents semantically identical notifications within a time window. Example: "user X mentioned you" shouldn't fire twice if two code paths trigger it for the same mention.

lib/send-mention.ts

await notify.send({
  recipientId: user.id,
  notificationId: "comment_mentioned",
  payload: { actorName: "Rey", postUrl: "/posts/42" },
  dedupeKey: "mention:post_42:user_rey",
  dedupeWindowMs: 5 * 60_000,
})

The composite key is scoped to (dedupeKey, notificationId, recipientId). The first send within the window succeeds; any subsequent send with the same composite key is skipped with reason "duplicate".

Dedup result

lib/send-mention.ts

const result = await notify.send({ ... dedupeKey, dedupeWindowMs })

if (result.skipped.some((item) => item.reason === "duplicate")) {
  console.log("duplicate within window")
}

Idempotency (retry-safe)

Prevents the same API call from being processed twice. Use when your caller might retry (network timeout, worker crash). The same logical operation returns the original result on replay.

workers/fulfillment.ts

await notify.send({
  recipientId: user.id,
  notificationId: "order_shipped",
  payload: { orderNumber: "ORD-123", trackingUrl: "..." },
  idempotencyKey: "ship:ORD-123",
})

The composite key is (idempotencyKey, notificationId, recipientId). A duplicate call within the TTL window returns the original NotificationRecord without re-processing.

TTL

Idempotency keys expire after a configurable TTL (default: 24 hours). After expiry, the same key can be reused.

lib/notifykit.ts

createNotifyKit({
  // ...
  idempotencyKeyTtlMs: 48 * 60 * 60_000,
})

Comparison

	Deduplication	Idempotency
Purpose	Prevent same logical event from notifying twice	Make retries safe
Key	`dedupeKey` (caller-provided)	`idempotencyKey` (caller-provided)
Window	`dedupeWindowMs` (per-call)	`idempotencyKeyTtlMs` (global config)
On duplicate	Create an audit record and skip provider delivery	Return the existing notification record
Skip reason	`"duplicate"`	`"idempotent_replay"`

Using both together

workers/process-mention.ts

await notify.send({
  recipientId: user.id,
  notificationId: "comment_mentioned",
  payload: { actorName: "Rey", postUrl: "/posts/42" },
  dedupeKey: "mention:post_42:user_rey",
  dedupeWindowMs: 10 * 60_000,
  idempotencyKey: "job:abc123",
})

Idempotency check

If the key already exists → return the original result immediately. No further processing.

Dedup check

If the dedup key exists within its window → skip provider delivery, write an audit record.

Normal send

Both checks passed → proceeds to rate limits, preferences, channels, delivery.

Key design guide

The key you choose determines what counts as "the same". Get it wrong and you either miss duplicates or suppress legitimate sends.

Scenario	Good key	Why
User mentioned in a comment	`mention:{postId}:{actorId}`	Same actor mentioning in same post = duplicate. Different actor = new notification.
Order shipped	`ship:{orderId}`	One shipment notification per order, regardless of retries.
Background job retry	`job:{jobId}`	Same job ID = same logical operation. Use as idempotency key.
Daily digest trigger	`digest:{userId}:{date}`	One per user per day. If cron fires twice, second is dropped.

Don't include timestamps in dedup keys. A key like mention:{postId}:{Date.now()} is unique every time — it defeats the purpose. Keys should represent the logical event, not the call.

Constructing a key from scratch

Ask these three questions about your event. Each answer becomes a segment of the key:

What happened?

The event verb: mention, like, ship, invite

To what?

The target entity: post_42, order_123, team_abc

By whom?

The actor (if relevant): user_rey. Omit if any actor doing the same thing is a duplicate.

Event	Key segments	Result	Dedup behavior
Rey mentions Alice in post 42	mention + post_42 + user_rey	`mention:post_42:user_rey`	Rey mentioning twice in same post = duplicate. Bob mentioning = new notification.
Someone likes Alice's post	like + post_42 + user_rey	`like:post_42:user_rey`	Rey liking twice = duplicate. Different user liking = new notification.
Order ships (system event)	ship + order_123	`ship:order_123`	No actor — shipping is a system event. One notification per order, period.
Daily summary cron	digest + user_alice + 2026-06-25	`digest:user_alice:2026-06-25`	Include date to allow one per day. If cron fires twice, second is dropped.

When in doubt, include the actor. A key without an actor means any user doing that action to the same target gets deduped. That's correct for "order shipped" (system event) but wrong for "user mentioned you" (you want each actor's mention to notify separately).

Testing your setup

Verify dedup and idempotency in your test suite by sending twice and checking the result flags:

tests/dedup.test.ts

const first = await notify.send({
  recipientId: "test_user",
  notificationId: "comment_mentioned",
  payload: { actorName: "Rey", postUrl: "/posts/1" },
  dedupeKey: "mention:post_1:rey",
  dedupeWindowMs: 60_000,
})
expect(first.deliveries.length).toBeGreaterThan(0)

const second = await notify.send({
  recipientId: "test_user",
  notificationId: "comment_mentioned",
  payload: { actorName: "Rey", postUrl: "/posts/1" },
  dedupeKey: "mention:post_1:rey",
  dedupeWindowMs: 60_000,
})
expect(second.skipped.some(s => s.reason === "duplicate")).toBe(true)
expect(second.deliveries.length).toBe(0)

Mistake	Symptom	Fix
Key too broad (`mention:{postId}`)	Different actors' mentions get suppressed	Include the actor: `mention:{postId}:{actorId}`
Key too narrow (includes timestamp)	Duplicates always go through	Remove any time-varying component from the key
Window too short (1 second)	Rapid retries still create duplicates	Use 5–10 minutes minimum for event dedup
Confusing dedup with idempotency	Using `dedupeKey` for retry safety	Use `idempotencyKey` for retries — it returns the original result instead of skipping

Choosing the right dedup window

The window should be longer than the maximum time between duplicate triggers, but shorter than the interval at which the event can legitimately recur. Get it wrong and you either miss duplicates or swallow real notifications.

Window too short	Window too long	Window just right
Retries or delayed webhooks create duplicates	User does the same action again (legitimately) and gets no notification	Covers the retry/propagation window without blocking real events

Use the event's natural recurrence interval to pick a window:

Event type	Typical duplicate source	Recommended window	Reasoning
Webhook-triggered (e.g. payment received)	Provider retries delivery 3× over 30 seconds	5 minutes	Covers retry window with margin. A new payment is a different `paymentId` in the key anyway.
User action (e.g. mentioned in comment)	Dual code paths fire for same edit	10 minutes	Edit + save + async worker can span several seconds. A new mention is a new comment ID.
Cron-triggered (e.g. daily summary)	Cron fires twice due to clock skew or restart	1 hour	Include the date in the key — window just needs to cover the cron jitter.
Realtime/streaming (e.g. typing indicator)	Rapid-fire events from the same action	30 seconds – 2 minutes	Events recur legitimately in seconds; window should be tight.
System event (e.g. deploy completed)	Multiple services report the same deploy	15 minutes	Deploy pipelines can span minutes. Key by deploy ID so a new deploy isn't suppressed.

webhooks/stripe.ts

await notify.send({
  recipientId: user.id,
  notificationId: "payment_received",
  payload: { amount: 49.99, orderId: "ORD-456" },
  dedupeKey: `payment:${payload.paymentId}`,
  dedupeWindowMs: 5 * 60_000,
})

webhooks/deploy.ts

await notify.send({
  recipientId: user.id,
  notificationId: "deploy_completed",
  payload: { service: "api", sha: "abc123" },
  dedupeKey: `deploy:${payload.deployId}`,
  dedupeWindowMs: 15 * 60_000,
})

When the key is unique per occurrence, window length matters less. If your dedup key includes a unique identifier (like paymentId or deployId), the window only needs to outlast the retry/propagation delay. A new occurrence generates a new key and passes dedup regardless of the window.

Monitoring dedup in production

Your dedup keys work perfectly in tests — but are they catching real duplicates in production, or silently swallowing legitimate notifications? Track dedup hits to find out.

lib/notifykit.ts

createNotifyKit({
  // ...
  on: {
    "send.completed": ({ result, input }) => {
      const wasDeduplicated = result.skipped.some(s => s.reason === "duplicate")

      if (wasDeduplicated) {
        metrics.inc("notifykit.dedup.hit", {
          notification: input.notificationId,
          dedupeKey: input.dedupeKey ?? "none",
        })
      } else if (input.dedupeKey) {
        metrics.inc("notifykit.dedup.miss", {
          notification: input.notificationId,
        })
      }
    },
  },
})

Signal	Meaning	Action
Hit rate 0% over 7 days	Dedup is configured but never triggers	Either your keys are too narrow (include timestamps?) or duplicates aren't occurring. Simplify or remove dedup if it's not needed.
Hit rate > 50%	More sends are duplicates than unique	Normal for webhook-triggered events (providers retry aggressively). Concerning for user actions — investigate upstream callers.
Hit rate spikes suddenly	A code path started double-firing	Check recent deploys. A new subscriber, webhook handler, or event listener may be duplicating triggers.
Users report missing notifications	Keys may be too broad — legitimate sends getting suppressed	Check if different actors or different targets share the same dedup key. Add more specificity.
Hit rate varies by notification type	Some events have noisier triggers than others	Expected. Webhook-triggered events should be high; manual user actions should be near zero.

Auditing suppressed sends

When a user reports a missing notification, check if dedup caught it. Use the timeline or query directly:

scripts/audit-dedup.ts

const result = await notify.send({
  recipientId: "user_123",
  notificationId: "comment_mentioned",
  payload: { actorName: "New Person", postUrl: "/posts/99" },
  dedupeKey: "mention:post_99:new_person",
  dedupeWindowMs: 10 * 60_000,
})

if (result.skipped.some(s => s.reason === "duplicate")) {
  console.log("Suppressed by dedup — key:", "mention:post_99:new_person")
}

const explanation = await notify.explain({
  recipientId: "user_123",
  notificationId: "comment_mentioned",
  payload: { actorName: "New Person", postUrl: "/posts/99" },
  dedupeKey: "mention:post_99:new_person",
  dedupeWindowMs: 10 * 60_000,
})
console.log("Would deduplicate:", explanation.wouldDeduplicate)

Dashboard query for ops. If you're tracking dedup hits in your metrics system, alert when dedup.hit / (dedup.hit + dedup.miss) > 0.8 for user-triggered events. High dedup rates on user actions usually mean a bug in your event pipeline — not a healthy dedup system.

Feature interactions

Dedup and idempotency run early in the send pipeline. Understanding their position relative to other stages prevents surprises:

Stage order	What happens	If dedup fires here
1. Idempotency	Returns cached result if key seen	N/A — idempotency runs first, replays the original outcome
2. Dedup	Skips if key within window	Send stops here — never reaches rate limit, digest, or delivery
3. Rate limit	Drops if over threshold	—
4. Digest	Buffers into batch window	—
5. Preferences	Skips channels user opted out of	—
6. Quiet hours	Defers push channels	—
7. Deliver	Queues to providers	—

Dedup + digests

Dedup runs before digest buffering. A deduplicated send never enters the digest — it's dropped before the engine considers batching. This means:

tests/dedup-digest.test.ts

// First send: enters digest buffer
await notify.send({
  recipientId: "user_1",
  notificationId: "activity_update",
  payload: { action: "liked your post" },
  dedupeKey: "like:post_42:user_rey",
  dedupeWindowMs: 10 * 60_000,
})

// Second send (within dedup window): SKIPPED — never reaches digest
const result = await notify.send({
  recipientId: "user_1",
  notificationId: "activity_update",
  payload: { action: "liked your post" },
  dedupeKey: "like:post_42:user_rey",
  dedupeWindowMs: 10 * 60_000,
})
expect(result.skipped.some(s => s.reason === "duplicate")).toBe(true)

Dedup + rate limits

Dedup also runs before rate limiting. A deduplicated send does not count against your rate limit budget:

# Notification with rate limit: max 5 per hour
# If 3 of 8 sends are deduped, only 5 unique sends count against the limit
# Deduped sends: skipped at stage 2, never reach stage 3

Dedup + quiet hours

A deduplicated send is dropped entirely — it won't be deferred by quiet hours. Only sends that pass dedup can be queued for later delivery. If the dedup window expires during quiet hours, a new send with the same key will pass dedup and get deferred normally.

Scenario	Outcome	Why
Send during quiet hours, first occurrence	Deferred — delivers when window ends	Dedup passes (first time), quiet hours defers
Same key sent again during quiet hours	Skipped — never delivers	Dedup fires at stage 2, quiet hours never reached
Dedup window expires, same key sent again	Treated as new — enters pipeline fresh	Key expired, engine has no memory of the first send
Digested send + dedup within digest window	First enters digest, second dropped	Dedup fires before digest buffering

Use explain() to see where a send stops. The explain response includes wouldDeduplicate, wouldRateLimit, and wouldDigest — check them in order to understand exactly which stage intercepted a send.

Explain & dry run Digests & rate limits Timeline debugging

PreviousQuiet hours NextFallback channels