Idempotency is not optional

The first time a duplicate message charges a customer twice, you stop thinking of idempotency as an advanced topic. It's table stakes.

Every reliable message broker — Azure Service Bus included — gives you at-least-once delivery. That phrase is doing a lot of work. It means a message will arrive at least once, and sometimes more than once: a handler crashes after doing its work but before acknowledging, a lock expires under load, a network blip triggers a redelivery. The broker is behaving correctly. Your handler has to behave correctly too.

The contract you're actually signing up for is this: the same message may be processed more than once, and the result must be the same as processing it once. That property is idempotency.

Where it goes wrong

public async Task HandleAsync(OrderPlaced message)
{
    await _payments.ChargeAsync(message.OrderId, message.Amount);
    await _email.SendConfirmationAsync(message.OrderId);
}

Looks fine. Now imagine the process dies after the charge but before the acknowledgement. Service Bus redelivers. The customer is charged twice and gets two emails. Nothing here is "broken" — it's just not safe to retry.

Strategy 1 — make the operation naturally idempotent

The best fix is to make the work idempotent at the data layer, so doing it twice is the same as doing it once. An upsert keyed on the business identity is the cleanest example:

MERGE Orders AS target
USING (SELECT @OrderId AS Id) AS source
ON target.Id = source.Id
WHEN NOT MATCHED THEN
    INSERT (Id, Status) VALUES (@OrderId, 'Placed');
-- a second run matches and does nothing

No bookkeeping required. The operation simply can't double-apply.

Strategy 2 — deduplicate with an idempotency key

When the work isn't naturally idempotent (charging money, sending an email, calling someone else's API), you need to remember what you've already done. Give every message a stable identifier and record it the moment you process it:

public async Task HandleAsync(OrderPlaced message)
{
    // Insert-if-absent. The unique constraint is the safety net.
    var firstTime = await _dedup.TryRecordAsync(message.MessageId);
    if (!firstTime)
        return; // already handled — acknowledge and move on

    await _payments.ChargeAsync(message.OrderId, message.Amount);
    await _email.SendConfirmationAsync(message.OrderId);
}

The unique constraint on MessageId is what makes this correct under concurrency — two redeliveries racing each other can't both win the insert.

Strategy 3 — the inbox pattern

The strongest version records the message and performs the work in one transaction, so they commit or fail together:

using var tx = await _db.Database.BeginTransactionAsync();

if (await _db.ProcessedMessages.AnyAsync(m => m.Id == message.MessageId))
    return;

_db.ProcessedMessages.Add(new ProcessedMessage(message.MessageId));
ApplyBusinessChange(message);

await _db.SaveChangesAsync();
await tx.CommitAsync();

If the process dies mid-flight, the transaction rolls back and redelivery starts cleanly. There's no window where the work happened but the record didn't.

Don't forget the poison messages

Some messages will never succeed — bad data, a downstream that's gone for good. Without a limit they redeliver forever, burning throughput. Configure max delivery count so Service Bus moves them to the dead-letter queue, and monitor that queue. A silent dead-letter queue is a stack of problems no one is looking at.

The mental model

Assume every message arrives twice. Design so the second time is a no-op.

Once you internalise that, distributed systems get a lot calmer. You stop chasing the impossible goal of exactly-once delivery and start building handlers that simply don't care how many times they run.