The Outbox Pattern: How to Publish Events Without Losing Them

TL;DR: When a service needs to save something to its database and tell other services about it at the same time, doing those two things separately can silently lose data. The outbox pattern fixes this by writing the “I need to send this event” note into the same database transaction as the actual change, then using a separate, reliable process to deliver it. It costs you a bit of extra plumbing, but it guarantees you never save something without eventually announcing it.

Imagine you’re building the order service for an online store. When a customer checks out, two things need to happen: the order gets saved to the database, and an “OrderPlaced” event needs to go out so the shipping service, the email service, and the inventory service can all react to it. That sounds simple. It is not.

The tricky part is that “save to the database” and “send a message to a queue” are two completely separate systems. There’s no single switch that turns both on at once. And whenever you have two separate systems that both need to succeed together, you’ve got a problem waiting to happen.

The problem with writing to two places at once

Let’s walk through what your order service code probably looks like without much thought:

save(order)
publish("OrderPlaced", order)

This works fine in a demo. It breaks in production, because there’s a gap between those two lines. Something can go wrong in that gap. What if the database save succeeds, but right after that, the network to your message queue has a hiccup and the publish call fails? Now you have an order sitting in your database with no one ever told about it. The customer paid, but shipping never got the memo. Nobody notices until they call asking where their package is.

You could try to flip the order and publish first, then save. That doesn’t help either — now you might tell three other services an order exists that never actually got saved, and when they go looking for it, it isn’t there.

This is called the “dual write” problem: whenever your code needs to update two different systems (here, a database and a message queue) as if it were one atomic action, but those two systems don’t share a way to guarantee that atomicity, you’re exposed to exactly this kind of silent data loss.

Once the database commit and the event publish are two separate steps, either one can fail without the other knowing.

How the outbox pattern fixes it

The core idea behind the outbox pattern is refreshingly simple: stop treating “save the order” and “send the event” as two separate actions. Instead, treat “save the order” and “record that an event needs to be sent” as one single action, because both of those can live in the same database, in the same transaction.

Here’s how it works. Alongside your normal orders table, you add a second table — the outbox table. It might just have columns like id, event_type, payload, and status. When the order service handles a checkout, it does this in one database transaction:

1. Insert the new row into the orders table.
2. Insert a row into the outbox table saying “an OrderPlaced event needs to go out, here’s the data.”

Because both inserts are in the same transaction, they either both succeed or both fail together. There’s no gap anymore. If the order got saved, the note to send the event got saved too. You’ve turned two separate operations into one atomic one, just by keeping them in the same database.

Of course, the row sitting in the outbox table isn’t the event itself — nobody outside your service is watching that table. You still need something to actually pick it up and deliver it to the message queue. That’s the job of a separate piece called the relay.

The order and the outbox row are saved together. A separate relay process handles getting the event out the door.

How the relay actually gets events out

The relay’s job is boring on purpose: it looks at the outbox table, finds rows that haven’t been sent yet, publishes them to the message queue, and then marks them as sent. There are two common ways to build this.

The simplest is polling: a background job runs every second or so, asks the database for outbox rows with status “pending,” publishes each one, and updates its status to “published.” This is easy to build and reason about, but it does add a small delay (however long your polling interval is) and some repeated load on your database.

The more advanced option is to use change data capture, often shortened to CDC. Instead of a job repeatedly asking “anything new?”, a tool like Debezium watches the database’s own transaction log — the internal record every database already keeps of every write — and reacts the instant a new outbox row appears. It’s faster and puts less strain on the database, but it’s also more infrastructure to run and understand, so plenty of teams start with polling and only move to CDC once they actually feel the pain of polling delay.

Either way, once a row has been published successfully, it’s marked done. Teams typically run a cleanup job afterward to archive or delete old published rows, since an outbox table that grows forever will eventually slow down your polling queries.

Each outbox row moves through a simple lifecycle: waiting to be sent, sent, then eventually cleaned up.

Things to watch out for

The outbox pattern solves the dual-write problem, but it trades it for a couple of smaller, more manageable problems you should know about going in.

First, you get “at-least-once” delivery, not “exactly-once.” If the relay publishes an event but crashes right before it marks the row as done, it’ll come back, see the row still says “pending,” and publish it again. That means the shipping service, the email service, and everyone else consuming these events need to handle receiving the same event twice without double-shipping the order or double-charging the customer. This is usually solved by giving each event a unique ID and having consumers check “have I already processed this ID?” before acting on it.

Second, ordering can get messy if you’re not careful. If your relay processes rows in parallel for speed, two events from the same order could arrive out of order at the consumer. Most teams sidestep this by keeping a consistent ordering key (like the order ID) and making sure events for the same key are always processed by the same worker, in the order they were written.

Third, this pattern adds real operational weight: a new table, a relay process to run and monitor, and a cleanup job. For a small side project with one service and no other systems that need to know about its changes, this is almost certainly overkill. It earns its keep once you have multiple services that genuinely need to react reliably to what happens in another service’s database.

The outbox pattern isn’t a clever trick — it’s really just an application of a boring but powerful idea: if two things need to happen together, put them in the same transaction. Everything else in the pattern (the relay, the polling or CDC, the cleanup job) exists purely to get that guaranteed event out of your database and into the world without losing it along the way. Next time you catch yourself writing a save-then-publish sequence, that’s the moment to ask whether an outbox table would save you a very confusing bug report six months from now.