SYSTEMS |
FEB 12, 2025 |
By Spareek |
20 MIN READ
Mastering Event-Driven Systems
Synchronous REST endpoints represent the weakest link in any distributed system. If Service A calls Service B, and Service B is down, Service A fails. Event-driven architectures invert this relationship to establish total isolation.
In a synchronous architecture, services are tightly coupled through HTTP gRPC contracts. If there is a spike in user traffic, downstream services can suffer from resource exhaustion. In an event-driven system, services communicate asynchronously using messages published to a broker, preventing load spikes from destroying the network.
The Event Stream
By producing events to a central broker (like Apache Kafka or AWS EventBridge), services no longer care who is listening. The order creation service just screams "ORDER_CREATED" into the void, and returns a 200 OK to the user.
Downstream—inventory, billing, and shipping modules pull from the stream at their own pace. If the billing service goes down for 3 hours, nobody panics. It simply reconnects and processes the backlog of events.
Message Queue (RabbitMQ) vs. Log Event Stream (Kafka)
| Feature |
Message Queue (e.g., RabbitMQ) |
Event Stream (e.g., Kafka) |
| Data Retention |
Deleted once consumer acknowledges |
Persistent (Retained for set duration) |
| Consumer Model |
Queue pushes messages to consumer |
Consumer pulls messages from log offset |
| Replay Messages |
No |
Yes (Can replay stream from any point) |
| Scale Pattern |
Compete consumer model per queue |
Partition-based parallel consumers |
idempotent-consumer.ts
interface EventPayload
eventId: string;
type: string;
data: Record;
async function handleEvent(event: EventPayload) {
// Check if we've already processed this event
const isDuplicate = await db.processedEvents.exists(event.eventId);
if (isDuplicate)
console.log('Skipping duplicate event:', event.eventId);
return;
// Execute processing logic
await db.transaction(async (trx) => {
await processOrder(event.data, trx);
await trx.processedEvents.insert( eventId: event.eventId );
});
}
Ensuring Idempotent Processing
The major challenge of distributed event-driven systems is network duplication. Because of the "at-least-once" delivery guarantee of brokers like Kafka, consumers can receive the same message twice.
To build robust systems, consumers must be idempotent—processing the same event multiple times must yield the same result. As shown in the TSX code snippet, we enforce this by storing processed event IDs inside our primary transactional database.
This paradigm shift from "commands" to "events" enables absolute horizontal scalability, but introduces complex challenges like idempotent processing and eventual consistency.