Designing Events: Event Driven Architecture EDA
Event-Driven Architecture (EDA) is an architectural pattern that focuses on the production, detection, consumption, and reaction to events. An event, in this context, is a immutable change or occurrence that has relevance within the system. In this post we will focus specifically on Event Design.
What we’ll cover?
- What do we mean by designing events and the benefits around it
- What does ‘good’ look like, design best practice
Before we go further, for more information on Event Driven Architectures (EDA) read my post on demystifying EDA:
What do we mean by event design?
By Event Design we mean ensuring an event is well-structured, scalable, and capable of efficiently handling and responding to events within a distributed system.
Event Schemas
One of the most important aspects that makes a good event is a well defined Event Schema. With Event Schemas you describe the event to potential consumers, building a contract that your system must conform to. You may want to include specific metadata to describe the intricacies of the event. You can also share and reuse the same event schema throughout your system, allowing for consistency and reducing the cognitive load of your teams consuming multiple events.
Event Identification
As well as providing a schema with a common structure, it is also important to identify your events well. Event Identification is crucial in Event-Driven Architecture (EDA) to ensure that the system captures and/or responds to relevant changes or occurrences properly.
Event identification allows for the right granularity of an event. You can link your events directly to business processes and a domain model, allowing your IT team to work closer to the business stakeholders. The events you publish can directly correlate to a change or “fact” that has happened within a business processes. You can now begin to understand why Domain-Driven-Design (DDD) works well with Event Driven Architecture (EDA).
Event Versioning
“It is not enough for code to work.”
― Robert C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship
As Robert C. Martin (Uncle Bob) says the software you build is never complete. The same goes for business processes and as a result, the events that are raised. Versioning of events helps to accommodate those changes over time. This is crucial for maintaining compatibility between different components that produce or consume events.
Event Channels
When designing events, think about how your events need to be consumed. Do they need to be processed in ‘real-time’ or perhaps in ‘order or sequence’ or maybe some events can be processed asynchronously ‘as and when’ they occur.
Whichever works correctly for your business process determines which technology you might want to use. Within AWS, we have AWS Kinesis for Real-time processing, SNS and SQS First-in-First-out (FIFO) for ordering or Amazon EventBridge asynchronous unordered event processing.
Event Handling
When designing events ask yourself what the detail will be. Are you going to use the Event Notification pattern, Event Carried State Transfer or do you need to support large event payloads.
As a consumer, consider how you will deal with a failure ingesting the event. Perhaps it is crucial that you retrieve all events — Like in Event Sourcing. If you do, I suggest using AWS SQS and dead-letter queues to capture and alert on processing failures.
We’ve mentioned a few patterns when looking at Event Handling, We’ll talk about them in later posts but for now let’s briefly look at each one:
Event Notification Pattern
Event Notification pattern is where the publisher only sends a brief payload, often only an identifier and known as sparse events, to notify consumers about something that has happened. The consumer in this pattern doesn’t care for a response and the consumer most often needs to call back to the source system for further information.
Event Carried State Transfer
With Event Carried State Transfer — All the data about a particular event is sent in the event payload to prevent consumers from having to contact the publishing/source system to gather more information.
Handling Large Payloads
If you’re using Event Carried State Transfer in particular, you may have to handling large payloads isn’t an EDA pattern as such, it’s a pattern that helps deal with the event size limits on Amazon EventBridge.
Here the publisher sends a location for the consumer to fetch the larger packet of information, typically Amazon S3 when using AWS. Jason Conway-Williams has a very good post on this, for more detail see https://jcdubs.medium.com/aws-large-payload-pattern-e3e4c57d402d
Event Sourcing
Event Sourcing pattern is where all events are recorded, showing a history in state. The idea of event sourcing is you can rebuild or replay events when necessary, at any time. Event sourcing is often used as the source of truth and systems will build their state around it.
What does ‘Good’ look like?
There are many guides and opinions on what makes a good event. If you’re familiar with AWS EventBridge we’ll take a look at the ‘source’, ‘detail’ and ‘detail-type’ sections of the event.
For those who are new to AWS EventBridge let’s first look at the PutEvents SDK command:
As part of the parameters we need to one required top level property, Entries — for sending multiple events at a time to the bus
Within the Entries array, here is where we can specify the source, detail and detail-type properties for each event.
Source
The source field should indicate where the event has come from. In practice this shouldn’t be needed for the consumer, but it is important for auditing. Perhaps you have one or more services that raise the same event.
{
...
// indicates it has orignated from order domains web service
source: "order.webservice"
...
}Detail-Type
DetailType should indicate what the event is — what immutable fact has happened. This works well with Domain Driven Design (DDD) and should be derived as part of Event Storming and your data model agreement with commercial business experts.
This field is one of the most important, consumers often use this to create their rule base and subscribe to your event. For example, consumers can listen for “OrderCreated” events:
{
...
// indicates it has orignated from order domains web service
source: "order.webservice" ,
// indicates the immutable fact (event) that occurred
detailType: "OrderCreated"
...
}Detail
Now we come to the real payload of the event. The detail section is where you want to let your consumer know what the state is or what has happened — depending on the patterns mentioned above.
What is common throughout the AWS community and an approach mentioned by Sheen Brisals at Lego.com is to split your detail into two parts, metadata and data.
The metadata portion allows you to introduce additional observability into your event-driven architectures. You can specify correlationIds and causationIds to help visualise the event as it flows through your system. idempotency keys are important for duplication handling or ordering if required.
You can extend metadata to be anything relevant to your system or to support your event. Click here to see another post by AO.com on how they use metadata and data.
{
// indicates it has orignated from order domains web service
source: "order.webservice" ,
// indicates the immutable fact (event) that occurred
detailType: "OrderCreated",
detail: {
metadata: {
"correlationId": "...",
"causationId": "...",
"idempotencyKey": "...",
"timestamp": "...",
...
},
data: {
orderId: "...",
orderLines: [...],
...
}
}
}How can we enforce these standards?
Event Identification — Involve commercial business experts or product owners who can give you context and insight into processes so you can correctly identify events. Run event storming sessions or similar, even if you’re not following strict DDD practices.
Schema Validation — Enforce the above with a schema validator, like AJV for Node, BEFORE you publish and AFTER you consume, in the code itself and you can ensure that teams are following event standardisation.
Event Documentation — Events should be well documented, as you can see there are lots of additional metadata and meanings behind the data attributes themselves that should be captured for teams to understand. Perhaps Event Catalog, an open source project by David Boyne can be used to document your events.
📣 Getting in touch!
Happy building!… 🚀 Thank you for reading
Hello I’m Robert Bulmer, a Technical Architect and Serverless Engineer with over five years of hands on experience in harnessing the capabilities of Amazon Web Services (AWS).
I am pleased to be an AWS Community Builder, in the Serverless domain, a community where I can connect and share with fellow advocates of cloud enthusiasts.
Read my Blogs at ArchiTech Insights
🔗 https://architechinsights.com/
Reach me on linkedIn here: https://www.linkedin.com/in/robertbulmer/
Disclaimer: The posts I write are designed to help the teams that I work with and echo my past experience in software development. My aim is to reduce the cognitive load on the teams so they can deliver maximum value to the business. While I hope this helps your journey, I accept no responsibility for your own use of the information.
