-
-
Notifications
You must be signed in to change notification settings - Fork 163
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
3 changed files
with
81 additions
and
0 deletions.
There are no files selected for viewing
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,33 @@ | ||
--- | ||
Aliases: [] | ||
Tags: [seedling] | ||
publish: true | ||
--- | ||
|
||
The event sourcing pattern captures all changes to an application as a sequence of event objects. This pattern creates a ledger of changes that can be used to replay events and view application state at any point in time. | ||
|
||
![Event Sourcing|500](event-sourcing.png) | ||
*Image source: [Event Sourcing and Event-Driven Architecture - In Under 5 Minutes!](https://ercin.medium.com/event-sourcing-and-event-driven-architecture-in-under-5-minutes-927f4e88351a)* | ||
|
||
## Event Sourcing Pattern Advantages | ||
|
||
- Provides an audit trail which is useful for auditing and compliance. | ||
- You can rebuild the state of any entity to any point in time by replaying events. | ||
- Aligns well with event-driven architectures and microservices where services communicate via events. | ||
|
||
## Event Sourcing Pattern Disadvantages | ||
|
||
- Managing the event store can be complex as the number of events grow and schema changes happen. | ||
- Leads to an eventually consistent system (which is fine if that's acceptable) but should be noted it can introduce latency. | ||
- Rebuilding an entity's current state by replaying all past events can resource intensive. Snapshotting can help mitigate this. | ||
|
||
[^1]: https://martinfowler.com/eaaDev/EventSourcing.html | ||
|
||
%% wiki footer: Please don't edit anything below this line %% | ||
|
||
## This note in GitHub | ||
|
||
<span class="git-footer">[Edit In GitHub](https://github.dev/data-engineering-community/data-engineering-wiki/blob/main/Concepts/Event%20Sourcing%20Pattern.md "git-hub-edit-note") | [Copy this note](https://raw.githubusercontent.com/data-engineering-community/data-engineering-wiki/main/Concepts/Event%20Sourcing%20Pattern.md "git-hub-copy-note")</span> | ||
|
||
<span class="git-footer">Was this page helpful? | ||
[👍](https://tally.so/r/mOaxjk?rating=Yes&url=https://dataengineering.wiki/Concepts/Event%20Sourcing%20Pattern) or [👎](https://tally.so/r/mOaxjk?rating=No&url=https://dataengineering.wiki/Concepts/Event%20Sourcing%20Pattern)</span> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,48 @@ | ||
--- | ||
Aliases: [Pub/sub messaging] | ||
Tags: [seedling] | ||
publish: true | ||
--- | ||
|
||
The Publisher-Subscriber pattern is an asynchronous messaging pattern that decouples sending events to one or more subscribers/consumers of those events. [^1] | ||
|
||
```mermaid | ||
%%{init: { "flowchart": { "useMaxWidth": true } } }%% | ||
graph LR | ||
A((Publisher))-->|Event| B[[Message Broker]] | ||
B -->|Event| C((Subscriber)) | ||
B -->|Event| D((Subscriber)) | ||
``` | ||
|
||
## Publisher-Subscriber Pattern Advantages | ||
|
||
- Decouples software and improves scalability due to async sending and ability to scale different parts of the system independently. [^2] | ||
- Push-based delivery reduces latency and eliminates the need for polling. | ||
- Simplifies integration code between systems. | ||
|
||
## Publisher-Subscriber Pattern Disadvantages | ||
|
||
- Message ordering isn't guaranteed and messages may be duplicated. Must design system to be [[Idempotence|idempotent]]. | ||
- Subscribers may need the ability to autoscale to keep up with the rate of messages. | ||
|
||
## Popular Publisher-Subscriber Tools | ||
|
||
- [[Apache Kafka]] | ||
- [[Redis]] | ||
- [AWS EventBridge](https://aws.amazon.com/eventbridge/) | ||
- [Azure Event Grid](https://azure.microsoft.com/en-us/products/event-grid/) | ||
- [GCP Eventarc](https://cloud.google.com/eventarc/) | ||
|
||
[^1]: https://learn.microsoft.com/en-us/azure/architecture/patterns/publisher-subscriber | ||
[^2]: https://aws.amazon.com/what-is/pub-sub-messaging/ | ||
|
||
%% wiki footer: Please don't edit anything below this line %% | ||
|
||
## This note in GitHub | ||
|
||
<span class="git-footer">[Edit In GitHub](https://github.dev/data-engineering-community/data-engineering-wiki/blob/main/Concepts/Publisher-Subscriber%20Pattern.md "git-hub-edit-note") | [Copy this note](https://raw.githubusercontent.com/data-engineering-community/data-engineering-wiki/main/Concepts/Publisher-Subscriber%20Pattern.md "git-hub-copy-note")</span> | ||
|
||
<span class="git-footer">Was this page helpful? | ||
[👍](https://tally.so/r/mOaxjk?rating=Yes&url=https://dataengineering.wiki/Concepts/Publisher-Subscriber%20Pattern) or [👎](https://tally.so/r/mOaxjk?rating=No&url=https://dataengineering.wiki/Concepts/Publisher-Subscriber%20Pattern)</span> |