Kora Engine, Data Quality Rules und mehr in unserem Q2 2023 Launch | FĂŒr die Demo registrieren

Utilizing Point-in-Time Queries in Event-Based Systems

« Current 2022

The arrival of events on a stream is the foundation of causal reasoning in an event-based system. Each event creates a ""tick"" of the system's causal ""clock"", allowing us to reason about the state of the system as of any particular event.

But what about our Read Models (e.g. tables, materialized views, database sinks)? Many data stores support efficient versioning and point-in-time querying, including Git, S3, Datomic, XTDB, and many others. How can our system design and implementation benefit by pairing the notion of time implicit in event-based systems with a data store capable of versioning and point-in-time queries?

In this talk, we'll discuss how the oNote team implemented a point-in-time queryable Event Model repository using Kafka, Git, and CRDTs. We'll also discuss some other technologies that facilitate this pattern. Finally, we'll explore the benefits of pairing immutable event streams with databases that support versioning and point-in-time querying, including:

  • Absolutely consistent reads for all system participants, even as writes continue to arrive
  • Uninterrupted read-availability, even in the face of partial system failure
  • Much better user experience in rapidly-changing or collaborative systems -- no more last-write-wins conflicts based on stale data!
  • Efficient reconstructions of system state as of any particular event, for audit, analysis, and business error recovery

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how