Utilizing Point-in-Time Queries in Event-Based Systems

The arrival of events on a stream is the foundation of causal reasoning in an event-based system. Each event creates a ""tick"" of the system's causal ""clock"", allowing us to reason about the state of the system as of any particular event.

But what about our Read Models (e.g. tables, materialized views, database sinks)? Many data stores support efficient versioning and point-in-time querying, including Git, S3, Datomic, XTDB, and many others. How can our system design and implementation benefit by pairing the notion of time implicit in event-based systems with a data store capable of versioning and point-in-time queries?

In this talk, we'll discuss how the oNote team implemented a point-in-time queryable Event Model repository using Kafka, Git, and CRDTs. We'll also discuss some other technologies that facilitate this pattern. Finally, we'll explore the benefits of pairing immutable event streams with databases that support versioning and point-in-time querying, including:

• Absolutely consistent reads for all system participants, even as writes continue to arrive
• Uninterrupted read-availability, even in the face of partial system failure
• Much better user experience in rapidly-changing or collaborative systems -- no more last-write-wins conflicts based on stale data!
• Efficient reconstructions of system state as of any particular event, for audit, analysis, and business error recovery