Rise of the Kafka Heroes! Join the Data Streaming Revolution | Read the Comic


All at Sea with Streams - Using Kafka to Detect Patterns in the Behaviour of Ships

« Kafka Summit Americas 2021

The great thing about streams of real-time events is that they can be used to spot behaviours as they happen and respond to them as needed. Instead of waiting until tomorrow to find out what happened yesterday, we can act on things straight away.

This talk will show a real-life example of one particular pattern that it's useful to detect—ships engaged in potentially suspicious behaviour at sea. Transhipping is often used for legitimate purposes to optimise efficiencies but can also be used for nefarious purposes such as illegal fishing.

By capturing streams of maritime AIS data in real-time into Kafka and processing it with ksqlDB, it's possible to detect the kind of characteristics that could indicate behaviour of interest, such as ships moving slowly at close proximity for a length of time.

I'll demonstrate how the data was ingested from a raw TCP feed, unified with reference data from CSV files, and then processed to spot patterns with the resulting real-time stream of matches written to a new Kafka topic for validation and analysis.

Related Links

How Confluent Completes Apache Kafka eBook

Leverage a cloud-native service 10x better than Apache Kafka

Confluent Developer Center

Spend less on Kafka with Confluent, come see how