Conferences

Confluent is proud to participate in the following conferences, trade shows, and meetups.

Devoxx Poland

Speaker: Tim Berglund, Sr Director, Developer ExperienceSession: Distributed Systems in One LessonRoom: Room 4 (Grand Parade)June 21, 12:10 to 13:00

Normally simple tasks like running a program or storing and retrieving data become much more complicated when we start to do them on collections of computers, rather than single machines. Distributed systems has become a key architectural concern, and affects everything a program would normally do—giving us enormous power, but at the cost of increased complexity as well.

Using a series of examples all set in a coffee shop, we’ll explore topics like distributed storage, computation, timing, messaging, and consensus. You'll leave with a good grasp of each of these problems, and a solid understanding of the ecosystem of open-source tools in the space.

Session Info
Speaker: Tim Berglund, Sr Director, Developer ExperienceSession: Four Distributed Systems Architectural PatternsRoom: Room 1June 21, 14:00 to 14:50

Developers and architects are increasingly called upon to solve big problems, and we are able to draw on a world-class set of open source tools with which to solve them. Problems of scale are no longer consigned to the web’s largest companies, but are increasingly a part of ordinary enterprise development. At the risk of only a little hyperbole, we are all distributed systems engineers now.

In this talk, we’ll look at four distributed systems architectural patterns based on real-world systems that you can apply to solve the problems you will face in the next few years. We’ll look at the strengths and weaknesses of each architecture and develop a set of criteria for knowing when to apply each one. You will leave knowing how to work with the leading data storage, messaging, and computation tools of the day to solve the daunting problems of scale in your near future.

Session Info
Speaker: Tim Berglund, Sr Director, Developer ExperienceSession: Distributed Systems in One LessonRoom: Room 4 (Grand Parade)June 23, 09:00 to 09:50

Normally simple tasks like running a program or storing and retrieving data become much more complicated when we start to do them on collections of computers, rather than single machines. Distributed systems has become a key architectural concern, and affects everything a program would normally do—giving us enormous power, but at the cost of increased complexity as well.

Using a series of examples all set in a coffee shop, we’ll explore topics like distributed storage, computation, timing, messaging, and consensus. You'll leave with a good grasp of each of these problems, and a solid understanding of the ecosystem of open-source tools in the space.

Session Info
Event Details

QCon New York

Speaker: Gwen Shapira, System Architect, ConfluentSession: Contracts and Compatibility in Streaming MicroservicesJune 27, 1:40pm - 2:30pm

In a world of microservices that communicate via unbounded streams of events, schemas are the contracts between the services. Having an agreed contract allows the teams developing those services to move fast, by reducing the risk involved in making changes. Yet delivering events with schema change in mind isn’t the common practice yet.

In this presentation, we’ll discuss patterns of schema design, schema storage and schema evolution that help development teams build better contracts through better collaboration - and deliver resilient applications faster. We’ll look at how schemas were used in the past, how their meaning has changed over the years and why they gained particular importance with the rise of the stream processing.

Session Info
Event Details

Meetup: Apache Kafka London

Speaker: Ben Stopford, Engineer at ConfluentSession: Stateful Stream Processing6:30 PM - 8:30 PM

For some Kafka is simply a conduit for low latency analytics. But stream processing is an increasingly popular tool for building transactional systems that run business logic for banks, telcos, consumer companies and more.

This talk walks through how Stateful Stream Processing can be used as a backbone to share state between services, breaking the chains that tie typical request-driven architectures. We’ll look at the benefits of a using a message bus that can retain state. Then layer atop stream processing tools to balance the dichotomy between consistency and the independence services need to iterate and get things done.

RSVP

Munich Apache Kafka Meetup

Speaker: Michael Noll, Product Manager, ConfluentSession: Rethinking Stream Processing with Apache Kafka: Applications vs. Clusters, Streams vs. Databases 5:30pm - 6:30pm

Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of information in real-time? The answer is stream processing, and the technology that has since become the core platform for streaming data is Apache Kafka. Among the thousands of companies that use Kafka to transform and reshape their industries are the likes of Netflix, Uber, PayPal, and AirBnB, but also established players such as Goldman Sachs, Cisco, and Oracle.

Unfortunately, today’s common architectures for real-time data processing at scale suffer from complexity: there are many technologies that need to be stitched and operated together, and each individual technology is often complex by itself. This has led to a strong discrepancy between how we, as engineers, would like to work vs. how we actually end up working in practice.

In this session we talk about how Apache Kafka helps you to radically simplify your data architectures. We cover how you can now build normal applications to serve your real-time processing needs — rather than building clusters or similar special-purpose infrastructure — and still benefit from properties such as high scalability, distributed computing, and fault-tolerance, which are typically associated exclusively with cluster technologies. We discuss common use cases to realize that stream processing in practice often requires database-like functionality, and how Kafka allows you to bridge the worlds of streams and databases when implementing your own core business applications (inventory management for large retailers, patient monitoring in healthcare, fleet tracking in logistics, etc), for example in the form of event-driven, containerized microservices.

Speaker: Kai Waehner, Technology Evangelist, ConfluentSession: How to Apply Machine Learning Models to Real Time Processing with Apache Kafka Streams6:30pm - 7:30pm

Big Data and Machine Learning are key for innovation in many industries today. The first part of this session explains how to build analytic models with R, Python or Scala leveraging open source machine learning / deep learning frameworks like Apache Spark, TensorFlow or H2O.ai. The second part discusses the deployment of these built analytic models to your own applications or microservices leveraging the Apache Kafka cluster and Kafka Streams. The session focuses on live demos and teaches lessons learned for executing analytic models in a highly scalable and performant way.

RSVP

DataStax Data 'n' Drinks at TopGolf Dallas

Join DataStax, Mesosphere and Confluent for free cocktails, food, and networking while learning about relevant real-world use cases and best practices for leveraging the always-on data platform, DataStax Enterprise (DSE).

You’ll also hear a DataStax technical evangelist discuss how to build scalable, always-on applications on a data later that lives seamlessly in multiple data centers and multiple public clouds.

This event will give you a strong foundation in this next-generation architecture and you’ll leave knowing how to bring this powerful technology to your next application.

Event Details

Dublin Apache Kafka Meetup

Speaker: Michael Noll, Product Manager, ConfluentSession: Rethinking Stream Processing with Apache Kafka: Applications vs. Clusters, Streams vs. Databases7:00pm - 7:30pm

Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of information in real-time? The answer is stream processing, and the technology that has since become the core platform for streaming data is Apache Kafka. Among the thousands of companies that use Kafka to transform and reshape their industries are the likes of Netflix, Uber, PayPal, and AirBnB, but also established players such as Goldman Sachs, Cisco, and Oracle.

Unfortunately, today’s common architectures for real-time data processing at scale suffer from complexity: there are many technologies that need to be stitched and operated together, and each individual technology is often complex by itself. This has led to a strong discrepancy between how we, as engineers, would like to work vs. how we actually end up working in practice.

In this session we talk about how Apache Kafka helps you to radically simplify your data architectures. We cover how you can now build normal applications to serve your real-time processing needs — rather than building clusters or similar special-purpose infrastructure — and still benefit from properties such as high scalability, distributed computing, and fault-tolerance, which are typically associated exclusively with cluster technologies. We discuss common use cases to realize that stream processing in practice often requires database-like functionality, and how Kafka allows you to bridge the worlds of streams and databases when implementing your own core business applications (inventory management for large retailers, patient monitoring in healthcare, fleet tracking in logistics, etc), for example in the form of event-driven, containerized microservices.

Speaker: Robin Moffatt, Partner Technology Evangelist, ConfluentSession: Real-time Data Integration at Scale with Kafka Connect7:30pm - 8:00pm

Apache Kafka is a streaming data platform. It enables integration of data across the enterprise, and ships with its own stream processing capabilities. But how do we get data in and out of Kafka in an easy, scalable, and standardised manner? Enter Kafka Connect. Part of Apache Kafka since 0.9, Kafka Connect defines an API that enables the integration of data from multiple sources, including MQTT, common NoSQL stores, and CDC from relational databases such as Oracle. By "turning the database inside out" we can enable an event-driven architecture in our business that reacts to changes made by applications writing to a database, without having to modify those applications themselves. As well as ingest, Kafka Connect has connectors with support for numerous targets, including HDFS, S3, and Elasticsearch.

This presentation will briefly recap the purpose of Kafka, and then dive into Kafka Connect, with practical examples of data pipelines that can be built with it and are in production at companies around the world already. We'll also look at the Single Message Transform (SMT) capabilities introduced with Kafka[masked] and how they can make Kafka Connect even more flexible and powerful.

RSVP

Reactive in Practice: Infrastructure Tooling for Microservices

Join us for the next event in a series of panel discussions as Cake Solutions & partners explore the latest in application design, development and deployment techniques, with a particular focus on Reactive programming principles.

At this event, the panel will talk about the approaches to developing, maintaining and evangelizing tooling for CI and CD pipelines of [containerised] microservices. The panelists have a long history of building CI and CD pipelines and infrastructure-as-code targeting AWS.

This event will be moderated by Mike O'Hara of The Realization Group with Panelists from Cake Solutions and partners.

We look forward to welcoming you for drinks, and what will essentially be an interesting, educational and thought-provoking evening.

Register

StampedeCon Big Data Conference

Speaker: Cliff GilmoreSession: Building Streaming Applications with Apache Kafka

Learn how the Apache Kafka’s Streams API allows you to develop next-generation applications and microservices services built upon the proven reliability, scalability, and low latency of Apache Kafka. In this session, you will learn about the architecture of the Streams API along with an overview use cases where it can be best applied.

Event Details

Kafka Summit San Francisco

Call for Papers opened. Submit proposal

Sponsorship opportunities available. Request more information

Join us

Strata Data Conference

Speaker: Gwen Shapira, System Architect, ConfluentSession: Architecting a next generation data platformRoom: 1E 12/13September 26, 2017, 1:30pm–5:00pm

Rapid advancements are causing a dramatic evolution in both the storage and processing capabilities in the open-source big data software ecosystem. These advancements include projects like:

  • Apache Kudu, a modern columnar data store that complements HDFS and Apache HBase by offering efficient analytical capabilities and fast inserts and updates with Hadoop.
  • Apache Kafka, which provides a high-throughput and highly reliable distributed message transport. Apache Impala (incubating), a highly concurrent, massively parallel processing query engine for Hadoop.
  • Apache Spark, which is rapidly replacing frameworks such as MapReduce for processing data on Hadoop due to its efficient design and optimized use of memory. Spark components such as * Spark Streaming and Spark SQL provide powerful near real-time processing.

Along with the Apache Hadoop platform, these storage and processing systems provide a powerful platform to implement powerful data processing applications on batch and streaming data. While these advancements are exciting, they also add a new array of tools that architects and developers need to understand when architecting solutions with Hadoop.

Using an example based on Customer 360 and the Internet of Things, we’ll explain how to architect a modern, real-time big data platform leveraging components like Kafka, Impala, Kudu, Spark Streaming, Spark SQL, and Hadoop to enable new forms of data processing and analytics. Along the way, we’ll discuss considerations and best practices for utilizing these components to implement solutions, cover common challenges and how to address them, and provide practical advice for building your own modern, real-time big data architectures.

Topics include:

  • Accelerating data processing tasks such as ETL and data analytics by building near real-time data pipelines using tools like Kafka, Spark Streaming, and Kudu.
  • Building a reliable, efficient data pipeline using Kafka and tools in the Kafka ecosystem such as Kafka Connect and Kafka Streams along with Spark Streaming.
  • Providing users with fast analytics on data with Impala and Kudu.
  • Illustrating how these components complement the batch processing capabilities of Hadoop.
  • Leveraging these capabilities along with other tools such as Spark MLlib and Spark SQL to provide sophisticated machine-learning and analytical capabilities for users.

Session Details
Speaker: Dustin Cote, Customer Operations Engineer, ConfluentSession: Mistakes were made, but not by us: Lessons from a year of supporting Apache KafkaRoom: 1E 07/08 September 27, 2017 2:05pm-2:45pm

The number of deployments of Apache Kafka at enterprise scale has greatly increased in the years since Kafka’s original development in 2010. Along with this rapid growth has come a wide variety of use cases and deployment strategies that transcend what Kafka’s creators imagined when they originally developed the technology. As the scope and reach of streaming data platforms based on Apache Kafka has grown, the need to understand monitoring and troubleshooting strategies has as well. Topics include: - Effective use of JMX for Kafka - Tools for preventing small problems from becoming big ones - Efficient architectures proven in the wild - Finding and storing the right information when it all goes wrong

Session Detail
Speaker: Jun Rao, Co-Founder, ConfluentSession: Apache Kafka Core Internals: A Deep DiveRoom: 1E 07/08September 27, 2017 2:55pm-3:35pm

In the last few years, Apache Kafka is a streaming platform and has been used extensively in enterprises for real-time data collecting, delivering, and processing. This talk will provide a deep dive on some of the key internals that help make Kafka popular and provide strong reliability guarantees. Companies like LinkedIn are now sending more than 1 trillion messages per day to Kafka. Learn about the underlying design in Kafka that leads to such high throughput. Many companies (e.g., financial institutions) are now storing mission critical data in Kafka. Learn how Kafka supports high reliability through its built-in replication mechanism. One common use case of Kafka is for propagating updatable database records. Learn how a unique feature called compaction in Apache Kafka is designed to solve this kind of problem more naturally.

Session Detail
Speaker: Neha Narkhede, CTO and Co-Founder, ConfluentSession: The Three Realities of Modern Programming: Cloud, Microservices, and the Explosion of DataRoom: 1A 23/24September 28, 2017 11:20am-12:00pm

Learn how the three realities of modern programming – the explosion of data and data systems, building business processes as microservices instead of monolithic applications and the rise of the public cloud – affect how developers and companies operate today and why companies across all industries are turning to streaming data and Apache Kafka for mission-critical applications.

Session Detail
Speaker: Gwen Shapira, Product Manager, ConfluentSession: One Cluster Does Not Fit All: Architecture Patterns for Multicluster Apache Kafka DeploymentsRoom: 1E 07/08September 28, 2017 2:05pm-2:45pm

In the last year, multicluster and cross-data center deployments of Apache Kafka have become the norm rather than an exception. The reasons are many and include:

  • Different groups in the same company using Kafka in different ways
  • Collecting information from many geographical regions and branches to a centralized analytics cluster
  • Planning for cases where an entire cluster or data center is not available
  • Using Kafka to assist in cloud migration

Robin Moffatt offers an overview of several use cases, including real-time analytics and payment processing, that may require multicluster solutions and discusses real-world examples with their specific requirements. Robin outlines the pros and cons of several common architecture patterns, including:

  • Multitenant Kafka clusters
  • Active-active multiclusters
  • Failover clusters
  • Stretching a single cluster between multiple data centers
  • Using Kafka to bridge between clouds or between on-premises and the cloud

Along the way, Robin explores the features of Apache Kafka and demonstrates how to use this understanding of Kafka to choose the right architecture for use cases from the financial, retail, and media industries.

Session Detail
Speaker: Tim Berglund, Sr Director of DevX, ConfluentSession: Heraclitus, Enterprise Architecture, and Streaming DataRoom: 1E 07/08September 28, 2017 2:55pm-3:35pm

Hailing from the Persian city of Ephesus in around 500 BC, the Greek philosopher Heraclitus is famous for his trenchant analysis of big data stream processing systems, saying “You never step into the same river twice.” Central to his philosophy was the idea that all things change constantly. His close readers also know him as the Weeping Philosopher—perhaps because dealing with constantly changing data at low latency is actually pretty hard. It doesn’t need to be that way.

Almost as famous as Heraclitus is Apache Kafka, the de facto standard open-source distributed stream processing system. Many of us know Kafka’s architectural and API particulars as well as we know the philosophy of Heraclitus, but that doesn’t mean we know how the most successful deployments of Kafka work. In this talk, I’ll present several real-world systems build on Kafka, not just as a giant message queue, but as a platform for distributed stream computation.

The talk will include a brief summary of Kafka architecture and (probably Java) APIs, then a detailed description of several architectures drawn from live customer deployments. The role of stream processing will be featured in each, with attention given to what computation gets done in the stream, how Kafka fills the role of persistence rather than merely a message queue, and what other persistence and computational technologies are present in the system.

Session Detail
Event Details

Strange Loop 2017

Speaker: Jason Gustafson, Apache Kafka Committer & Confluent EngineerSession: EOS In Kafka: Listen Up, I Will Only Say This Once!

Apache Kafka's rise in popularity as a streaming platform has demanded a revisit of its traditional at-least-once message delivery semantics. In this talk, we present the recent additions to Kafka to achieve exactly-once semantics (EoS) including support for idempotence and transactions in the Kafka clients. The main focus will be the specific semantics that Kafka distributed transactions enable and the underlying mechanics which allow them to scale efficiently. We will discuss Kafka's spin on standard two-phase commit protocols, how transaction state is maintained and replicated, and how different failure scenarios are handled. We will also share our view of future improvements that will make exactly-once stream processing with Kafka even better!

Session Info
Event Details

Software Architecture Conference

Speaker: Ben Stopford, Engineer at ConfluentSession: Rethinking Microservices with Stateful StreamsRoom: King's Suite - SandringhamOctober 16, 15:50–16:40

When building service-based systems, we don’t generally think too much about data. If we need data from another service, we ask for it. This pattern works well for whole swathes of use cases, particularly ones where datasets are small and requirements are simple. But real business services have to join and operate on datasets from many different sources. This can be slow and cumbersome in practice.

These problems stem from an underlying dichotomy. Data systems are built to make data as accessible as possible — a mindset that focuses on getting the job done. Services, instead, focus on encapsulation — a mindset that allows independence and autonomy as we evolve and grow. But these two forces inevitably compete in most serious service-based architectures.

Ben Stopford explains why understanding and accepting this dichotomy is an important part of designing service-based systems at any significant scale. Ben looks at how companies use log-backed architectures to build an immutable narrative that balances data that sits inside their services with data that is shared, an approach that allows the likes of Uber, Netflix, and LinkedIn to scale to millions of events per second.

Ben concludes with a set implementation patterns, starting lightweight and gradually getting more functional, paving the way for an evolutionary approach to building log-backed microservices.

Session Info
Event Details

Big Data LDN

Speaker: Neha Narkhede, CTO and Co-Founder, ConfluentSession: Keynote: The Rise of the Streaming PlatformNovember 15, 2017
Details

Ready to Talk to Us?

Have someone from Confluent contact you.

Contact Us