Microservices & Apache Kafka

Putting the Micro into Microservices: Thursday, October 27, 2022

10 AM PDT | 1 PM EDT | 10 AM BST | 10:30 AM IST | 1 PM SGT | 3 PM AEST

Application architecture is shifting from monolithic enterprise systems to flexible, scalable, event-driven approaches. Welcome to the microservices era.

What are microservices, and how do they work?

Microservices separate monolithic systems into a collection of independent, self-contained services that allow easier deployment, testing, and maintenance.

Pros and cons of microservices architecture:

Microservices have many benefits: they are faster to build, easier to maintain and avoid the bottlenecks that come with monolithic architectures. Most importantly, microservices offer a flexible, and scalable development model that keeps up with modern business requirements.

These benefits don’t come for free though. Because Microservices are distributed systems they face a number of challenges not faced by monolithic systems: more complex failure modes lead to an increased dependency on automation and DevOps practices. In addition, more resilient design patterns are typically required so that, when a failure inevitably occurs, it can be managed without a system outage. But the traditional approach to microservices–one based on HTTP and REST–limits the options you have available because all services are coupled together in a single, synchronous runtime. Adding event streams lets you decouple different bounded contexts, resulting in a more scalable solution with fewer failure modes. One where different teams can evolve and change with less risk of affecting one another. But doing this well requires a new set of tools and infrastructure which is what we’ll be looking at in this series of talks.

Why Kafka is used in Microservices:

When it comes to event-driven microservice architecture Apache Kafka® is by far the most popular tool for event-driven microservices, whether it’s self-managed as an open source tool or uses the richer feature-set available on Confluent. Kafka blends together concepts seen in traditional messaging systems, Big Data infrastructure, and traditional databases and Confluent expands on this with an online platform with better scalability, infinite storage, and event streaming features such as data lineage, schemas, and advanced security.

In many of the most significant digital-native businesses, you’ll find Kafka and Confluent separating the online world where users click buttons and expect things to happen, from the asynchronous world where the majority of the business is run. Making this distinction is the key to running large microservice architectures successfully.

This is a three-part series which introduces key concepts, use cases, and best practices for finding success with event-driven microservices. Each session is recorded so if you missed a session you’ll have a chance to watch on-demand.

Additional Resources:

• Designing Event-Driven Systems (Download for Free)