Stream Processing

November Update of KSQL Developer Preview Available

Update: KSQL is now available as a component of the Confluent Platform.

Today we are releasing the first update to KSQL since its launch as a Developer Preview at Kafka Summit San Francisco in August. Since then we have seen an amazing amount of interest and great feedback from the community. We have gained a lot of insight and see a bright future ahead for KSQL.  From this month onwards, we will be releasing KSQL on a monthly basis, and we will also kick-off a KSQL beta program.

What is KSQL?

KSQL is the streaming SQL engine for Apache Kafka. It provides an easy and completely interactive SQL interface for stream processing on Kafka—no need to write any code in a programming language such as Java or Python. KSQL supports a wide range of powerful stream processing operations including filtering, transformations, aggregations, joins, windowing, sessionization, and much more.  KSQL is open source (Apache 2.0 licensed), distributed, scalable, fault-tolerant, and real-time.

What do you get in this KSQL update?

This update has been focused on addressing many community-raised issues and requests for the Developer Preview.  It includes a total of 369 pull requests, 50 closed issues, and participation from more than a thousand members of the Confluent Slack community.  Many thanks to all of you for your great feedback and contributions thus far!

Which use cases are companies solving with KSQL?

In the past two months we have spoken with dozens of companies about how KSQL enables them to solve their real-time use cases.  Already we’ve seen such diverse use cases powered by KSQL such as monitoring of application behavior and business events, real-time ETL, data exploration, testing and validating machine learning, driving automated ad-tech auctioning, and more.

And we have learned that people can be very creative (and have a lot of fun) with KSQL and Kafka.  Perhaps the most entertaining use case has been shared by Rittman Mead, where KSQL is used to visualize real-time device data.  Racing and Kafka are definitely an exciting combination!

If you are interested in further reads, you might be interested in how KSQL fits synergistically with microservice-oriented stream processing strategies that have thus far been the exclusive province of Kafka’s Streams API, or how KSQL makes real-time analytics problems much easier.

How can you get involved in the KSQL community?

KSQL is a community-driven effort to make the world of Kafka and stream processing accessible to a broader audience.  KSQL is an open source project (Apache 2.0 licensed), and all development including design discussions, issue tracking, etc. is managed in the KSQL project repository on GitHub.

The easiest way to get involved is by sharing your feedback via the KSQL issue tracker.  If you want to take it to the next step you can also contribute code and pull requests to make KSQL even better.  And, at any time, if you have questions or need a helping hand, you can team up with fellow KSQL users and developers in the #ksql channel in our public Confluent Slack community.

Next steps for KSQL: monthly releases and beta program

While KSQL is still considered a developer preview, we will be releasing KSQL on a monthly basis from now on.  This will make it even easier for you to get up and running with the latest-and-greatest functionality of KSQL to solve your own business problems.

Also, we have begun working hand-in-hand with closed beta customers on implementing very cool use cases with KSQL.  And if you, too, are interested in joining the KSQL beta, please reach out to us!

Where to go from here

If you have enjoyed this article, you might want to continue with the following resources to learn more about KSQL:

Did you like this blog post? Share it now

Subscribe to the Confluent blog

More Articles Like This

Real-Time Data Replication with ksqlDB

Real-Time Data Replication with ksqlDB

Data can originate in a number of different sources—transactional databases, mobile applications, external integrations, one-time scripts, etc.—but eventually it has to be synchronized to a central data warehouse for analysis […]

How to Make Your Open Source Apache Kafka Connector Available on Confluent Hub

How to Make Your Open Source Apache Kafka Connector Available on Confluent Hub

Do you have data you need to get into or out of Apache Kafka®? Kafka connectors are perfect for this. There are many connectors out there, usually for well-known and […]

ksqlDB: The Missing Link Between Real-Time Data and Big Data Streaming

ksqlDB: The Missing Link Between Real-Time Data and Big Data Streaming

Is event streaming or batch processing more efficient in data processing? Is an IoT system the same as a data analytics system, and a fast data system the same as […]

This website uses cookies to enhance user experience and to analyze performance and traffic on our website. We also share information about your use of our site with our social media, advertising, and analytics partners.

More Information