88 Percent of Survey Respondents Say Kafka Will Be Mission-Critical to Their Data and Application Infrastructures by 2017
SAN FRANCISCO, KAFKA SUMMIT—April 26, 2016—Today at the inaugural Kafka Summit, Confluent, the company founded by the creators of Apache™ Kafka®, shared results from a recent survey of the Kafka community that illuminates Kafka’s growing impact on the way organizations collect and process their data. Eighty eight percent of the survey respondents indicated that Kafka would be a mission-critical part of their data and application infrastructure by 2017. Nearly 1 in 3 respondents (29%) work for organizations with more than $1 billion in annual sales, illustrating how quickly this open source technology has gained traction across large enterprises.
“We see more and more organizations embracing real-time data and stream processing, and Kafka is at the heart of that shift,” said Jay Kreps, one of Kafka’s co-creators and the CEO and co-founder of Confluent. ”Kafka’s popularity comes from its scalability, its reliability, and the fact that it has proven itself in production in some of the world’s most massive real-time environments.”
The global survey was conducted in April 2016 by Researchscape International and sponsored by Confluent. It represents the views of more than 100 Kafka users from 20 countries and spanning 16 industries, including technology, media and entertainment, banking/financial services, telecommunications, energy/utilities, consumer products/manufacturing and automotive.
The Rise of Stream Processing
The survey underscores the pace at which organizations are adopting Kafka for stream processing to enable all incoming data to flow in a continuous stream. According to the results, Apache Kafka is most commonly used for stream processing (72% of respondents).
In addition, nearly seven of ten (68%) Kafka users surveyed say they plan to incorporate more stream processing over the next 6 to 12 months. Of those:
- 74% say they will add it to new applications in development now
- 69% will add it to existing applications
- 60% will build net-new applications using stream processing and Kafka
This growth is also influencing hiring, with 65 percent of responding organizations planning to hire employees with Kafka skills in the next 12 months as the number of applications using Kafka continues its upward trajectory.
How Kafka Is Being Used
Kafka powers a wide variety of applications today based on survey respondents, including application monitoring (60%), data warehousing (51%) and asynchronous applications (47%) to system monitoring (39%), recommendation/decisioning engines (35%), security/fraud detection (26%), Internet of Things applications (20%) and dynamic pricing applications (12%), to name a few.
Among organizations using Kafka, the median number of applications powered by Kafka is four, but nearly a quarter (23%) use Kafka to power 11 or more applications. Meanwhile, more than a third of respondents (37%) said their organization is already using Kafka to send more than 1 billion messages per day, and a two percent of respondents noted that Kafka now powers upwards of 1 trillion messages within their organization daily.
Benefits of Kafka
Among the many benefits reported, several were echoed by a majority of Kafka users:
- 67% said “Kafka helps our applications work together in a loosely coupled manner”
- 59% said “We use Kafka as underlying data infrastructure for stream processing”
- 58% noted the “improved scalability of applications” Kafka brings
- 51% said “High volumes of data are now available in real-time—we’ve been able to move beyond batch processing”
The survey also revealed that 100 percent of respondents expressed satisfaction with Kafka, and 70 percent said they were ‘very satisfied’ or ‘completely satisfied’ with it. Users with enterprise support for Kafka were likely to be more satisfied with their experience—92 percent said they were very or completely satisfied.
“We’re incredibly pleased with the feedback we’ve received from the community—this level of insight helps us make Kafka even better,” said Neha Narkhede, co-founder and CTO of Confluent and one of Kafka’s co-creators. “As Kafka Summit begins, we’re excited for members of the community to have the chance to share their stories and learn from one another as we work together to ensure that Kafka reaches its full potential as core infrastructure technology.”
About Kafka Summit
Confluent is hosting the inaugural Kafka Summit in San Francisco on April 26. The conference is bringing together Kafka’s core committers, leading production users and more than 500 members of the Kafka community to learn about how companies are using Kafka today for large-scale, real-time data integration and stream processing in a variety of applications, and hear about where project development is headed in the future.
About Apache Kafka
Apache Kafka is an open source technology that acts as a real-time, fault tolerant, highly scalable messaging system. It is widely adopted for use cases ranging from collecting user activity data, logs, application metrics, stock ticker data and device instrumentation. Its key strength is its ability to make high volume data available as a real-time stream for consumption in systems with very different requirements—from batch systems like Hadoop, to real-time systems that require low-latency access, to stream processing engines that transform the data streams as they arrive. This infrastructure lets you build around a single central nervous system transmitting messages to all the different systems and applications within a company.
Confluent, founded by the creators of Apache™ Kafka®, enables organizations to harness business value from stream data. The Confluent Platform manages the barrage of stream data and makes it available throughout an organization. It provides various industries, from retail, logistics and manufacturing, to financial services and online social networking, a scalable, unified, real-time data pipeline that enables applications ranging from large volume data integration to big data analysis with Hadoop to real-time stream processing. Backed by Benchmark, Data Collective, Index Ventures and LinkedIn, Confluent is based in Palo Alto. To learn more, please visitwww.confluent.io. Download Apache Kafka and Confluent Platform at www.confluent.io/download.
Jill Reed or Pete Johnson
Highwire PR for Confluent
(415) 963-4174, ext. 5