Confluent Cloud Q1 Launch: Build a Secure Shared Services Data Streaming Platform | Learn more

Apache Kafka as a Service with Confluent Cloud Now Available on GCP Marketplace

Get started with Confluent, for free

Watch demo: Kafka streaming in 10 minutes

Escrito por

Following Google鈥檚 announcement to provide leading open source services with a cloud-native experience by partnering with companies like Confluent, we are delighted to share that Confluent Cloud is now available on Google Cloud Platform (GCP) Marketplace.

The integrated experience and billing model makes it easier for developers to get started with Confluent Cloud, our fully managed event streaming platform built around Apache Kafka, by leveraging their existing billing service on GCP. With Confluent Cloud available natively in the Google Cloud Console, this is the first step towards making Kafka a first-class-citizen service on GCP so that developers can be even more productive.

Developer joy with zero hassle

Considering Apache Kafka鈥檚 tremendous adoption throughout the years, either developers are fully convinced that only Kafka can grant them the scalability and performance their workloads demand, or perhaps the ecosystem of APIs around Kafka has grown so much that pretty much all major developer stacks support Kafka, which in turn fuels further adoption. But it may also mean that developers are building their applications more oriented to streams rather than state, because it allows them to create applications that can react more quickly to opportunities or threats that the business behind the application may face, as well as make instant decisions.

Regardless, we鈥檝e always heard that managing Apache Kafka is a job for the pros鈥攂ut that doesn鈥檛 have to be the case.

Confluent Cloud is a fully managed Apache Kafka service that removes the operational complexities of having to manage Kafka on your own (you can read this blog post for more on the importance of a fully managed service for Apache Kafka). Built as a cloud-native service, Confluent Cloud offers developers a serverless experience with elastic scaling and pricing that charges only for what you stream.

While Confluent Cloud has always been available on GCP, it requires developers to sign up and provide a credit card as a form of payment to get started. This meant having to manage an additional bill in addition to their GCP bill. This could be a point of friction since developers familiar with GCP are not used to the concept of swapping a credit card for every service or resource they spin up for usage.

We are happy to say that having two separate bills is no longer necessary, and you can now use Confluent Cloud with your existing project鈥檚 billing and credits. Any consumption of Confluent Cloud is now a line in your monthly GCP bill.

Event streaming applications made easy and fun

An event streaming platform such as Confluent Cloud is made up of components and other layers that make developing event streaming applications easier and fun. Just like a Kafka cluster, these other pieces in the ecosystem are available as fully managed services as well, such as Confluent Schema Registry and connectors like the newly available BigQuery connector. There is also support for ksqlDB as a managed service.

When developers use these tools as part of their Confluent Cloud subscription, they don鈥檛 have to write code or worry about technical decisions. All this to say, if you are developing event applications with GCP, the amount of effort required by your architecture can significantly decrease because this is all built into Confluent Cloud. This is particularly useful if your use case is all about event ingestion into BigQuery, and events need to be curated (e.g., filtered, partitioned, adjusted, aggregated, or enriched) before getting into a BigQuery table.

Getting started with Confluent Cloud on GCP Marketplace

In honor of Linus Torvalds, the creator of Linux kernel who once said, 鈥淭alk is cheap, show me the code,鈥 let鈥檚 see how GCP developers can start streaming with Confluent Cloud.

Step 1: Spin up Confluent Cloud

First, log into your account and select the project where you want to spin up Confluent Cloud. Then click on the 鈥淢arketplace鈥 option, as shown in Figure 1.


Figure 1. GCP console showing the home page of the selected project

Step 2: Select Confluent Cloud in Marketplace

In Marketplace, simply use this page or search for 鈥淎pache Kafka鈥 or 鈥淐onfluent Cloud.鈥 Select the option shown in Figure 2.

Apache Kafka庐 on Confluent Cloud鈩

Figure 2. Selecting Confluent Cloud in Marketplace

Step 3: Purchase Confluent Cloud

After selecting Confluent Cloud, you will be served with a page that allows you to purchase the service. Simply click on the button 鈥淧urchase鈥 as shown in Figure 3.


Figure 3. Purchasing Confluent Cloud

To pay for Confluent Cloud, GCP has enabled Confluent Consumption Units (CCUs). Each CCU equals $0.0001. Pricing in GCP Marketplace is no different than our direct purchase pricing. For example, if you stream 1 GB of data into a cluster in region us-east1, you would pay $0.11 or the equivalent of 1,100 CCUs.

Pick 鈥淐onfluent Consumption Unit鈥 as the pricing option and make sure to select the box that authorizes Google to make this purchase on your behalf. Now, let鈥檚 enable Confluent Cloud.

Step 4: Enable Confluent Cloud on GCP

After some seconds, GCP will show that Confluent Cloud has been successfully purchased and that you can now enable its API for usage. Note that there are absolutely no charges until you enable the Confluent Cloud API in GCP. Click on the 鈥淓nable鈥 button to enable Confluent Cloud on GCP, as shown in Figure 4.

Figure 4. Enabling the Confluent Cloud API on GCP

Figure 4. Enabling the Confluent Cloud API on GCP

Enabling Confluent Cloud on GCP can take up to one minute. You should see the overview page display that GCP now has Confluent Cloud enabled, as shown in Figure 5.

Figure 5. Confluent Cloud properly enabled on GCP

Figure 5. Confluent Cloud properly enabled on GCP

Step 5: Register the admin user

Click on the 鈥淢anage via Confluent鈥 link to start working with Confluent Cloud. By clicking that link, you will be brought outside of the GCP console to a signup page where you will register yourself as the main user of Confluent Cloud. During the registration process, please use the same email that you use to log into GCP. Fill in all the fields required and then click on the 鈥淪ubmit鈥 button as shown in Figure 6.

Figure 6. Registering the administrator for Confluent Cloud

Figure 6. Registering the administrator for Confluent Cloud

Step 6: Verify your email

You should receive an email asking you to verify your email address. This step is required for security purposes, so Confluent can verify if you are the person that initiated the registration process. Click on the 鈥淰erify your email address鈥 link as shown in Figure 7.

Figure 7. Verifying your email address

Figure 7. Verifying your email address

After verification, you will be automatically logged into Confluent Cloud with your new account (which happens to be the administrator of the account) and you can start playing with the service immediately.

For new users, the first $50.00 of the service usage will be free each month for the first three months. That means if you have a steady flow of events that on average computes ~50 KB per second (an estimate for region us-east1 with the topic retention policy configured to seven days), you won鈥檛 pay anything because the total bill would be ~$37.60, and the $50 allowance covers that. In other words, you can try the service for free.

Figure 8. Now the fun begins. Create as many clusters you need.

Figure 8. Now the fun begins. Create as many clusters you need.

When it comes to creating clusters, you have some decisions to make, but none that are as scary as they would be if you were managing Apache Kafka by yourself. Instead, you just have to name your cluster, choose which region from GCP to spin up the cluster from (it should be as close as possible to your apps to minimize latency), and choose the availability of your cluster. Figure 10 shows an example of cluster creation.

Figure 9. Creating a new cluster in Confluent Cloud

Figure 9. Creating a new cluster in Confluent Cloud

Creating a cluster in Confluent Cloud enabled by GCP is no different than if users were to sign up directly with Confluent, except for the fact that any cluster you create won鈥檛 ask for credit card details. The Confluent Cloud documentation provides more details on how to use the service.


Confluent Cloud makes it easy and fun to develop event streaming applications. Now, we鈥檙e taking another step towards putting Apache Kafka at the heart of every organization by making Confluent Cloud a first-class citizen on GCP, so you can use the service with your existing GCP billing account.

If you haven鈥檛 already, sign up for Confluent Cloud today!

  • Ricardo is a Developer Advocate at Confluent, the company founded by the creators of Apache Kafka. He has +21 years of experience working with Software Engineering, where he specialized in different types of Distributed Systems architectures such as Integration, SOA, NoSQL, Messaging, In-Memory Caching, and Cloud Computing. Prior to Confluent, he worked for other vendors such as Oracle, Red Hat and IONA Technologies, as well as several consulting firms. While at Oracle, he used to be part of the Alpha Team, otherwise known as 鈥淭he A-Team鈥 鈥 a special unit from the Engineering organization that handles projects using the following philosophy: when all else fails, we don鈥檛. Currently, he lives in Apex, North Carolina, with his wife, son and two dogs.

Get started with Confluent, for free

Watch demo: Kafka streaming in 10 minutes

驴Te ha gustado esta publicaci贸n? Comp谩rtela ahora

Announcing the Source Available Confluent CLI

Confluent is pleased to announce that the Confluent CLI鈥攖he leading command-line tool for managing enterprise Kafka deployments and modern data flow鈥攊s now source available under the Confluent Community License.