Show Me How: Build Streaming Data Pipelines for Real-Time Data Warehousing | Register Today
With so many technologies in the modern development ecosystem, a common complaint is having to go through the mental gymnastics of adopting new products and keeping up with ever-expanding feature sets. At the same time, modern tech stacks such as CI/CD pipelines require large amounts of automation, which is often particularly painful to achieve with low-level infrastructure like databases and data platforms.
Confluent solves these challenges by empowering developers, operators, and architects with intuitive command line interfaces (CLIs) for managing cloud and on-prem Confluent deployments. The
confluent CLIs help you harness the full power of data in motion with a rich set of management and automation capabilities.
Here’s an example that creates a fully managed Apache Kafka® cluster in Confluent Cloud on AWS:
$ ccloud kafka cluster create <name> --cloud aws --region us-west-2
This blog post covers some of the exciting new features that have been engineered into our cloud and on-prem CLI tools and how you can benefit from these new developments.
Confluent’s CLIs have always worked on Windows, Mac, and Linux. We have recently added support for new operating systems and distribution mechanisms.
Users with Apple’s newest generation of Mac computers, which use the arm64 CPU architecture, can seamlessly run Confluent’s CLIs on their machines. Native
darwin/arm64 builds are available as of
confluent v1.27.0. Customers can follow the installation instructions for cloud or on-prem, respectively, to install the latest CLI version on their computer.
Both of Confluent’s CLIs now natively support Alpine Linux, which is commonly used in Docker-based workflows. To that end, we have also published official images on Docker Hub for both
confluent. You can now build powerful Docker-based automation infrastructure without ever explicitly installing one of our CLIs. For example, in order to list your current Kafka clusters with JSON-formatted output, you can run:
$ docker run -e CCLOUD_EMAIL='your email address' \ -e CCLOUD_PASSWORD='your password' \ -it \ confluentinc/ccloud-cli:latest \ /ccloud kafka cluster list -o json
Both of Confluent’s CLIs now support non-interactive login as well, which is particularly useful in automation scenarios.
ccloud, you can specify
CCLOUD_PASSWORD, which correspond to your Confluent Cloud credentials
confluent, you can specify
CONFLUENT_PASSWORD, which correspond to the credentials used to log in to your Confluent Platform environment (i.e., metadata service credentials)
confluent, you can specify the
CONFLUENT_MDS_URLenvironment variable, which provides the host and port of your metadata service (MDS) instance, as well as the
CONFLUENT_CA_CERT_PATHenvironment variable, in case you have configured MDS to use self-signed or custom certificates that are not part of the system truststore
Most CLI commands can now be executed in a stateless fashion, which saves you from a lot of unnecessary typing.
For instance, rather than running this long sequence of commands:
$ ccloud environment use <environment-id> $ ccloud kafka cluster use <kafka-cluster-id> $ ccloud kafka topics list
You can now simply run:
$ ccloud kafka topic list --environment abc123 --cluster lkc-zyx987
One advantage of stateless command execution is that multiple users can execute CLI commands on the same computer. For example, a server used for continuous integration jobs can support multiple parallel jobs that use the
ccloud CLI without worrying about the values stored in or corrupting the
~/.ccloud/config.json configuration file.
It is now possible to sign up for Confluent Cloud right from the CLI. If you haven’t tried Confluent Cloud yet, you can get started by installing
ccloud with a simple
curl command and then running
ccloud signup. After your email is verified, you can create Kafka clusters, configure ksqlDB apps, and more. Payment information can also be entered and managed using the subcommands of
ccloud admin payment.
The video below shows how easy it is to get started with Confluent Cloud using the latest version of the CLI:
ccloud shell drops you into an interactive environment with rich contextual information and helpful autocompletion for working with your Confluent Cloud infrastructure. The shell environment now supports server-side autocompletion for values of positional and flag arguments. For instance, once you’re in the
ccloud shell, beginning to type “kafka cluster update” will result in the CLI shell suggesting IDs of Kafka clusters in your current environment, as shown in the video below.
With the addition of Docker images, support for logins based on environment variables, and stateless command execution, Confluent’s CLIs are well suited for a wide array of automation tasks and other production use cases. At the same time, M1 Mac support and improvements to the
ccloud shell are improving the CLIs’ interactive experiences for end users. Finally, newer commands like
ccloud signup and user management make the CLIs more complete interfaces for Confluent’s rich functionality.
If you are looking to get started with Confluent’s CLIs or with Confluent in general, you can find quick installation guides for our cloud and on-prem CLIs in the documentation.
The CLI team is highly responsive to user feedback and is interested in hearing about both bug reports and feature requests—any way to help us serve users better. If you would like to provide any suggestions or feedback, feel free to file a reach out on the Confluent Forum, send us an email at email@example.com, or file a ticket through Confluent Support. We look forward to hearing from you!
Use the Confluent CLI and API to create Stream Designer pipelines from SQL source code.
This post details how to minimize internal messaging within Confluent platform clusters. Service mesh and containerized applications have popularized the idea of control and data planes. This post applies it to the Confluent platform clusters and highlights its use in Confluent Cloud.