Confluent Platform enables unparalleled Kafka performance and elasticity through automated partition rebalancing, decoupling of the compute and storage layers, and infinite data retention.
Auto Data Balancer (ADB) provides an automated solution for partition rebalancing to optimize the cluster’s throughput. ADB determines the best cluster-wide partition rebalancing plan without manual, time-consuming calculations or risk of human error.
Tiered Storage allows Kafka to recognize two tiers of storage: local disks and cost-efficient object stores (Amazon S3). Brokers can offload older topic data to object storage, enabling virtually infinite retention.
ADB provides fully-automated partition rebalancing to ensure your cluster’s throughput is optimized without the need for time-consuming calculations or risk of manual error.
With ADB, you can also efficiently rebalance your entire cluster rather than needing to explicitly reference each individual topic one-by-one as you would with the Apache Kafka tooling.
Limit the impact of a partition rebalance on your cluster’s throughput through ADB’s ability to optimize partition movement during the rebalance and throttle its allocated bandwidth.
Achieve dramatically longer periods of data retention for your cluster without a significant increase in operating costs. By leveraging Tiered Storage, you can back up data for replay in the future or use it to make Kafka the central nervous system of your organization.
Tiered Storage also allows you to achieve the regulatory compliance for data retention and durability requirements specific to your industry, without needing to build additional infrastructure into your architecture.
Scale storage resources without having to scale compute resources, and vice versa, by effectively decoupling compute and storage resources allocated to your cluster through Tiered Storage.
Minimize the time needed to rebalance partitions when adding new brokers. Because less data is stored on the broker when using Tiered Storage, smaller partitions need to be moved to newly added brokers using ADB, reducing rebalancing times from potentially hours down to a few seconds.