Deep Dive into SASL PLAIN and SCRAM in Kafka: Login Modules and Config Hot-Reload

Modern Kafka clusters are long‑lived, business‑critical systems. That means two things for authentication:

• Credentials must be secure.

• Credentials must be changeable without downtime.

If you’re using SASL PLAIN or SASL SCRAM in Kafka, both goals are achievable, but only if you understand what’s really going on under the hood: which login module you use, where credentials are stored, and whether that mechanism supports hot‑reloading configuration.

This post walks through:

• What SASL PLAIN and SCRAM actually are (beyond “auth mechanisms”)

• How different Kafka login modules handle credentials

• Why some combinations support hot‑reload and others do not

• A concrete, end‑to‑end example using three listeners and three login modules

• Practical recommendations for production deployments

SASL PLAIN and SCRAM in Kafka: The “What”

SASL PLAIN is the simplest SASL mechanism: it sends a username and password to the server, typically protected by TLS in production. In Kafka, it’s commonly used for straightforward client authentication when ease of configuration is the top priority.

On the broker, PLAIN uses a login module to validate credentials. The most common is: PlainLoginModule – usernames and passwords are defined in JAAS config (or backed by LDAP).

On the client, you configure something like:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config= \
  org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="alice" \
  password="alice-secret";

SASL SCRAM (Salted Challenge Response Authentication Mechanism) is a more secure SASL method. Instead of sending raw passwords, it uses a challenge‑response protocol based on salted password hashes and cryptographic proofs (e.g., SCRAM-SHA-256, SCRAM-SHA-512).

Compared to PLAIN, SCRAM provides stronger protection against credential theft and replay attacks, and in Kafka it comes with an additional operational benefit we’ll explore: native hot‑reload of credentials via cluster metadata.

On the client side, a typical SCRAM config looks like:

security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config= \
  org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="alice" \
  password="alice-secret";

Why “Hot-Reload” Matters for Authentication

Hot‑reload means a running broker can detect and apply authentication config changes without a restart. In the context of SASL auth, that implies:

• You can add new users.

• You can rotate passwords or keys.

• You can revoke or update credentials.

…...all while brokers stay online and continue serving traffic.

This is not a “nice to have” in production; it directly affects:

• Zero‑downtime credential rotation – rotate passwords or keys regularly, without rolling restarts.

• Incident response – quickly revoke compromised credentials.

• Operational simplicity – fewer restarts, fewer chances to disrupt clients.

Whether hot‑reload is possible depends entirely on:

1. Where credentials are stored, and

2. How the chosen login module reads and refreshes them.

That’s where the key differences between PlainLoginModule, ScramLoginModule, and a file‑backed module come in.

The Three Login Modules: Plain, SCRAM, and File-Based

To understand hot‑reload behavior, you need to look at credential storage and refresh semantics.

PlainLoginModule (Static JAAS, No Hot-Reload)

• Mechanism: SASL/PLAIN

• Credential source: Static JAAS configuration (or LDAP in some setups)

• Behavior:

  • Credentials are typically loaded once at broker startup.

  • No built‑in file watching or reload.

• Result: No hot‑reload - changing credentials in JAAS requires a broker restart unless an external system like LDAP provides dynamic validation.

• Why: The module assumes configuration immutability and keeps credentials in memory after initialization.

ScramLoginModule (Cluster Metadata, Native Hot-Reload)

• Mechanism: SASL/SCRAM (SCRAM-SHA-256, SCRAM-SHA-512)

• Credential source: Cluster metadata

  • Historically ZooKeeper, now the KRaft metadata log in modern Kafka/Confluent Platform.

• Behavior:

  • Credentials are stored as metadata records.

  • Brokers fetch and update SCRAM credentials dynamically from the quorum.

  • When you use Admin APIs to add/alter users, brokers learn the changes automatically.

• Result: Hot‑reload is naturally supported - no broker restart required.

• Why: Credentials aren’t tied to static files; they live in the distributed metadata managed by the cluster itself.

FileBasedLoginModule (Watched File, Hot-Reload)

• Mechanism: SASL/PLAIN (but with a different backend)

• Credential source: An external credential file (e.g., JSON) on disk.

• Behavior:

  • The module periodically checks or reloads the credential file.

  • Updates to users/passwords can be applied while brokers are running.

• Result: Hot‑reload is supported.

• Why: The module is explicitly designed with file‑watching / reload semantics, unlike PlainLoginModule.

A Concrete Example: Three Listeners, Three Behaviors

To make this real, let’s look at a single Confluent Platform (CP) Kafka broker configured with three SASL listeners:

• CUSTOMER_PLAIN – SASL/PLAIN via PlainLoginModule (static JAAS)

• CUSTOMER_SCRAM – SASL/SCRAM via ScramLoginModule (metadata‑backed)

• CUSTOMER_FILE – SASL/PLAIN via FileBasedLoginModule (file‑backed, hot‑reload)

Listener Setup in server.properties

First, define advertised listeners and security protocol mappings:

advertised.listeners=INTERNAL://...:9092,BROKER://...:9091,CUSTOM://...:9093,\
CUSTOMER_PLAIN://broker-host:9094,\
CUSTOMER_SCRAM://broker-host:9095,\
CUSTOMER_FILE://broker-host:9096

listener.security.protocol.map=CONTROLLER:PLAINTEXT,INTERNAL:PLAINTEXT,\
BROKER:PLAINTEXT,\
CUSTOM:PLAINTEXT,\
CUSTOMER_PLAIN:SASL_PLAINTEXT,\
CUSTOMER_SCRAM:SASL_PLAINTEXT,\
CUSTOMER_FILE:SASL_PLAINTEXT

listeners=INTERNAL://:9092,BROKER://:9091,CUSTOM://:9093,\
CUSTOMER_PLAIN://:9094,\
CUSTOMER_SCRAM://:9095,\
CUSTOMER_FILE://:9096

Then configure each listener’s SASL settings.

PLAIN (Static JAAS, PlainLoginModule)

listener.name.customer_plain.sasl.enabled.mechanisms=PLAIN
listener.name.customer_plain.plain.sasl.jaas.config= \
  org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="broker" \
  password="broker-secret" \
  user_alice="alice-secret" \
  user_bob="bob-secret";

SCRAM (Metadata-backed, ScramLoginModule)

listener.name.customer_scram.sasl.enabled.mechanisms=SCRAM-SHA-512
listener.name.customer_scram.scram-sha-512.sasl.jaas.config= \
  org.apache.kafka.common.security.scram.ScramLoginModule required;

FILE-Based PLAIN (FileBasedLoginModule)

listener.name.customer_file.sasl.enabled.mechanisms=PLAIN
listener.name.customer_file.plain.sasl.jaas.config= \
  io.confluent.kafka.server.plugins.auth.FileBasedLoginModule required \
  config_path="/etc/kafka/apikeys.json" \
  refresh_ms="10000";  

Important gotcha: If you configure both CUSTOMER_PLAIN (PlainLoginModule) and CUSTOMER_FILE (FileBasedLoginModule) using overlapping settings, the file‑based module can effectively “win” and your static‑JAAS PLAIN listener may not behave as expected. It’s best to test them one at a time when validating behavior.

File-Based Credentials: apikeys.json

For the file‑based PLAIN listener, credentials reside in a JSON file, e.g.:

{
"keys": {
  "customer": {
    "sasl_mechanism": "PLAIN",
    "hashed_secret": "customer-secret",
    "hash_function": "none",
    "logical_cluster_id": "my-cluster",
    "user_id": "customer",
    "service_account": false
    },
  "alice": {
    "sasl_mechanism": "PLAIN",
    "hashed_secret": "alice-secret",
    "hash_function": "none",
    "logical_cluster_id": "my-cluster",
    "user_id": "alice",
    "service_account": false
    }
  }
}

Client Configs

PLAIN Client

# plain_client.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config= \
  org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="customer" \
  password="customer-secret";

SCRAM Client

# scram_client.properties
security.protocol=SASL_PLAINTEXT
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config= \
  org.apache.kafka.common.security.scram.ScramLoginModule required \
  username="customer" \
  password="customer-secret";

Scenarios: How Each Auth Path Handles Changes

With the above setup, you can run simple operations like listing topics to validate behavior:

# PLAIN / FILE listener
kafka-topics.sh --bootstrap-server broker-host:9096 \
--list --command-config plain_client.properties

# SCRAM listener
kafka-topics.sh --bootstrap-server broker-host:9095 \
--list --command-config scram_client.properties

Now compare how each mechanism behaves when credentials change.

SASL PLAIN (PlainLoginModule → No Hot-Reload)

Scenario:

1. alice/alice-secret is present in server.properties at broker startup.

2. Client connects via CUSTOMER_PLAIN as alice → SUCCESS.

3. You add a new user customer/customer-secret to the JAAS section without broker restart.

4. Client attempts to connect as customer → FAILS.

5. You restart the broker.

6. Client connects as customer → SUCCESS.

Takeaway: PlainLoginModule reads credentials once; changes only take effect after broker restart.

SASL SCRAM (ScramLoginModule → Hot-Reload)

With SCRAM, you manage users via Kafka’s Admin APIs, e.g.:

# Add or reset SCRAM user
kafka-configs.sh \
--bootstrap-server broker-host:9093 \
--alter \
--add-config
'SCRAM-SHA-512=[iterations=8192,password=customer-secret]' \
--entity-type users \
--entity-name customer

Scenario:

1. alice/alice-secret exists in cluster metadata.

2. Client connects via CUSTOMER_SCRAM as alice → SUCCESS.

3. You add customer/customer-secret using kafka-configs.sh (no broker restart).

4. Client connects as customer → SUCCESS.

You can even inspect SCRAM credentials in the KRaft metadata log with kafka-dump-log for full transparency.

Takeaway: Because SCRAM credentials live in cluster metadata, brokers automatically pick up changes. No restart, full hot‑reload.

SASL PLAIN (FileBasedLoginModule → Hot-Reload)

For the file‑based PLAIN listener:

Scenario:

1. alice/alice-secret is present in apikeys.json.

2. Client connects via CUSTOMER_FILE as alice → SUCCESS.

3. You add customer/customer-secret to apikeys.json and save the file; the broker keeps running.

4. After the refresh_ms interval (or next file check), client connects as customer → SUCCESS.

Takeaway: FileBasedLoginModule is designed to watch and reload the credential file, so you can add/remove users without touching the broker process.

Operational Recommendations

Based on the behaviors above, here are practical guidelines for production:

1. Prefer SCRAM for strong security + native hot‑reload

   1. Use ScramLoginModule with SCRAM-SHA-256 or SCRAM-SHA-512.

   2. Manage users via kafka-configs.sh or Admin APIs.

   3. Let credentials live in cluster metadata; avoid static JAAS for users.

2. If you must use PLAIN, avoid static JAAS for dynamic users

   1. Static PLAIN via PlainLoginModule is acceptable only when:

      1. Users rarely change, and

      2. You can tolerate restarts for changes.

   2. For dynamic users on PLAIN, prefer a file‑backed module with hot‑reload semantics.

3. Be explicit about which listener uses which login module

   1. Keep listener responsibilities clear (e.g., one listener per login module for testing).

   2. Document which paths support hot‑reload and which do not.

4. Design for zero‑downtime credential rotation

   1. Make rotation runbooks that:

      1. Update credentials through SCRAM Admin APIs or file‑backed stores.

      2. Do not rely on cluster restarts.

   2. Test scenarios in lower environments that mirror production listeners and modules.

5. Observe and debug via metadata and logs

   1. Use kafka-configs.sh and kafka-dump-log to verify SCRAM users in metadata.

   2. For file‑based PLAIN, monitor broker logs around file reload and auth failures.

Done right, you get stronger security and smoother operations: secure mechanisms, controlled storage, and hot‑reload as a first‑class property of your Kafka authentication layer.

Call to Action

Who is the target persona?

• Platform/SRE engineers operating Kafka or Confluent Platform in production

• Security engineers/architects responsible for auth on Kafka

• Kafka operators and administrators (including CFK / containerized deployments)

• Advanced Kafka developers who own cluster configuration

What is the target persona’s next step(s) in their journey?

• Audit your current Kafka SASL configuration:

  • Identify which listeners use PLAIN vs SCRAM.

  • Determine which login modules and credential backends you rely on today.

• Plan a migration path where:

  • User credentials move to SCRAM (cluster metadata) or a file‑based PLAIN backend that supports hot‑reload.

  • Static JAAS PLAIN is limited to cases where restarts are acceptable.

• Document and test a zero‑downtime credential rotation runbook in a lower environment before rolling it out to production.

What is the key takeaway for the reader?

The real difference between SASL PLAIN and SCRAM in Kafka isn’t just about cryptography, it’s about where credentials live and how they change. ScramLoginModule (cluster metadata) and file‑backed PLAIN modules give you safe, zero‑downtime hot‑reload, while static JAAS PLAIN (PlainLoginModule) does not. Choosing the right combination is essential for both security and operability.

Is this post tied to an event/campaign?

This post is best aligned with Kafka security hardening, platform reliability, and operational best practices content themes. It can complement campaigns or initiatives focused on:

• Kafka/Confluent Platform security best practices

• Migrating to KRaft‑based clusters

• Modernizing CI/CD and operational runbooks around Kafka auth and access control.