I am trying to enable SASL_PLAINTEXT authentication between Kafka broker and client, while not requiring it between Kafka and Zookeeper. Currently I am using confluent offering of Kafka, with CDH Zookeeper. Is there a way to pass in a flag to disable SASL for Kafka <-> Zookeeper?
You need to set the environment variable ZOOKEEPER_SASL_ENABLED=false. This is configured for Kafka broker
Related
We are trying to read messages from Kafka cluster(AWS MSK) and produce events using Knative eventing.
Everything in sink and works well if the MSK brokers have no authentication, But failed to communicate/connect MSK from KafkaSource if SASL authentication enabled.
We have followed steps for SASL authentication as mentioned in the below blog :
https://knative.dev/docs/eventing/sources/kafka-source/#enabling-sasl-for-kafkasources
Please suggest a way to connect to MKS brokers having SASL enabled from KafkaSource
I have created a kinesis source connector which is supposed to pull data from the kinesis stream and push the same data to kafka topic.
The use case here is connector will be deployed on Amazon MSK connect and where as the topic is created on some other kafka cluster which is hosted on Azure.
The roadblocker I have is the authentication related properties are not supported on MSK connect worker configurations, Is there any other way where we can integrate kafka security using MSK connect?
Below are the properties which are not supported MSK connect
admin.
admin.listeners.https.
client.
connect.
inter.worker.
internal.
listeners.https.
metrics.
metrics.context.
rest.
sasl.
security.
socket.
ssl.
topic.tracking.
worker.
bootstrap.servers
config.storage.topic
connections.max.idle.ms
connector.client.config.override.policy
group.id
listeners
metric.reporters
plugin.path
receive.buffer.bytes
response.http.headers.config
scheduled.rebalance.max.delay.ms
send.buffer.bytes
status.storage.topic
I am playing with Kafka locally in docker. I would like to set up the communication between the zookeeper and Kafka in a SASL_PLAINTEXT mode. But I want to set up the communication between Kafka and the Kafka producer using org.apache.kafka.common.security.scram.ScramLoginModule this protocol.
So, in the Kafka producer client config, I can say
String saslJaasConfigValue =
String.format(
"org.apache.kafka.common.security.scram.ScramLoginModule required username=%s password=%s",
username, pass);
put(SaslConfigs.SASL_JAAS_CONFIG, saslJaasConfigValue)
I think I've read somewhere that Kafka will store encrypted passwords in Zookeeper. Does that mean what I'm looking for is not possible?
If it's possible how do I do it?
I have configured 3 nodes kafka cluster. Now we want to setup security with Keycloak for kafka. Please let me know what are the ways to do the same.
Question 1: How to implement security for kafka broker to kafka broker with keycloak?
Question 2: How to implement security for kafka client to kafka broker with keycloak?
Note: We had already Keycloak setup.
You can configure Kafka to use AUTHBEARER which is implemented in latest kafka release , You can find more info how to configure here .
And also get more information about the feature from Kafka doc
You need to implement org.apache.kafka.common.security.auth.AuthenticateCallbackHandler to get token from keycloak and validate token from Keycloak.
I am using Kafka Version 0.10.1. I connected Kafka brokers and its clients via SSL and its working fine.Now I have a query with some limitations.
My limitations are
No Plain text communications allowed
The connection between Kafka-brokers and its clients be SSL.
The connection between Kafka-brokers and zookeeper via SASL (since zookeeper doesn't support SSL).
Since all inter-broker communications are set to SSL. I have a query that, Whether SASL connection between Zookeeper and Kafka-Broker is possible without enabling plaintext in Kafka-Broker.
Thanks in advance.
Yes it is possible to setup a Kafka cluster with Zookeeper with all the requirements you listed.
You'll need to have 2 listeners SSL and SASL_SSL (No PLAINTEXT) in your Kafka config:
listeners=SASL_SSL://host.name:port,SSL://host.name:port
Set inter broker to SSL
security.inter.broker.protocol=SSL
I suggest you check the Security section in the Kafka documentation to see what you need to do exactly to get this working, including how to configure clients so they connect over SASL_SSL: http://kafka.apache.org/documentation/#security
It also contains a section about securing Zookeeper:
http://kafka.apache.org/documentation/#zk_authz