How Kafka Connect will communicate with a SASL enabled Broker? - apache-kafka

I've enabled SASL PLAIN authentication for my Zookeper and Broker. It seems working, I can only see topics and their content by using the credentials I set. The problem is, even though the status for all connectors were "RUNNING", there wasn't any data coming to kafka topics. So I restarted Kafka Connect and now I can't connect it, a connection refused error occurs.
It was already confusing me, how does Kafka Connect establish a connection with a SASL activated broker? It needs to be authenticated to be able to write data to a topic right? How can I do that? For example; I've provided the Schema Registry basic authentication information for Kafka Connect in connect-distributed.properties file like that:
schema.registry.basic.auth.user.info=admin:secret
key.converter.basic.auth.user.info=admin:secret
value.converter.basic.auth.user.info=admin:secret
schema.registry.basic.auth.credentials.source=USER_INFO
key.converter.basic.auth.credentials.source=USER_INFO
value.converter.basic.auth.credentials.source=USER_INFO
I believe I need to do something similar. But in tutorials I didn't see anything about that.
EDIT:
Connect service seems to be runnning, but connectors can't fetch the metadata of topics. That means there is a problem with authentication to Kafka.

It seems to be working with below configuration. I am not sure if you need to add producer. and consumer. parts but they don't cause any problems. I've added these lines to connect-distributed.properties file.
sasl.mechanism=PLAIN
security.protocol=SASL_PLAINTEXT
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="secret";
producer.sasl.mechanism=PLAIN
producer.security.protocol=SASL_PLAINTEXT
producer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="secret";
consumer.sasl.mechanism=PLAIN
consumer.security.protocol=SASL_PLAINTEXT
consumer.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="admin" \
password="secret";

Related

confluent kafka s3 connector worker failed in connecting to kafka authenticated by krb5

I am working on leveraging confluent kafka s3 connector to download kafka record and save them as parquet file in minio. It worked fine with a dummy kafka without authentication.
Now I'm trying to verify the flow with a real kafka instance that requires kerberos authentication.
I assured my kerberos credentials are in place and set up properly.
env | grep OPTS
KAFKA_OPTS=-Djava.security.krb5.conf=/plugins/krb5.conf -Djava.security.auth.login.config=/plugins/kafka_client_jaas.conf
in my connector.properties file I specified the following
#connecting to kafka
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
Now I start the connector-standalone.sh
root#2c553a4e0b7c:/opt/bitnami/kafka/bin# ./connect-standalone.sh /plugins/connector.properties /plugins/s3-sink.properties
[2022-09-21 10:37:43,559] WARN [Consumer clientId=connector-consumer-s3-sink-0, groupId=connect-s3-sink] Bootstrap broker broker:9030 (id: -1 rack: n
ull) disconnected (org.apache.kafka.clients.NetworkClient:1024)
It seems that the krb5 authentication is not enabled.
I've tried kafka-console-consumer.sh with the same krb configs and credentials and it all worked ok.
It's likely that the three lines of configuration in connector.properties did not take effective. Are the correct configurations to to notify worker to use krb5?
security.protocol=SASL_PLAINTEXT
sasl.mechanism=GSSAPI
sasl.kerberos.service.name=kafka
Kafka-connect, Bootstrap broker disconnected
Answers to this question apply to here as well.
There's no document or whatsoever that talks about CONNECT_CONSUMER env vars must be set and repeat the jvm parameters.

Debezium fails when using it with Kerberos

I'm trying to configure the Oracle Connector (debezium 1.9) with a Kerberized Kafka cluster (from Cloudera Private CDP) and have some weird troubles.
I first tried to configure Debezium with a PLAINTEXT security protocol (using an Apache Kafka 3.1.0) to validate everything was fine (Oracle, Connect config... ) and everything runs perfectly.
Next, I deployed the same connector, using the same Oracle DB instance on my On Premises Cloudera CDP platform, which is kerberized, and updating the connector config by adding :
"database.history.kafka.topic": "schema-changes.oraclecdc",
"database.history.consumer.sasl.jaas.config": "com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab=\"/tmp/debezium.keytab\" principal=\"debezium#MYREALM\";",
"database.history.consumer.security.protocol": "SASL_PLAINTEXT",
"database.history.consumer.sasl.kerberos.service.name": "kafka",
"database.history.producer.sasl.jaas.config": "com.sun.security.auth.module.Krb5LoginModule required useKeyTab=true keyTab=\"/tmp/debezium.keytab\" principal=\"debezium#MYREALM\";",
"database.history.producer.security.protocol": "SASL_PLAINTEXT",
"database.history.producer.sasl.kerberos.service.name": "kafka"
In this case, the topic schema-changes.oraclecdc is automatically created when the connector starts (auto creation enabled) and the DDL definitions are correctly reported. But that's it. So I suppose the JAAS config is OK and the producer config is correctly set as the connector has been able to create the topic and publish something in it.
But I can't get my updates/inserts/deletes being published. And the corresponding topics are not created. Instead kafka connect reports me the producer is disconnected, as soon as the connector starts.
Activating the TRACE level into kafka-connect, I can check that the updates/inserts/... are correctly detected by debezium from the redo log.
The fact the producer is being disconnected makes me think there's a problem of authentication. But if I understand the debezium documentation, the producer config is the same for either schema topic and tables cdc topics. So I can't understand why the "schema changes topic" is created with messages published, but the "CDC mechanism" doesn't create topics...
What am I missing here?

How to connect Redpanda cluster and topic to Redpanda Schema Registry?

I am trying to connect a running redpanda kafka cluster to a redpanda schema registry, so that the schema registry verifies incoming messages to the topic, and/or messages being read from the topic.
I am able to add a schema to the registry and read it back with curl requests, as well as add messages to the kafka topic I created in the redpanda cluster.
My question is, how to implement the schema registry with a topic in a kafka cluster? Like how do I instruct the schema registry and/or the kafka topic to validate incoming messages against the schema I added to the registry?
Thanks for your help or a point in the right direction!
Relevant info:
Cluster & topic creation:
https://vectorized.io/docs/guide-rpk-container
rpk container start -n 3
rpk topic create -p 6 -r 3 new-topic --brokers <broker1_address>,<broker2_address>...
Schema Registry creation:
https://vectorized.io/blog/schema_registry/
Command to add a schema:
curl -s \
-X POST \
"http://localhost:8081/subjects/sensor-value/versions" \
-H "Content-Type: application/vnd.schemaregistry.v1+json" \
-d '{"schema": "{\"type\":\"record\",\"name\":\"sensor_sample\",\"fields\":[{\"name\":\"timestamp\",\"type\":\"long\",\"logicalType\":\"timestamp-millis\"},{\"name\":\"identifier\",\"type\":\"string\",\"logicalType\":\"uuid\"},{\"name\":\"value\",\"type\":\"long\"}]}"}' \
| jq
The client is responsible for such integration. For example, the Confluent Schema Registry includes KafkaAvroSerializer Java class that wraps an HTTP Client that handles the schema registration and message validation. The broker doesn't handle "topic schemas", since schemas are really per-record. Unless RedPanda has something similar, broker-side validation is only offered by Enterprise "Confluent Server."
RedPanda is primarily a server that exposes a Kafka-compatible API; I assume it is up to you to create (de)serializer interfaces for your respective client languages. There is a Python example on Vectorized Github.
That being said, the Confluent Schema Registry should work with RedPanda as well, and so you can use its serializers and HTTP client libraries with it.

Kafka zookeeper authentication not working

I am trying to enable SASL username and password for a Kafka cluster with no ssl. I followed the steps on this Stackoverflow:
Kafka SASL zookeeper authentication
SASL authentication seems to be working for Kafka brokers. consumers and producers have to authenticate before writing to or reading from a topic. So far so good.
The problem is with creating and deleting topics on kafka. when I try to use the following command for example:
~/kafka/bin/kafka-topics.sh --list --zookeeper 10.x.y.z:2181
I am able to list all topics in the kafka cluster and create or delete any topic with no authentication at all.
I tried to follow the steps here:
Super User Authentication and Authorization
but nothing seem to work.
Any help in this matter is really appreciated.
Thanks & Regards,
Firas Khasawneh
You need to add zookeeper.set.acl=true to your Kafka server.properties so that Kafka will create everything in zookeeper with ACL set. For the topics which are already there, there will be no ACL and everyone can remove them directly from zookeeper.
Actually because of that mess, I had to delete everything from my zookeeper and Kafka and start from scratch.
But once everything is set, you can open zookeeper shell to verify that the ACL is indeed set:
KAFKA_OPTS="-Djava.security.auth.login.config=/path/to/your/jaas.conf" bin/zookeeper-shell.sh XXXXX:2181
From the shell you can run: getAcl /brokers/topics and check that not anyone from world have cdrwa
On a side note, the link you provided doesn't seem to reflect how the current version of Kafka stores information in zookeeper. I briefly looked at the codes and for those kafka-topics.sh commands, the topics information is from /brokers/topics instead of /config/topics

Not Kerberized Kafka broker connection to Kerberized Zookeeper

I couldn't find any info about this issue, so I'd be glad if someone could help me on this.
I have a Kerberized cluster with services such as Hbase, MapReduce, HDFS, Zookeeper,... all kerberized and working.
Let's imagine I want to add some kafka brokers to the cluster, but I do not want to Kerberize Kafka, since a shot in the testicles makes me feel better than the idea of a kerberized Kafka.
I don't know if I'm missing something, some parameter... probably I am.. but can the zookeeper be told that also has to accept PLAINTEXT petitions for some nodes, or for some specific directories, such as kafka in the example:
zookeeper:2181/kafka
Resuming, the question is:
Is there any option to include a non kerberized Kafka Broker and make it work against the already kerberized Zookeeper in the cluster?
If you need configuration like:
[zookeeper] <----- SASL ----> [kafka] <----- non-authenticated request ---> [clients]
then yes, it's possible. You need just to
Create principal (with keytabs) for brokers that will be used to communicate with Zookeeper.
Configure Zookeeper ACLs, setting cdrwa access to the node zookeeper:2181/kafka to that user
Copy the keytab to brokers and configure Kafka jaas file like this:
ZookeeperClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
storeKey=true
keyTab="/path/to/keytab"
principal="user#REALM";
};
Then, set zookeeper.set.acl=true in Kafka configuration, but do not set any authorizer.class.name (this would enable authentication for Kafka consumers and producers)