Connect to Kafka on Unix from Windows with Kerberos - apache-kafka

Question

I'm quite new to Kafka so please bear with me. Here is my set up.
I have kafka hosted on a unix box. Clustered. and in a domain say B.
client is on windows. and am trying to connect to kafka hosted on B using a domain A.
I have the keytab. and krb5. both these are set up in the envt.
krb5.ini(and is set to envt variable KRB5_CONFIG)
[logging]
default = CONSOLE
admin_server = CONSOLE
kdc = CONSOLE
[libdefaults]
renew_lifetime = 7d
clockskew = 324000
forwardable = true
proxiable = true
renewable = true
default_realm = some.something.COM
dns_lookup_realm = true
dns_lookup_kdc = false
default_tgs_enctypes = somethingelse
default_tkt_enctypes = somethingelse
[appdefaults]
renewable = true
[realms]
some.something.COM = {
kdc = some.something.COM
admin_server = some.something.COM
}
I also have set up Jaas.config(Kafka.client.ini in my case and is set to envt variable KAFKA_CLIENT_KERBEROS_PARAMS) below is the config
KafkaClient {
com.sun.security.auth.module.Krb5LoginModule required
useKeyTab=true
keyTab="sample.keytab"
storeKey=true
useTicketCache=true
serviceName="kafka"
principal="svcacc#some.something.COM";
};
downloaded apache kafka_2.12-0.10.2.1.tgz and am executing this command.
kafka-console-producer.bat --broker-list <broker list> --topic <mytopic> --security-protocol SASL_PLAINTEXT
no matter what i change i keep getting below error
"security-protocol is not a recognised option"
can someone please help me in this?
i also added below props in producer.properties. but nothing seems to change. I'm not sure what i'm missing
security.protocol=SASL_PLAINTEXT
sasl.kerberos.service.name=kafka
I even tried setting this property in kafka-console-producer.bat but with no luck
set KAFKA_CLIENT_KERBEROS_PARAMS=- Djava.security.auth.login.config=..\..\config\kafka_Connection.ini
looking forward for your inputs. Many thanks (i've no control as of now on kafka server nor i will be able to explain why its hosted on domain B)

Answer 1

Disclaimer: I'm not too familiar with Kafka, and that error message does not clearly hint at a Kerberos problem.
But given that this is a cross-realm situation, you will probably hit a Kerberos snag sooner or later...
From Kerberos MIT documentation about krb5.conf section [capaths]
In order to perform direct (non-hierarchical) cross-realm
authentication, configuration is needed to determine the
authentication paths between realms.
A client will use this section to find the authentication path between
its realm and the realm of the server.
In other words, you get a Kerberos TGT (ticket-granting-ticket) for principal wtf#USERS.CORP.DMN but need a Kerberos service ticket for kafka/brokerhost.some.where#SERVERS.CORP.DMN. Each realm has its own KDC servers. Your Kerberos client (the Java implementation in this case) must have a way to "hop" from one domain to the others
Scenario 1 >> both realms are "brother" AD domains with mutual trust, and they use the default hierarchical relationship -- meaning that there is a "father" AD domain named CORP.DMN that is in the path from USERS to SERVERS.
Your krb5.conf should look like this...
[libdefaults]
default_realm = USERS.CORP.DMN
kdc_timeout = 3000
...
...
[realms]
USERS.CORP.DMN = {
kdc = roundrobin.siteA.users.corp.dmn
kdc = roundrobin.bcp.users.corp.dmn
}
SERVERS.CORP.DMN = {
kdc = dc1.servers.corp.dmn
kdc = dc2.servers.corp.dmn
kdc = roundrobin.bcp.servers.corp.dmn
}
CORP.DMN = {
kdc = roundrobin.corp.dmn
kdc = roundrobin.bcp.corp.dmn
}
...assuming you have multiple AD Domain Controllers in each domain, sometimes behind DNS aliases doing round-robin assignment, plus another set of DC on a separate site for BCP/DRP. It could be more simple than that :-)
Scenario 2 >> there is trust enabled but the relationship does not use the default, hierarchical path.
In that case you must define explicitly that "path" in a [capaths] section, as explained in the Kerberos documentation.
Scenario 3 >> there is no trust between realms. You are screwed.
Or rather, you must obtain a different user that can authenticate on the same domain as the Kafka broker, e.g. xyz#SERVERS.CORP.DMN.
And maybe use a specific krb5.conf that states default_realm = SERVERS.CORP.DMN (I've seen weird behaviors of some JDK versions on Windows, for example)
Bottom line: you must require assistance from your AD administrators. Maybe they are not familiar with raw Kerberos conf, but they will know about the trust and about the "paths"; at this point it's just a matter of following the proper krb5.conf syntax.
Or, maybe, that conf has already been done by the Linux administrators; so you should require an example of their standard krb5.conf to check whether there is cross-domain stuff in there.
And of course you should enable Kerberos debug traces in your Kafka producer:
-Dsun.security.krb5.debug=true
-Djava.security.debug=gssloginconfig,configfile,configparser,logincontext
Just for the record, but not useful here... when using Keberos over HTTP (SPNego) there's an additional flag-Dsun.security.spnego.debug=true