Enable SSL for Kafka Clients - apache-kafka

Kafka allows clients to connect over SSL. By default SSL is disabled, but I have enabled by referring the below link.
http://docs.confluent.io/2.0.0/kafka/ssl.html
After all configuration was done ,Producer/Consumer unable to produce/consume the message.
[2016-02-29 09:20:49,189] ERROR Error when sending message to topic ssltopic with key: null, value: 2 bytes with error: Failed to update metadata after 60000 ms. (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
sas
[2016-02-29 09:21:16,031] WARN Failed to send SSL Close message (org.apache.kafka.common.network.SslTransportLayer)
java.io.IOException: Connection reset by peer

The above answer by supermonk clarifies most of the places to check. I faced the similar problem as the OP and the mistake was not in the broker configuration but the client side configuration.
In the official documentation, although they implicitly mentioned to create the client.keystore as step 1, I missed signing the certificate with the CA as done for the server.keystore. This was causing the Kafka broker to refuse the connection from the clients (producer/consumer).
Performing these two steps has eliminated the problem in my case.
keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days $VALIDITY -CAcreateserial -passin pass:$PASSWORD
keytool -keystore kafka.client.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore kafka.client.keystore.jks -alias localhost -import -file cert-signed
This will sign the certificate using the CA-cert and add the CARoot as well as signed certificates to the client.keystore.
Reference: Confluent blog on securing Apache Kafka

Below are some of verification steps.Since the error log is not detailed, Could you try the below step to validate if the setup is good.
To verify if the server’s keystore and truststore are setup correctly you can run the following command:
openssl s_client -debug -connect localhost:9093 -tls1 Note: TLSv1 should be listed under ssl.enabled.protocols.
In the output of this command you should see the server’s certificate:
-----BEGIN CERTIFICATE-----
{variable sized random bytes}
-----END CERTIFICATE-----
subject=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=Joe Smith issuer=/C=US/ST=CA/L=Santa Clara/O=org/OU=org/CN=kafka/emailAddress=test#test.com
If the certificate does not show up or if there are any other error messages then your keystore is not setup correctly.
Check the server.properties
echo "############################# Security #############################" >>server.properties
echo "listeners=SSL://:9093" >>server.properties
echo "security.inter.broker.protocol=SSL" >> server.properties
echo "ssl.client.auth=required" >> server.properties
echo "ssl.keystore.location=/home/vagrant/securityDemo/kafka.server.keystore.jks" >> server.properties
echo "ssl.keystore.password=test1234" >> server.properties
echo "ssl.key.password=test1234" >> server.properties
echo "ssl.truststore.location=/home/vagrant/securityDemo/kafka.server.truststore.jks" >> server.properties
echo "ssl.truststore.password=test1234" >> server.properties
echo "ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1" >> server.properties
echo "authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer" >>, if acl enabled ** server.properties
Make sure you have only one CA root.
Reference 1. and Reference 2.

Related

Faild when I try to send an email in Jenkins

I'm trying to send an email in Jenkins. I'm doing next steps:
Jenkins -> Configure System
In "E-mail Notification" area I'm providing next data, click "Test Configuration" and many errors appears:
Could you please help me with this issue?
It looks like you need gmail smtp server's certificate
Please try the following steps from http://notepad2.blogspot.com/2012/04/import-gmail-certificate-into-java.html (1)
Copied from the (1):
"
The following procedures are to import the gmail smtp certificate into the default Java keystore (Depends on the java mail application, the location of keystore may be vary):
Connect to smtp.gmail.com:465 to display the certificate in a terminal window:
For Linux:
openssl s_client -connect smtp.gmail.com:465
For Mac OS:
openssl s_client -connect smtp.gmail.com:465
For Windows
Install openssl first
Run command:
s_client -connect smtp.gmail.com:465
Copy and save the lines between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" into a file, say, gmail.cert
Import the certificate into java keystore(Default location):
sudo keytool -import -alias smtp.gmail.com -keystore /path/to/keystore -file /Users/wilson/gmail.cert
For Windows:
keytool -import -alias smtp.gmail.com -keystore "%JAVA_HOME%/jre/lib/security/cacerts" -file C:\Users\wilson\gmail.cert
For Mac OS:
sudo keytool -import -alias smtp.gmail.com -keystore /System/Library/Frameworks/JavaVM.framework//Versions/CurrentJDK/Home/lib/security/cacerts -file /Users/wilson/gmail.cert
For Linux:
sudo keytool -import -alias smtp.gmail.com -keystore $JAVA_HOME/jre/lib/security/cacerts -file /Users/wilson/gmail.cert
Note: your need to provide the password to access the keystore. The password for the default java keystore is changeit
Answer Yes when it ask "Trust this certificate? [no]: yes"
Note: if your java mail client application uses its own keystore, you need to change the location of the application's keystore rather than JVM's keystore in the keytool command.
"

How to create Kafka-python producer with ssl configuration

I'm trying to create kafka producer with ssl. I need information on how to set SSL parameters in the constructor, the information provided in kafka-python client is not descriptive enough.
What are the ssl_certfile, ssl_cafile, ssl_keyfile parameters. I'm not sure where to look for these files.
producer = KafkaProducer(bootstrap_servers=kafka_broker,
value_serializer=lambda v: json.dumps(v).encode('utf-8'),
security_protocol='SSL',
api_version=(0,10),
ssl_cafile='ca-certs.pem',ssl_certfile='server.pem',
ssl_keyfile='server.pem',ssl_password='xxx')
producer.send('rk976772_topic',{"test":0})
Traceback (most recent call last): File "", line 1, in
File
"/usr/lib/python2.7/site-packages/kafka/producer/kafka.py", line 543,
in send
self._wait_on_metadata(topic, self.config['max_block_ms'] / 1000.0) File "/usr/lib/python2.7/site-packages/kafka/producer/kafka.py", line 664,
in _wait_on_metadata
"Failed to update metadata after %.1f secs." % max_wait) kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update
metadata after 60.0 secs.
I was having this issue as well as many other while trying to configure kafka with SSL or SASL_SSL. I'm posting a full tutorial here in case anyone else runs into the same issues. I am using kafka-python 1.4.6 with kafka 2.2.0 on CentOS 6.
Below are the configurations that worked for me for SASL_SSL using kafka-python client. These configurations can be used for PLAINTEXT and SSL security protocols along with SASL_SSL and SASL_PLAINTEXT.
Bash script to generate key files, CARoot, and self-signed cert for use with SSL:
#!/bin/bash
#Step 1
keytool -keystore server.keystore.jks -alias localhost -validity 365 -keyalg RSA -genkey
#Step 2
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
#Step 3
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:admin123
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed
You can then use the following command to extract the CARoot.pem:
keytool -exportcert -alias CARoot -keystore server.keystore.jks -rfc -file CARoot.pem
In my server.properties file I have:
listeners=PLAINTEXT://localhost:9091,SASL_PLAINTEXT://localhost:9092,SSL://localhost:9093,SASL_SSL://localhost:9094
security.protocol=SSL
sasl.enabled.mechanisms=PLAIN
ssl.truststore.location=/var/private/ssl/server.truststore.jks
ssl.truststore.password=admin123
ssl.keystore.location=/var/private/ssl/server.keystore.jks
ssl.keystore.password=admin123
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
advertised.listeners=PLAINTEXT://localhost:9091,SASL_PLAINTEXT://localhost:9092,SSL://localhost:9093,SASL_SSL://localhost:9094
In my JAAS configuration file(/etc/kafka/kafka_plain_jaas.conf):
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=kafka
password=kafka-secret
user_username=password;
};
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=username
password=password;
};
Before starting the Kafka server, need to run the following:
export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_plain_jaas.conf"
Python consumer and producer:
The ssl_context and api_version are what caused SSL handshake errors to occur for me, leading to a timeout. So I commented those out. (There were some tutorials out there that mentioned to use those.)
from kafka import KafkaConsumer, KafkaProducer
import kafka
import ssl
import logging
logging.basicConfig(level=logging.DEBUG)
try:
topic = "sendMessage"
sasl_mechanism = "PLAIN"
username = "username"
password = "password"
security_protocol = "SASL_SSL"
#context = ssl.create_default_context()
#context.options &= ssl.OP_NO_TLSv1
#context.options &= ssl.OP_NO_TLSv1_1
consumer = KafkaConsumer(topic, bootstrap_servers='localhost:9094',
#api_version=(0, 10),
security_protocol=security_protocol,
#ssl_context=context,
ssl_check_hostname=True,
ssl_cafile='../keys/CARoot.pem',
sasl_mechanism = sasl_mechanism,
sasl_plain_username = username,
sasl_plain_password = password)
#ssl_certfile='../keys/certificate.pem',
#ssl_keyfile='../keys/key.pem')#,api_version = (0, 10))
producer = KafkaProducer(bootstrap_servers='localhost:9094',
#api_version=(0, 10),
security_protocol=security_protocol,
#ssl_context=context,
ssl_check_hostname=True,
ssl_cafile='../keys/CARoot.pem',
sasl_mechanism=sasl_mechanism,
sasl_plain_username=username,
sasl_plain_password=password)
#ssl_certfile='../keys/certificate.pem',
#ssl_keyfile='../keys/key.pem')#, api_version = (0,10))
# Write hello world to test topic
producer.send(topic, bytes("Hello World SSL"))
producer.flush()
for msg in consumer:
print(msg)
except Exception as e:
print e
Dudes, watch carefully and follow the instructions...
Step 1: Run all scripts (if necessary, set the values)
keytool -keystore kafka.server.keystore.jks -alias localhost -keyalg RSA -validity {validity} -genkey
openssl req -new -x509 -keyout ca-key -out ca-cert -days {validity}
keytool -keystore kafka.client.truststore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days {validity} -CAcreateserial -passin pass:{ca-password}
keytool -keystore kafka.server.keystore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias localhost -importcert -file cert-signed
keytool -exportcert -alias CARoot -keystore kafka.server.keystore.jks -rfc -file CARoot.pem
As a result, you will get:
kafka.server.keystore.jks, kafka.server.truststore.jks, kafka.client.truststore.jks, ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, CARoot.pem
Step 2: Copy kafka.server.keystore.jks and kafka.server.truststore.jks to the server and modify server.properties file (it is located in the config folder)
listeners=PLAINTEXT://MYSERVER:9092,SSL://MYSERVER:9093
advertised.listeners=PLAINTEXT://MYSERVER:9092,SSL://MYSERVER:9093
ssl.keystore.location=../store/kafka.server.keystore.jks
ssl.keystore.password=qwerty
ssl.truststore.location=../store/kafka.server.truststore.jks
ssl.truststore.password=qwerty
ssl.client.auth=required
ssl.endpoint.identification.algorithm=
Step 3: Create the python program
def kafka_consumer_ssl():
consumer = KafkaConsumer('test_topic',
bootstrap_servers=['MYSERVER:9093'],
auto_offset_reset='earliest',
enable_auto_commit=True,
value_deserializer=lambda x: x.decode('utf-8'),
security_protocol='SSL',
ssl_check_hostname=False,
ssl_cafile='CARoot.pem',
ssl_certfile='ca-cert',
ssl_keyfile='ca-key',
ssl_password='qwerty'
)
for event in consumer:
print(event.value)
kafka_consumer_ssl()
Step 4: Enjoy !!!
I had to publish the message over SASL_SSL
Used below code to create a producer with SASL_SSL protocol.
from kafka import KafkaProducer
security_protocol=environment_params.kafka_security_protocol
if env=='dev':
if security_protocol=='SASL_SSL':
producer = KafkaProducer(bootstrap_servers=environment_params.dev_kafka_broker,value_serializer=lambda v: json.dumps(v).encode('utf-8'),security_protocol=security_protocol,ssl_cafile='ca-certs.pem',sasl_mechanism='GSSAPI',api_version=environment_params.dev_kafka_api_version)
elif security_protocol=='PLAINTEXT':
producer = KafkaProducer(bootstrap_servers=environment_params.dev_kafka_broker,value_serializer=lambda v: json.dumps(v).encode('utf-8'))
Thanks Alot. I was having jks file and my kafka-producer was giving continuously error SSL Certification verify error 897
Though converted the CARoot.pem file but it was not working.
What helped is I converted using the below command and used on producer and it worked.
kafka.server.keystore.jks,
kafka.server.truststore.jks,
kafka.client.truststore.jks,
ca-cert,
ca-cert.srl,
ca-key, cert-file,
cert-signed,
CARoot.pem

Script to Generate Self signed SSL certificate

Question is around how to quickly generate a self signed certificate that you can use with Agg Cat services OR with other intuit services.
Is there a simple script.
Here is a simple script I created for this :
#!/bin/bash
if (( $# != 2 ))
then
echo "Usage: aliasname password"
exit 1
fi
keytool -genkey -keystore ./$1.p12 -deststoretype PKCS12 -storepass $2 -alias $1 -keyalg "RSA" -keysize 2048 -validity 9000
openssl pkcs12 -in $1.p12
keytool -v -importkeystore -srckeystore ./$1.p12 -srcstoretype PKCS12 -srcstorepass $2 -destkeystore ./$1.jks -deststoretype JKS -deststorepass $2
This will create an SSL certificate and print the private key and public cert.
You use the pub cert during app creation
Use the private key in the sample app of API Explorer.

How do I Import a .pem file?

I have the exact problem reported in
Jarsigner: certificate chain not found for
My starting point was a .pem file. My sense is that this does have the private key also. I used the following command to import this into a keystore:
keytool -importcert -alias myalias -file myfile.pem
For "Trust this certificate? [no]", if I choose "no" the import fails. So, I went with "yes". The import does succeed. My
keytool -list
produces output similar to the one listed in Jarsigner: certificate chain not found for.
My sense is that I do have the right certificate bit am not importing this correctly. In other words, I am suspecting that a 'trusted certificate entry' is being created instead of a 'key entry' but don't know how to force keytool to create a 'key entry'.
How can I solve this problem?
Additional Info:
After further work, I am leaning towards exactly the opposite conclusion than the one above. I now think that something is wrong with my pem file. I looked at a previous keystore entry with an expired key. It clearly states PrivateKeyEntry while my import states trustedCertEntry.
You can try to create a pkcs12 from your files that would contain the entire certificate chain. You'll need your public cert and the root CA cert. Command is like this:
openssl pkcs12 -export -inkey file.pem -in file.crt -out file.p12 \
-CAfile root-CA.pem -chain -name mykey
Once you have the entire file.p12 file, you can export the full cert to pem format:
openssl pkcs12 -in file.p12 -out new-cert.pem -nodes -clcerts
Or if you want to export to a Java keystore format that has the entire chain, the command is:
keytool -importkeystore -srcstoretype pkcs12 -srckeystore file.p12 \
-srcstorepass <password> -keystore keystore.jks

How to list the certificates stored in a PKCS12 keystore with keytool?

I wanted to list the certificates stored in a PKCS12 keystore.
The keystore has the extension .pfx
If the keystore is PKCS12 type (.pfx) you have to specify it with -storetype PKCS12 (line breaks added for readability):
keytool -list -v -keystore <path to keystore.pfx> \
-storepass <password> \
-storetype PKCS12
You can also use openssl to accomplish the same thing:
$ openssl pkcs12 -nokeys -info \
-in </path/to/file.pfx> \
-passin pass:<pfx's password>
MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Bag Attributes
localKeyID: XX XX XX XX XX XX XX XX XX XX XX XX XX 48 54 A0 47 88 1D 90
friendlyName: jedis-server
subject=/C=US/ST=NC/L=Raleigh/O=XXX Security/OU=XXX/CN=something1
issuer=/C=US/ST=NC/L=Raleigh/O=XXX Security/OU=XXXX/CN=something1
-----BEGIN CERTIFICATE-----
...
...
...
-----END CERTIFICATE-----
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048
You can list down the entries (certificates details) with the keytool and even you don't need to mention the store type. Also, the .p12 and .pfx are both PKCS#12 files. Assume that you've the keystore file cert.pfx or cert.p12 then you can use the following command to list down the content.
keytool -list -v -keystore cert.pfx -storepass <password>
or
keytool -list -v -keystore cert.p12 -storepass <password>
Keystore type: PKCS12
Keystore provider: SunJSSE
Your keystore contains 1 entry
Alias name: 1
Creation date: Jul 11, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 2
openssl pkcs12 -info -in keystore_file
What is missing in the question and all the answers is that you might need the passphrase to read public data from the PKCS#12 (.pfx) keystore. If you need a passphrase or not depends on how the PKCS#12 file was created. You can check the ASN1 structure of the file (by running it through a ASN1 parser, openssl or certutil can do this too), if the PKCS#7 data (e.g. OID prefix 1.2.840.113549.1.7) is listed as 'encrypted' or with a cipher-spec or if the location of the data in the asn1 tree is below an encrypted node, you won't be able to read it without knowledge of the passphrase. It means your 'openssl pkcs12' command will fail with errors (output depends on the version).
For those wondering why you might be interested in the certificate of a PKCS#12 without knowledge of the passphrase. Imagine you have many keystores and many phassphrases and you are really bad at keeping them organized and you don't want to test all combinations, the certificate inside the file could help you find out which password it might be. Or you are developing software to migrate/renew a keystore and you need to decide in advance which procedure to initiate based on the contained certicate without user interaction.
So the latter examples work without passphrase depending on the PKCS#12 structure.
Just wanted to add that, because I didn't find an answer myself and spend a lot of time to figure it out.