I'm trying to send an email in Jenkins. I'm doing next steps:
Jenkins -> Configure System
In "E-mail Notification" area I'm providing next data, click "Test Configuration" and many errors appears:
Could you please help me with this issue?
It looks like you need gmail smtp server's certificate
Please try the following steps from http://notepad2.blogspot.com/2012/04/import-gmail-certificate-into-java.html (1)
Copied from the (1):
"
The following procedures are to import the gmail smtp certificate into the default Java keystore (Depends on the java mail application, the location of keystore may be vary):
Connect to smtp.gmail.com:465 to display the certificate in a terminal window:
For Linux:
openssl s_client -connect smtp.gmail.com:465
For Mac OS:
openssl s_client -connect smtp.gmail.com:465
For Windows
Install openssl first
Run command:
s_client -connect smtp.gmail.com:465
Copy and save the lines between "-----BEGIN CERTIFICATE-----" and "-----END CERTIFICATE-----" into a file, say, gmail.cert
Import the certificate into java keystore(Default location):
sudo keytool -import -alias smtp.gmail.com -keystore /path/to/keystore -file /Users/wilson/gmail.cert
For Windows:
keytool -import -alias smtp.gmail.com -keystore "%JAVA_HOME%/jre/lib/security/cacerts" -file C:\Users\wilson\gmail.cert
For Mac OS:
sudo keytool -import -alias smtp.gmail.com -keystore /System/Library/Frameworks/JavaVM.framework//Versions/CurrentJDK/Home/lib/security/cacerts -file /Users/wilson/gmail.cert
For Linux:
sudo keytool -import -alias smtp.gmail.com -keystore $JAVA_HOME/jre/lib/security/cacerts -file /Users/wilson/gmail.cert
Note: your need to provide the password to access the keystore. The password for the default java keystore is changeit
Answer Yes when it ask "Trust this certificate? [no]: yes"
Note: if your java mail client application uses its own keystore, you need to change the location of the application's keystore rather than JVM's keystore in the keytool command.
"
I'm trying to create kafka producer with ssl. I need information on how to set SSL parameters in the constructor, the information provided in kafka-python client is not descriptive enough.
What are the ssl_certfile, ssl_cafile, ssl_keyfile parameters. I'm not sure where to look for these files.
producer = KafkaProducer(bootstrap_servers=kafka_broker,
value_serializer=lambda v: json.dumps(v).encode('utf-8'),
security_protocol='SSL',
api_version=(0,10),
ssl_cafile='ca-certs.pem',ssl_certfile='server.pem',
ssl_keyfile='server.pem',ssl_password='xxx')
producer.send('rk976772_topic',{"test":0})
Traceback (most recent call last): File "", line 1, in
File
"/usr/lib/python2.7/site-packages/kafka/producer/kafka.py", line 543,
in send
self._wait_on_metadata(topic, self.config['max_block_ms'] / 1000.0) File "/usr/lib/python2.7/site-packages/kafka/producer/kafka.py", line 664,
in _wait_on_metadata
"Failed to update metadata after %.1f secs." % max_wait) kafka.errors.KafkaTimeoutError: KafkaTimeoutError: Failed to update
metadata after 60.0 secs.
I was having this issue as well as many other while trying to configure kafka with SSL or SASL_SSL. I'm posting a full tutorial here in case anyone else runs into the same issues. I am using kafka-python 1.4.6 with kafka 2.2.0 on CentOS 6.
Below are the configurations that worked for me for SASL_SSL using kafka-python client. These configurations can be used for PLAINTEXT and SSL security protocols along with SASL_SSL and SASL_PLAINTEXT.
Bash script to generate key files, CARoot, and self-signed cert for use with SSL:
#!/bin/bash
#Step 1
keytool -keystore server.keystore.jks -alias localhost -validity 365 -keyalg RSA -genkey
#Step 2
openssl req -new -x509 -keyout ca-key -out ca-cert -days 365
keytool -keystore server.truststore.jks -alias CARoot -import -file ca-cert
keytool -keystore client.truststore.jks -alias CARoot -import -file ca-cert
#Step 3
keytool -keystore server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days 365 -CAcreateserial -passin pass:admin123
keytool -keystore server.keystore.jks -alias CARoot -import -file ca-cert
keytool -keystore server.keystore.jks -alias localhost -import -file cert-signed
You can then use the following command to extract the CARoot.pem:
keytool -exportcert -alias CARoot -keystore server.keystore.jks -rfc -file CARoot.pem
In my server.properties file I have:
listeners=PLAINTEXT://localhost:9091,SASL_PLAINTEXT://localhost:9092,SSL://localhost:9093,SASL_SSL://localhost:9094
security.protocol=SSL
sasl.enabled.mechanisms=PLAIN
ssl.truststore.location=/var/private/ssl/server.truststore.jks
ssl.truststore.password=admin123
ssl.keystore.location=/var/private/ssl/server.keystore.jks
ssl.keystore.password=admin123
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
advertised.listeners=PLAINTEXT://localhost:9091,SASL_PLAINTEXT://localhost:9092,SSL://localhost:9093,SASL_SSL://localhost:9094
In my JAAS configuration file(/etc/kafka/kafka_plain_jaas.conf):
KafkaServer {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=kafka
password=kafka-secret
user_username=password;
};
KafkaClient {
org.apache.kafka.common.security.plain.PlainLoginModule required
username=username
password=password;
};
Before starting the Kafka server, need to run the following:
export KAFKA_OPTS="-Djava.security.auth.login.config=/etc/kafka/kafka_plain_jaas.conf"
Python consumer and producer:
The ssl_context and api_version are what caused SSL handshake errors to occur for me, leading to a timeout. So I commented those out. (There were some tutorials out there that mentioned to use those.)
from kafka import KafkaConsumer, KafkaProducer
import kafka
import ssl
import logging
logging.basicConfig(level=logging.DEBUG)
try:
topic = "sendMessage"
sasl_mechanism = "PLAIN"
username = "username"
password = "password"
security_protocol = "SASL_SSL"
#context = ssl.create_default_context()
#context.options &= ssl.OP_NO_TLSv1
#context.options &= ssl.OP_NO_TLSv1_1
consumer = KafkaConsumer(topic, bootstrap_servers='localhost:9094',
#api_version=(0, 10),
security_protocol=security_protocol,
#ssl_context=context,
ssl_check_hostname=True,
ssl_cafile='../keys/CARoot.pem',
sasl_mechanism = sasl_mechanism,
sasl_plain_username = username,
sasl_plain_password = password)
#ssl_certfile='../keys/certificate.pem',
#ssl_keyfile='../keys/key.pem')#,api_version = (0, 10))
producer = KafkaProducer(bootstrap_servers='localhost:9094',
#api_version=(0, 10),
security_protocol=security_protocol,
#ssl_context=context,
ssl_check_hostname=True,
ssl_cafile='../keys/CARoot.pem',
sasl_mechanism=sasl_mechanism,
sasl_plain_username=username,
sasl_plain_password=password)
#ssl_certfile='../keys/certificate.pem',
#ssl_keyfile='../keys/key.pem')#, api_version = (0,10))
# Write hello world to test topic
producer.send(topic, bytes("Hello World SSL"))
producer.flush()
for msg in consumer:
print(msg)
except Exception as e:
print e
Dudes, watch carefully and follow the instructions...
Step 1: Run all scripts (if necessary, set the values)
keytool -keystore kafka.server.keystore.jks -alias localhost -keyalg RSA -validity {validity} -genkey
openssl req -new -x509 -keyout ca-key -out ca-cert -days {validity}
keytool -keystore kafka.client.truststore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.truststore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file cert-file
openssl x509 -req -CA ca-cert -CAkey ca-key -in cert-file -out cert-signed -days {validity} -CAcreateserial -passin pass:{ca-password}
keytool -keystore kafka.server.keystore.jks -alias CARoot -importcert -file ca-cert
keytool -keystore kafka.server.keystore.jks -alias localhost -importcert -file cert-signed
keytool -exportcert -alias CARoot -keystore kafka.server.keystore.jks -rfc -file CARoot.pem
As a result, you will get:
kafka.server.keystore.jks, kafka.server.truststore.jks, kafka.client.truststore.jks, ca-cert, ca-cert.srl, ca-key, cert-file, cert-signed, CARoot.pem
Step 2: Copy kafka.server.keystore.jks and kafka.server.truststore.jks to the server and modify server.properties file (it is located in the config folder)
listeners=PLAINTEXT://MYSERVER:9092,SSL://MYSERVER:9093
advertised.listeners=PLAINTEXT://MYSERVER:9092,SSL://MYSERVER:9093
ssl.keystore.location=../store/kafka.server.keystore.jks
ssl.keystore.password=qwerty
ssl.truststore.location=../store/kafka.server.truststore.jks
ssl.truststore.password=qwerty
ssl.client.auth=required
ssl.endpoint.identification.algorithm=
Step 3: Create the python program
def kafka_consumer_ssl():
consumer = KafkaConsumer('test_topic',
bootstrap_servers=['MYSERVER:9093'],
auto_offset_reset='earliest',
enable_auto_commit=True,
value_deserializer=lambda x: x.decode('utf-8'),
security_protocol='SSL',
ssl_check_hostname=False,
ssl_cafile='CARoot.pem',
ssl_certfile='ca-cert',
ssl_keyfile='ca-key',
ssl_password='qwerty'
)
for event in consumer:
print(event.value)
kafka_consumer_ssl()
Step 4: Enjoy !!!
I had to publish the message over SASL_SSL
Used below code to create a producer with SASL_SSL protocol.
from kafka import KafkaProducer
security_protocol=environment_params.kafka_security_protocol
if env=='dev':
if security_protocol=='SASL_SSL':
producer = KafkaProducer(bootstrap_servers=environment_params.dev_kafka_broker,value_serializer=lambda v: json.dumps(v).encode('utf-8'),security_protocol=security_protocol,ssl_cafile='ca-certs.pem',sasl_mechanism='GSSAPI',api_version=environment_params.dev_kafka_api_version)
elif security_protocol=='PLAINTEXT':
producer = KafkaProducer(bootstrap_servers=environment_params.dev_kafka_broker,value_serializer=lambda v: json.dumps(v).encode('utf-8'))
Thanks Alot. I was having jks file and my kafka-producer was giving continuously error SSL Certification verify error 897
Though converted the CARoot.pem file but it was not working.
What helped is I converted using the below command and used on producer and it worked.
kafka.server.keystore.jks,
kafka.server.truststore.jks,
kafka.client.truststore.jks,
ca-cert,
ca-cert.srl,
ca-key, cert-file,
cert-signed,
CARoot.pem
Question is around how to quickly generate a self signed certificate that you can use with Agg Cat services OR with other intuit services.
Is there a simple script.
Here is a simple script I created for this :
#!/bin/bash
if (( $# != 2 ))
then
echo "Usage: aliasname password"
exit 1
fi
keytool -genkey -keystore ./$1.p12 -deststoretype PKCS12 -storepass $2 -alias $1 -keyalg "RSA" -keysize 2048 -validity 9000
openssl pkcs12 -in $1.p12
keytool -v -importkeystore -srckeystore ./$1.p12 -srcstoretype PKCS12 -srcstorepass $2 -destkeystore ./$1.jks -deststoretype JKS -deststorepass $2
This will create an SSL certificate and print the private key and public cert.
You use the pub cert during app creation
Use the private key in the sample app of API Explorer.
I have the exact problem reported in
Jarsigner: certificate chain not found for
My starting point was a .pem file. My sense is that this does have the private key also. I used the following command to import this into a keystore:
keytool -importcert -alias myalias -file myfile.pem
For "Trust this certificate? [no]", if I choose "no" the import fails. So, I went with "yes". The import does succeed. My
keytool -list
produces output similar to the one listed in Jarsigner: certificate chain not found for.
My sense is that I do have the right certificate bit am not importing this correctly. In other words, I am suspecting that a 'trusted certificate entry' is being created instead of a 'key entry' but don't know how to force keytool to create a 'key entry'.
How can I solve this problem?
Additional Info:
After further work, I am leaning towards exactly the opposite conclusion than the one above. I now think that something is wrong with my pem file. I looked at a previous keystore entry with an expired key. It clearly states PrivateKeyEntry while my import states trustedCertEntry.
You can try to create a pkcs12 from your files that would contain the entire certificate chain. You'll need your public cert and the root CA cert. Command is like this:
openssl pkcs12 -export -inkey file.pem -in file.crt -out file.p12 \
-CAfile root-CA.pem -chain -name mykey
Once you have the entire file.p12 file, you can export the full cert to pem format:
openssl pkcs12 -in file.p12 -out new-cert.pem -nodes -clcerts
Or if you want to export to a Java keystore format that has the entire chain, the command is:
keytool -importkeystore -srcstoretype pkcs12 -srckeystore file.p12 \
-srcstorepass <password> -keystore keystore.jks
I wanted to list the certificates stored in a PKCS12 keystore.
The keystore has the extension .pfx
If the keystore is PKCS12 type (.pfx) you have to specify it with -storetype PKCS12 (line breaks added for readability):
keytool -list -v -keystore <path to keystore.pfx> \
-storepass <password> \
-storetype PKCS12
You can also use openssl to accomplish the same thing:
$ openssl pkcs12 -nokeys -info \
-in </path/to/file.pfx> \
-passin pass:<pfx's password>
MAC Iteration 2048
MAC verified OK
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Bag Attributes
localKeyID: XX XX XX XX XX XX XX XX XX XX XX XX XX 48 54 A0 47 88 1D 90
friendlyName: jedis-server
subject=/C=US/ST=NC/L=Raleigh/O=XXX Security/OU=XXX/CN=something1
issuer=/C=US/ST=NC/L=Raleigh/O=XXX Security/OU=XXXX/CN=something1
-----BEGIN CERTIFICATE-----
...
...
...
-----END CERTIFICATE-----
PKCS7 Data
Shrouded Keybag: pbeWithSHA1And3-KeyTripleDES-CBC, Iteration 2048
You can list down the entries (certificates details) with the keytool and even you don't need to mention the store type. Also, the .p12 and .pfx are both PKCS#12 files. Assume that you've the keystore file cert.pfx or cert.p12 then you can use the following command to list down the content.
keytool -list -v -keystore cert.pfx -storepass <password>
or
keytool -list -v -keystore cert.p12 -storepass <password>
Keystore type: PKCS12
Keystore provider: SunJSSE
Your keystore contains 1 entry
Alias name: 1
Creation date: Jul 11, 2020
Entry type: PrivateKeyEntry
Certificate chain length: 2
openssl pkcs12 -info -in keystore_file
What is missing in the question and all the answers is that you might need the passphrase to read public data from the PKCS#12 (.pfx) keystore. If you need a passphrase or not depends on how the PKCS#12 file was created. You can check the ASN1 structure of the file (by running it through a ASN1 parser, openssl or certutil can do this too), if the PKCS#7 data (e.g. OID prefix 1.2.840.113549.1.7) is listed as 'encrypted' or with a cipher-spec or if the location of the data in the asn1 tree is below an encrypted node, you won't be able to read it without knowledge of the passphrase. It means your 'openssl pkcs12' command will fail with errors (output depends on the version).
For those wondering why you might be interested in the certificate of a PKCS#12 without knowledge of the passphrase. Imagine you have many keystores and many phassphrases and you are really bad at keeping them organized and you don't want to test all combinations, the certificate inside the file could help you find out which password it might be. Or you are developing software to migrate/renew a keystore and you need to decide in advance which procedure to initiate based on the contained certicate without user interaction.
So the latter examples work without passphrase depending on the PKCS#12 structure.
Just wanted to add that, because I didn't find an answer myself and spend a lot of time to figure it out.