Maybe I miss something, if so forgive my ignorance.
Here what we have:
We use TLS authentication listeners in Kafka cluster (this can be changed, we can add new type of listeners).
When connect to Kafka topic from Java code I use SSL certificate generated for the Kafka user.
If I decide to avoid using SSL certificate, because of 2 reasons:
I will connect to Kafka topic only from trusted OpenShift cluster PODs
To avoid updating on producer/consumer side re-generareated yearly user's SSL certificate (because Kafka generates user certificate 1 year valid period)
Would be the SCRAM-SHA-512 authentication type for KafkaUser a better (and the only ?) choice for the two reasons above? Or SCRAM-SHA-512 also requires SSL certificates?
Another approach I saw was no authentication, but I am not sure how can ACL be used for such users? How I pass to server information which user is connecting. Is it possible to use both ACL and not authenticated by SSL certificate or by password Kafka user?
[UPD] Environment is built on Strimzi (Apache Kafka cluster in OpenShift)
Using SCRAM-SHA-512 does not require TLS. So you can just disable the TLS encryption in the Kafka custom resource (.spec.kafka.listeners -> set tls: false), enable he SCRAM-SHA-512 authentication (same place, in the authentication section). And then you just use the KafkaUser to create the user and get the password.
In general, TLS encryption is normally always recommended. But the SCRAM-SHA mechanisms do not send the password over the network directly, so using it without encryption should not leak the password. At the end, it is up to you to decide.
Also, just as a sidenote - the certificates are for 1 year by default. You can change it in the Kafka CR.
Related
I followed the tutorial of the answer of this question:
Kafka SASL zookeeper authentication
And i setted zookeeper.set.acl=true in the server.propeties, but i still can access the zookeeper on port 2181 and this is available for anyone through the: kafka-topics --zookeeper <server-name>:2181 --list
ps: instead of <server-name> i put the DN of my server.
Authentication enforcement feature has recently been submitted in the ZooKeeper codebase and afaik there's no stable version released yet which supports it.
When you turn on SASL authentication, it will be available, but clients are still able to connect without it. Hence the recommendation is to use ACLs side by side with authentication to prevent non-authenticated user from accessing sensitive data.
After reviewing several MongoDB official documents (see list at the bottom) I understand that MongoDB security in communications (as in community version 4.2) works as follows:
For internal communication authentication (i.e. between the members of a replica set or between mongos and the replica sets which implement the different shards) there are two mechanisms available:
shared keyfile (--keyFile)
x.509 certificates
For internal communication encryption, SSL/TLS is the only possibility. In other words, shared keyfile (--keyFile) provides only authentication, but if you want encryption you need to use SSL/TLS alternative. Using SSL/TLS requires to use x.509 certificates also (so, we can say that encryption also provides authentication)
For client to MongoDB communications (either standalone, replica set or mongos in a shard cluster):
there isn't keyfile option
the only way to secure communication (which provides both authentication and encryption) is SSL/TLS with x.509 certificates
I'd like just to confirm my understanding, as the documentation I have browsed is a little "disperse" and I'm not sure if I have got the point. Any feedback, comment, extra info or documentation pointers is really welcome!
PD: the statement which I'm most unsure is this one: "shared keyfile (--keyFile) provides only authentication, but if you want encryption you need to use SSL/TLS alternative"
References checked:
https://docs.mongodb.com/manual/tutorial/configure-ssl
https://docs.mongodb.com/manual/tutorial/configure-ssl-clients/
https://docs.mongodb.com/manual/core/security-internal-authentication
For client to MongoDB communications (either standalone, replica set or mongos in a shard cluster) there are several authentication methods:
No authentication
Internal authentication with username/password
Kerberos Authentication
LDAP Proxy Authentication
LDAP Authentication
Note, you can always connect to MongoDB database, even without an account. However, you are not permitted to execute any command unless you are authenticated.
I'm having some mongodb connection securtity concerns for my env.
Here is my environment:
one ECS hosted on cloud that has a public IP but no domain and no ssl certificate neither.
installed mongodb service on this ECS that needs username/password to authenticate
only specific IPs in the whitelist can access the ECS/mongodb
I'm wondering if the data transfer between this mongodb and my local pc is safe or not?
Will the data be encrpyted during the transmission or just plain text so that everyone on the internet can catch and read it? (As I don't have https so it's not using TLS/SSL)
Can canyone explain the machanism or give some some doc links?
Thanks!
As your not using SSL, your data on fly is not encrypted. You need to use TLS/SSL to encrypt the network transmission. You must have the TLS/SSL certificates as PEM files, which are concatenated certificate containers
In addition to encrypting connections, TLS/SSL allows for authentication using certificates, both for client authentication and for internal authentication of members of replica sets and sharded clusters
Does the client x.509 certificate encrypt the data as well as handle authorization?
Documentation says it handles authorization and message signing. But does that mean the data is encrypted in transit?
It is NOT encrypted when using Secure-Cluster with Certificates (Node2Node + Client2Node) with default Rpc-Endpoints. In Wireshark you can see the whole communication. It seems just for authorization.
Endpoints with https are encrypted of course.
Yes, a given x509 certificate will be used to encrypt the data while communication happens between a client and the cluster. As for authorization, it means that you could set what client certificates will posses 'SF Cluster Admin' privileges, and the ones that will allow only to query the info about your cluster.
In addition to the cluster certificates, you can add client
certificates to perform management operations on a service fabric
cluster. You can add two kinds of client certificates - Admin or
Read-only. These then can be used to control access to the admin
operations and Query operations on the cluster. By default, the
cluster certificates are added to the allowed Admin certificates list.
you can specify any number of client certificates. Each
addition/deletion results in a configuration update to the service
fabric cluster
The REST API for Kafka Connect is not secured and authenticated.
Since its not authenticated, the configuration for a connector or Tasks are easily accessible by anyone. Since these configurations may contain about how to access the Source System [in case of SourceConnector] and destination system [in case of SinkConnector], Is there a standard way to restrict access to these APIs?
In Kafka 2.1.0, there is possibility to configure http basic authentication for REST interface of Kafka Connect without writing any custom code.
This became real due to implementation of REST extensions mechanism (see KIP-285).
Shortly, configuration procedure as follows:
Add extension class to worker configuration file:
rest.extension.classes = org.apache.kafka.connect.rest.basic.auth.extension.BasicAuthSecurityRestExtension
Create JAAS config file (i.e. connect_jaas.conf) for application name 'KafkaConnect':
KafkaConnect {
org.apache.kafka.connect.rest.basic.auth.extension.PropertyFileLoginModule required
file="/your/path/rest-credentials.properties";
};
Create rest-credentials.properties file in above-mentioned directory:
user=password
Finally, inform java about you JAAS config file, for example, by adding command-line property to java:
-Djava.security.auth.login.config=/your/path/connect_jaas.conf
After restarting Kafka Connect, you will be unable to use REST API without basic authentication.
Please keep in mind that used classes are rather examples than production-ready features.
Links:
Connect configuratin
BasicAuthSecurityRestExtension
JaasBasicAuthFilter
PropertyFileLoginModule
This is a known area in need of improvement in the future but for now you should use a firewall on the Kafka Connect machines and either an API Management tool (Apigee, etc) or a Reverse proxy (haproxy, nginx, etc.) to ensure that HTTPS is terminated at an endpoint that you can configure access control rules on and then have the firewall only accept connections from the secure proxy. With some products the firewall, access control, and SSL/TLS termination functions can be all done in a fewer number of products.
As of Kafka 1.1.0, you can set up SSL and SSL client authentication for the Kafka Connect REST API. See KIP-208 for the details.
Now you are able to enable certificate based authentication for client access to the REST API of Kafka Connect.
An example here https://github.com/sudar-path/kc-rest-mtls