I have a enterprise level Kafka hosted in AWS Cluster. I'm trying to consume a topic from AWS Cluster. I need to use SSL protocol for connecting to servers.
From documentation i found that, i need to enable few properties,
ssl.keystore.password=test1234
ssl.key.password=test1234
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=test1234
I have a problem here, i cannot store the keystore.jks and truststore.jks in the source. Our security does not allow storing sensitive data in the source.
Instead I have a encrypted keystore file, which I pull it from Vault.
Is there a possibility that i can use this encrypted keystore? I don't see such feasibility in the documentation.
A local file path is needed for SSL certificates. You'll need a wrapper script before your code starts (or before the main method consumer logic) that will download the necessary information from Vault and fills in the properties files/configs
Or use a library that handles this, such as Spring Vault
Related
Does Keycloak 17 and above powered by Quarkus distribution has standalone mode?
The keycloak documentation says, that i can still use it, to setting up HTTPS/SSL. In the documentation there is a procedure, to edit the standalone.xml file, that no longer exist in this new version of keycloak.
Does standalone mode still exist? Or is there a different documentation in this not deprecated, new version that should be used? How to set up HTTPS/SSL then?
See https://www.keycloak.org/server/all-config?q=https
Use these parameters to customize TLS configuration based on your needs:
https-certificate-file
The file path to a server certificate or certificate chain in PEM format.
https-certificate-key-file
The file path to a private key in PEM format.
https-cipher-suites
The cipher suites to use.
https-client-auth
Configures the server to require/request client authentication.
https-key-store-file
The key store which holds the certificate information instead of specifying separate files.
https-key-store-password
The password of the key store file.
https-key-store-type
The type of the key store file.
https-port
The used HTTPS port.
https-protocols
The list of protocols to explicitly enable.
https-trust-store-file
The trust store which holds the certificate information of the certificates to trust.
https-trust-store-password
The password of the trust store file.
https-trust-store-type
The type of the trust store file.
Container deployement has also support for TLS, see Keycloak Docker HTTPS required
We are using Quorum and Hashicorp vault in one of our systems. We have been able to successfully integrate these two i.e. we have put the Tessera private and public keys in the Vault and successfully ran the Quorum server.
The problem is, when we are trying to use passphrase for the private key, we could not find a way through which we can achieve this. Even we have observed that when we are using the tessera key generation tool for Hashicorp vault where it generates the keys and internally saves the same in the Vault as well, it does not ask for any passphrase. But when we use the normal key generation tool where it generates the keys and puts the same in the specified directory, it asks for the passphrase.
May you please help us how we can achieve this leveraging Hashicorp Vault and Tessera i.e. we generate a key pair where private is protected with passphrase.
We could not find any help in the Wiki and also we tried to analyze the source code and our impression is if we want to use passphrase protected private key for Tessera, we can't use Hashicorp Vault now.
Please help.
Tessera does not support the storing of passphrase-protected private keys in a Hashicorp Vault as Vault already encrypts the data that it stores.
However, to get access to the data stored in a Vault, the Tessera instance must possess the correct set of credentials (provided as environment variables) in order to authenticate with the Vault. Using these credentials offers more flexibility and control in comparison to the passphrases used to secure file-stored keys.
For example, configuring an authentication method (e.g. AppRole authentication) makes it possible to define the authorisation for a particular Tessera instance, ensuring it is only allowed to access the secrets that it needs. Additionally these credentials can be configured to expire after a certain number of uses or length of time.
Finally, TLS should be enabled on the Vault server to ensure secure communication between Vault and Tessera. The necessary TLS certificates and keys should be included in the Tessera start-up config.
The Tessera wiki provides more details on the exact configuration and environment variables to provide:
https://github.com/jpmorganchase/tessera/wiki/Setting-up-a-Hashicorp-Vault
https://github.com/jpmorganchase/tessera/wiki/Keys#4-hashicorp-vault-key-pairs
I am using a JBoss based vault to secure sensitive data such as the database credentials.
I use a Java based HTTP REST client to create distributed Kafka connectors but ended up with a security concern such that a request for the connector's "config" exposes the sensitive credentials in the response.
I referred this official documentation but could not get much help in the context of JBoss vault.
Any pointers or references that directly addresses this specific problem is very much appreciated.
Any references to alternate open source (and free to use) Vault based solutions would also be of great help.
You'd have to write code that implements the ConfigProvider interface of the Connect API, mentioned there.
You can browse Kafka source code on Github to see the existing File one, but that KIP (which references Hashicorp Vault) and the source files are the only such documentation for now.
Connect doesn't use JBoss, either, so you'd have to find a way around that
Spring Boot - 2.0.0.M3
Spring cloud - Finchley.M1
I want to know if someone is using Spring Cloud config server with both vault and git support in a production setup using Database storage backend.
I have evaluated Spring cloud config using vault and contemplating whether to go for Oracle JCE to encrypt username/pwd or Vault and seek suggestions on the same. we are working on Springboot/microservices.
Following are my findings -
Vault will introduce an additional layer and thus will introduce additional usecases of security, auditing while communicating with Vault.
Spring cloud Config actuator endpoints are broken for the milestone release at this point for generation of encrypted values and /encrypt /decrypt may not work if we go for Oracle JCE support so we generate encrypted values through stable versions.
We do not wish to use consul server and are trying to use Cassandra as Storage backend.
I used Vault Authentication backend using AppRole and generated a Token (different from root token as it's unsafe to use the same) with read permissions. However, Spring Cloud config at the moment support only Token based authentication from client side. That means we first generate token from Vault and then pass it as commandline/env variable.
Some additional points of concern are expiry of token (though we can have non-expiry token not sure about pros/cons), restarts, safety issues, instantiating new microservices. There is no provision of dynamic tokens/authentication at cloud config side.
For milestone release i found that the client side encryption/decryption is not working as of now using recommended inclusion of RSA jar. Here is the ticket i opened.
https://github.com/spring-cloud/spring-cloud-config/issues/805#issuecomment-332491536
These are some of my observations, please share your thoughts if there is any case study/whitepaper that address spring cloud config vault usecases, setup and challenges for production micro-services environment.
Thanks
Thanks for reaching out to me. One think I would state is that the App Role backend utilizes two distinct tokens, and indeed spring-cloud-config-vault does indeed support this functionality, see: http://cloud.spring.io/spring-cloud-vault/single/spring-cloud-vault.html#_approle_authentication. I leverage vault in the same way I leverage config server, as per the documentation. I don't encrypt any values in my config, I just don't put them there. I put the secret values in vault and let it serve config. As long as keys don't collide, you don't have to mess with anything, otherwise you may need adjust the priority so vault wins, again see the documentation that I pointed to above. I wouldn't mess with encryption/decryption in spring-cloud-config personally. Because you have to check the keys into SCM or distribute them to your teams for local development, you lose the value of having these keys IMO.
Thanks Spring Cloud vault does support but not Spring cloud config with Vault. Only way seems to be passing X-Config-token from Microservice to Config Server. We are bit skeptical with this part of generating tokens manually or through script. Especially with containerization and when new MS instances will be spawn. Not sure about this approach especially in production setup.
I want to use WAL-E to backup my PostgreSQL data to Amazon S3. I am trying to determine if the data is encrypted in transit from Postgres to S3 using SSL/TLS but the documentation is not entirely clear on this point. I see WAL-E uses boto library and I believe by default it uses SSL/TLS but can anybody confirm or tell me how to configure WAL-E to ensure it does use SSL/TLS?
HTTPS is the default, but you can manually specify the S3 endpoint to force the protocol.
https://github.com/wal-e/wal-e#manually-specifying-the-s3-endpoint
The format is that of:
protocol+convention://hostname:port Where valid protocols are http and
https, and conventions are path, virtualhost, and subdomain.
Example:
Turns off encryption and specifies us-west-1 endpoint.
WALE_S3_ENDPOINT=http+path://s3-us-west-1.amazonaws.com:80
For radosgw.
WALE_S3_ENDPOINT=http+path://hostname
As seen when using Deis, which uses radosgw.
WALE_S3_ENDPOINT=http+path://deis-store-gateway:8888