I want to keep my smtp login information private. And its a hassle to edit application-prod.yml every time I have to deploy to production.
What is the correct method to avoid storing sensitive details in application-prod.yml on github ?
Many different ways depending on where and how you deploy to:
Don't store application-prod.yml in git and don't package it in your jar
Don't store secrets in application-prod.yml, use environment variables or command line options. See Spring Boot doc.
Encrypt secrets using git-crypt in application-prod.yml
Store secrets in external Spring Cloud Config Server (e.g. JHipster Registry) or HashiCorp Vault
and many other ways...
Related
I am currently working on a solution that centralizes connections to external databases.
For that, I initialize a database with flyway and connect to other postgresql sources with the postrgres_fdw extension to create my table projections (with foreign tables).
Everything works fine if I put my credentials, hosts, etc... in my application.yml configuration file (we are on spring), putting these values in placeholders and reusing them in my sql migration scripts. But we want to fetch this data from vault, where we store all this data.
However, although I have followed the flyway documentation on vault integration (https://flywaydb.org/blog/integrating-vault-to-secure-flyway-parameters), I cannot achieve my goal. I've tried putting my placeholders in vault (my secrets are of the form flyway.placeholders.[...]=), and connecting to my instance via my data in my configuration file
spring:
flyway:
vault:
url: https://localhost:8200/v1/
token: root
secrets: secret/data/...
but without success. Has anyone had this problem before? Is it possible to retrieve any secret value via placeholders to use in sql scripts or do we have to go through the java API to have a little more flexibility?
I have a enterprise level Kafka hosted in AWS Cluster. I'm trying to consume a topic from AWS Cluster. I need to use SSL protocol for connecting to servers.
From documentation i found that, i need to enable few properties,
ssl.keystore.password=test1234
ssl.key.password=test1234
ssl.truststore.location=/var/private/ssl/kafka.server.truststore.jks
ssl.truststore.password=test1234
I have a problem here, i cannot store the keystore.jks and truststore.jks in the source. Our security does not allow storing sensitive data in the source.
Instead I have a encrypted keystore file, which I pull it from Vault.
Is there a possibility that i can use this encrypted keystore? I don't see such feasibility in the documentation.
A local file path is needed for SSL certificates. You'll need a wrapper script before your code starts (or before the main method consumer logic) that will download the necessary information from Vault and fills in the properties files/configs
Or use a library that handles this, such as Spring Vault
I have configured my service fabric services to use Azure Key Vault for configuration. If, after the app is deployed, I change the config in Key Vault, how do I then restart the affected service so it can pick up the new config value?
Or is there another way altogether?
The best way to handle configuration on SF is use your application parameters file for this, if you use a continuous deployment pipeline like VSTS, you could use release variables to set these values for you and deploy a new version of your configuration file and let SF do the rest.
But in case you still need to use Key vault:
if you are using asp.net core, Using Azure Key Vault to store secrets are like loading configuration files, the values are cached until you reload it.
You can use the IConfigurationRoot.Reload() to reload the secrets from your key vault new values. Check it Here
The trick now is to make it automatically you have to:
Enable Key Vault Logging to track the changes, this will emit logs once you update the key vault. check it here and here .
And then:
Create an endpoint in your API to be called and refresh the secrets. Make it secure to avoid abuse.
Create an Azure function to process these logs and trigger the endpoint
Or:
Create a message queue to receive the command and the system read the message to refresh the settings
Or:
Make a timer to refresh on specific periods(I would not recommended this approach because you might end up with outdated config, but it is easy and useful for quick test scenarios, not production)
Or if you prefer more custom designed solution, you could create your own ConfigurationProvider based on KeyVault and do the cache logic according to your app architecture and you don't have to bother with the rest. Please refer to the Asp.Net source here for this.
The documented way to provide configuration to your services is by using the 'configuration' part of your application package.
As this is versioned, it can be upgraded, without requiring your services to be upgraded or even be restarted.
More info here and here.
Spring Boot - 2.0.0.M3
Spring cloud - Finchley.M1
I want to know if someone is using Spring Cloud config server with both vault and git support in a production setup using Database storage backend.
I have evaluated Spring cloud config using vault and contemplating whether to go for Oracle JCE to encrypt username/pwd or Vault and seek suggestions on the same. we are working on Springboot/microservices.
Following are my findings -
Vault will introduce an additional layer and thus will introduce additional usecases of security, auditing while communicating with Vault.
Spring cloud Config actuator endpoints are broken for the milestone release at this point for generation of encrypted values and /encrypt /decrypt may not work if we go for Oracle JCE support so we generate encrypted values through stable versions.
We do not wish to use consul server and are trying to use Cassandra as Storage backend.
I used Vault Authentication backend using AppRole and generated a Token (different from root token as it's unsafe to use the same) with read permissions. However, Spring Cloud config at the moment support only Token based authentication from client side. That means we first generate token from Vault and then pass it as commandline/env variable.
Some additional points of concern are expiry of token (though we can have non-expiry token not sure about pros/cons), restarts, safety issues, instantiating new microservices. There is no provision of dynamic tokens/authentication at cloud config side.
For milestone release i found that the client side encryption/decryption is not working as of now using recommended inclusion of RSA jar. Here is the ticket i opened.
https://github.com/spring-cloud/spring-cloud-config/issues/805#issuecomment-332491536
These are some of my observations, please share your thoughts if there is any case study/whitepaper that address spring cloud config vault usecases, setup and challenges for production micro-services environment.
Thanks
Thanks for reaching out to me. One think I would state is that the App Role backend utilizes two distinct tokens, and indeed spring-cloud-config-vault does indeed support this functionality, see: http://cloud.spring.io/spring-cloud-vault/single/spring-cloud-vault.html#_approle_authentication. I leverage vault in the same way I leverage config server, as per the documentation. I don't encrypt any values in my config, I just don't put them there. I put the secret values in vault and let it serve config. As long as keys don't collide, you don't have to mess with anything, otherwise you may need adjust the priority so vault wins, again see the documentation that I pointed to above. I wouldn't mess with encryption/decryption in spring-cloud-config personally. Because you have to check the keys into SCM or distribute them to your teams for local development, you lose the value of having these keys IMO.
Thanks Spring Cloud vault does support but not Spring cloud config with Vault. Only way seems to be passing X-Config-token from Microservice to Config Server. We are bit skeptical with this part of generating tokens manually or through script. Especially with containerization and when new MS instances will be spawn. Not sure about this approach especially in production setup.
I have a case where the RunDeck scripts do need some credentials in order to run. Obviously we do not want to store these in the job definitions because these are visible and also stored in SCM.
While I was able to use the Key Storage vault to put these secrets in, I was not able to find a way to access them from the job itself.
Rundeck 2.6.2 (released 2015-12-02) allows you to specify key storage secrets as default values for secure job options. See Secure Options using Key Storage