Is it possible to use AWS KMS and a tool like credstash without the use of EC2 or equivalent or does it rely solely on IAM roles?
I've got a server elsewhere where I am testing some things out and ultimately I will be looking at migrating an app to EC2 etc. to make use of scaling. But for now whilst I'm setting up my deployment pipeline etc. I wondered if it was still possible to make use of KMS on my non-aws provisioned server?
The only possible way I can think of is by installing the AWS CLI tools on the server in question. Does this sounds like the right approach?
What #Viccari said is correct (in the comments). In terms of what you want to do (store passwords), the AWS Parameter Store would be a good fit for you. See https://docs.aws.amazon.com/systems-manager/latest/userguide/systems-manager-paramstore.html for more information. The guide explicitly calls out your use-case:
Parameter Store offers the following benefits and features.
Use a secure, scalable, hosted secrets management service (No servers to manage).
In the end, if you end up using Parameter Store or KMS, you will need some sort of credentials stored somewhere to grab an AWS STS token to use to call the underlying AWS services. If working outside of AWS EC2, you will need the AWS Access Key and AWS Secret Key from an IAM user. If you are in EC2, the IAM instance role will magically provide you the credentials and use that role to call those AWS services. The AWS SDK does this for you behind the scenes.
But, as you state, you don't want to run this in EC2 (to save money, or other reasons). The quickest way to store these credentials is to have them in a un-tracked file (added to your .gitignore) you can source from as environment variables, which your program will then read. This allows you to do local testing, and easily run it in EC2
with zero code changes. See https://docs.aws.amazon.com/cli/latest/userguide/cli-environment.html for what variables to set. Note that this doc talks about the CLI; the SDK's follow the same behavior.
Related
We have a bunch of Windows server applications that currently handle secrets as follows; our apps are in C#.
We store them in settings files in code
We store them encrypted, using a certificate
The servers have this certificate with the private key, so they can decrypt the secret
We're looking at implementing Hashicorp Vault. It seems easy enough to simply replace the encrypt-store-decrypt with storing the secret in Vault in the KV engine, and just grabbing it in our apps - that takes that certificate out of the picture entirely. Since we're on-prem, I'll need to figure out our auth method.
We have different apps running on different machines, and it's somewhat dynamic (not as much as an autoscaling scenario, but not permanent - so we can't just assign servers to roles one time and depend on Kerberos auth).
I'm unsure how to make AppRole work in our scenario. We don't have one of the example "trusted platforms" or "trusted entities", there's no Nomad, Chef, Terraform, etc. We have Windows machines, in a domain, and we have a homegrown orchestrator that could be queried to say "This machine name runs these apps", so maybe there's something that can be done there?
Am I in "write your own auth plugin" territory, to speak to our homegrown orchestrator?
Edit - someone on Reddit suggested that this is a simple solution if our apps are all 1-to-1 with the Windows domain account they run under, because then we can just use kerb authentication. That's not currently the way we're architected, but we've got to solve this somehow, and that might do it nicely.
2nd edit - replaced "services" with "apps", since most of our services aren't actually running as Windows services, just processes. The launcher is a Windows service but the individual processes it launches are not.
How about Group Managed Service Accounts?
https://learn.microsoft.com/en-us/windows-server/security/group-managed-service-accounts/group-managed-service-accounts-overview
Essentially you created one "trusted platform" (to your key vault service).
Your service can still has its own identity but delegation to the gMSA when you want to retrieve the secrets.
For future visibility, here's what we landed on:
TLS certificate authentication. Using Vault, we issue a handful of certs, each will correspond to a security policy/profile, so that any machine that holds that certificate will be able to authenticate and retrieve the secrets they should have access to.
Kerberos ended up being a dead-end for two reasons. The vault.exe agent (which is part of this use case) can't use the native Windows Kerberos SSPI, so we'd have to manage and distribute keytab files. Also, if we used machine authentication, it would blow up our client count (we're using the cloud-hosted HCP Vault, where pricing is partially based on client count).
Custom plugins can't be loaded into the HCP, of course
Azure won't work, it requires Managed Identities which you can't assign to on-prem machines. Otherwise this might have been a great fit
I have my production database with some specific master password and a specific user. My DB is AWS RDS with Postgres.
I'm automatically cloning it every day to some dev environment. Where I need multiple developers to have access to it, but they should not have access to the production environment.
How can I give it a new, non-production password during cloning? I can obviously use some automatic tool I can write. But I prefer something simpler, or optionally to use AWS API
You can use IAM to provide access to any RDS instance without having to share any passwords.
https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.html
Spring Boot - 2.0.0.M3
Spring cloud - Finchley.M1
I want to know if someone is using Spring Cloud config server with both vault and git support in a production setup using Database storage backend.
I have evaluated Spring cloud config using vault and contemplating whether to go for Oracle JCE to encrypt username/pwd or Vault and seek suggestions on the same. we are working on Springboot/microservices.
Following are my findings -
Vault will introduce an additional layer and thus will introduce additional usecases of security, auditing while communicating with Vault.
Spring cloud Config actuator endpoints are broken for the milestone release at this point for generation of encrypted values and /encrypt /decrypt may not work if we go for Oracle JCE support so we generate encrypted values through stable versions.
We do not wish to use consul server and are trying to use Cassandra as Storage backend.
I used Vault Authentication backend using AppRole and generated a Token (different from root token as it's unsafe to use the same) with read permissions. However, Spring Cloud config at the moment support only Token based authentication from client side. That means we first generate token from Vault and then pass it as commandline/env variable.
Some additional points of concern are expiry of token (though we can have non-expiry token not sure about pros/cons), restarts, safety issues, instantiating new microservices. There is no provision of dynamic tokens/authentication at cloud config side.
For milestone release i found that the client side encryption/decryption is not working as of now using recommended inclusion of RSA jar. Here is the ticket i opened.
https://github.com/spring-cloud/spring-cloud-config/issues/805#issuecomment-332491536
These are some of my observations, please share your thoughts if there is any case study/whitepaper that address spring cloud config vault usecases, setup and challenges for production micro-services environment.
Thanks
Thanks for reaching out to me. One think I would state is that the App Role backend utilizes two distinct tokens, and indeed spring-cloud-config-vault does indeed support this functionality, see: http://cloud.spring.io/spring-cloud-vault/single/spring-cloud-vault.html#_approle_authentication. I leverage vault in the same way I leverage config server, as per the documentation. I don't encrypt any values in my config, I just don't put them there. I put the secret values in vault and let it serve config. As long as keys don't collide, you don't have to mess with anything, otherwise you may need adjust the priority so vault wins, again see the documentation that I pointed to above. I wouldn't mess with encryption/decryption in spring-cloud-config personally. Because you have to check the keys into SCM or distribute them to your teams for local development, you lose the value of having these keys IMO.
Thanks Spring Cloud vault does support but not Spring cloud config with Vault. Only way seems to be passing X-Config-token from Microservice to Config Server. We are bit skeptical with this part of generating tokens manually or through script. Especially with containerization and when new MS instances will be spawn. Not sure about this approach especially in production setup.
We have an application that is deployed into CloudFoundry/Bluemix. The application reads its database connections from the VCAP_SERVICES environment variable. The db password stored in the environment variable is encrypted and we decrypt it when the application boots up.
We are looking at Spring Cloud Service Connectors. Do the cloud connectors provide any hook, so that we can decrypt the password from VCAP_SERVICES before the DataSource instance is created?
Why do you want to do this? Where does the app get its decryption key from? If it's hard-coded in the app, that's an antipattern that will make it hard to rotate the key. If it's through an environment variable, then it's no more secure than storing the database credentials unencrypted as services in Cloud Foundry - services in CF are nothing more than domain-specific groups of environment variables. I can't see that encrypting them adds any security.
To answer the question: Not out-of-the-box, but you could probably intercept the flow of Spring components that act on the environment variables that Cloud Foundry provides to your app.
The abstract class that creates ServiceInfo instances is CloudFoundryServiceInfoCreator. You could look at maybe providing a custom implementation of this? There is a blog post describing how Spring Cloud Connectors works. You might be able to extend CloudFoundryConnector too.
In the time since this question was answered, AWS Tools for Powershell has been released and I basically have the same problem: I have an RDS snapshot on one AWS account that I would like to transfer to another.
So far I've been able to select the snapshot that I want with the Get-RDSDBSnapshot cmdlet, and I'd like to take that Amazon.RDS.Model.DBSnapshot object and use it in the other account.
I've been looking around and I think the Restore-RDSDBInstanceFromDBSnapshot cmdlet (maps to rds-restore-db-instance-from-db-snapshot) might be what I'm looking for, but I'm not confident that I understand its behavior -- can this cmdlet be used to take my snapshot from my first account, and restore it to an instance in the second account?
I'm specifically concerned if there are any account-specific details in a Snapshot object or the handling of the cmdlet that would prevent that data from moving across accounts. I would be okay with a more general solution than powershell, if one exists.
Update 2015/10/29:
AWS has added native support for this functionality since my original posting (link to announcement). This is supported for unencrypted MySQL, Oracle, SQL Server, and PostgreSQL.
You are given the option to share your RDS snapshot publicly, or privately (by managing specific AWS Account IDs with permission to view your snapshot). By default, snapshots can be privately shared with up to 20 accounts.
This can be managed from the RDS console by clicking 'Snapshots (left navigation bar) > Share Snapshot (top toolbar)', which leads you to the following UI:
This is also available in the RDS API and CLI.
Original Answer:
I also posted this to the AWS Developer Forums, and got a response from PhilP#AWS. It seems like we can't do this at all, via powershell or any other means. He did have a couple of alternate suggestions, though:
It's not possible to directly share an RDS Snapshot from one account
to another. However I can make a couple of suggestions here (depending
on your current configuration):
If your RDS Instance is publicly accessible:
Launch a new RDS DB onto your second account
Install the appropriate DB management tools onto a PC, and give this PC network access to both RDS instances (security groups and DB user access for read and write)
Using the database management tools to copy the data from one DB to the other DB
Copy data through an EC2 instance as an intermediary:
Launch an EC2 instance configured with appropriate DB server software
Copy the RDS DB Data from your RDS instance up to your EC2 instance
Then launch your new RDS instance into the second account
Configure appropriate access (security groups and DB user access for read and write)
Copy the database data from your EC2 instance to your newly created RDS instance
My RDS instance isn't publicly accessible, and of his suggestions the EC2 solution would be preferable. We could alternate back to using a mysqldump, per the Server Fault solution.
Edit: I wanted to update that I've successfully been able to implement the EC2 intermediary suggestion. This can be automated several ways, but the solution I chose involved passing a bash script to the (linux AMI) EC2 instance as user-data, and the details of data transfer were handled in the script.
This solution ended up being fairly cost-effective, with the caveat that you want the RDS instance and the EC2 instance to be in the same availability zone. This is in large part because data transfer between RDS-EC2 in the same availability zone is free with a private IP address.
Amazon finally made it possible to accomplish this. You can share the snapshot with another account using the Edit-RDSDBSnapshotAttribute cmdlet (example here), then you can restore it to an account the snapshot was shared with using the Restore-RDSDBInstanceFromDBSnapshot cmdlet.
You can even share encrypted snapshots now. Here's a good walkthrough on how to do that.