What is the best practice for storing password and API keys with Chef? It's really tempting to store database passwords, AWS api keys, and other sensitive credentials as Chef Server Attributes for use in recipes -- but what about security considerations? What's the best practice for this?
From the #chef IRC channel, many people store this kind of data in a data bag on the chef server.
For example, a data bag might be 'aws', with an item 'main', referring to the primary AWS account. Separate keys in the item would be for each particular value. E.g.:
{
"id": "main",
"aws_secret_key": "The secret access key",
"aws_access_key": "The access key"
}
You may also be interested in encrypted data bags. I wrote about them in more detail for managing postfix SASL authentication.
Update: I've written blog posts about Chef Vault on my blog and sysadvent.
This question is old and has no accepted answer, however, the correct answer to this question is that Chef allows the use of Encrypted Data Bags for storing sensitive data in Data Bags.
I think Hashicorp's Vault is really promising as a way to dynamically retrieve encrypted information and leave behind some of the oddities of Chef workflow in this area.
This is an interesting post that starts to touch the subject.
https://www.hashicorp.com/blog/using-hashicorp-vault-with-chef.html
The best practice is to keep keys and passwords in chef data_bags. A data bag contains databag items. Individual data_bag item are in json format.
For exmaple:
{
/* This is a supported comment style */
// This style is also supported
"id": "ITEM_NAME",
"key": "value"
}
Encrypt Data Bag Item:
data bag item may be encrypted using shared secret encryption. This allows each data bag item to store confidential information (such as a database password or ssh keys) or to be managed in a source control system (without plain-text data appearing in revision history). This can be done as follow:
Crete Secret Keys:
Create a secret key called encrypted_data_bag_secret for example
$ openssl rand -base64 512 | tr -d '\r\n' > encrypted_data_bag_secret
where encrypted_data_bag_secret is the name of the file which will contain the secret key
Encrypt the data_bag:
A data bag item is encrypted using a knife command similar to:
$ knife data bag create passwords mysql --secret-file /tmp/my_data_bag_key
where “passwords” is the name of the data bag, “mysql” is the name of the data bag item, and “/tmp/my_data_bag_key” is the path to the location in which the file that contains the secret-key is locate
Verify Encryption:
When the contents of a data bag item are encrypted, they will not be readable until they are decrypted. Encryption can be verified with a knife command similar to:
$ knife data bag show passwords mysql
Decrypt data Bag:
An encrypted data bag item is decrypted with a knife command similar to:
$ knife data bag show --secret-file /tmp/my_data_bag_key passwords mysql
Chef Encrypted data_bags is indeed a legitimate solution.
Adding to that, you can also use a ruby Gem that allows you to encrypt a Chef Data Bag Item using the public keys of a list of chef nodes. This allows only those chef nodes to decrypt the encrypted values.
cf. https://github.com/Nordstrom/chef-vault
Chef Vault can be a good choice. It provides simple interface for storing encrypted data on chef-server, access management. Upload, edit, update data with knife vault ... commands.
To get data from recipe use ChefVault::Item.load command
chef_gem "chef-vault"
require 'chef-vault'
item = ChefVault::Item.load("passwords", "root")
item["password"]
To set users, which can update data use knife vault_admins property.
knife[:vault_admins] = [ 'example-alice', 'example-bob', 'example-carol' ]
I've never tried databags, but that's probably because I find everything apart from chef-solo a little too complicated. Which is why I'm using chef recipies with a service called Scalarium.
So the issue with passwords, or e.g. private keys and all kinds of other credentials is a pretty tough one. I too have a bunch of recipes where passwords need to be created, or set correctly.
Usually what I do is, I specify what the scalarium folks call custom json. This json is similar to the node.json some people give to chef-solo using chef-solo -j node.json.
So e.g. in my custom json on Scalarium web interface, I have the following:
{"super_secure_password":"foobar"}
What this does is, my super secure password is available during my chef run in node[:super_secure_password] and I can use it in recipes or templates.
This works fine as long as I only deploy my server using Scalarium but we also use our recipes in local vagrant boxes for a development environment and easier testing. And when I use vagrant (or even chef-solo by itself), I don't have access to the custom json on Scalarium.
This is what I do to fix that, in my_recipe/attributes/default:
set_unless[:super_secure_password] = "test123"
This means that when my recipe is run outside of scalarium, the password is still available in node[:super_secure_password] and my recipes work and so on. When the recipe is executed in the scalarium context, it will not override what they provide.
Currently most widely used approach and for most cases secure enough is to use chef-vault.
It uses shared secret to encrypt your data (similarity to chef-encrypted databag). This shared secret is encrypted for every client and/or user that will use it(if You allow to use it).
Benefits:
on test environment you can use unencrypted data
One doesn't store shared secret as a plain text
One may grant access only few out their servers to read and write some databags
Example
export EDITOR=vi #sets your favourite text editor
knife vault create secret_data john_doe --admins "admin" --search "*:*" --mode client
Command above creates in secret_data databag item: john_doe that can modified by admin and used by all clients. After that command EDITOR will open so you can type o paste your secret data(in json ).
Search query can be: "role:basic" - Which means only servers with role basic can read this data
knife vault need some extra installation
In Your Cookbook
chef_gem 'chef-vault' do
compile_time true if respond_to?(:compile_time)
end
require 'chef-vault'
item = ChefVault::Item.load("secret_data", "john_doe")
item["password"]
and in metadata.rb:
depends 'chef-vault', '1.3.0'
more info here: https://blog.chef.io/2016/01/21/chef-vault-what-is-it-and-what-can-it-do-for-you/
and here: https://github.com/chef/chef-vault
I would Suggest use IAM role with chef provisioning
require 'chef/provisioning/aws_driver'
iam = AWS::Core::CredentialProviders::EC2Provider.new
puts iam.credentials.inspect
with_driver(
'aws:IAM:eu-west-1',
:aws_credentials => { 'IAM' => iam.credentials }
)
Related
We use vault to store our credentials, I've successfully grabbed S3 Access key ID and Secret Access key using the vault API, and used channelMap.put to create mappings: ${access_key} and ${secret_key}.
aws_s3_file_writer
However when I use these in the S3 file writer I get the error:
"The AWS Access Key Id you provided does not exist in our records."
I know the Access Key Id is valid, it works if I plug it in directly in the S3 file writer destination.
I'd appreciate any help on this. thank you.
UPDATE: I had to convert the results to a string, that fixed it.
You can try using the variable to a higher map. You can use globalChannelMap, globalMap or configurationMap. I would use this last one since it can store password not in plain text mode. You are currently using a channelMap, it scope is only applied to the current message while it is traveling through the channel.
You can check more about variable maps and their scopes in Mirth User guide, Section Variable Maps, page 393. I think that part of the manual is really important to understand.
See my comment, it was a race condition between Vault, Mirth and AWS.
MongoDB => Holochain Rust DHT
How to import, if possible
If I am using a different app backend, like mongo, and I get my holochain set up correctly and configured, is there a way to get the data from mongo to holochain? How would I do that?
Here is the question in context
Definitely technologically possible; you could write a nodejs script, fire up a Holochain container with the holochain-nodejs library, and import all the data as one agent. Then when users join the HC-based network, they vouch for their identity in some way and 'claim' all the data as theirs.
Here's a sketch of how it could look:
you (let's call you 'agent 0') import all the data.
For each user, you create an 'anchor' with the user's ID (I'll explain anchors in a
sec) and link each piece of data to the anchor.
You also record that
user's password hash as a private entry on your own source chain. A
user joins the network and is required to prove continuity of
identity.
They do this by using node-to-node messaging to send their
user ID and their password hash to you privately. You authorise them
to claim their identity by publishing an entry that says that "agent
public key x = user ID". (You would probably want to link from your
authorisation entry to their user ID anchor and their public key too,
for convenience's sake.)
The user collects all their data by asking
for all the links to their user ID anchor.
The user then publishes
each piece of their data to their own source chain as a way of
'claiming' ownership of it.
Now, every redundant copy of the data in
the DHT has two authors in its metadata fields -- you and the user
that actually owns the data. Peers validate that piece of data by
saying, "Is agent 0 already the author of this piece of data?
If so,
has agent 0 published an authorisation entry that says that the new
author of this data is allowed to claim/republish it?"
Problems with this approach (not insurmountable):
Agent 0 has to be online all the time cuz they never know when a new
user is going to sign up and try to claim their data. Agent 0 has to
import a ton of data. (I don't think it'd be vastly
time-prohibitive though)
For relational data, there's the chicken-and-egg problem of how to
create links if the data doesn't exist. I'm thinking not of linking
data to data -- that can be done on initial import -- but linking
data to humans, who now have a public key which might not exist on
the DHT yet because they haven't joined the network. That would
always have to happen per-user once they join, and it could create
some cyclic dependency problems.
Anchors
Re: anchors, an anchor is just a pattern that consists of a base and a link -- the base is a simple string, so it's easy for anyone who knows the string to find it by hash. It acts as, well, an anchor to hang links off of. That's why I'm recommending using it to connect legacy user IDs to pieces of content. You can get sample source code for implementing the anchor pattern at https://github.com/holochain/mixins/tree/master/anchors (note that this is for the legacy version of Holochain, so it's written in JavaScript).
( answer provided by
pauldaoust )
I'm developing an API with Sails, and now I need to securize some variables from an entity. Those variable will be accesed only from Admin or own user.
I have an structure like this:
Employee (contains your employee records)
fullName
hourlyWage
phoneNumber
accountBank
Location (contains a record for each location you operate)
streetAddress
city
state
zipcode
...
I need to encrypt phonenumber and accountbank, to avoid anyone to see the values of this fields in the DataBase. Only the owner or the admin.
How I can do that? Thanks
You are looking for a way to encrypt data so that people with no required access right could not see it.
The solution for that is not Sails.js specific and Node actually comes with tools to encrypt data :https://nodejs.org/api/crypto.html.
The key rule here is to always keep your secret password safe.
As for integration in your Sails.js application, I would use callbacks in Models. The official documentation provides a good example here : http://sailsjs.org/documentation/concepts/models-and-orm/lifecycle-callbacks
Basically you just define a function that will be called each time the record is about to be created, fetched or updated. You can then apply your encrypt/decrypt functions there.
This will encrypt/decrypt your phone numbers and bank account numbers automatically.
Regarding access control, you can use Sails' policies along with authentication to determine if the client has the right to access the resource. If not you can always remove attributes from the response sent back to the client.
I'm building a system that has a Java Swing front end accessed a postgres database. Prior to discovering Jasypt this week, I had originally planned to use Postgres' own encryption mechanism. It worked fine, but I also now wanted the passwords over the network to be encrypted, hence why I turned to Jasypt.
Problem is, I need a fixed password to be entered into my Postgres stored function. I.e. If the input password is 'aaa' then any other inputed password into the Postgres stored function (other than 'aaa') will not match.
Is there a way to get these two encryption mechanisms to work in tandem together or do I have to dump Postgres'?
My user table:
CREATE TABLE "user"
(
id serial NOT NULL,
cryptpwd text NOT NULL,
md5pwd text NOT NULL,
...
)
Encrypting password:
cryptedPassword = crypt(passwordIn, gen_salt('md5'));
md5Password = md5(passwordIn);
INSERT INTO "user"(username, cryptpwd, md5pwd, ...)
VALUES (usernameIn, cryptedPassword, md5Password, ...);
Decrypting password:
select ..... from "user" .... where username = usernameIn and cryptpwd = crypt(passwordIn, cryptpwd);
If I cannot get the two of them to work together then I would have to dump Postgres' mechanism as I need to have encryption over the network.
Also, with regards to the database connection string and database username and password (not using any framework ... plain old jdbc connection hopefully with SSL - yet to implement), I don't think I'll be able to use Jasypt because I'd need to decrypt it at database level. Would SSL alone be sufficient for this case?
Thanks.
I think SSL alone, on every piece of the path, would be sufficient. In LedgerSMB (although we are Perl-based) we do something different and rely on SSL protected links between servers and between servers and clients. There are a few things to think about with your approach though.
We actually pass the db username and password to the middleware from the client in re-usable format (plain text) over an SSL connection, and then use another SSL connection to log into PostgreSQL to authenticate this way. This works fine, but the problem areas we face are somewhat similar to the problem areas you will. These include:
Logging. Is it possible passwords will get accidently logged? This is a concern with LedgerSMB and we take what steps we can but a badly configured server or a tampered-with program could log usernames and passwords. In our case this comes primarily on the middleware level, but in your case, query logging could do this too, right?
Is it possible credentials can be re-used unintentionally? We prevent this in a couple of ways, but it is worth considering.
On the whole, we trust SSL. There isn't much to be gained from adding additional encryption beyond that, and key management adds a lot of complexity that is not worth the marginal gains IMO.
I found a question and that is : to implement a key-value server
User should be able to connect to server and be able to run command SET a = b.
On running command GET a, it should print b.
First of all, I didn't really understand what the question is all about.
In its simplest form, a Key-Value server is nothing more but a server that holds keys in a dictionary structure and associates a value with said key.
If it helps, you can think of a key as a variable name in a programming language or as an environment variable in the bash shell.
A client to the Key-Value server would either tell the server what value the key has, or request the current value of the key from the server.
As Ramon mentioned in his comment, memcached.org is such example of a Key-Value server.
Of course, the server can be much more complex that what I described above. Keys could be more than just values (for instance, objects) and the server/client could have a lot more functionality than the basic set/get.
Note that the term Key-Value server is very broad and doesn't mean anything concrete by itself. NoSQL systems make use of key-value stores, for example, so you could technically call any NoSQL database system a Key-Value server.