I have an API Token that I am storing in an Azure Key Vault. I want to provide access to this secret so it can be used from various pipelines. But I don't necessarily want anyone editing/executing those pipelines to be able to retain the Token. If a pipeline just echoes the secret value then won't the build agent logs simply display it, making the whole key vault process a useless step?
If that is the case, is there a way to safeguard a value if I don't necessarily want to trust everyone that can create pipelines?
It depends on how the secret is being retrieved and turned into a variable. In general, secrets retrieved via the standard Azure Pipelines tasks will be masked as **** when being displayed.
However, there's nothing to stop someone malicious from base64 encoding the value and printing it. Or ROT encoding it. Or reversing it. Or splitting the string in half and printing both halves. Or running a script that does this: az keyvault secret show. And so on and so forth.
If there's a secret value in memory as plain text, and people have the ability to manipulate the environment on which the secret is stored in memory as plain text, it's trivial to get the plaintext value of the secret. That's true for any computer under any circumstances.
If this is a concern, by far the best approach is to not have pipelines read secrets. If your application can be configured to integrate directly with keyvault and retrieve the secrets as needed at runtime, there's no need to have the pipeline retrieve the secret as an intermediate step.
Related
I have a stack that utilizes AWS KMS key for, I believe, secrets and state encryption in Pulumi stack configuration file Pulumi..yaml
Is it safe to expose this in a public repository? As I understand secrets are stored within stack configuration files as well in encrypted form; would it be reasonably safe to expose those as well in a public repository?
How exactly this key is generated and what are the inner mechanics behind secrets management in Pulumi?
Yes, exposing these values in your code is complete safe.
The key is asymmetrically encrypted using your key provider, in this case an AWS KMS key, its only possible to retrieve the value if someone has access to the AWS KMS key itself to decrypt the value, and even then, is a bit of a hoop jumping exercise.
I expose these values myself in source control, so you should be absolutely okay to leave them in your repo
I have created CMK in ADF. I dont know how to consume CMK in ADF, either pipelines or dataflow.
When tried to add the pipeline, I wont get option to select the CMK. Any information is helpful.
Let's review the entire process.
First we can use to Azure key vault generate a RSA 2048 key.
Add the key to the Azure Data Factory.
By default, data is encrypted with a randomly generated Microsoft-managed key. After we generate a RSA 2048 key via key vault, the data will be encrypted by Customer-managed keys(CMK).
According to this document, we can know Customer-managed keys(CMK) is a type of Server-side encryption. It gives us control over the keys, including Bring Your Own Keys (BYOK) support, or allows you to generate new ones.
Similar to transparent Data Encryption with customer managed keys in Azure SQL Database.
So we can conclude that Customer-managed keys(CMK) is transparent to users.This encryption only acts on the network layer or the transport layer. In other words, there is no different to us.
I'm not finding any answers to this on Google but I may just not know what terms to search for.
In a CRD, is there a way to define a field in the spec that is a secret (and therefore shouldn't be stored in plain text)? For example, if the custom resource needs to have an API token included in it, how do you define that in the CRD?
One thought I had was to just have the user create a Secret outside of the CRD and then provide the secret's name in a custom resource field so the operator can query it from the K8s API on demand when needed (and obviously associated RBAC needs to be configured so the operator has read access to the Secret). So the field in the CRD would just be a normal string that is the name of the target Secret.
But is there a better way? Any existing best practices around this?
You do indeed just store the value in an actual Secret and reference it. You'll find the same pattern all over k8s. Then in your controller code you get your custom object, find the ref, get that secret, and then you have your data.
The kubernetes dashboard allows one to see secrets in plain text (not base64 encoded) and make an easy change to any key-value pair within a Secret. I cannot find a way to easily make a similar change on the command line.
My best attempt has been to write a script which uses kubectl get secret to pull all of the data in Json format, grab each key-value pair, base64 decode the values, update the one I actually want, then feed them all back in to kubectl apply. After running into multiple issues I figured there is probably a kubectl option that I'm overlooking which will allow me to update just one key-value pair in a given Secret.
How can I do this?
Usually you would have your secrets manifest stashed somewhere where it is secure.
I dont usually change the secrets using the dashboard, but instead do a kubectl apply -f mysecret.yaml. the mysecret.yaml file keeps the latest and greatest values. No in-place editing. This way you get consistency across deployments.
Have been using Kubernetes secrets up to date.
Now we have ConfigMaps as well.
What is the preferred way forward - secrets or config maps?
P.S. After a few iterations we have stabilised at the following rule:
configMaps are per solution domain (can be shared across microservices within the domain, but ultimately are single purpose config entries)
secrets are shared across solution domains, usually represent third party systems or databases
I'm the author of both of these features. The idea is that you should:
Use Secrets for things which are actually secret like API keys, credentials, etc
Use ConfigMaps for not-secret configuration data
In the future, there will likely be some differentiators for secrets like rotation or support for backing the secret API w/ HSMs, etc. In general, we like intent-based APIs, and the intent is definitely different for secret data vs. plain old configs.
One notable difference in the implementation is that kubectl apply -f:
ConfigMaps are "unchanged" if the data hasn't changed.
Secrets are always "configured" - even if the file hasn't changed
Both, ConfigMaps and Secrets store data as a key value pair. The major difference is, Secrets store data in base64 format meanwhile ConfigMaps store data in a plain text.
If you have some critical data like, keys, passwords, service accounts credentials, db connection string, etc then you should always go for Secrets rather than Configs.
And if you want to do some application configuration using environment variables which you don't want to keep secret/hidden like, app theme, base platform url, etc then you can go for ConfigMaps