When using Kubernetes .yml files, I can do the following:
$ cat configmap.yml
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
foo: ${FOO}
bar: ${BAR}
static: doesNotChange
$ export FOO=myFooVal
$ export BAR=myBarVal
$ cat configmap.yml | envsubst | kubectl apply -f -
This would replace ${FOO} and ${BAR} in the configmap.yml file before actually applying the file to the cluster.
How could I achieve the very same behavior with a Kubernetes secret which has it's data values base64 encoded?
I would need to read all the keys in the data: field, decode the values, apply the environment variables and encode it again.
A tool to decode and encode the data: values inplace would be much appreciated.
It is actually possible, to store the secret.yml with stringData instead of data which allows to keep the files in plain text (SOPS encryption is still possible and encouraged)
$ cat secret.yml
apiVersion: v1
kind: Secret
metadata:
name: test-secret
namespace: default
type: Opaque
stringData:
dotenv: |
DATABASE_URL="postgresql://test:test#localhost:5432/test?schema=public"
API_PORT=${PORT}
FOO=${FOO}
BAR=${BAR}
$ export PORT=80
$ export FOO=myFooValue
$ export BAR=myBarValue
$ cat secret.yml | envsubst | kubectl apply -f -
A plus is for sure, that this not only allows for creation of the secret, but updating is also possible.
Just for documentation, here would be the full call with SOPS:
$ sops --decrypt secret.enc.yml | envsubst | kubectl apply -f -
Related
I've a kubernetes sealed secret with encrypted data in it. How can I edit the sealed secret like editing a deployment using command "kubectl edit deployment".
I know kubectl edit secret works on normal secrets not on sealed secrets.
apiVersion: bitnami.com/v1alpha1
kind: SealedSecret
metadata:
creationTimestamp: null
name: my-secret
namespace: test-ns
spec:
encryptedData:
password: AgCEP9MHaeVGelL3jSbnfUF47m+eR0Vv4ufzPyPXBvEjYcJluI0peLgilSqWUMnDLjp00UJq186nRYufEf7Bi1Jxm39t3nDbW13+wTSbblVzb9A2iKJI2VbxgTG/IDodNFxFWdKefZdZgVct2hTsuwlivIxpdxDZtcND4h+Cx8YFQMUpT5oO26oqISzRTh5Ewa6ehWtv6krfeFmbMQCF70eg0yioe6Op0YaQKloiFVInc1JK5KTR5iQYCeKb2R0ovKVa/lipbqjHCYSRhLR/3q5wz/ZuWz7g7ng6G9Q7o1pVv3udQYUvp2B6XvK1Nezc85wbhGgmuz5kcZUa36uF+eKMes6UdPcD7q58ndaj/0KWozdTAuk1OblV7mrUaK8Q45GIf+JqaBfzVt52INMT07P4MId/KB31sZDeE+OwEXhCDVTBAlxSRM0U9NjxDDb+mwUzxHNZHL1sY8M1YCoX+rr6n1+yW1HG42VHLCRzeBa2V31OFuQTNjoNxDEfUq+CSTRNDCmt8UvercSkgyM3mBa6JpHdkySllpqyEJDYKM1YvVRrjVvg1qGTF5dOCx6x3ROXnZtA3NBIafTu0+pHovVo+X7nUkl7hyupd0KKzBG+afgNpYQOxeuei5A+o++o92G5lexxk2v4bQt6ANYBxMlvT0LdBUW9e/L2y+TuNAHL23Xa/aTq1lagNBi9JTowX0lx0br2CqDbKg==
username: AgArwZm3qh83Fpzles1r/PjTDKQ2/SZ482IKC84T72/kI4M29aG2VT4SCXcqbmtVDYuVUN0wTbsFYsnwY1DSRrL4oup2xRg6N34IxHjj0ZtF1q0YtBKIM/DPcF2bBVAYc9/vOI0L3+VVSF9r93XYEMUWX6hY9eHa8VUHBM/Y65Sj3Il7Pmx/qoEcZ+e9UJhqWEJPotz6W5OMh/Al/QPJZknwUulM4coZ3C0J4TmrBVexPturcRCimDEQnd9UitotnGDoNAp2O28ovhXoImNsJBhNK5LykesRxEfIp4UJOb3I0CpLdoz9khEcb2r31j+KTtxifLez7Rg3Pg7BGpR3EKC3INZWrR8S/aUm5u/dP12ELgW3nq4WbafRitrZcHhLFZkHma/Er8miFbuTXvpFcXE1g+BnG2vIs4kHSl2QcP32HPGKHJJt0KEd1dUJrXXTjS9eXHJ2KsA5DZk4TcFA5dPAG76ZdKo0GCIQwvNeT0Ao4ntqmeOiijAQgmhXdCtD2WVavXi54h0f8F2ue6b0mBFCgTGKZyypjbXznzB/MPAZxgIu+UWQzV1CczwKlitPy638s/9iSan2/u2rhKu2SP0JFMZ6pPnfO51nMpDHtCDGFc1unjsjM4ZpnNXtaQJJmXo7Hw0L4dW2/N3uxCfxNtmYuBxE1t4GCefSUCTIleDgmAbB00nKkja+ml9bidcxawlIgHnoq/XNCqy2R3PkEw==
template:
data: null
metadata:
creationTimestamp: null
name: my-secret
namespace: test-ns
You can update the existing SealedSecret by using --merge-into option in SealedSecret service. You can simply copy & paste the encrypted data into a json and merge this to the existing SealedSecret like this
$ echo -n bar | kubectl create secret generic mysecret --dry-run=client --from-file=foo=/dev/stdin -o json \
| kubeseal > mysealedsecret.json
$ echo -n baz | kubectl create secret generic mysecret --dry-run=client --from-file=bar=/dev/stdin -o json \
| kubeseal --merge-into mysealedsecret.json
When the sealedsecret needs to decrypt and work the same as normal secrets in kubernetes then both sealedsecrets and secrets need to be in the same namespace.
For more detailed information refer this official sealedsecrets github page
To know more about the usage of SealedSecret refer to this document
I have a file with three configmaps in it, like the one below.
apiVersion: v1
data:
TEST: "one"
kind: ConfigMap
metadata:
name: test-config-one
---
apiVersion: v1
data:
TEST: "two"
kind: ConfigMap
metadata:
name: test-config-two
---
apiVersion: v1
data:
TEST: "three"
kind: ConfigMap
metadata:
name: test-config-three
I'm trying to apply only test-config-three to the cluster. I know I can break that out into its own file and run kubectl apply -f test-config-three.yaml, but is there a way to do that without having to create a new file?
I was hoping to be able to do something like:
cat file.yml | yq <get only test-config-three> | kubectl apply -f -
But yq doesn't seem to support finding a single resource in a file. I also looked at tools like kubesplit but they tend to output all resources to separate files.
Is there a way to isolate and output a single resource from a yaml file containing multiple resources without creating a new file?
Update
Thanks to #Inian's answer below, I was able to get this full command working.
cat file.yml | yq e 'select(.data.TEST == "three")' - | kubectl apply -f -
There are two versions of yq implemented, one in Python and one in Go as I've highlighted in my answer at How can I parse a YAML file from a Linux shell script?
Using the Python version - kislyuk/yq
yq -y 'select(.data.TEST == "three")' yaml
Go version - mikefarah/yq
yq e 'select(.data.TEST == "three")' yaml
If you have python you can try following
export MYFILE=file.yml
export DOC_NUMBER=2
python3 -c "import yaml; print(yaml.dump(list(yaml.safe_load_all(open('"$MYFILE"')))["$DOC_NUMBER"]))" | kubectl apply -f -
yaml.safe_load_all(open('"$MYFILE"')) loads all the documents in the yaml file into a list. Then you are selecting "$DOC_NUMBER" document with ["$DOC_NUMBER"].
yaml.dump would dump the loaded object back into yaml format which is then printed.
When managing secrets in argocd, I encode the value in argocd-secret with base64 and set it to manifest.
In that case, although I use kusotomize, build and apply it, secret encoded by base64 will be encoded to base64 further.
$ echo -n "clientid" | base64
Y2xpZW50aWQ=
$ echo -n "clientsecret" | base64
Y2xpZW50c2VjcmV0
---
apiVersion: v1
kind: Secret
metadata:
name: argocd-secret
type: Opaque
stringData:
dex.github.clientID: Y2xpZW50aWQ=
dex.github.clientSecret: Y2xpZW50c2VjcmV0
If you use kustomize build to build and check the difference, it looks like this
$ kustomize build --load_restrictor none overlays/dev/ap-northeast-1/argocd | k diff -f -
+ dex.github.clientID: WTJ4cFpXNTBhV1E9
+ dex.github.clientSecret: WTJ4cFpXNTBjMlZqY21WMA==
I don't understand why something encoded by base64 is encoded further.
I'm going to ask for someone's help.
Thanks.
I reproduced your case and it looks like it isn't further encoded by kustomize but by kubectl (either by kubectl client itself or by kube-apiserver performing the operation requested by e.g. kubectl apply command).
And here you can find a fragment that sheds some light on why this is actually happening:
The Secret contains two maps: data and stringData. The data field is
used to store arbitrary data, encoded using base64. The stringData
field is provided for convenience, and allows you to provide secret
data as unencoded strings.
The data field serves for storing the data already encoded using base64 and value of this field isn't further encoded when we apply the Secret. The stringData however behaves quite differently. It allows you to provide secret data as unencoded strings and for that reason it is further encoded when you run kubectl apply. In your example you use stringData field and although it contains already encoded data, it is treated as any other string and in consequence is encoded again.
Solution: simply use data rather than stringData map in your Secret and your base64 encoded strings won't be encoded again during the resource creation.
It's actually secret controller who encodes the string. So, you can't pass an encoded string into a yaml. You need to leave it in plain text:
$ echo -n demo | base64
ZGVtbw==
$ echo -n ZGVtbw== | base64
WkdWdGJ3PT0=
$ kubectl create secret generic demo --from-literal=key=demo
secret/demo created
$ kubectl create secret generic demo2 --from-literal=key=ZGVtbw==
secret/demo2 created
$ kubectl get secret demo demo2 -oyaml
apiVersion: v1
items:
- apiVersion: v1
data:
key: ZGVtbw== <- word demo got encoded
kind: Secret
metadata:
creationTimestamp: "2020-06-04T13:41:27Z"
name: demo
namespace: default
resourceVersion: "10118413"
selfLink: /api/v1/namespaces/default/secrets/demo
uid: bc39444c-e4f6-43ba-b151-705e15811831
type: Opaque
- apiVersion: v1
data:
key: WkdWdGJ3PT0= <- encoded string of demo got encoded further
kind: Secret
metadata:
creationTimestamp: "2020-06-04T13:41:47Z"
name: demo2
namespace: default
resourceVersion: "10118469"
selfLink: /api/v1/namespaces/default/secrets/demo2
uid: 2fe63343-06b4-4f47-94cd-9fe8e12fb388
type: Opaque
kind: List
metadata:
resourceVersion: ""
selfLink: ""
Note: this was not always like that. And it can actually change again. Back in the days, you had to encode it, then you must not, then again...
I want to edit a configmap from aws-auth during a vagrant deployment to give my vagrant user access to the EKS cluster. I need to add a snippet into the existing aws-auth configmap. How do i do this programmatically?
If you do a kubectl edit -n kube-system configmap/aws-auth you get
apiVersion: v1
data:
mapRoles: |
- groups:
- system:bootstrappers
- system:nodes
rolearn: arn:aws:iam::123:role/nodegroup-abc123
username: system:node:{{EC2PrivateDNSName}}
kind: ConfigMap
metadata:
creationTimestamp: "2019-05-30T03:00:18Z"
name: aws-auth
namespace: kube-system
resourceVersion: "19055217"
selfLink: /api/v1/namespaces/kube-system/configmaps/aws-auth
uid: 0000-0000-0000
i need to enter this bit in there somehow.
mapUsers: |
- userarn: arn:aws:iam::123:user/sergeant-poopie-pants
username: sergeant-poopie-pants
groups:
- system:masters
I've tried to do a cat <<EOF > {file} EOF then patch from file. But that option doesn't exist in patch only in the create context.
I also found this: How to patch a ConfigMap in Kubernetes
but it didn't seem to work. or perhaps i didn't really understand the proposed solutions.
There are a few ways to automate things. The direct way would be kubectl get configmap -o yaml ... > cm.yml && patch ... < cm.yml > cm2.yml && kubectl apply -f cm2.yml or something like that. You might want to use a script that parses and modifies the YAML data rather than a literal patch to make it less brittle. You could also do something like EDITOR="myeditscript" kubectl edit configmap ... but that's more clever that I would want to do.
First, note that the mapRoles and mapUsers are actually treated as a string, even though it is structured data (yaml).
While this problem is solvable by jsonpatch, it is much easier using jq and kubectl apply like this:
kubectl get cm aws-auth -o json \
| jq --arg add "`cat add.yaml`" '.data.mapUsers = $add' \
| kubectl apply -f -
Where add.yaml is something like this (notice the lack of extra indentation):
- userarn: arn:aws:iam::123:user/sergeant-poopie-pants
username: sergeant-poopie-pants
groups:
- system:masters
See also https://docs.aws.amazon.com/eks/latest/userguide/add-user-role.html for more information.
Here is a kubectl patch one-liner for patching the aws-auth configmap:
kubectl patch configmap -n kube-system aws-auth -p '{"data":{"mapUsers":"[{\"userarn\": \"arn:aws:iam::0000000000000:user/john\", \"username\": \"john\", \"groups\": [\"system:masters\"]}]"}}'
I want to edit the configmap and replace the values. But it should be done using a different YAML in I ll specify overriding values as part of that file.
I was trying using kubectl edit cm -f replace.yaml but this didn't work so i want to know the structure in which the new file should be.
apiVersion: v1
kind: ConfigMap
metadata:
name: int-change-change-management-service-configurations
data:
should_retain_native_dn: "False"
NADC_IP: "10.11.12.13"
NADC_USER: "omc"
NADC_PASSWORD: "hello"
NADC_PORT: "991"
plan_compare_wait_time: "1"
plan_prefix: ""
ingress_ip: "http://10.12.13.14"
Now lets us assume NADC_IP should be changed and So I would like to know how should be structure of the YAML file and using which command it can be served?
The override taking place should only be during helm test for example when i run
helm test <release-name>?
kubectl replace -f replace.yaml
If you have a configmap in place like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
should_retain_native_dn: "False"
NADC_IP: "10.11.12.13"
and you want to change the value of NADC_IP create a manifest file like this:
apiVersion: v1
kind: ConfigMap
metadata:
name: my-configmap
data:
should_retain_native_dn: "False"
NADC_IP: "12.34.56.78" # the new IP
and run kubectl replace -f replace.yaml
To update variable in configmap you need to take two steps:
First, update the value of variable:
kubectl create configmap <name_of_configmap> --from-literal=<var_name>=<new_value> -o yaml --dry-run | kubectl replace -f -
So in your case it will looks like this:
kubectl create configmap int-change-change-management-service-configurations --from-literal=NADC_IP=<new_value> -o yaml --dry-run | kubectl replace -f -
Second step, restart the pod:
kubectl delete pod <pod_name>
App will use new value from now. Let me know, if it works for you.
kubectl get cm {configmap name} -o=yaml --export > filename.yaml
You can try this it will give you yaml format
kubectl get configmap
int-change-change-management-service-configurations -o yaml
You can copy the content and replace it inside new yaml file and apply the changes
EDIT : 1
If you want to edit over terminal you can run
kubectl edit configmap {configmap name}
It will use vim editor and you can replace value from terminal using edit command.
EDIT : 2
kubectl get cm {configmap name} -o=yaml --export > filename.yaml