I cannot access a secret I created. I inserted a secret in K8s secret store and am simply trying to test access to it with this yaml...
apiVersion: v1
kind: Namespace
metadata:
name: space1
---
apiVersion: v1
kind: Pod
metadata:
name: space1
namespace: space1
spec:
containers:
- name: space1-pod
image: repo/python-image:latest
imagePullPolicy: Always
command: ['sh', '-c', 'echo "Username: $USER" "Password: $PASSWORD"']
env:
- name: USER
valueFrom:
secretKeyRef:
name: tool-user
key: username
- name: PASSWORD
valueFrom:
secretKeyRef:
name:tool-user
key: password
restartPolicy: Always
The status of the "pod is waiting to start: CreateContainerConfigError". And I receive this error...
Error: secret "tool-user" not found
Despite the result I get from "kubectl get secrets" which clearly shows...
NAME TYPE DATA AGE
tool-user Opaque 2 4d1h
kubectl get secrets shows secrets from a default namespace, add -n space1 to see secrets from the namespace your pod runs in.
secrets are namespaced objects. Make sure the secret "tool-user" is created on the "secret1" namespace.
Related
I have below ConfigMap code which pulls secrets from GSM.
kind: ConfigMap
metadata:
name: db-config
labels:
app: poc
data:
entrypoint.sh: |
#!/usr/bin/env bash
set -euo pipefail
echo $(gcloud secrets versions access --project=<project> --secret=<secret-name>) >> /var/config/dburl.env
---
apiVersion: v1
kind: Pod
metadata:
name: poc-pod
namespace: default
spec:
initContainers:
- image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
name: init
command: ["/tmp/entrypoint.sh"]
volumeMounts:
- mountPath: /tmp
name: entrypoint
- mountPath: /var/config
name: secrets
volumes:
# volumes mounting
...
containers:
- image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
name: my-container
volumeMounts:
- mountPath: /var/config
name: secrets
env:
- name: HOST
?? # Assign value fetched in configmap
How to assign values from CM created files to container's env variables? Or, is there any other approach available to achieve this?
I need send couple of env variable to Spring cloud config service. It's hard to find any guide/documentation for this. Any help is appreciated!
One of the best ways to access secrets in google secret manager from GKE is by using the operator External Secret Operator, you can install it easily using helm.
Once is installed, you create a service account with the role roles/secretmanager.secretAccessor, then you download the creds (key file) and save them in a k8s secret:
kubectl create secret generic gcpsm-secret --from-file=service-account-credentials=key.json
Then you can define your secret store (it is not execlusive for GCP, it works with other secret manager such as AWS secret manager...):
apiVersion: external-secrets.io/v1alpha1
kind: SecretStore
metadata:
name: gcp-secret-manager
spec:
provider:
gcpsm:
auth:
secretRef:
secretAccessKeySecretRef:
name: gcpsm-secret # the secret you created in the first step
key: service-account-credentials
projectID: <your project id>
Now, you can create an external secret, and the operator will read the secret from the secret manager and create a k8s secret for you
apiVersion: external-secrets.io/v1alpha1
kind: ExternalSecret
metadata:
name: gcp-external-secret
spec:
secretStoreRef:
kind: SecretStore
name: gcp-secret-manager
target:
name: k8s-secret # the k8s secret name
data:
- secretKey: host # the key name in the secret
remoteRef:
key: <secret-name in gsm>
Finally in your pod, you can access the secret by:
apiVersion: v1
kind: Pod
metadata:
name: poc-pod
namespace: default
spec:
initContainers:
- image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
name: init
command: ["/tmp/entrypoint.sh"]
volumeMounts:
- mountPath: /tmp
name: entrypoint
- mountPath: /var/config
name: secrets
volumes:
# volumes mounting
...
containers:
- image: gcr.io/google.com/cloudsdktool/cloud-sdk:slim
name: my-container
volumeMounts:
- mountPath: /var/config
name: secrets
env:
- name: HOST
valueFrom:
secretKeyRef:
name: k8s-secret
key: host
If you update the secret value in the secret manager, you should recreate the external secret to update the k8s secret value.
I'm trying to figure it out, how to change one string inside configmap in Kubernetes. I have pretty simple configmap:
apiVersion: v1
data:
config.cfg: |-
[authentication]
USERNAME=user
PASSWORD=password
[podname]
PODNAME=metadata.podName
kind: ConfigMap
metadata:
name: name_here
And I need to mount the configmap inside a couple of pods. But PODNAME should be matched to current podname. Is it possible in any another way? thanks!
I do not think it could be done with ConfigMap. But you can set environment variables in your pod spec that references a pod fields.
apiVersion: v1
kind: Pod
metadata:
name: test-ref-pod-name
spec:
containers:
- name: test-container
image: busybox
command: [ "sh", "-c"]
args:
- env | grep PODNAME
env:
- name: PODNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
restartPolicy: Never
See official documentation: https://kubernetes.io/docs/tasks/inject-data-application/environment-variable-expose-pod-information/#use-pod-fields-as-values-for-environment-variables
This doesn't answer you question exactly but the pod name normally ends up as the host name inside the pod and can be accessed as a standard environment variable
echo $HOSTNAME
I'm trying to get secrets in init container and pass them to the main container. I have 2 questions here,
How to pass external secrets in init container
How to call them as env variables in main container
you can mount the same secret as Environment variables on both initContainer and mainContainer.
Given a secret such as:
k create secret generic mysecret --from-literal=key=secret
That will be something like:
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: mypod
name: mypod
spec:
initContainers:
- name: init
image: busybox
env:
- name: SECRET
valueFrom:
secretKeyRef:
name: mysecret
key: key
command: ['sh','-c','echo $SECRET']
containers:
- image: busybox
name: mypod
resources: {}
env:
- name: SECRET
valueFrom:
secretKeyRef:
name: mysecret
key: key
command: ['sh','-c','echo $SECRET']
dnsPolicy: ClusterFirst
restartPolicy: Never
status: {}
When running the pods, see them both printing the content of the same secret.
kubectl logs mypod -c init
secret
kubectl logs mypod
secret
I am working on operator-sdk, in the controller, we often need to create a Deployment object, and Deployment resource has a lot of configuration items, such as environment variables or ports definition or others as following. I am wondering what is best way to get these values, I don't want to hard code them, for example, variable_a or variable_b.
Probably, you can put them in the CRD as spec, then pass them to Operator Controller; Or maybe you can put them in the configmap, then pass configmap name to Operator Controller, Operator Controller can access configmap to get them; Or maybe you can put in the template file, then in the Operator Controller, controller has to read that template file.
What is best way or best practice to deal with this situation? Thanks for sharing your ideas or points.
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: m.Name,
Namespace: m.Namespace,
Labels: ls,
},
Spec: appsv1.DeploymentSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: ls,
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{{
Image: "....",
Name: m.Name,
Ports: []corev1.ContainerPort{{
ContainerPort: port_a,
Name: "tcpport",
}},
Env: []corev1.EnvVar{
{
Name: "aaaa",
Value: variable_a,
},
{
Name: "bbbb",
Value: variable_b,
},
Using enviroment variables
It can be convenient that your app gets your data as environment variables.
Environment variables from ConfigMap
For non-sensitive data, you can store your variables in a ConfigMap and then define container environment variables using the ConfigMap data.
Example from Kubernetes docs:
Create the ConfigMap first. File configmaps.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: special-config
namespace: default
data:
special.how: very
---
apiVersion: v1
kind: ConfigMap
metadata:
name: env-config
namespace: default
data:
log_level: INFO
Create the ConfigMap:
kubectl create -f ./configmaps.yaml
Then define the environment variables in the Pod specification, pod-multiple-configmap-env-variable.yaml:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
env:
- name: SPECIAL_LEVEL_KEY
valueFrom:
configMapKeyRef:
name: special-config
key: special.how
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: env-config
key: log_level
restartPolicy: Never
Create the Pod:
kubectl create -f ./pod-multiple-configmap-env-variable.yaml
Now in your controller you can read these environment variables SPECIAL_LEVEL_KEY (which will give you special.how value from special-config ConfigMap) and LOG_LEVEL (which will give you log_level value from env-config ConfigMap):
For example:
specialLevelKey := os.Getenv("SPECIAL_LEVEL_KEY")
logLevel := os.Getenv("LOG_LEVEL")
fmt.Println("SPECIAL_LEVEL_KEY:", specialLevelKey)
fmt.Println("LOG_LEVEL:", logLevel)
Environment variables from Secret
If your data is sensitive, you can store it in a Secret and then use the Secret as environment variables.
To create a Secret manually:
You'll first need to encode your strings using base64.
# encode username
$ echo -n 'admin' | base64
YWRtaW4=
# encode password
$ echo -n '1f2d1e2e67df' | base64
MWYyZDFlMmU2N2Rm
Then create a Secret with the above data:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
Create a Secret with kubectl apply:
$ kubectl apply -f ./secret.yaml
Please notice that there are other ways to create a secret, pick one that works best for you:
Creating a Secret using kubectl
Creating a Secret from a generator
Creating a Secret from files
Creating a Secret from string literals
Now you can use this created Secret for environment variables.
To use a secret in an environment variable in a Pod:
Create a secret or use an existing one. Multiple Pods can reference the same secret.
Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in env[].valueFrom.secretKeyRef.
Modify your image and/or command line so that the program looks for values in the specified environment variables.
Here is a Pod example from Kubernetes docs that shows how to use a Secret for environment variables:
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password
restartPolicy: Never
Finally, as stated in the docs:
Inside a container that consumes a secret in an environment variables, the secret keys appear as normal environment variables containing the base64 decoded values of the secret data.
Now in your controller you can read these environment variables SECRET_USERNAME (which will give you username value from mysecret Secret) and SECRET_PASSWORD (which will give you password value from mysecret Secret):
For example:
username := os.Getenv("SECRET_USERNAME")
password := os.Getenv("SECRET_PASSWORD")
Using volumes
You can also mount both ConfigMap and Secret as a volume to you pods.
Populate a Volume with data stored in a ConfigMap:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "ls /etc/config/" ]
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
# Provide the name of the ConfigMap containing the files you want
# to add to the container
name: special-config
restartPolicy: Never
Using Secrets as files from a Pod:
To consume a Secret in a volume in a Pod:
Create a secret or use an existing one. Multiple Pods can reference the same secret.
Modify your Pod definition to add a volume under .spec.volumes[]. Name the volume anything, and have a .spec.volumes[].secret.secretName field equal to the name of the Secret object.
Add a .spec.containers[].volumeMounts[] to each container that needs the secret. Specify .spec.containers[].volumeMounts[].readOnly = true and .spec.containers[].volumeMounts[].mountPath to an unused directory name where you would like the secrets to appear.
Modify your image or command line so that the program looks for files in that directory. Each key in the secret data map becomes the filename under mountPath.
An example of a Pod that mounts a Secret in a volume:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecret
The example has:
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: usernamekey
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: passwordkey
restartPolicy: Never
the above from:
https://kubernetes.io/docs/concepts/configuration/secret/
I've created a secret like this:
kubectl --namespace=mycustomnamespace create secret generic mysecret --from-literal=passwordkey="abc123" --from-literal=usernamekey="mememe"
I understand that the above secrets exist under the namespace.
But if I try this:
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
namespace: mycustomnamespace
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: usernamekey
namespace: mycustomnamespace
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: passwordkey
namespace: mycustomnamespace
restartPolicy: Never
(note that I added a namespace declaration under metadata)
I get this error:
Error validating data:
[ValidationError(Pod.spec.containers[1].env[2].valueFrom.secretKeyRef):
unknown field "namespace" in io.k8s.api.core.v1.SecretKeySelector,
ValidationError(Pod.spec.containers[1].env[6].valueFrom.secretKeyRef):
unknown field "namespace" in io.k8s.api.core.v1.SecretKeySelector];
If I take out the namespace(s) (under the secretKeyRef(s))....the pod fails..with
Warning Failed 2s (x8 over 1m) kubelet, minikube Error:
secret "mysecret" not found
Yes, my secrets are in the namespace:
kubectl get secrets --namespace mycustomnamespace
NAME TYPE DATA AGE
default-token-55bzp kubernetes.io/service-account-token 3 10m
mysecret Opaque 2 10m
APPEND : (resolution)
It was an error on my part. Check my comment under Vasily's answer.
But basically, the magic-sauce is that the below yml....
metadata:
name: secret-env-pod
namespace: mycustomnamespace
the above yml should "drive" the namespaces (aka, set the scope of the namespace) for the rest of the configuration (yml) ....
(if you are a future reader of this question, double and triple check that you have everything under the correct namespace. ALL of your normal "get" statements need to use -n (aka --namespace) as a part.
example
kubectl get pods
the above will only get pods under "default".
you have to do
kubectl get pods --namespace mycustomnamespace
Simply remove namespace: mycustomnamespace from pod secretKeyRef definitions.
Also your secret create command should be like that:
kubectl --namespace=mycustomnamespace create secret generic mysecret --from-literal=passwordkey="abc123" --from-literal=usernamekey="mememe"
Try this You can run your pod
kubectl apply -f podconfigfile.yaml -n mycustomnamespace
This will run pod in the same namespace and find secret if the secret is created
If any error will be there then check for the namespace having secrets and proper name of the namespace.
Check secret in namespace :
kubectl get secret -n mycustomnamespace