I want to make some deployments in kubernetes using helm charts. Here is a sample override-values yaml that I use:
imageRepository: ""
ocbb:
imagePullPolicy: IfNotPresent
TZ: UTC
logDir: /oms_logs
tnsAdmin: /oms/ora_k8
LOG_LEVEL: 3
wallet:
client:
server:
root:
db:
deployment:
imageName: init_db
imageTag:
host: 192.168.88.80
port:
service:
alias:
schemauser: pincloud
schemapass:
schematablespace: pincloud
indextablespace: pincloudx
nls_lang: AMERICAN_AMERICA.AL32UTF8
charset: AL32UTF8
pipelineschemauser: ifwcloud
pipelineschemapass:
pipelineschematablespace: ifwcloud
pipelineindextablespace: ifwcloudx
pipelinealias:
queuename:
In this file I have to set some values involving credentials, for example schemapass, pipelineschemapass...
Documentation states I have to generate kubernetes secrets to do this and add this key to my yaml file with the same path hierarchy.
I generated some kubernetes secrets, for example:
kubectl create secret generic schemapass --from-literal=password='pincloud'
Now I don't know how to reference this newly generated secret in my yaml file. Any tip about how to set schemapass field in yaml chart to reference kubernetes secret?
You cannot use Kubernetes secret in your values.yaml. In values.yaml you only specify the input parameters for the Helm Chart, so it could be the secret name, but not the secret itself (or anything that it resolved).
If you want to use the secret in your container, then you can insert it as an environment variable:
env:
- name: SECRET_VALUE_ENV
valueFrom:
secretKeyRef:
name: schemapass
key: password
You can check more in the Hazelcast Enterprise Helm Chart. We do exactly that. You specify the secret name in values.yaml and then the secret is injected into the container using environment variable.
You can reference K8S values -whether secrets or not- in Helm by specifying them in your container as environment variables.
let your deployment be mongo.yml
--
kind: Deployment
--
--
containers:
--
env:
- name: DB_URL
valueFrom:
configMapKeyRef:
name: mongo-config
key: mongo-url
- name: MONGO_INITDB_ROOT_PASSWORD
valueFrom:
secretKeyRef:
name: mongo-secret
key: mongo-password
Where mongo-secret is
apiVersion: v1
kind: Secret
metadata:
name: mongo-secret
type: Opaque
data:
mongo-user: bW9uZ291c2Vy
mongo-password: bW9uZ29wYXNzd29yZA==
and mongo-config is
apiVersion: v1
kind: ConfigMap
metadata:
name: mongo-config
data:
mongo-url: mongo-service
Related
I have a configMap file:
apiVersion: v1
kind: ConfigMap
metadata:
labels:
owner: testdb
name: testdb-configmap
data:
host: postgres
port: "5432"
and a secret file:
aapiVersion: v1
kind: Secret
type: Opaque
metadata:
labels:
owner: testdb
name: testdb-secret
namespace: test
data:
user: dGVzdA==
pwd: dGVzdA==
and I want to build an environment variable CONNECTION_STRING as below:
env:
- name: CONNECTION_STRING
value: "Host=<host-from-configmap>;Username=<user-from-secret>;Password=<password-from-secret>;Port=<port-from-configmap>;Pooling=False;"
I want to know if this is possible and if yes, then how? I have also looked at using .tpl (named templates) but couldn't figure out a way.
NOTE
Since I don't have access to the image which requires CONNECTION_STRING I have to build it this way. These configmap and secret files are also going to remain like this.
Kubernetes can set environment variables based on other environment variables. This is a core Kubernetes Pod capability, and doesn't depend on anything from Helm.
Your value uses four components, two from the ConfigMap and two from the Secret. You need to declare each of these as separate environment variables, and then declare a main environment variable that concatenates them together.
env:
- name: TESTDB_HOST
valueFrom:
configMapRef:
name: testdb-configmap # {{ include "chart.name" . }}
key: host
- name: TESTDB_PORT
valueFrom:
configMapRef:
name: testdb-configmap
key: port
- name: TESTDB_USER
valueFrom:
secretKeyRef:
name: testdb-secret
key: user
- name: TESTDB_PASSWORD
valueFrom:
secretKeyRef:
name: testdb-secret
key: password
- name: CONNECTION_STRING
value: Host=$(TESTDB_HOST);Username=$(TESTDB_USER);Password=$(TESTDB_PASSWORD);PORT=$(TESTDB_PORT);Pooling=False;
I do not believe what you're asking to do is possible.
Furthermore, do not use configs maps for storing information like this. It's best practice to use secrets and then mount them to your container as files or ENV variables.
I would abandon whatever you're thinking and re-evaluate what you're trying to accomplish.
I'm using helm chart to deploy pods to multiple environments. I would like to have one secret file for each environment like dev, sit. I have created secrets.yaml file which is referencing values.yaml of each environment.
apiVersion: v1
kind: Secret
metadata:
name: my-secret
type: Opaque
data:
DB_URL: {{ .Values.secret.db.url }}
And values.yaml for each environment looks like:
templates:
env:
- name: DB_URL
valueFrom:
secretKeyRef:
name: my-secret
key: DB_URL
secret:
db:
url: <base64Encoded_value>
The secret is not getting applied into the environment. What wrong am I doing here?
First of all, Is the value being successfully referenced by the Secret object from the values.yaml file? Try running a "helm template" command to locally render the templates to see the result.
If it's working as expected, it'll be good to see how you're using the secret to apply in your environment.
I have been trying to figure out how to consume a ConfigMap created using a ConfigMap generator via Kustomize.
When created using Kustomize generators, the configMaps are named with a special suffix. See here:
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#create-a-configmap-from-generator
Question is how can this be referenced?
You don't reference it yourself. Kustomize recognizes where the configMap is used in the other resources (like a Deployment) and changes those references to use the name+hash.
The reason for this is so that if you change the configMap, Kustomize generates a new hash and updates the Deployment, causing a rolling restart of the Pods.
If you don't want this behavior, you can add the following to your kustomization.yaml file:
generatorOptions:
disableNameSuffixHash: true
It is specified there in the doc. When you do kubectl apply -k . a configmap created named game-config-4-m9dm2f92bt.
You can check that the ConfigMap was created like this: kubectl get configmap. This ConfigMap will contains a field data where your given datas will belong.
Now as usual you can use this configmap in a pod. Like below:
Ex from k8s:
apiVersion: v1
kind: Pod
metadata:
name: test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
env:
# Define the environment variable
- name: SPECIAL_LEVEL_KEY
valueFrom:
configMapKeyRef:
# The ConfigMap containing the value you want to assign to SPECIAL_LEVEL_KEY
name: special-config
# Specify the key associated with the value
key: special.how
restartPolicy: Never
You can use ConfigMap as volume also, like this example from k8s doc:
apiVersion: v1
kind: Pod
metadata:
name: configmap-demo-pod
spec:
containers:
- name: demo
image: alpine
command: ["sleep", "3600"]
env:
# Define the environment variable
- name: PLAYER_INITIAL_LIVES # Notice that the case is different here
# from the key name in the ConfigMap.
valueFrom:
configMapKeyRef:
name: game-demo # The ConfigMap this value comes from.
key: player_initial_lives # The key to fetch.
- name: UI_PROPERTIES_FILE_NAME
valueFrom:
configMapKeyRef:
name: game-demo
key: ui_properties_file_name
volumeMounts:
- name: config
mountPath: "/config"
readOnly: true
volumes:
# You set volumes at the Pod level, then mount them into containers inside that Pod
- name: config
configMap:
# Provide the name of the ConfigMap you want to mount.
name: game-demo
# An array of keys from the ConfigMap to create as files
items:
- key: "game.properties"
path: "game.properties"
- key: "user-interface.properties"
path: "user-interface.properties
You can see k8s official doc
I was struggling with this too. I could not figure out why kustomize was not updating the configmap name for the volume in the deployment to include the hash. What solved this for me was to add namespace: <namespace> in the kustomization.yaml for both the base and overlay.
I am working on operator-sdk, in the controller, we often need to create a Deployment object, and Deployment resource has a lot of configuration items, such as environment variables or ports definition or others as following. I am wondering what is best way to get these values, I don't want to hard code them, for example, variable_a or variable_b.
Probably, you can put them in the CRD as spec, then pass them to Operator Controller; Or maybe you can put them in the configmap, then pass configmap name to Operator Controller, Operator Controller can access configmap to get them; Or maybe you can put in the template file, then in the Operator Controller, controller has to read that template file.
What is best way or best practice to deal with this situation? Thanks for sharing your ideas or points.
deployment := &appsv1.Deployment{
ObjectMeta: metav1.ObjectMeta{
Name: m.Name,
Namespace: m.Namespace,
Labels: ls,
},
Spec: appsv1.DeploymentSpec{
Replicas: &replicas,
Selector: &metav1.LabelSelector{
MatchLabels: ls,
},
Template: corev1.PodTemplateSpec{
ObjectMeta: metav1.ObjectMeta{
Labels: ls,
},
Spec: corev1.PodSpec{
Containers: []corev1.Container{{
Image: "....",
Name: m.Name,
Ports: []corev1.ContainerPort{{
ContainerPort: port_a,
Name: "tcpport",
}},
Env: []corev1.EnvVar{
{
Name: "aaaa",
Value: variable_a,
},
{
Name: "bbbb",
Value: variable_b,
},
Using enviroment variables
It can be convenient that your app gets your data as environment variables.
Environment variables from ConfigMap
For non-sensitive data, you can store your variables in a ConfigMap and then define container environment variables using the ConfigMap data.
Example from Kubernetes docs:
Create the ConfigMap first. File configmaps.yaml:
apiVersion: v1
kind: ConfigMap
metadata:
name: special-config
namespace: default
data:
special.how: very
---
apiVersion: v1
kind: ConfigMap
metadata:
name: env-config
namespace: default
data:
log_level: INFO
Create the ConfigMap:
kubectl create -f ./configmaps.yaml
Then define the environment variables in the Pod specification, pod-multiple-configmap-env-variable.yaml:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "env" ]
env:
- name: SPECIAL_LEVEL_KEY
valueFrom:
configMapKeyRef:
name: special-config
key: special.how
- name: LOG_LEVEL
valueFrom:
configMapKeyRef:
name: env-config
key: log_level
restartPolicy: Never
Create the Pod:
kubectl create -f ./pod-multiple-configmap-env-variable.yaml
Now in your controller you can read these environment variables SPECIAL_LEVEL_KEY (which will give you special.how value from special-config ConfigMap) and LOG_LEVEL (which will give you log_level value from env-config ConfigMap):
For example:
specialLevelKey := os.Getenv("SPECIAL_LEVEL_KEY")
logLevel := os.Getenv("LOG_LEVEL")
fmt.Println("SPECIAL_LEVEL_KEY:", specialLevelKey)
fmt.Println("LOG_LEVEL:", logLevel)
Environment variables from Secret
If your data is sensitive, you can store it in a Secret and then use the Secret as environment variables.
To create a Secret manually:
You'll first need to encode your strings using base64.
# encode username
$ echo -n 'admin' | base64
YWRtaW4=
# encode password
$ echo -n '1f2d1e2e67df' | base64
MWYyZDFlMmU2N2Rm
Then create a Secret with the above data:
apiVersion: v1
kind: Secret
metadata:
name: mysecret
type: Opaque
data:
username: YWRtaW4=
password: MWYyZDFlMmU2N2Rm
Create a Secret with kubectl apply:
$ kubectl apply -f ./secret.yaml
Please notice that there are other ways to create a secret, pick one that works best for you:
Creating a Secret using kubectl
Creating a Secret from a generator
Creating a Secret from files
Creating a Secret from string literals
Now you can use this created Secret for environment variables.
To use a secret in an environment variable in a Pod:
Create a secret or use an existing one. Multiple Pods can reference the same secret.
Modify your Pod definition in each container that you wish to consume the value of a secret key to add an environment variable for each secret key you wish to consume. The environment variable that consumes the secret key should populate the secret's name and key in env[].valueFrom.secretKeyRef.
Modify your image and/or command line so that the program looks for values in the specified environment variables.
Here is a Pod example from Kubernetes docs that shows how to use a Secret for environment variables:
apiVersion: v1
kind: Pod
metadata:
name: secret-env-pod
spec:
containers:
- name: mycontainer
image: redis
env:
- name: SECRET_USERNAME
valueFrom:
secretKeyRef:
name: mysecret
key: username
- name: SECRET_PASSWORD
valueFrom:
secretKeyRef:
name: mysecret
key: password
restartPolicy: Never
Finally, as stated in the docs:
Inside a container that consumes a secret in an environment variables, the secret keys appear as normal environment variables containing the base64 decoded values of the secret data.
Now in your controller you can read these environment variables SECRET_USERNAME (which will give you username value from mysecret Secret) and SECRET_PASSWORD (which will give you password value from mysecret Secret):
For example:
username := os.Getenv("SECRET_USERNAME")
password := os.Getenv("SECRET_PASSWORD")
Using volumes
You can also mount both ConfigMap and Secret as a volume to you pods.
Populate a Volume with data stored in a ConfigMap:
apiVersion: v1
kind: Pod
metadata:
name: dapi-test-pod
spec:
containers:
- name: test-container
image: k8s.gcr.io/busybox
command: [ "/bin/sh", "-c", "ls /etc/config/" ]
volumeMounts:
- name: config-volume
mountPath: /etc/config
volumes:
- name: config-volume
configMap:
# Provide the name of the ConfigMap containing the files you want
# to add to the container
name: special-config
restartPolicy: Never
Using Secrets as files from a Pod:
To consume a Secret in a volume in a Pod:
Create a secret or use an existing one. Multiple Pods can reference the same secret.
Modify your Pod definition to add a volume under .spec.volumes[]. Name the volume anything, and have a .spec.volumes[].secret.secretName field equal to the name of the Secret object.
Add a .spec.containers[].volumeMounts[] to each container that needs the secret. Specify .spec.containers[].volumeMounts[].readOnly = true and .spec.containers[].volumeMounts[].mountPath to an unused directory name where you would like the secrets to appear.
Modify your image or command line so that the program looks for files in that directory. Each key in the secret data map becomes the filename under mountPath.
An example of a Pod that mounts a Secret in a volume:
apiVersion: v1
kind: Pod
metadata:
name: mypod
spec:
containers:
- name: mypod
image: redis
volumeMounts:
- name: foo
mountPath: "/etc/foo"
readOnly: true
volumes:
- name: foo
secret:
secretName: mysecret
I use minikube on windows 10 and try to test Kubernetes ConfigMap with both literal type and outer file type. First I make below manifest yaml file to make ConfigMap.
apiVersion: v1
kind: ConfigMap
metadata:
name: simple-config
data:
mysql_root_password: password
mysql_password: password
mysql_database: test
---
apiVersion: v1
kind: Pod
metadata:
name: blog-db
labels:
app: blog-mysql
spec:
containers:
- name: blog-mysql
image: mysql:latest
env:
- name: MYSQL_ROOT_PASSWORD
valueFrom:
configMapKeyRef:
name: simple-config
key: mysql_root_password
- name: MYSQL_PASSWORD
valueFrom:
configMapKeyRef:
name: simple-config
key: mysql_password
- name: MYSQL_DATABASE
valueFrom:
configMapKeyRef:
name: simple-config
key: mysql_database
ports:
- containerPort: 3306
The above configmap yaml file throws no errors. It works successfully. This time I try to test kubernetes configmap with file.
== configmap.properties
mysql_root_password=password
mysql_password=password
mysql_database=test
But I am stuck with this part. Most of configmap examples use kubectl command with --from-file option like below,
kubectl create configmap simple-config --from-file=configmap.properties
But I have no idea how to mount the properties file using manifest yaml file grammer. Any advice?
You can not directly mount a properties file in a pod without first creating a ConfigMap from the properties file.You can create configMap from env file as below
kubectl create configmap simple-config \
--from-env-file=configmap.properties