I want to update a tag in a kubernetes kustomization.yaml with sed. the original file looks like this:
resources:
- ../../base
namePrefix: prod-
commonLabels:
env: production
images:
- name: my-service
newTag: current-version
patchesStrategicMerge:
- deployment.yaml
when i use my sed command it just does not work and i'm not sure why:
sed -r 'name: my-service\s*(newTag:\s*).*/\1new-version/g' overlays/production/kustomization.yaml
as far as i understand this it should match the newTag key if it follows a name: my-service element. I dont get any errors, it just does not work.
I'm currently testing this on MacOS
As RavinderSingh13 comments, yq will be an appropriate tool to handle
yaml file. If yq is available, would you please try:
yq -y '(.images[] | select(.name == "my-service") | .newTag) |= "new-version"' yourfile.yaml
Output:
resources:
- ../../base
namePrefix: prod-
commonLabels:
env: production
images:
- name: my-service
newTag: new-version
patchesStrategicMerge:
- deployment.yaml
If yq is not available and you have a specific reason to use sed, then try the alternative:
sed -E '
/my-service/{ ;# if the line matches "my-service", then execute the block
N ;# append the next line to the pattern space
s/(newTag:[[:space:]]*).*/\1new-version/ ;# replace the value
} ;# end of the block
' yourfile.yaml
The reason why your sed command does not work is because sed is a
line-oriented tool and process the input line by line. Your regex crosses
the lines and will not match.
Related
I am learning kubernetes and have the following question related to command and argument syntax for POD.
Are there any specific syntax that we need to follow to write a shell script kind of code in the arguments of a POD? For example
In the following code, how will I know that the while true need to end with a semicolon ; why there is no semi colon after do but after If etc
while true;
do
echo $i;
if [ $i -eq 5 ];
then
echo "Exiting out";
break;
fi;
i=$((i+1));
sleep "1";
done
We don't write shell script in the similar way from semicolon prespective so why do we have to do this in POD.
I tried the command in /bin/bash format as well
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: bash
name: bash
spec:
containers:
- image: nginx
name: nginx
args:
- /bin/bash
- -c
- >
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
Error with new code
/bin/bash: -c: line 2: syntax error near unexpected token `echo'
/bin/bash: -c: line 2: ` echo "Welcome $i times"'
Are there any specific syntax that we need to follow to write a shell script kind of code in the arguments of a POD?
No, shell syntax is the same across.
...how will I know that the while true need to end with a semicolon
Used | for your text block to be treated like an ordinary shell script:
...
args:
- /bin/bash
- -c
- |
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done
When you use > your text block is merge into a single line where newline is replaced with white space. Your command become invalid in such case. If you want your command to be a single line, then write them with ; like you would in ordinary terminal. This is shell scripting standard and is not K8s specific.
If you must use >, you need to either add empty line or indented the next line correctly:
apiVersion: v1
kind: Pod
metadata:
labels:
run: bash
name: bash
spec:
containers:
- image: nginx
name: nginx
args:
- /bin/bash
- -c
- >
for i in 1 2 3 4 5
do
echo "Welcome $i times"
done
restartPolicy: Never
kubectl logs bash to see the 5 echos and kubectl delete pod bash to clean-up.
A common requirement when deploying Kubernetes manifests to a cluster is to prefix the container names with a trusted registry prefix that mirrors the allowed images. Usually used along with an admission controller.
Is there a sensible way to do this using Kustomize without having to list every single image by name in the kustomization.yaml images: transformer stanza?
Given this kustomization.yaml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- "https://github.com/prometheus-operator/kube-prometheus"
if I want to prefix all the images it references with mytrusted.registry/ I need to append this to my kustomization.yaml:
images:
- name: grafana/grafana
newName: mytrusted.registry/grafana/grafana
- name: jimmidyson/configmap-reload
newName: mytrusted.registry/jimmidyson/configmap-reload
- name: k8s.gcr.io/kube-state-metrics/kube-state-metrics
newName: mytrusted.registry/k8s.gcr.io/kube-state-metrics/kube-state-metrics
- name: k8s.gcr.io/prometheus-adapter/prometheus-adapter
newName: mytrusted.registry/k8s.gcr.io/prometheus-adapter/prometheus-adapter
- name: quay.io/brancz/kube-rbac-proxy
newName: mytrusted.registry/quay.io/brancz/kube-rbac-proxy
- name: quay.io/prometheus/alertmanager
newName: mytrusted.registry/quay.io/prometheus/alertmanager
- name: quay.io/prometheus/blackbox-exporter
newName: mytrusted.registry/quay.io/prometheus/blackbox-exporter
- name: quay.io/prometheus/node-exporter
newName: mytrusted.registry/quay.io/prometheus/node-exporter
- name: quay.io/prometheus-operator/prometheus-operator
newName: mytrusted.registry/quay.io/prometheus-operator/prometheus-operator
- name: quay.io/prometheus/prometheus
newName: mytrusted.registry/quay.io/prometheus/prometheus
which I generated with this putrid, fragile monstrosity (which WILL break if your containers are specified by hash, or you have a port in your registry prefix):
kustomize build | \
grep 'image:' | \
awk '$2 != "" { print $2}' | \
sort -u | \
cut -d : -f 1 | \
jq --raw-input '{ name: ., newName: ("mytrusted.registry/" + .) }' | \
faq -s -fjson -oyaml '{ images: .}'
(Note that the above will also NOT WORK completely, because Kustomize doesn't recognise images outside PodTemplates, such as those in the kind: Alertmanager spec.image or the kind: Prometheus spec.image; it'd still be better than the current situation).
What I want instead is to able to express this in the image transformer without generating and maintaining lists of images, with something like the imaginary, does not work example:
images:
- name: "(*)"
newName: "mytrusted.registry/$1"
i.e. use a capture group. Or something functionally similar, like an image transformer "prependName" option or similar.
This must be such a common problem to have, but I can't for the life of me find a well established way this is done by convention in the k8s world. Just lots of DIY fragile hacks.
This answer is probably too late to help the original asker, but maybe it will help others who stumble upon this question through Google, like I did.
Kustomize has a built-in PrefixTransformer that can add a prefix to all your images, or indeed to any arbitrary field in your specs.
Create a file named image-prefix.yaml with the following contents:
apiVersion: builtin
kind: PrefixTransformer
metadata:
name: image-prefix
prefix: mytrusted.registry/
fieldSpecs:
- path: spec/template/spec/containers/image
- path: spec/template/spec/initContainers/image
- path: spec/image # for kind Prometheus and Alertmanager
Then add this transformer to your kustomization.yaml as follows:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- "https://github.com/prometheus-operator/kube-prometheus"
transformers:
- image-prefix.yaml
That should do it.
When you build this, you should see your prefix automatically added to all the images:
$ kubectl kustomize | grep image:
...
image: mytrusted.registry/quay.io/prometheus/blackbox-exporter:v0.22.0
image: mytrusted.registry/jimmidyson/configmap-reload:v0.5.0
image: mytrusted.registry/quay.io/brancz/kube-rbac-proxy:v0.13.0
image: mytrusted.registry/grafana/grafana:my-tag
image: mytrusted.registry/k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.6.0
...
I tested this with kubectl 1.25 and the version of Kustomize that comes bundled with it:
$ kubectl version --short --client
...
Client Version: v1.25.0
Kustomize Version: v4.5.7
You can further restrict the PrefixTransformer by using GVK (group/version/kind) triplets. For example, if for some reason you wanted to apply your image prefix only to Deployments, but not to DaemonSets, StatefulSets, or others, you would put something like this in your image-prefix.yaml file:
fieldSpecs:
- kind: Deployment
path: spec/template/spec/containers/image
- kind: Deployment
path: spec/template/spec/initContainers/image
Also note that the ImageTransformer runs before the PrefixTransformer, so if you wanted to override the tag of a particular image in your kustomization.yaml, you should use the original image name without the prefix:
images:
- name: grafana/grafana
newTag: my-tag
Unfortunately there is no clear documentation for PrefixTransformer that I could find, or I would have linked it here. I discovered all this by digging through Kustomize source code.
There are quite a few other built-in transformers that might be of interest, you can glean their usage by looking at the *_test.go files in each of the subfolders here:
https://github.com/kubernetes-sigs/kustomize/tree/master/plugin/builtin
As I know the way to create a configMap in Kubernetes from a file is to use:
--from-file option for kubectl
What I am looking for is a way to only load part of the yaml file into the configMap.
Example:
Let's say I have this yml file:
family:
Boys:
- name: Joe
- name: Bob
- name: dan
Girls:
- name: Alice
- name: Jane
Now I want to create a configMap called 'boys' which will include only the 'Boys' section.
Possible?
Another thing that could help if the above is not possible is when I am exporting the configMap as environment variables to a pod (using envFrom) to be able to only export part of the configMap.
Both options will work for me.
Any idea?
The ConfigMap uses a key and value for its configuration. Based on your example, you get multiple arrays of data where there are multiple values with their own keys. But you can create multiple ConfigMap from different file for these issues.
First you need to create .yaml files to create a ConfigMap guided by the documentation.
First file call Boys.yaml
# Env-files contain a list of environment variables.
# These syntax rules apply:
# Each line in an env file has to be in VAR=VAL format.
# Lines beginning with # (i.e. comments) are ignored.
# Blank lines are ignored.
# There is no special handling of quotation marks (i.e. they will be part of the ConfigMap value)).
name=Joe
name=Bob
name=Dan
Second file call Girls.yaml
name=Alice
name=Jane
Create your ConfigMap
kubectl create configmap NmaeOfYourConfigmap --from-env-file=PathToYourFile/Boys.yaml
where the output is similar to this:
apiVersion: v1
kind: ConfigMap
metadata:
creationTimestamp:
name: NmaeOfYourConfigmap
namespace: default
resourceVersion:
uid:
data:
name: Joe
name: Bob
name: Dan
Finally, you can pass these ConfigMap to pod or deployment using configMapRef entries:
envFrom:
- configMapRef:
name: NmaeOfYourConfigmap-Boys
- configMapRef:
name: NmaeOfYourConfigmap-Girls
Configmaps cannot contain rich yaml data. Only key value pairs. So if you want to have a list of things, you need to express this as a multiline string.
With that in mind you could use certain tools, such a yq to query your input file and select the part you want.
For example:
podman run -rm --interactive bluebrown/tpl '{{ .family.Boys | toYaml }}' < fam.yaml \
| kubectl create configmap boys --from-file=Boys=/dev/stdin
The result looks like this
apiVersion: v1
kind: ConfigMap
metadata:
name: boys
namespace: sandbox
data:
Boys: |+
- name: Joe
- name: Bob
- name: dan
You could also encode the file or part of the file with base64 and use that as an environment variable, since you get a single string, which is easily processable, out of it. For example:
$ podman run --rm --interactive bluebrown/tpl \
'{{ .family.Boys | toYaml | b64enc }}' < fam.yaml
# use this string as env variable and decode it in your app
LSBuYW1lOiBKb2UKLSBuYW1lOiBCb2IKLSBuYW1lOiBkYW4K
Or with set env which you could further combine with dry run if required.
podman run --rm --interactive bluebrown/tpl \
'YAML_BOYS={{ .family.Boys | toYaml | b64enc }}' < fam.yaml \
| kubectl set env -e - deploy/myapp
Another thing is, that YAML is a superset of JSON, in many cases you are able to convert YAML to JSON or at least use JSON like syntax.
This can be useful in such a scenario in order to express this as a single line string rather than having to use multiline syntax. It's less fragile.
Every YAML parser will be able to parse JSON just fine. So if you are parsing the string in your app, you won't have problems.
$ podman run --rm --interactive bluebrown/tpl '{{ .family.Boys | toJson }}' < fam.yaml
[{"name":"Joe"},{"name":"Bob"},{"name":"dan"}]
Disclaimer, I created the above used tool tpl. As mentioned, you might as well use alternative tools such as yq.
Variable substitution in Gitlab only seem to work inside the gitlab-ci.yml file.
However I have a configuration file (k8s secret) where I would like to set different values depending if I'm in a staging or a production environment.
apiVersion: v1
kind: Secret
metadata:
name: tls-config
namespace: myNamespace
type: kubernetes.io/tls
data:
tls.crt: |
${TLS_CRT}
tls.key: |
${TLS_KEY}
Where TLS_CRT & TLS_KEY would be variables defined in /settings/ci_cd/variables.
How should one handle variable substitution in this case?
You could use an editor like sed
For example, if you had the file like:
data:
tls.crt: |
#TLS_CRT#
tls.key: |
#TLS_KEY#
You could use sed like this as part of your GitLab job to replace its contents:
myjob:
script:
- sed -i "s|#TLS_CRT#|${TLS_CRT}|" ./path/to/file
- sed -i "s|#TLS_KEY#|${TLS_KEY}|" ./path/to/file
Then the file contents will be something like this:
data:
tls.crt: |
MIIC2DCCAcCgAwIBAgIBATANBgkqh ...
tls.key: |
MIIEpgIBAAKCAQEA7yn3bRHQ5FHMQ ...
Using the pattern #word# is not totally necessary. However, it does make using the sed easier, as it avoids needing to escape the literal ${} characters in the shell command.
As a short explanation of the sed command used here:
# v shorthand for --in-place; edits the file directly
sed -i "s|FIRST_ARGUMENT|SECOND_ARGUMENT|" ./path/to/file
# ^ replace this ^ with this ^ in this file
Fore more info, see here: https://unix.stackexchange.com/a/112024/453397
I'm trying to pass in a command argument in Kubernetes Daemonset with Helm Charts which performs export of an new environment variable passed in from a cURL result.
command: ["/bin/bash","-c","export MACHINE_TYPE=$(curl --unix-socket /var/run/docker.sock http://docker/containers/rancher-agent/json | grep -oP 'CATTLE_HOST_LABELS=.+?\w+' | awk -F '=' '{print $2}')"]
The result should be that the variable is set in the container e.g. MACHINE_TYPE=compute
I have also tried using command + args like so:
command: ["/bin/bash","-c"]
args: ["export MACHINE_TYPE=$(`curl --unix-socket /var/run/docker.sock http://docker/containers/rancher-agent/json | grep -oP 'CATTLE_HOST_LABELS=.+?\w+' | awk -F = '{print $2}'`)"]
When I try to deploy the daemonset, I get an error message
"Error: YAML parse error on /templates/daemonset.yaml: error converting YAML to JSON: yaml: line 46: found unknown escape character"
The export command works if I run it from within the container.
My aim is to be able to set a final container environment variable (LABEL) from the daemonset.yaml which is concatenate of two environment variables e.g.
containers:
- name: {{ .Chart.Name }}
image: "{{.Values.image.repository}}:{{.Values.image.tag}}"
imagePullPolicy: {{.Values.image.pullPolicy}}
env:
- name: LABEL
value: $MACHINE_TYPE-$HOSTNAME
command: ["/bin/bash","-c"]
args: ["export MACHINE_TYPE=$(`curl --unix-socket /var/run/docker.sock http://docker/containers/rancher-agent/json | grep -oP 'CATTLE_HOST_LABELS=.+?\w+' | awk -F = '{print $2}'`)"]
so the 'env' output in the container for LABEL variable would be
LABEL=compute-ip-x-x-x-x.ap-southeast-2.compute.internal
I know that the value value: $MACHINE_TYPE-$HOSTNAME will not work, so hoping for assistance with that as well.
found unknown escape character 'w'
... rep -oP 'CATTLE_HOST_LABELS=.+?\w+' | awk -F '=' '{print $2}')"]
^)
The error message appears to be pretty straightforward: backslash is magic in a double-quoted string
If you use the more straightforward yaml structure, without using the double-quoted strings, you can use the single backslash as you wish:
command:
- /bin/bash
- -c
- export MACHINE_TYPE=$(curl --unix-socket /var/run/docker.sock http://docker/containers/rancher-agent/json | grep -oP 'CATTLE_HOST_LABELS=.+?\w+' | awk -F '=' '{print $2}')
and benefiting from the infinitely more legible syntax
Separately,
final container environment variable (LABEL) from the daemonset.yaml
In that case, you want the valueFrom: fieldPath: status.nodeName in your env: block to set HOSTNAME, and then (as you are currently doing) build up the LABEL from its MACHINE_TYPE component right before executing the actual in-container command. You cannot (that I am aware of) declare LABEL in the kubernetes descriptor's env block because you are mixing metaphors trying to run a command in a container that affects the kubernetes descriptor for that command.
I know that the value value: $MACHINE_TYPE-$HOSTNAME will not work, so hoping for assistance with that as well. Thanks in advance.
There are plenty of existing SO answers about that, but the syntax is $(MACHINE_TYPE)-$(HOSTNAME) assuming those two env-vars are declared in the Pod's env: block, and not otherwise