Following this Github Action auto deploy to GKE workflow:
https://docs.github.com/en/actions/deployment/deploying-to-your-cloud-provider/deploying-to-google-kubernetes-engine
In the last step notice these lines:
./kustomize edit set image gcr.io/PROJECT_ID/IMAGE:TAG=gcr.io/$PROJECT_ID/$IMAGE:$GITHUB_SHA
./kustomize build . | kubectl apply -f -
How does Kustomize knows to which file to change the image?
Does he finds the file by searching and then grab it fully and just apply on it?
How does it work?
How does Kustomize knows to which file to change the image? Does he finds the file by searching and then grab it fully and just apply on it?
Kustomize doesn't know in which file to change the image. For the most part, Kustomize doesn't care about "files". In this case, the command is adding a configuration for the image transformer. Running that kustomize edit command...
kustomize edit set image gcr.io/PROJECT_ID/IMAGE:TAG=gcr.io/$PROJECT_ID/$IMAGE:$GITHUB_SHA
Adds a configuration like this to your kustomization.yaml:
images:
- name: gcr.io/PROJECT_ID/IMAGE:TAG
newName: gcr.io/my_project/my_image
newTag: 12345678
This means "whenever you find an image reference for gcr.io/PROJECT_ID/IMAGE:TAG, replace it with the given values". This will work for deployments, pods, statefulsets, daemonsets, and all other native kubernetes resources that contain image references.
Related
When operating a k8s cluster with an admission controller that limits the allowed registries, it's desirable to check manifests to verify they only refer to such images.
Is there a well established and correct way to process a Kubernetes manifest stream such as Kustomize output and list all container images it references? Including all Deployments, StatefulSets, Jobs, CRDs that embed PodTemplate, etc?
I landed up writing my own, then realised this must be a solved problem. kustomize has the images: transformer to rewrite images, but it doesn't seem to be able to list the candidates the transformer inspects. Surely there's something?
For example, if I have the kustomization.yaml:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- "https://github.com/prometheus-operator/kube-prometheus"
I want to be able to run some getimages filter such that
kustomize build | getimages
returns the same list as this hacky shell pipeline example:
$ kustomize build|grep 'image:' | awk '$2 != "" { print $2}' | sort -u
grafana/grafana:8.4.6
jimmidyson/configmap-reload:v0.5.0
k8s.gcr.io/kube-state-metrics/kube-state-metrics:v2.4.2
k8s.gcr.io/prometheus-adapter/prometheus-adapter:v0.9.1
quay.io/brancz/kube-rbac-proxy:v0.12.0
quay.io/prometheus/alertmanager:v0.24.0
quay.io/prometheus/blackbox-exporter:v0.20.0
quay.io/prometheus/node-exporter:v1.3.1
quay.io/prometheus-operator/prometheus-operator:v0.55.1
quay.io/prometheus/prometheus:v2.34.0
... but in a robust and correct manner, unlike said hacky shell command.
I expected tools like kubeval or kustomize to be able to do this, but have drawn blanks in all searching.
Edit: There is kustomize cfg tree --image to list images, but:
It doesn't integrate with the kustomize image transformer's configuration so it won't recognise images in CRDs. You have to add additional --field specs for each one manually.
Its output format is painful if you just want the image names.
In my GitHub repo I have 2 yaml files:
k8s/deploy-all-secrets.yaml
k8s/deploy-edge.yaml
I make use of cloud.okteto.com to deploy this deployment-file. But I don't want Okteto to deploy the deploy-all-secrets.yaml file. Is there any way I can exclude this file from Okteto?
I tried using a .stignore file, but this had no result.
Another option you have is to create an okteto-pipeline.yaml file at the root of your repo. This allows you to control how Okteto deploys your pipeline. For the scenario you describe, it would look like this:
deploy:
- kubectl apply -f deploy-all-secrets.yaml
More information on how to customize your pipelie is available here.
Note: The .stignore file is only used by the okteto up command, during the file synchronization phase. More information on that is available here.
According to the documentation https://okteto.com/docs/cloud/okteto-pipeline/.
You can place any k8s manifest file that needs to be executed using kubectl apply in this folder. So by simply removing the deploy-all-secrets.yaml from the k8s folder, it won't be executed by Okteto.
The official kubernetes guidelines, instructs on updating the deployment either by performing a command line set:
kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
or by inline editing (that will launch the default editor I guess)
kubectl edit deployment/nginx-deployment
However both processes make consistency more difficult given that one needs to go and udpate offline the my-deployment.yml file, where the up & running deployment came from. (and this deprives one from the advantage of keeping their manifests version-controlled).
Is there a way to
launch a deployment via the file
perform (when needed) updates to the same file
update the deployment by pointing to the same, updated file?
You can do it simply by following steps -
Edit the deployment.yaml file
Run below command -
kubectl apply -f deployment.yaml
This is what I usually follow. You can use a kubectl patch or edit also.
Kubernetes newbie question: Can I somehow include my pod definition inside my deployment definition?
I currently have a pod.yml, a service.yml and a deployment.yml file. Both in pod.yml and deployment.yml I specify my docker image and tag: user/image:v1.
To do a rolling update, I tried doing kubectl set image deployment/api api=user/image:v2
However, that doesnt work alone.. It seems to conflict with the image tag in the pod definition. I need to also update the pod with tag v2 for kubectl set image to work.. I feel like I'm doing something wrong. Thoughts?
Yes, you can include all definitions in one file. Have a look at the guestbook-all-in-one.yaml example.
The recommended way to do a rolling update is to change the file and then use apply:
$ vim guestbook-all-in-one.yaml # make the desired changes
$ kubectl apply -f guestbook-all-in-one.yaml # apply these changes
If possible, you should also have this file under version control, so that the file with the current status is always easily accessible.
So in order to update the images running on a pod, I have to modify the deployment config (yaml file), and run something like kubectl apply -f deploy.yaml.
This means, if I'm not editing the yaml file manually I'll have to use some template / search and replace functionality. Which isn't really ideal.
Are there any better approaches?
It seems there is a kubectl rolling-update command, but I'm not sure if this works for 'deployments'.
For example running the following: kubectl rolling-update wordpress --image=eu.gcr.io/abcxyz/wordpress:deploy-1502443760
Produces an error of:
error: couldn't find a replication controller with source id == default/wordpress
I am using this for changing images in Deployments:
kubectl set image deployment/nginx-deployment nginx=nginx:1.9.1
If you view the yaml files as source of truth then use a tag like stable in the yaml and only issue kubectl set image commands when the tag is moved (use the sha256 image id to actually trigger a rollout; the image names are matched like a string so updating from :stable to :stable is a noop even if the tag now points to a different image).
See updating a deployment for more details.
The above requires the deployment replica count to be set more then 1, which is explained here: https://stackoverflow.com/a/45649024/1663462.