I have an image pushed to Google Container Registry, named gcr.io/$(PROJECT_ID)/img-name:46d49ab.
In my replication controller I have:
apiVersion: v1
kind: ReplicationController
metadata:
name: go-server-rc
spec:
replicas: 3
selector:
name: go-server
version: v8
template:
metadata:
labels:
name: go-server
version: v8
spec:
containers:
- name: go-server
image: gcr.io/$(PROJECT_ID)/img-name:46d49ab
ports:
- containerPort: 5000
This works, but it doesn't when I remove the commit hash tag 46d49ab. I don't want to have to change the tag every single time I commit.
I have also set up a trigger on Google Container Builder to pull the master branch of my repository after every commit, and create an image gcr.io/$(PROJECT_ID)/img-name:$(COMMIT_HASH).
How can I edit my replication controller file to just get the most recent? What workflows do people use?
It's possible to use the latest tag to ensure Kubernetes pulls the image each time it runs. Every time you create a new image tag it with latest and push it to the container registry. However, I would not recommend this.
You won't know which pods are running which version of your code. I do exactly as you mention in your question. I find it's better to update your deployment object each time your image updates. This will ensure the deployment is in the state you expect and troubleshooting will be clearer when looking at images.
Related
I'm currently looking at GKE and some of the tutorials on google cloud. I was following this one here https://cloud.google.com/solutions/integrating-microservices-with-pubsub#building_images_for_the_app (source code https://github.com/GoogleCloudPlatform/gke-photoalbum-example)
This example has 3 deployments and one service. The example tutorial has you deploy everything via the command line which is fine and all works. I then started to look into how you could automate deployments via cloud build and discovered this:
https://cloud.google.com/build/docs/deploying-builds/deploy-gke#automating_deployments
These docs say you can create a build configuration for your a trigger (such as pushing to a particular repo) and it will trigger the build. The sample yaml they show for this is as follows:
# deploy container image to GKE
- name: "gcr.io/cloud-builders/gke-deploy"
args:
- run
- --filename=kubernetes-resource-file
- --image=gcr.io/project-id/image:tag
- --location=${_CLOUDSDK_COMPUTE_ZONE}
- --cluster=${_CLOUDSDK_CONTAINER_CLUSTER}
I understand how the location and cluster parameters can be passed in and these docs also say the following about the resource file (filename parameter) and image parameter:
kubernetes-resource-file is the file path of your Kubernetes configuration file or the directory path containing your Kubernetes resource files.
image is the desired name of the container image, usually the application name.
Relating this back to the demo application repo where all the services are in one repo, I believe I could supply a folder path to the filename parameter such as the config folder from the repo https://github.com/GoogleCloudPlatform/gke-photoalbum-example/tree/master/config
But the trouble here is that those resource files themselves have an image property in them so I don't know how this would relate to the image property of the cloud build trigger yaml. I also don't know how you could then have multiple "image" properties in the trigger yaml where each deployment would have it's own container image.
I'm new to GKE and Kubernetes in general, so I'm wondering if I'm misinterpreting what the kubernetes-resource-file should be in this instance.
But is it possible to automate deploying of multiple deployments/services in this fashion when they're all bundled into one repo? Or have Google just over simplified things for this tutorial - the reality being that most services would be in their own repo so as to be built/tested/deployed separately?
Either way, how would the image property relate to the fact that an image is already defined in the deployment yaml? e.g:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: photoalbum-app
name: photoalbum-app
spec:
replicas: 3
selector:
matchLabels:
name: photoalbum-app
template:
metadata:
labels:
name: photoalbum-app
spec:
containers:
- name: photoalbum-app
image: gcr.io/[PROJECT_ID]/photoalbum-app#[DIGEST]
tty: true
ports:
- containerPort: 8080
env:
- name: PROJECT_ID
value: "[PROJECT_ID]"
The command that you use is perfect for testing the deployment of one image. But when you work with Kubernetes (K8S), and the managed version of GCP (GKE), you usually never do this.
You use YAML file to describe your deployments, services and all other K8S object that you want. When you deploy, you can perform something like this
kubectl apply -f <file.yaml>
If you have several file, you can use wildcard is you want
kubectl apply -f config/*.yaml
If you prefer to use only one file, you can separate the object with ---
apiVersion: v1
kind: Service
metadata:
name: my-nginx-svc
labels:
app: nginx
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
labels:
app: nginx
spec:...
...
I'm using K8s on GCP.
Here is my deployment file
apiVersion: apps/v1
kind: Deployment
metadata:
name: simpleapp-direct
labels:
app: simpleapp-direct
role: backend
stage: test
spec:
replicas: 1
selector:
matchLabels:
app: simpleapp-direct
version: v0.0.1
template:
metadata:
labels:
app: simpleapp-direct
version: v0.0.1
spec:
containers:
- name: simpleapp-direct
image: gcr.io/applive/simpleapp-direct:latest
imagePullPolicy: Always
I first apply the deployment file with kubectl apply command
kubectl apply -f deployment.yaml
The pods were created properly.
I was expecting that every time I would push a new image with the tag latest, the pods would be automatically killed and restart using the new images.
I tried the rollout command
kubectl rollout restart deploy simpleapp-direct
The pods restart as I wanted.
However, I don't want to run this command every time there is a new latest build.
How can I handle this situation ?.
Thanks a lot
Try to use image hash instead of tag in your Pod Definition.
Generally: there is no way to automatically restart pods when the new image is ready. It is generally advisable not to use image:latest (or just image_name) in Kubernetes as it can cause difficulties with rollback of your deployment. Also you need to make sure that the flag: imagePullPolicy is set to Always. Normally when you use CI/CD or git-ops your deployment is updated automatically by these tools when the new image is ready and passed thru the tests.
When your Docker image is updated, you need to setup a trigger on this update within your CI/CD pipilne to re-run the deployment. Not sure about the base system/image where you build your docker image, but you can add there kubernetes certificates and run the above commands like you do on your local computer.
This question already has answers here:
Kubernetes: kubectl apply does not update pods when using "latest" tag
(4 answers)
Closed 3 years ago.
Problem
A frequent question that comes up on Slack and Stack Overflow is how to trigger an update to a Deployment/RS/RC when the image tag hasn't changed but the underlying image has.
Consider:
There is an existing Deployment with image foo: latest
The user builds a new image foo: latest
The user pushes foo: latest to their registry
The user wants to do something here to tell the Deployment to pull the new image and do a rolling-update of existing pods
For Example:
apiVersion: apps/v1
kind: Deployment
metadata:
name: worker-deployment
spec:
replicas: 1
selector:
matchLabels:
component: worker
template:
metadata:
labels:
component: worker
spec:
containers:
- name: worker
image: stephengrider/multi-worker
env:
- name: REDIS_HOST
value: redis-cluster-ip-service
- name: REDIS_PORT
value: "6379"
And now I have made changes in my multiWorker image
and redeploys the deployment, but it doesn't fetch the latest image?
The default pull policy is IfNotPresent which causes the Kubelet to skip pulling an image if it already exists. Refer Link
If you would like to always force a pull, you can do one of the following:
1: set the imagePullPolicy of the container to Always.
2: omit the imagePullPolicy and use :latest as the tag for the image to use.
3: omit the imagePullPolicy and the tag for the image to use.
4: enable the AlwaysPullImages admission controller.
Note that you should avoid using :latest tag, we should avoid using the :latest tag when deploying containers in production as it is harder to track which version of the image is running and more difficult to roll back properly.
Various imagePullPolicy and the tag of the image affect when the kubelet attempts to pull the specified image listed below. Refer Link for Details
1) imagePullPolicy: IfNotPresent: the image is pulled only if it is not already present locally.
2) imagePullPolicy: Always: the image is pulled every time the pod is started.
3) imagePullPolicy is omitted and either the image tag is :latest or it is omitted: Always is applied.
4) imagePullPolicy is omitted and the image tag is present but not :latest: IfNotPresent is applied.
5) imagePullPolicy: Never: the image is assumed to exist locally. No attempt is made to pull the image.
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: test
spec:
replicas: 1
template:
metadata:
labels:
app: test
spec:
containers:
- name: test
image: xxx:latest
ports:
- containerPort: 80
imagePullPolicy: Always
imagePullSecrets:
- name: aaaa
I use the tag "latest". When I update the image,and the new image is still "latest". When I "kubectl set image deployments/test test=xxx:latest",nothing happend. What should I do?
a RollingUpdate is always triggered when the PodTemplateSpec under template is changed.
While using the :latest tag is not suggested it can still work when using imagePullPolicy: Always and a label which is changed with every image adjustment. Sth like this:
kubectl patch deployment test -p "{\"spec\":{\"template\":{\"metadata\":{\"labels\":{\"date\":\"$(date +%s)\"}}}}}"
Rolling update depends on docker image tags. If you're using the latest tag in your deployment, you need to cut a new image with a new version.
The deployment resource can't determine if the images has changed if you're always using the latest tag. As far as k8s is concerned, you're already running an image with the tag latest, so it doesn't have anything to do.
It's highly recommended not to use latest for deployments for this reason. You'll have a much easier time if you version your docker images correctly.
I have a Deployment with 3 replicas of a pod running the image 172.20.20.20:5000/my_app, that is located in my private registry.
I want do a rolling-update in the deployment when I push a new latest version of that image to my registry.
I push the new image this way (tag v3.0 to latest):
$ docker tag 172.20.20.20:5000/my_app:3.0 172.20.20.20:5000/my_app
$ docker push 172.20.20.20:5000/my_app
But nothing happens. Pods' images are not upgraded. This is my deployment definition:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: myapp-deployment
spec:
replicas: 3
template:
metadata:
labels:
app: myapp
spec:
containers:
- name: app
image: 172.20.20.20:5000/my_app:latest
ports:
- containerPort: 8080
Is there a way to do that automatically? Should I run a command like rolling-update like in ReplicaControllers?
In order to upgrade a Deployment you have to modify the Deployment resource with the new image. So for example, change 172.20.20.20:5000/my_app:v1 to 172.20.20.20:5000/my_app:v2. Since you're just modifying the image within the Docker registry doesn't notice the change.
If you (manually) kill the individual Pods, the Deployment will restart them. Since the Deployment image specifies the "latest" tag Kubernetes will download the latest version (now "v3" in your case) due to the implied "Always" ImagePullPolicy.