kubernetes: selector field missing in kompose - kubernetes

I have a docker-compose.yml file we have been using to set up our development environment.
The file declares some services, all of them more or less following the same pattern:
services:
service_1:
image: some_image_1
enviroment:
- ENV_VAR_1
- ENV_VAR_2
depends_on:
- another_service_of_the_same_compose_file
In the view of migrating to kubernetes, when running:
kompose convert -f docker-compose.yml
produces, for each service, a pair of deployment/service manifests.
Two questions about the deployment generated:
1.
the examples in the official documentation seem to hint that the selector field is needed for a Deployment to be aware of the pods to manage.
However the deployment manifests created do not include a selector field, and are as follows:
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
kompose.cmd: kompose convert -f docker-compose.yml
kompose.version: 1.6.0 (e4adfef)
creationTimestamp: null
labels:
io.kompose.service: service_1
name: service_1
spec:
replicas: 1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
io.kompose.service: service_1
spec:
containers:
- image: my_image
name: my_image_name
resources: {}
restartPolicy: Always
status: {}
2.
the apiVersion in the generated deployment manifest is extensions/v1beta1, however the examples in the Deployments section of the official documentation default to apps/v1.
The recommendation seems to be
for versions before 1.9.0 use apps/v1beta2
Which is the correct version to use? (using kubernetes 1.8)

Let's begin by saying that Kubernetes and Kompose are two different independent systems. Kompose is trying to match all of the dependency with kubernetes.
At the moment all of the selector's fields are generated by kubernetes.In future, It might be done by us.
If you would like to check your selector's fields use following commands
kubectl get deploy
kubectl describe deploy DEPLOY_NAME
After version k8s 1.9 all of the long-running objects would be part of /apps group.
We’re excited to announce General Availability (GA) of the apps/v1 Workloads API, which is now enabled by default. The Apps Workloads API groups the DaemonSet, Deployment, ReplicaSet, and StatefulSet APIs together to form the foundation for long-running stateless and stateful workloads in Kubernetes. Note that the Batch Workloads API (Job and CronJob) is not part of this effort and will have a separate path to GA stability.
I have attached the link for further research
kubernetes-19-workloads

As a selector field isn't required for deployments and Kompose doesn't know your cluster's nodes, it doesn't set a selector (which basically tells k8s in which nodes run pods).
I wouldn't edit apiversion cause Kompose assumes that version defining the rest of the resource. Also, if you are using kubernetes 1.8 read 1.8 docs https://v1-8.docs.kubernetes.io/docs/

In kubernetes 1.16 the deployment's spec.selector became required. Kompose (as of 1.20) does not yet do this automatically. You will have to add this to every *-deployment.yaml file it creates:
selector:
matchLabels:
io.kompose.service: alignment-processor
If you use an IDE like jetbrains you can use the following search/replace patterns on the folder where you put the conversion results:
Search for this multiline regexp:
io.kompose.service: (.*)
name: \1
spec:
replicas: 1
template:
Replace with this pattern:
io.kompose.service: $1
name: $1
spec:
replicas: 1
selector:
matchLabels:
io.kompose.service: $1
template:
The (.*) captures the name of the service, the \1 matches the (first and only) capture, and the $1 substitutes the capture in the replacement.
You will also have to substitute all extensions/v1beta1 with apps/v1 in all *-deployment.yaml files.
I also found that secrets have to massaged a bit but this goes beyond the scope of this question.

Related

Kubectl error upon applying agones fleet: ensure CRDs are installed first

I am using minikube (docker driver) with kubectl to test an agones fleet deployment. Upon running kubectl apply -f lobby-fleet.yml (and when I try to apply any other agones yaml file) I receive the following error:
error: resource mapping not found for name: "lobby" namespace: "" from "lobby-fleet.yml": no matches for kind "Fleet" in version "agones.dev/v1"
ensure CRDs are installed first
lobby-fleet.yml:
apiVersion: "agones.dev/v1"
kind: Fleet
metadata:
name: lobby
spec:
replicas: 2
scheduling: Packed
template:
metadata:
labels:
mode: lobby
spec:
ports:
- name: default
portPolicy: Dynamic
containerPort: 7600
container: lobby
template:
spec:
containers:
- name: lobby
image: gcr.io/agones-images/simple-game-server:0.12 # Modify to correct image
I am running this on WSL2, but receive the same error when using the windows installation of kubectl (through choco). I have minikube installed and running for ubuntu in WSL2 using docker.
I am still new to using k8s, so apologies if the answer to this question is clear, I just couldn't find it elsewhere.
Thanks in advance!
In order to create a resource of kind Fleet, you have to apply the Custom Resource Definition (CRD) that defines what is a Fleet first.
I've looked into the YAML installation instructions of agones, and the manifest contains the CRDs. you can find it by searching kind: CustomResourceDefinition.
I recommend you to first try to install according to the instructions in the docs.

GKE automating deploy of multiple deployments/services with different images

I'm currently looking at GKE and some of the tutorials on google cloud. I was following this one here https://cloud.google.com/solutions/integrating-microservices-with-pubsub#building_images_for_the_app (source code https://github.com/GoogleCloudPlatform/gke-photoalbum-example)
This example has 3 deployments and one service. The example tutorial has you deploy everything via the command line which is fine and all works. I then started to look into how you could automate deployments via cloud build and discovered this:
https://cloud.google.com/build/docs/deploying-builds/deploy-gke#automating_deployments
These docs say you can create a build configuration for your a trigger (such as pushing to a particular repo) and it will trigger the build. The sample yaml they show for this is as follows:
# deploy container image to GKE
- name: "gcr.io/cloud-builders/gke-deploy"
args:
- run
- --filename=kubernetes-resource-file
- --image=gcr.io/project-id/image:tag
- --location=${_CLOUDSDK_COMPUTE_ZONE}
- --cluster=${_CLOUDSDK_CONTAINER_CLUSTER}
I understand how the location and cluster parameters can be passed in and these docs also say the following about the resource file (filename parameter) and image parameter:
kubernetes-resource-file is the file path of your Kubernetes configuration file or the directory path containing your Kubernetes resource files.
image is the desired name of the container image, usually the application name.
Relating this back to the demo application repo where all the services are in one repo, I believe I could supply a folder path to the filename parameter such as the config folder from the repo https://github.com/GoogleCloudPlatform/gke-photoalbum-example/tree/master/config
But the trouble here is that those resource files themselves have an image property in them so I don't know how this would relate to the image property of the cloud build trigger yaml. I also don't know how you could then have multiple "image" properties in the trigger yaml where each deployment would have it's own container image.
I'm new to GKE and Kubernetes in general, so I'm wondering if I'm misinterpreting what the kubernetes-resource-file should be in this instance.
But is it possible to automate deploying of multiple deployments/services in this fashion when they're all bundled into one repo? Or have Google just over simplified things for this tutorial - the reality being that most services would be in their own repo so as to be built/tested/deployed separately?
Either way, how would the image property relate to the fact that an image is already defined in the deployment yaml? e.g:
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
name: photoalbum-app
name: photoalbum-app
spec:
replicas: 3
selector:
matchLabels:
name: photoalbum-app
template:
metadata:
labels:
name: photoalbum-app
spec:
containers:
- name: photoalbum-app
image: gcr.io/[PROJECT_ID]/photoalbum-app#[DIGEST]
tty: true
ports:
- containerPort: 8080
env:
- name: PROJECT_ID
value: "[PROJECT_ID]"
The command that you use is perfect for testing the deployment of one image. But when you work with Kubernetes (K8S), and the managed version of GCP (GKE), you usually never do this.
You use YAML file to describe your deployments, services and all other K8S object that you want. When you deploy, you can perform something like this
kubectl apply -f <file.yaml>
If you have several file, you can use wildcard is you want
kubectl apply -f config/*.yaml
If you prefer to use only one file, you can separate the object with ---
apiVersion: v1
kind: Service
metadata:
name: my-nginx-svc
labels:
app: nginx
spec:
type: LoadBalancer
ports:
- port: 80
selector:
app: nginx
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-nginx
labels:
app: nginx
spec:...
...

Google Stackdriver - how can I use my Kubernetes YAML labels for Stackdriver Log Query?

When using Google Stackdriver I can use the log query to find the exact log statements I am looking for.
This might look like this:
resource.type="k8s_container"
resource.labels.project_id="my-project"
resource.labels.location="europe-west3-a"
resource.labels.cluster_name="my-cluster"
resource.labels.namespace_name="dev"
resource.labels.pod_name="my-app-pod-7f6cf95b6c-nkkbm"
resource.labels.container_name="container"
However as you can see in this query argument resource.labels.pod_name="my-app-pod-7f6cf95b6c-nkkbm" that I am looking for a pod with the id 7f6cf95b6c-nkkbm. Because of this I can not use this Stackdriver view with this exact query if I deployed a new revision of my-app therefore having a new ID and the one in the curreny query becomes invalid or not locatable.
Now I don't always want to look for the new ID every time I want to have the current view of my my-app logs. So I tried to add a special label stackdriver: my-app to my Kubernetes YAML file.
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-app
spec:
template:
metadata:
labels:
stackdriver: my-app <<<
Revisiting my newly deployed Pod I can assure that the label stackdriver: my-app is indeed existing.
Now I want to add this new label to use as a query argument:
resource.type="k8s_container"
resource.labels.project_id="my-project"
resource.labels.location="europe-west3-a"
resource.labels.cluster_name="my-cluster"
resource.labels.namespace_name="dev"
resource.labels.pod_name="my-app-pod-7f6cf95b6c-nkkbm"
resource.labels.container_name="container"
resource.labels.stackdriver=my-app <<< the kubernetes label
As you can guess this did not work otherwise I'd have no reason to write this question ;)
Any idea how the thing I am about to do can be achieved?
Any idea how the thing I am about to do can be achieved?
Yes! In fact, I've prepared an example to show you the whole process :)
Let's assume:
You have a GKE cluster named: gke-label
You have a Cloud Operations for GKE enabled (logging)
You have a Deployment named nginx with a following label:
stackdriver: look_here_for_me
deployment.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
stackdriver: look_here_for_me
replicas: 1
template:
metadata:
labels:
app: nginx
stackdriver: look_here_for_me
spec:
containers:
- name: nginx
image: nginx
You can apply this definition and send some traffic from the other pod so that the logs could be generated. I've done it with:
$ kubectl run -it --rm --image=ubuntu ubuntu -- /bin/bash
$ apt update && apt install -y curl
$ curl NGINX_POD_IP_ADDRESS/NONEXISTING # <-- this path is only for better visibility
After that you can go to:
GCP Cloud Console (Web UI) -> Logging (I used new version)
With the following query:
resource.type="k8s_container"
resource.labels.cluster_name="gke-label"
-->labels."k8s-pod/stackdriver"="look_here_for_me"
You should be able to see the container logs as well it's label:

How to deploy a bunch of yaml files?

I would like to deploy a bunch of yaml files https://github.com/quay/quay/tree/master/deploy/k8s on my kubernetes cluster and would like to know, what is the best approach to deploy these at once.
You can directly apply folder
kubectl create -f ./<foldername>
kubectl apply -f ./<foldername>
You can also add mutiliple files in one command
kubectl apply -f test.yaml,test-1.yaml
You can also merge all YAML files into a single file and manage it further.
Marge YAML file using ---
For example :
apiVersion: v1
kind: Service
metadata:
name: test-data
labels:
app: test-data
spec:
ports:
- name: http
port: 80
targetPort: 9595
- name: https
port: 9595
targetPort: 9595
selector:
app: test-data
tier: frontend
---
apiVersion: v1
kind: Service
metadata:
name: test-app
labels:
app: test-app
spec:
ports:
- name: http
port: 80
targetPort: 9595
- name: https
port: 9595
targetPort: 9595
selector:
app: test-app
tier: frontend
kubectl apply -f <folder-name>
A simple way to deploy all files in a given folder.
You may consider using Helm (The package manager for Kubernetes). Just like we use yum or apt-get for Linux, we use helm for k8s.
Using Helm, you can deploy multiple resources (bunch of YAMLs) in one go. Helm Charts help you define, install, and upgrade even the most complex Kubernetes application. Helm uses a packaging format called charts. A chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on. Also, you don't need to combine all your YAMLs; they can remain separate as part of a given chart. Besides, if one chart depends on another, you can use the helm dependency feature.
The reason why i use Helm is because whenever i deploy a chart, helm tracks it as a release. Any change to a chart get a new release version. This way, upgrade (or rollback) becomes very easy and you can confidently say what went as part of a given release.
Also, if you have different microservices that have stuff in common, then helm provides a feature called Library Chart using which you can create definitions that can be re-used across charts, thus keeping your charts DRY.
Have a look at this introductory video: https://www.youtube.com/watch?v=Zzwq9FmZdsU&t=2s
I would advise linking the yaml's into one. The purpose of a deployment and service yaml is to deploy your application onto the cluster in one fell swoop. You can define many deployments and services within the one file. In your case, a tool such as Kustomize will help you combine them. Kustomize comes preinstalled with kubectl.
You can combine your yamls called a Multi-Resource yaml into one file using the --- operator. i.e.
apiVersion: v1
kind: Service
metadata:
name: foo
spec:
...
---
apiVersion: v1
kind: Service
metadata:
name: bar
spec:
...
Then make a kustomization.yaml which combines all your multi-resource yamls. There is a good guide on this here: https://levelup.gitconnected.com/kubernetes-merge-multiple-yaml-into-one-e8844479a73a
The documentation from k8 is here: https://kubernetes.io/docs/tasks/manage-kubernetes-objects/kustomization/

how to set different environment variables of Deployment replicas in kubernetes

I have 4 k8s pods by setting the replicas of Deployment to 4 now.
apiVersion: v1
kind: Deployment
metadata:
...
spec:
...
replicas: 4
...
The POD will get items in a database and consume it, the items in database has a column class_name.
now I want one pod only get one class_name's item.
for example pod1 only get item which class_name equals class_name_1, and pod2 only get item which class_name equals class_name_2...
So I want to pass different class_name as environment variables to different Deployment PODs. Can I define it in the yaml file of Deployment?
Or is there any other way to achieve my goal?(like something other than Deployment in k8s)
For distributed job processing Deployments are not very good, because they don't have any type of ordering or consistent pod hostnames. You'd better use StatefulSet for it, because they have consistent naming, like pod-0, pod-1, pod-2. You can rely on that hostname index.
For example, if your class_name_idx - is the index of class name in class names list, num_replicas - is the number of replicas in StatefulSet and pod_idx - is the index of pod in StatefulSet, then pod should run the job only if: class_name_idx % num_replicas == pod_idx.
Unfortunately number of StatefulSet replicas cannot be obtained within the pod dynamically using Downward API, so you can either hardcode it or use Kubernetes API to obtain it from cluster.
Neither Deployment nor anything else won't help to achieve your goal. Your goal is some kind of logic and it should be implemented via code in your application.
Since the Deployment is some instances of the same application the only thing that might be useful for you is: using multiple deployments, each for its own task. The first could get class_name_1 item, while other class_name_2, class_name_3 etc. But it is not a good idea
I would not recommend this approach, but the closest thing to do what you want is using the stateful-set and use the pod name as the index.
When you deploy a stateful set, the pods will be named after their statefulset name, in the following sample:
apiVersion: v1
kind: Service
metadata:
name: kuard
labels:
app: kuard
spec:
type: NodePort
ports:
- port: 8080
name: web
selector:
app: kuard
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: kuard
spec:
serviceName: "kuard"
replicas: 3
selector:
matchLabels:
app: kuard
template:
metadata:
labels:
app: kuard
spec:
containers:
- name: kuard
image: gcr.io/kuar-demo/kuard-amd64:1
ports:
- containerPort: 8080
name: web
The pods created by the statefulset will be named as:
kuard-0
kuard-1
kuard-2
This way you could either, name the stateful-set according to the classes, i.e: class-name and the pod created will be class-name-0 and you can replace the _ by -. Or just strip the name out to get the index at the end.
To get the name just read the environment variable HOSTNAME
This naming is consistent, so you can make sure you always have 0, 1, 2, 3 after the name. And if the 2 goes down, it will be recreated.
Like I said, I would not recommend this approach because you tie the infrastructure to your code, and also can't scale(if needed) because each service are unique and adding new instances would get new ids.
A better approach would be using one deployment for each class and pass the proper values as environment variables.