As I have seen few related posts but none answered my question, I thought I would ask a new question based on suggestions from other users as well here.
I have the need to make a selector label for a network policy for a running cronjob that is responsible to connect to some other services within the cluster, as far as I know there is no easy straight forward way to make a selector label for the jobs pod as that would be problematic with duplicate job labels if they ever existed. Not sure why the cronjob can't have a selector itself, and then can be applied to the job and the pod.
also there might be a possibility to just set this cronjob in its own namespace and then allow all from that one namespace to whatever needed in the network policy but does not feel like the right way to overcome that problem.
Using k8s v1.20
First of all, to select pods (spawned by your CronJob) that should be allowed by the NetworkPolicy as ingress sources or egress destinations, you may set specific label for those pods.
You can easily set a label for Jobs spawned by CronJob using labels field (another example with an explanation can be found in the OpenShift CronJobs documentation):
---
apiVersion: batch/v1beta1
kind: CronJob
metadata:
name: mysql-test
spec:
...
jobTemplate:
spec:
template:
metadata:
labels:
workload: cronjob # Sets a label for jobs spawned by this CronJob.
type: mysql # Sets another label for jobs spawned by this CronJob.
...
Pods spawned by this CronJob will have the labels type=mysql and workload=cronjob, using this labels you can create/customize your NetworkPolicy:
$ kubectl get pods --show-labels
NAME READY STATUS RESTARTS AGE LABELS
mysql-test-1615216560-tkdvk 0/1 Completed 0 2m2s ...,type=mysql,workload=cronjob
mysql-test-1615216620-pqzbk 0/1 Completed 0 62s ...,type=mysql,workload=cronjob
mysql-test-1615216680-8775h 0/1 Completed 0 2s ...,type=mysql,workload=cronjob
$ kubectl describe pod mysql-test-1615216560-tkdvk
Name: mysql-test-1615216560-tkdvk
Namespace: default
...
Labels: controller-uid=af99e9a3-be6b-403d-ab57-38de31ac7a9d
job-name=mysql-test-1615216560
type=mysql
workload=cronjob
...
For example this mysql-workload NetworkPolicy allows connections to all pods in the mysql namespace from any pod with the labels type=mysql and workload=cronjob (logical conjunction) in a namespace with the label namespace-name=default :
NOTE: Be careful to use correct YAML (take a look at this namespaceSelector and podSelector example).
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: mysql-workload
namespace: mysql
spec:
podSelector: {}
policyTypes:
- Ingress
ingress:
- from:
- namespaceSelector:
matchLabels:
namespace-name: default
podSelector:
matchLabels:
type: mysql
workload: cronjob
To use network policies, you must be using a networking solution which supports NetworkPolicy:
Network policies are implemented by the network plugin. To use network policies, you must be using a networking solution which supports NetworkPolicy. Creating a NetworkPolicy resource without a controller that implements it will have no effect.
You can learn more about creating Kubernetes NetworkPolicies in the Network Policies documentation.
Related
I have tried defining LoadBalancer in my service type and creating a deployment for it with 3 replicas:
kind: Service
apiVersion: v1
metadata:
name: springboot-postgres-k8s
labels:
name: springboot-postgres-k8s
spec:
ports: # ...
selector: # type: ...
type: LoadBalancer # <=====
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: springboot-postgres-k8s
spec:
selector:
matchLabels:
app: springboot-postgres-k8s
replicas: 3 # <=====
template: # ...
This starts up three pod and a load balancer which successfully load balances request among these three pods.
I want to know if k8s allows to manually autoscale the pods. That is if my cluster with 3 replicas + a load balancer is up and running, how can I manually increase replicas and still make the existing load balancer to scale across all 4 replicas (3 old and one newly created)?
Do I have to run (ref1 ref2):
kubectl scale --current-replicas=3 --replicas=4 deployment/springboot-postgres-k8s
Q1. Will above command notify existing load balancer of newly created pod?
or I have to run following (as specified in ref2)
kubectl scale --replicas=4 -f foo.yaml
Q2. Will above command notify existing load balancer of newly created pod?
Q3. What if my foo.yaml contains both service and deployment definition?
Yes its allows manual autoscale.
When you create a service in Kubernetes, k8s automatically creates kind: Endpoints type resource(s) for your pods with a matching label selector. This resource referenced by a Service to define which Pods the traffic can be sent to and periodically updated by the k8s when pods are created or deleted.
So regardless of the creation time of resources, k8s will handle the update and pods will be able to receive traffic from loadbalancer.
I have this Deployment object:
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment-webserver-nginx
annotations:
description: This is a demo deployment for nginx webserver
labels:
app: deployment-webserver-nginx
spec:
replicas: 3
selector:
matchLabels:
app: deployment-webserver-pods
template:
metadata:
labels:
app: deployment-webserver-pods
spec:
containers:
- name: nginx
image: nginx:alpine
ports:
- containerPort: 80
My understanding on this Deployment object is that any Pod with app:deployment-webserver-pods label will be selected. Of course, this Deployment object is creating 3 replicas, but I wanted to add one more Pod explicitly like this, so I created a Pod object and had its label as app:deployment-webserver-pods, below is its Pod definition:
apiVersion: v1
kind: Pod
metadata:
name: deployment-webserver-nginx-extra-pod
labels:
app: deployment-webserver-pods
spec:
containers:
- name: nginx-alpine-container-1
image: nginx:alpine
ports:
- containerPort: 81
My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.
I even tried to first create this pod with this label, and then created my Deployment and thought that maybe now this explicit Pod will be picked but still that didn't happen.
Doesn't Labels and Selectors work like this?
I know I can scale by deployment to 4 Replicas, but I am trying to understand how Pods / other Kubernetes objects are selected using Labels and Selectors.
From the official docs:
Note: You should not create other Pods whose labels match this
selector, either directly, by creating another Deployment, or by
creating another controller such as a ReplicaSet or a
ReplicationController. If you do so, the first Deployment thinks that
it created these other Pods. Kubernetes does not stop you from doing
this.
As described further in docs, it is not recommended to scale replicas of the deployments using the above approach.
Another important point to note from same section of docs:
If you have multiple controllers that have overlapping selectors, the
controllers will fight with each other and won't behave correctly.
My expectation was that continuously running Deployment Controller will pick this new Pod, and when I do kubectl get deploy then I will see 4 pods running. But that didn't happen.
The Deployment Controller does not work like that, it listen for Deployment-resources and "drive" them to desired state. That typically means, if any change in the template:-part, then a new ReplicaSet is created with the number of replicas. You cannot add a Pod to a Deployment in another way than changing replicas: - each instance is created from the same Pod-template and is identical.
Doesn't Labels and Selectors work like this?
... but I am trying to understand how Pods / other Kubernetes objects are selected using Labels and Selectors.
Yes, Labels and Selectors are used for many things in Kubernetes, but not for everything. When you create a Deployment with a label, and a Pod with the same label and finally a Service with a selector - then the traffic addressed to that Service will distribute traffic to your instances of your Deployment as well as to your extra Pod.
Example:
apiVersion: v1
kind: Service
metadata:
name: my-service
spec:
selector:
app: deployment-webserver-pods
ports:
- protocol: TCP
port: 80
targetPort: 8080
Labels and Selector are also useful for management when using e.g. kubectl. You can add labels for Teams or e.g. App, then you can select all Deployments or Pods belonging to that Team or App (e.g. if the app consist of App-deployment and a cache-deployment), e.g:
kubectl get pods -l team=myteam,app=customerservice
My expectation was that continuously running Deployment Controller
will pick this new Pod, and when I do kubectl get deploy then I will
see 4 pods running. But that didn't happen.
Kubernetes is a system that operates "Declaratively" and not "Imperatively" which means you write down the desired state of the application in the cluster typically through a YAML file, and these declared desired states define all of the pieces of your application.
If a cluster were to configured imperatively like the way you are expecting it to be, it would have been very difficult to understand and replicate how the cluster came to be in that state.
Just to add in the above explanations that if we are trying to manually create pod and manage then what is the purpose of having controllers in K8s.
My expectation was that continuously running Deployment Controller
will pick this new Pod, and when I do kubectl get deploy then I will
see 4 pods running. But that didn't happen.
As per your yaml replicas:3 was already set so deployment would not take a new pod as the 4th replica.
I was wondering if there is a way to create a service for a pod on a specific node.
For example:
Lets say I have a cluster with 4 worker nodes [compute-0 ... compute-3].
Nodes "compute-0" and "compute-1" have a label "app=firstApplication"
Nodes "compute-2" and "compute-3" have a different label "app=secondApplication"
I have a single Daemonset running across all 4 nodes.
I want to create 2 services, one for each couple of nodes.
Is this possible somehow?
Thanks!
EDIT
The reason for what we are trying to do is that we have an Openshift4.6 cluster, and for security reasons we have VXLAN port blocked off between 2 groups of nodes. When pods try to resolve DNS queries using the default dns service (172.30.0.10), sometimes they access the dns pods on the blocked off nodes.
No - this is not possible! Since services are referencing their Pods by Labels and all Pods in a DaemonSet are labelled the same, you can't do that. Of course, you could label your Pods after creation, but since this would be lost after recreation of the DaemonSet, I would not go down that route.
You could split your DaemonSet into parts and use Node Selectors or Affinity to control the distribution of Pods over Nodes.
If you specify a .spec.template.spec.nodeSelector, then the DaemonSet controller will create Pods on nodes which match that node selector. Likewise if you specify a .spec.template.spec.affinity, then DaemonSet controller will create Pods on nodes which match that node affinity.
That way, each DaemonSet can have its own Service.
You just need to patch existing pods. Add those label in your pods. May be you need to handle another operator. The job of the operator is to get the pods first. Then check if the desire label exist or not . If not exist patch the label of the pod. this is just like kubectl patch. With the help of kubeclient just update the label if the label is not exist in the pods. do some research about kubeclient. There are also an example sample-controller in kubernetes. Here is the link :
ref
if there are some extra label in pod just add them in selector.
---
kind: Service
apiVersion: v1
metadata:
name: first-svc
labels:
app: firstApplication
spec:
selector:
app: firstApplication
ports:
- name: http
port: 8080
targetPort: 8080
---
kind: Service
apiVersion: v1
metadata:
name: second-svc
labels:
app: secondApplication
spec:
selector:
app: secondApplication
ports:
- name: http
port: 8080
targetPort: 8080
---
I understand that services use a selector to identify which pods to route traffic to by thier labels.
apiVersion: v1
kind: Service
metadata:
name: svc
spec:
ports:
- name: tcp
protocol: TCP
port: 443
targetPort: 443
selector:
app: nginx
Thats all well and good.
Now what is the difference between this selector and the one of the spec.selector from the deployment. I understand that it is used so that the deployment can match and manage its pods.
I dont understand however why i need the extra matchLabels declaration and cant just do it like in the service. Whats the use of this semantically?
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
spec:
selector:
matchLabels:
app: nginx
replicas: 1
template:
metadata:
labels:
app: nginx
spec:
containers:
- name: nginx
image: nginx
Thanks in advance
In the Service's spec.selector, you can identify which pods to route traffic to only by their labels.
On the other hand, in the Deployment's spec.selector you have two options to decide on which node the pods will be scheduled on, which are: matchExpressions, matchLabels.
How Deployment uses spec.selector
When a Deployment is changed, a new ReplicaSet is created. The ReplicaSet is responsible to manage the Pods. It uses the spec.selector to know what Pods it should manage.
Example:
If the replicas: 1 is changed in the Deployment to e.g. replicas: 2 a new ReplicaSet is created, and it observes the Pods using spec.selector to match Pods with matching labels. It only see 1 replica initially, but its desired state is now replicas: 2 so it is responsible for creating additionally one Pod from the template in the Deployment.
Selector syntax
There is two ways to declare the labels under the spec.selector in `Deployment.
matchLabels - you declare the labels
matchExpressions - you write an expression for labels
See kubectl explain deployment.spec.selector for full explanation of spec.selector alternatives.
Labels and Selectors
Labels and Selectors is a generic concept in Kubernetes and is used in multiple places. Another example is how you can filter what resources you want to see or use with kubectl. E.g. you can select the Pods for an app with:
kubectl get pod -l=app=myappname
(if your Pods is labelled with app: myappname.
why i need the extra matchLabels declaration and cant just do it like in the service. Whats the use of this semantically?
Because service spec only support equality-based selectors and the deployment is a newer resource that supports two syntax (equality-based and set-based).
The API currently supports two types of selectors: equality-based and set-based. A label selector can be made of multiple requirements which are comma-separated. In the case of multiple requirements, all must be satisfied so the comma separator acts as a logical AND (&&) operator.
Reference
The Service spec uses just the "equality-based" label selector syntax.
Newer resources, such as Job, Deployment, ReplicaSet, and DaemonSet, support set-based requirements...
Reference
My understanding is that earlier the only supported syntax was the equality-based one, like we have on the service spec, and that now, when the resource you are using supports the new syntax, you are required to use matchLabels or matchExpressions.
What is the command to delete replication controller and its pods?
I am taking a course to learn k8s on pluralsight. I am trying to delete the pods that I have just created using Replication controller. Following is my YAML:
apiVersion: v1
kind: ReplicationController
metadata:
name: hello-rc
spec:
replicas: 2
selector:
app: hello-world
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: hello-ctr
image: nigelpoulton/pluralsight-docker-ci:latest
ports:
- containerPort: 8080
If I do 'kubectl get pods' following is the how it looks on my mac:
I have tried the following two commands to delete the pods that are created in the Minikube cluster on my mac, but they are not working:
kubectl delete pods hello-world
kubectl delete pods hello-rc
Could someone help me understand what I am missing?
you can delete the pods by deleting the replication controller that created them
kubectl delete rc hello-rc
also, because pods created are just managed by ReplicationController, you can delete only theReplicationController and leave the pods running
kubectl delete rc hello-rc --cascade=false
this means the pods are no longer managed .you can create a new ReplicationController with the
proper label selector and manage them again
Also,instead of replicationcontrollers, you can use replica sets.
They behave in a similar way, but they have more expressive
pod selectors. For example, a ReplicationController can’t match pods with 2 labels
below command is just enough
kubectl delete rc hello-rc
One more thing is that ReplicationController is deprecated rather ReplicaSets is preferred