I have a pod with 2 closely related services running as containers. I am running as a StatefulSet and have set replicas as 5. So 5 pods are created with each pod having both the containers.
Now My requirement is to have the second container run only in 1 pod. I don't want it to run in 5 pods. But my first service should still run in 5 pods.
Is there a way to define this in the deployment yaml file for Kubernetes? Please help.
a "pod" is the smallest entity that is managed by kubernetes, and one pod can contain multiple containers, but you can only specify one pod per deployment/statefulset, so there is no way to accomplish what you are asking for with only one deployment/statefulset.
however, if you want to be able to scale them independently of each other, you can create two deployments/statefulsets to accomplish this. this is imo the only way to do so.
see https://kubernetes.io/docs/concepts/workloads/pods/ for more information.
Containers are like processes,
Pods are like VMs,
and Statefulsets/Deployments are like the supervisor program controlling the VM's horizontal scaling.
The only way for your scenario is to define the second container in a new deployment's pod template, and set its replicas to 1, while keeping the old statefulset with 5 replicas.
Here are some definitions from documentations (links in the references):
Containers are technologies that allow you to package and isolate applications with their entire runtime environment—all of the files necessary to run. This makes it easy to move the contained application between environments (dev, test, production, etc.) while retaining full functionality. [1]
Pods are the smallest, most basic deployable objects in Kubernetes. A Pod represents a single instance of a running process in your cluster. Pods contain one or more containers. When a Pod runs multiple containers, the containers are managed as a single entity and share the Pod's resources. [2]
A deployment provides declarative updates for Pods and ReplicaSets. [3]
StatefulSet is the workload API object used to manage stateful applications. Manages the deployment and scaling of a set of Pods, and provides guarantees about the ordering and uniqueness of these Pods. [4]
Based on all that information - this is impossible to match your requirements using one deployment/Statefulset.
I advise you to try the idea #David Maze mentioned in a comment under your question:
If it's possible to have 4 of the main application container not having a matching same-pod support container, then they're not so "closely related" they need to run in the same pod. Run the second container in a separate Deployment/StatefulSet (also with a separate Service) and you can independently control the replica counts.
References:
Documentation about Containers
Documentation about Pods
Documentation about Deployments
Documentation about StatefulSet
Related
If I have a k8s deployment file for a service with multiple containers like api and worker1, can I make it so that there is a configmap with a variable worker1_enabled, such that if my service is restarted, container worker1 only runs if worker1_enabled=true in the configmap?
The short answer is No.
According to k8s docs, Pods in a Kubernetes cluster are used in two main ways:
Pods that run a single container. The "one-container-per-Pod" model is the most common Kubernetes use case; in this case, you can think of a Pod as a wrapper around a single container; Kubernetes manages Pods rather than managing the containers directly.
Pods that run multiple containers that need to work together. A Pod can encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers form a single cohesive unit of service—for example, one container serving data stored in a shared volume to the public, while a separate sidecar container refreshes or updates those files. The Pod wraps these containers, storage resources, and an ephemeral network identity together as a single unit.
Unless your application requires it, it is better to separate the worker and api containers into their own pod. So you may have one deployment for worker and one for api.
As for deploying worker when worker1_enabled=true, that can be done with helm. You have to create a chart such that when the value of worker1_enabled=true is set, worker is deployed.
Last note, a service in kubernetes is an abstract way to expose an application running on a set of Pods as a network service.
When i scale the application by creating deployment .Let's say i am running nginx service on 3 cluster.
Nginx is running in containers in multiple pods .
If i change nginx configuration in one of the pod ,does it propagate to all the nodes and pods because it is running in cluster and scaled.
does it propagate to all the nodes and pods because it is running in
cluster and scaled.
No. Only when you change the deployment yaml. Then it re-creates pods 1 by 1 with the new configuration.
I would like to add a few more things to what was already said. First of all you are even not supposed to do any changes to Pods which are managed let's say by ReplicaSet, ReplicationController or Deployment. This are objects which provide additional abstraction layer and it is their responsibility to ensure that there are given number of Pods of a certain kind running in your kubernetes cluster.
It doesn't matter how many nodes your cluster consists of as mentioned controllers span across all nodes in the cluster.
Changes made in a single Pod will not only not propagate to other Pods but may be easily lost if such newly created Pod with changed configuration crashes.
Remember that one of the tasks of the Deployment is to make sure that certain number of Pods of a given type ( specified in a Pod template section of the Deployment ) are always up and running. When your manually reconfigured Pod goes down then your Deployment (actually ReplicaSet created by the Deployment) acts behind the scenes and recreates such Pod. But how does it recreate it ? Does it take into consideration changes introduced by you to such Pod ? Of course not, it will recreate it based on the template it is given in the Deployment.
If you want to make changes in your Pods one by one kubernetes allows you to do so by providing so called rolling update mechanism.
Here you can read about old-fashioned approach using ReplicationController which is not used any more as it is replaced by Deployments and ReplicaSets but I think it's still worth reading just to grasp the concept.
Currently Deployment is the way to go. About updating a Deployment you can read here. Note that the default update strategy is RollingUpdate which ensures that changes are not applied to all Pods at once but one by one.
I am trying to deploy multiple pods in k8s like say MySQL, Mango, Redis etc
Can i create a single deployment resource for this and have multiple containers defined in template section? Is this allowed? If so, how will replication behave in this case?
Thanks
Pavan
I am trying to deploy multiple pods in k8s like say MySQL, Mango,
Redis etc
From microservices architecture perspective it is actually quite a bad idea to place all those containers in a single Pod. Keep in mind that a Pod is a smallest deployable unit that can be created and managed by Kubernetes. There are quite many good reasons you don't want to have all above mentioned services in a single Pod. Difficulties in scaling such solution is just one of them.
Can i create a single deployment resource for this and have multiple
containers defined in template section? Is this allowed? If so, how
will replication behave in this case?
No, it is not allowed in Kubernetes. As to Deployments and StatefulSets, (which you need for statefull applications such as databases) both manage Pods that are based on identical container spec so it is not possible to have a Deployment or StatefulSet consisting of different types of Pods, based on different specs.
To sum up:
Many Deployments and StatefulSets objects, serving for different purposes are the right solution.
A deployment can have multiple containers inside of it.
Generaly it's used to have one master container for the app and some sidecar container that are needed for the app. I don't have an example right now.
Still it's a best practice to split deployments for scalling purpose, your front may need to scale more than the back depending on cache and you may not want to have pods too big. For cahing purpose like redis it's better to have a cluster on the side as each time a pod start or stop, you will loose data.
It's common having multiple containers per Pod in order to share namespaces and volumes between them: take as example the Ambassador pattern that is used to present the application to outside adding a layer for the authentication, making it totally transparent to the main app.
Other examples using the sidecar pattern consist of log parsers or configurators that hot reload credentials without the main app to worry about it.
That's the theory, according to your needs you have to use one deployment per component, so a Deployment for your app, a StatefulSet for the DB and so on. Keep in mind to use a container per process and a Kubernetes resource per backing service.
I am developing an application for dealing with kubernetes runtime microservices. I actually did some cool things, like moving a microservice from a node to another one. The problem is that all replicas go together.
So, Imagine that a microservice has two replicas and it is running on a namespaces with two nodes.
I want to set one replica in each node. Is that possible? Even in a yaml file, is that possible?
I am trying to do my own scheduler to do that, but I got no success until now.
Thank you all
I think what you are looking for is a NodeSelector for your replica Set. From the documentation:
Inter-pod affinity and anti-affinity allow you to constrain which nodes your pod is eligible to be scheduled based on labels on pods that are already running on the node rather than based on labels on nodes.
Here is the documentation: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#inter-pod-affinity-and-anti-affinity-beta-feature
I can't find where it's documented, but I recently read somewhere that replicas will be distributed across nodes when you create the kubernetes service BEFORE the deployment / replicaset.
somebody could please help me to create a yaml config file for Kubernetes in order to face a situation like: one pod with 3 containers (for example) and these containers have to be deployed on 3 nodes of a cluster (Google GCE).
|P| |Cont1| ----> |Node1|
|O| ---> |Cont2| ----> |Node2| <----> GCE cluster
|D| |Cont3| ----> |Node3|
Thanks
From Kuberenets Concepts,
Pods in a Kubernetes cluster can be used in two main ways: Pods that
run a single container. The “one-container-per-Pod” model is the most
common Kubernetes use case; in this case, you can think of a Pod as a
wrapper around a single container, and Kubernetes manages the Pods
rather than the containers directly. Pods that run multiple containers
that need to work together. A Pod might encapsulate an application
composed of multiple co-located containers that are tightly coupled
and need to share resources. These co-located containers might form a
single cohesive unit of service–one container serving files from a
shared volume to the public, while a separate “sidecar” container
refreshes or updates those files. The Pod wraps these containers and
storage resources together as a single manageable entity.
In short, most likely, you should place each container in a single Pod to truly benefit from the microservices architecture vs the monolithic architecture commonly deployed in VMs. However there are some cases where you want to consider co-locating containers. Namely, as described in this article (Patterns for Composite Containers) some of the composite containers applications are:
Sidecar containers
extend and enhance the "main" container
Ambassador containers
proxy a local connection to the world
Adapter containers
standardize and normalize output
Once you define and run the Deployments, the Scheduler will be responsible to select the most suitable placement for your Pods, unless you manually assign Nodes by defining Labels in Deployment's YAML (not recommended unless you know what you're doing).
You can assign multiple containers to a single pod. You can assign pods to a specific node-pool. But I am not sure whether is it possible to assign multiple containers to multiple nodes running in side a single pod.
What you can do here is to assign each container to different pods (3 containers --> 3 pods) and then assign each pod to a different node-pool by adding this code to your deployment's .yaml file.
nodeSelector:
nodeclass: pool1