My understanding is since pod is defined as a group of containers which provides shared resources such as storage and network among those containers, can it be thought of as a namespace in a worker node that is to say, different pods are representing different namespaces in a worker node machine?
Or otherwise is pod actually a process which is first started (or run or executed) by the deployment and then it starts the containers inside it?
Can i see it through ps command? (I did try it, there are only docker containers running so I am ruling out pod being a process)
If we start from the basics
What is a namespace (in a generic manner)?
A namespace is a declarative region that provides a scope to the identifiers (the names of types, functions, variables, etc) inside it. Namespaces are used to organize code into logical groups and to prevent name collisions that can occur especially when your code base includes multiple libraries.
What is a Pod (in K8s)?
A pod is a group of one or more containers (such as Docker containers), with shared storage/network, and a specification for how to run the containers. A pod’s contents are always co-located and co-scheduled, and run in a shared context. A pod models an application-specific “logical host” - it contains one or more application containers which are relatively tightly coupled — in a pre-container world, being executed on the same physical or virtual machine would mean being executed on the same logical host.
While Kubernetes supports more container runtimes than just Docker, Docker is the most commonly known runtime, and it helps to describe pods in Docker terms.
The shared context of a pod is a set of Linux namespaces, cgroups, and potentially other facets of isolation - the same things that isolate a Docker container. Within a pod’s context, the individual applications may have further sub-isolations applied.
Some deep dive into Pods
What is a Namespace (in k8s terms)?
Namespaces are intended for use in environments with many users spread across multiple teams, or projects.
Namespaces provide a scope for names. Names of resources need to be unique within a namespace, but not across namespaces.
Namespaces are a way to divide cluster resources between multiple users.
So I think its suffice to say:
Yes Pods have a namespace :
Pods kind of represent a namespace but on a container level (where they share the same context of networks, volumes/storage only among a set of containers)
But namespaces (in terms of K8s) are a bigger level of isolation -- on a cluster level which shared by all the containers (services, deployments, dns-names, IPs, config-maps, secrets, roles, etc).
Also you should see this link
Hope this clears a bit of fog on the issue.
Yes, you could say that a pod is a namespace that is shared by containers. When using the Docker executor, a pause container is created which establishes the network, file-system, and process namespace for subsequent containers to utilise.
This is because Docker doesn't understand pods as a first class primitive, and you won't see the pause container with an other run-time.
Related
I have a k8s cluster that runs the main workload and has a lot of nodes.
I also have a node (I call it the special node) that some of special container are running on that that is NOT part of the cluster. The node has access to some resources that are required for those special containers.
I want to be able to manage containers on the special node along with the cluster, and make it possible to access them inside the cluster, so the idea is to add the node to the cluster as a worker node and taint it to prevent normal workloads to be scheduled on it, and add tolerations on the pods running special containers.
The idea looks fine, but there may be a problem. There will be some other containers and non-container daemons and services running on the special node that are not managed by the cluster (they belong to other activities that have to be separated from the cluster). I'm not sure that will be a problem, but I have not seen running non-cluster containers along with pod containers on a worker node before, and I could not find a similar question on the web about that.
So please enlighten me, is it ok to have non-cluster containers and other daemon services on a worker node? Does is require some cautions, or I'm just worrying too much?
Ahmad from the above description, I could understand that you are trying to deploy a kubernetes cluster using kudeadm or minikube or any other similar kind of solution. In this you have some servers and in those servers one is having some special functionality like GPU etc., for deploying your special pods you can use node selector and I hope you are already doing this.
Coming to running separate container runtime on one of these nodes you need to consider two points mainly
This can be done and if you didn’t integrated the container runtime with
kubernetes it will be one more software that is running on your server
let’s say you used kubeadm on all the nodes and you want to run docker
containers this will be separate provided you have drafted a proper
architecture and configured separate isolated virtual network
accordingly.
Now comes the storage part, you need to create separate storage volumes
for kubernetes and container runtime separately because if any one
software gets failed or corrupted it should not affect the second one and
also for providing the isolation.
If you maintain proper isolation starting from storage to network then you can run both kubernetes and container runtime separately however it is not a suggested way of implementation for production environments.
Kubernetes documentation describes pod as a wrapper around one or more containers. containers running inside of a pod share a set of namespaces (e.g. network) which makes me think namespaces are nested (I kind doubt that). What is the wrapper here from container runtime's perspective?
Since containers are just processes constrained by namespaces, Cgroups e.g. Perhaps, pod is just the first container launched by Kubelet and the rest of containers are started and grouped by namespaces.
The main difference is networking, the network namespace is shared by all containers in the same Pod. Optionally, the process (pid) namespace can also be shared. That means containers in the same Pod all see the same localhost network (which is otherwise hidden from everything else, like normal for localhost) and optionally can send signals to processes in other containers.
The idea is the Pods are groups of related containers, not really a wrapper per se but a set of containers that should always deploy together for whatever reason. Usually that's a primary container and then some sidecars providing support services (mesh routing, log collection, etc).
Pod is just a co-located group of container and an Kubernetes object.
Instead of deploying them separate you can do deploy a pod of containers.
Best practices is that you should not actually run multiple processes via single container and here is the place where pod idea comes to a place. So with running pods you are grouping containers together and orchestrate them as single object.
Containers in a pod runs the same Network namespace (ip address and port space) so you have to be careful no to have the same port space used by two processes.
This differs for example when it comes to filesystem, since the containers fs comes from the image fs. The file systems are isolated unless they will share one Volume.
I am a beginner and learning about Kubernetes.
As per my understanding, namespace is a virtual cluster backed by the same physical cluster.
In which usecases do we go for separate physical Kubernetes cluster?
What are the main resources that can be saved by opting for namespace instead of physical Kubernetes cluster? (Kubernetes objects present in one namespace of the physical cluster can be shared by all other namespaces, like the ones in kube-system? And are the nodes in the physical Kubernetes cluster shared by all the namespaces but it is not possible to share nodes between multiple physical Kubernetes clusters?)
A namespace isn't a "virtual cluster" in any meaningful way; it's just a way to group together resources. For instance, these Services are different because they're in different namespaces:
kubectl describe service --namespace n1 foo
kubectl describe service --namespace n2 foo
But a service in n1 can make a call to foo.n2.svc.cluster.local without doing any special setup.
A namespace is a natural boundary for Kubernetes RBAC settings. If one object directly references another (e.g., a pod mounts a persistent volume claim or gets environment variables from a config map) they generally must be in the same namespace.
In an actual cluster, the nodes are shared. A given node can run any pod from any namespace (unless that's specifically configured at the pod level); kubectl describe node will show this. If a pod makes very heavy use of some resource (CPU, memory, disk I/O) this can impact other pods running on the same node. That completely ignores namespace boundaries.
You want different clusters when you want things to actually be separated: when a service in one environment shouldn't be able to call a service in a different environment, when cluster-level resources like NodePort services need to be separated, if you have different policies around things like PersistentVolume allocation.
Sharing a cluster means that you need fewer copies of some cluster-global processes (the Kubernetes core, service meshes like Istio) and you can share nodes. That could result in better utilization of large nodes.
You might, for example, separate your test and production environments into separate clusters. These would have different external-DNS settings, separate ingress controllers, and separate node pools. You couldn't accidentally send requests into the test environment from outside, and a load test on the test environment wouldn't impact the production environment.
Generally a separate physical cluster is necessary
To meet compliance and security standards such as PCI DSS, HIPPA etc.
To provide dedicated physical resources to critical workloads.
To separate different environments such as DEV, TEST, PROD
A multi tenant cluster shared by many tenants using their own namespace is useful for saving cost. Namespace separation is logical where the resources of all namespaces still reside in same ETCD storage but with different keys. This is not a problem in separate dedicated physical cluster because in that case the cluster will have separate ETCD as well.
Access to resources across namespaces is controlled by RBAC via kubernetes API Server. But you can access everything from all namespaces if you get access to ETCD directly bypassing the API Server.
You need to put lot of best practices and protection in a multi tenant cluster so that tenants from different namespaces do not step on each other toes. This not that much necessary in a separate dedicated physical cluster.
What's the benefit of having multiple containers in a pod versus having standalone containers?
If you have multiple containers in the same pod, they can speak to each other as localhost and can share mounted volumes.
If you have multiple pods of one container each, you can restart one without restarting the other. Assuming they're controlled by deployments, you can add additional replicas of one without necessarily scaling the other. If the version or some other characteristic of one of them changes, you're not forced to restart the other. You'd need to set up a service to talk from one to the other, and they can't communicate via a filesystem.
The general approach I've always seen is to always have one container per pod within a deployment, unless you have a specific reason to need an additional container. Usually this is some kind of special-purpose "sidecar" that talks to a credentials service, or manages logging, or runs a network proxy, or something else that's secondary to the main thing the pod does (and isn't a separate service in its own right).
Apart from the points pointed out , the CPU and Memory(under technical preview) are associated with a POD so if we have a single container in a POD it is easy to understand and implement the application resourcerequirement inside the POD with more than one container inside the POD we could face issues/challenges when we want to do a horizontal scale
Secondly the deployments (Blue/Green,Canary,A/B) are also more aligned with the approach of single container/POD
From the Kubernetes documentation
A Pod might encapsulate an application composed of multiple co-located containers that are tightly coupled and need to share resources. These co-located containers might form a single cohesive unit of service–one container serving files from a shared volume to the public, while a separate “sidecar” container refreshes or updates those files. The Pod wraps these containers and storage resources together as a single manageable entity.
somebody could please help me to create a yaml config file for Kubernetes in order to face a situation like: one pod with 3 containers (for example) and these containers have to be deployed on 3 nodes of a cluster (Google GCE).
|P| |Cont1| ----> |Node1|
|O| ---> |Cont2| ----> |Node2| <----> GCE cluster
|D| |Cont3| ----> |Node3|
Thanks
From Kuberenets Concepts,
Pods in a Kubernetes cluster can be used in two main ways: Pods that
run a single container. The “one-container-per-Pod” model is the most
common Kubernetes use case; in this case, you can think of a Pod as a
wrapper around a single container, and Kubernetes manages the Pods
rather than the containers directly. Pods that run multiple containers
that need to work together. A Pod might encapsulate an application
composed of multiple co-located containers that are tightly coupled
and need to share resources. These co-located containers might form a
single cohesive unit of service–one container serving files from a
shared volume to the public, while a separate “sidecar” container
refreshes or updates those files. The Pod wraps these containers and
storage resources together as a single manageable entity.
In short, most likely, you should place each container in a single Pod to truly benefit from the microservices architecture vs the monolithic architecture commonly deployed in VMs. However there are some cases where you want to consider co-locating containers. Namely, as described in this article (Patterns for Composite Containers) some of the composite containers applications are:
Sidecar containers
extend and enhance the "main" container
Ambassador containers
proxy a local connection to the world
Adapter containers
standardize and normalize output
Once you define and run the Deployments, the Scheduler will be responsible to select the most suitable placement for your Pods, unless you manually assign Nodes by defining Labels in Deployment's YAML (not recommended unless you know what you're doing).
You can assign multiple containers to a single pod. You can assign pods to a specific node-pool. But I am not sure whether is it possible to assign multiple containers to multiple nodes running in side a single pod.
What you can do here is to assign each container to different pods (3 containers --> 3 pods) and then assign each pod to a different node-pool by adding this code to your deployment's .yaml file.
nodeSelector:
nodeclass: pool1