I found specifying like kubectl --context dev --namespace default {other commands} before kubectl client in many examples. Can I get a clear difference between them in a k8's environment?
The kubernetes concept (and term) context only applies in the kubernetes client-side, i.e. the place where you run the kubectl command, e.g. your command prompt.
The kubernetes server-side doesn't recognise this term 'context'.
As an example, in the command prompt, i.e. as the client:
when calling the kubectl get pods -n dev, you're
retrieving the list of the pods located under the namespace 'dev'.
when calling the kubectl get deployments -n dev, you're
retrieving the list of the deployments located under the namespace 'dev'.
If you know that you're targetting basically only the 'dev' namespace at the moment, then instead of adding "-n dev" all the time in each of your kubectl commands, you can just:
Create a context named 'context-dev'.
Specify the namespace='dev' for this context.
Set the current-context='context-dev'.
This way, your commands above will be simplified to as followings:
kubectl get pods
kubectl get deployments
You can set different contexts, such as 'context-dev', 'context-staging', etc., whereby each of them is targeting different namespace. BTW it's not obligatory to prefix the name with 'context'. You can just set the name with 'dev', 'staging', etc.
Just as an analogy where a group of people are talking about films. So somewhere within the conversation the word 'Rocky' was used. Since they're talking about films, it's clear and there's no ambiguity that 'Rocky' here refers to the boxing film 'Rocky' and not about the "bumpy, stony" terrains. It's redundant and unnecessary to mention 'the movie Rocky' each time. Just one word, 'Rocky', is enough. The context is obviously about film.
The same thing with Kubernetes and with the example above. If the context is already set to a certain cluster and namespace, it's redundant and unnecessary to set and / or mention these parameters in each of your commands.
My explanation here is just revolving around namespace, but this is just an example. Other than specifying the namespace, within the context you will actually also specify which cluster you're targeting and the user info used to access the cluster. You can have a look inside the ~/.kube/config file to see what information other than the namespace is associated to each context.
In the sample command in your question above, both the namespace and the context are specified. In this case, kubectl will use whatever configuration values set within the 'dev' context, but the namespace value specified within this context (if it exists) will be ignored as it will be overriden by the value explicitly set in the command, i.e. 'default'.
Meanwhile, the namespace concept is used in both sides: server and client sides. It's a logical grouping of Kubernetes objects. Just like how we group files inside different folders in Operating Systems.
You use multiple contexts to target multiple different Kubernetes clusters.You can quickly switch between clusters by using the kubectl config use-context command.
Namespaces are a way to divide cluster resources between multiple users (via resource quota).Namespaces are intended for use in environments with many users spread across multiple teams, or projects.
A context in Kubernetes is a group of access parameters. Each context contains a Kubernetes cluster, a user, and a namespace. The current context is the cluster that is currently the default for kubectl: all kubectl commands run against that cluster. Each of the context that have been used will be available on your .kubeconfig.
Meanwhile a namespace is a way to support multiple virtual cluster within the same physical cluster. This usually will be related to resource quota as well as RBAC management.
A context is the connection to a specific cluster (username/apiserver host) used by kubectl. You can manage multiple clusters that way.
Namespace is a logical partition inside a specific cluster to manage resources and constraints.
Related
I have an svc running eg my-svc-1,
and I run a deployment that makes an svc of the same name my-svc-1. What would happen?
Service should be unique within a namespace but not across a namespaces. The uniqueness applies to all namespace-based scoping objects for example Deployment, service, secrets etc.
Namespaces provides a mechanism for isolating groups of resources within a single cluster. Names of resources need to be unique within a namespace, but not across namespaces.
If you really need the same service with the different software version, you can create another namespace named B and then you can create a service having name my-svc-1.
working-with-objects-namespaces
I am still bit of kubernetes newbe. But I am looking for a way to give developers controlled access to kubectl exec command . I want to give them run most of the read-only command but prevent some high risk command and also prevent interactive download/install etc. Also want to log all their action during sessions for audit purpose.
I am not seeing any straight forward way to do it using rbac. Also not seeing those options in rancher either. I am looking for some guidance and direction to achieve such capability.
I am sure some of you have achieved it some way.
Kubernetes RBAC can only validate whenever you can or cannot exec into pods, (by checking create verb on pods/exec resource), after that it switches to SPDY protocol and passes your input and returning back output from analog of docker exec on your container runtime, without actually caring about what's going in and out
With rbac you also have to specify pod name, which might be problematic if you are using Deployments, where each new revision will generate a different pod name. Since pattern matching is not implemented in rbac - you would have to change your role every time new pod name is generated.
So the answer is "No, you can' do it with rbac"
An alternative solution would be to use some kind of CI/CD (jenkins,gitlab-ci etc.) or orchestration tool (rundeck, ansible-tower etc) where you will create some kind of script, where your developers would pass arguments to a job, controlled by you, i.e.
kubectl exec deploy/foo -- /bin/bar baz "$DEV_ARGUMENT"
Which, essentially, means, that you would be responsible for managing access to that job/script, creating and maintaining serviceAccount for that script, etc.
If you are afraid of image mutability, i.e. you don't want your developers to install something in running container, but otherwise are okay with giving them shell on it (remember, they can still read any secrets/env vars/configMaps and even serviceAccount tokens that pod uses of you mount them by default), you should consider the following:
Don't run your containers as root. Try to use images, that support rootles operation, and then either specify correct non-root UID in runAsUser field in securityContext, or configure runAsNonRoot: true flag to deny containers running as root.
Better general solution would be to utilize PodSecurityPolicy (deprecated, removed in 1.25), Pod Security Admission or some 3rd party admission contoller like OPA Gatekeeper to deny containers running as root in your namespace
You can also make your pods immutable by using readOnlyRootFilesystem in security context, which will deny write operation to pod ephemeral storage (but if you mounted any volume as RW - they still will be accessible to write operations). Feasibility of this approach depends on whenever your apps use some kind of temporary files of not
Relevant links:
kubernetes RBAC role verbs to exec to pod
https://github.com/kubernetes/kubernetes/issues/44703#issuecomment-324826356 - issue, discussing current rbac limitations
https://itnext.io/how-it-works-kubectl-exec-e31325daa910
https://erkanerol.github.io/post/how-kubectl-exec-works/ - bot links explaining how exec actually works
We are going to provide customers a function by deploying and running a container in customers kubernetes environment. After the job is done, we will clean up the container. Currently, the plan is to use k8s default namespace, but I'm not sure whether it can be a concern for customers. I don't have much experience in k8s related field. Should we give customers' an option to specify a namespace to run container, or just use the default namespace? I appreciate your suggestions!
I would recommend you not use (!?) the default namespace for anything ever.
The following is more visceral than objective but it's drawn from many years' experience of Kubernetes. In 2016, a now former colleague and I blogged about the use of namespaces:
https://kubernetes.io/blog/2016/08/kubernetes-namespaces-use-cases-insights/
NB since then, RBAC was added and it permits enforcing separation, securely.
Although it exists as a named (default) namespace, it behaves as if there is (the cluster has) no namespace. It may be (!?) that it was retcon'd into Kubernetes after namespaces were added
Unless your context is defined to be a specific other namespace, kubectl ... behaves as kubectl ... --namespace=default. So, by accident it's easy to pollute and be impacted by pollution in this namespace. I'm sure your team will use code for your infrastructure but mistakes happen and "I forgot to specify the namespace" is easily done (and rarely wanted).
Using non-default namespaces becomes very intentional, explicit and, I think, precise. You must, for example (per #david-maze answer) be more intentional about RBAC for the namespace's resources.
Using namespaces is a mechanism that promotes multi-tenancy which is desired for separation of customers (business units, versions etc.)
You can't delete the default namespace but you can delete (and by consequence delete all the resources constrained by) any non-default namespace.
I'll think of more, I'm sure!
Update
Corollary: generally don't constrain resources to namespace in specs but use e.g. kubectl apply --filename=x.yaml --namespace=${NAMESPACE}
I'd consider the namespace name pretty much a required option. I would default to the namespace name specified in the .kube/config file, if that's at all a choice for you. (That may not be default.)
RBAC rules or organizational policies also might mean the default namespace can't or shouldn't be used. One of the clusters I work with is a shared cluster where each user has their own namespace, enforced by RBAC policies; except for cluster admins, nobody gets to use default, and everybody needs to be able to configure the namespace to run in their own.
I have an application namespace with 30 services. Most are stateless Deployments, mixed with some StatefulSets etc. Fairly standard stuff that is.
I need to grant a special user a Role that can only exec into certain Pod. Currently RBAC grants the exec right to all pods in the namespace, but I need to tighten it down.
The problem is Pod(s) are created by a Deployment configurator, and the Pod name(s) are thus "generated", configurator-xxxxx-yyyyyy. Since you cannot use glob (ie. configurator-*), and Role cannot grant exec for Deployments directly.
So far I've thought about:
Converting Deployment into StatefulSet or a plain Pod, so Pod would have a known non-generated name, and glob wouldn't be needed
Moving the Deployment into separate namespace, so the global exec right is not a problem
Both of these work, but neither is optimal. Is there a way to write a proper Role for this?
RBAC, as it is meant by now, doesn't allow to filter resources by other attributes than namespace and resource name. The discussion is open here.
Thus, namespaces are the smallest piece at authorizing access to pods. Services must be separated in namespaces thinking in what users could need access to them.
The optimal solution right now is to move this deployment to another namespace since it needs different access rules than other deployments in the original namespace.
I'm just getting started with kubernetes and setting up a cluster on AWS using kops. In many of the examples I read (and try), there will be commands like:
kubectl run my-app --image=mycompany/myapp:latest --replicas=1 --port=8080
kubectl expose deployment my=app --port=80 --type=LoadBalancer
This seems to do several things behind the scenes, and I can view the manifest files created using kubectl edit deployment, and so forth However, i see many examples where people are creating the manifest files by hand, and using commands like kubectl create -f or kubectl apply -f
Am I correct in assuming that both approaches accomplish the same goals, but that by creating the manifest files yourself, you have a finer grain of control?
Would I then have to be creating Service, ReplicationController, and Pod specs myself?
Lastly, if you create the manifest files yourself, how do people generally structure their projects as far as storing these files? Are they simply in a directory alongside the project they are deploying?
The fundamental question is how to apply all of the K8s objects into the k8s cluster. There are several ways to do this job.
Using Generators (Run, Expose)
Using Imperative way (Create)
Using Declarative way (Apply)
All of the above ways have a different purpose and simplicity. For instance, If you want to check quickly whether the container is working as you desired then you might use Generators .
If you want to version control the k8s object then it's better to use declarative way which helps us to determine the accuracy of data in k8s objects.
Deployment, ReplicaSet and Pods are different layers which solve different problems.All of these concepts provide flexibility to k8s.
Pods: It makes sure that related containers are together and provide efficiency.
ReplicaSet: It makes sure that k8s cluster has desirable replicas of the pods
Deployment: It makes sure that you can have different version of Pods and provide the capability to rollback to the previous version
Lastly, It depends on use case how you want to use these concepts or methodology. It's not about which is good or which is bad.
There is a little more nuance to the difference between apply and create than what is already mentioned here. Kubectl create can be used imperatively on the command line or declaratively against a manifest file.
Kubectl apply is used declaratively against a manifest file. You can't use kubectl apply imperatively.
One key difference is when you already have an object and you want to update something. Even if you used a manifest file with kubectl create, you will get an error when you use kubectl create again to update the same resource. But, if you use kubectl apply, you will not get an error. It will update the resource without any issues.
So, the convention is to use kubectl apply to create AND update resources, kubectl create is used to create resources, and kubectl run is used to create a pod with a specific image, namespace, etc. for experimentation and testing with the --dry-run=client option.