We have multiple development teams who work and deploy their applications on kuberenetes. We use helm to deploy our application on kubernetes.
Currently the challenge we are facing with one of our shared clusters. We would like to deploy tiller separate for each team. So they have access to their resources. default Cluster-admin role will not help us and we don't want that.
Let's say we have multiple namespaces for one team. I would want to deploy tiller which has permission to work with resources exist or need to be created in these namespaces.
Team > multiple namespaces
tiller using the service account that has the role ( having full access to namespaces - not all ) associated with it.
I would want to deploy tiller which has permission to work with resources exist or need to be created in these namespaces
According to the fine manual, you'll need a ClusterRole per team, defining the kinds of operations on the kinds of resources, but then use a RoleBinding to scope those rules to a specific namespace. The two ends of the binding target will be the team's tiller's ServiceAccount and the team's ClusterRole, and then one RoleBinding instance per Namespace (even though they will be textually identical except for the namespace: portion)
I actually would expect you could make an internal helm chart that would automate the specifics of that relationship, and then helm install --name team-alpha --set team-namespaces=ns-alpha,ns-beta my-awesome-chart and then grant your tiller cluster-admin or whatever more restrictive ClusterRole you wish.
Related
I am trying to understand security implication of granting kubernetes serviceaccount permission to perform deployments, service creation, etc. The role is clusterrole with namespaced rolebinding.
Use case are using service account to automate/orchestrate some tasks inside cluster. Version is 1.16
Service account can used to grant different access level to different purpose. for developer ClusterRole need to list, get and watch resources but not delete. but Admin can delete and create resources.
If you are developing K8 operator, operator need to communicate with cluster for create, update and delete resources then operator need service with ClusterRoleBinding of all verbs. so that operator can perform all operation on resources. but not good practice to assign full permission to regular deployment.
Let me discuss about this.
According to the docs.
A Role always _sets permissions within a particular namespace__; when you create a Role, you have to specify the namespace it belongs in
ClusterRole, by contrast, is a non-namespaced resource. The resources have different names (Role and ClusterRole) because a Kubernetes object always has to be either namespaced or not namespaced; it can't be both.
A role binding grants the permissions defined in a role to a user or set of users. It holds a list of subjects (users, groups, or service accounts), and a reference to the role being granted. A RoleBinding grants permissions within a specific namespace whereas a ClusterRoleBinding grants that access cluster-wide.
If you want to define a role within a namespace, use a Role; if you want to define a role cluster-wide, use a ClusterRole.
A RoleBinding may reference any Role in the same namespace. Alternatively, a RoleBinding can reference a ClusterRole and bind that ClusterRole to the namespace of the RoleBinding. If you want to bind a ClusterRole to all the namespaces in your cluster, you use a ClusterRoleBinding.
So answering your question - handling ClusterRoles it could be definitively security risk.
The best solution is to grant as minimal permissions as possible.
Grant a role to an application-specific service account (best practice)
Grant a role to the "default" service account in a namespace
Grant a role to all service accounts in a namespace
Grant a limited role to all service accounts cluster-wide (discouraged)
Grant super-user access to all service accounts cluster-wide (strongly discouraged)
There is no one answer for this question because you need to plan and test action/permissions between your application/deployment and kubernetes API. F.e at namesapaced or non namesapced resources, inside one namespace or across the entire cluster.
In you example you can simply use Role/Rolebinding if you are working inside the same namespace. You can use ClusterRole/Rolebinding and extend permissions by additional RoleBinding allowing the ServiceAccount to create new k8s objects into another namespace.
Assuming we are talking about ServiceAccount for deployment, here you can find good advice for "RBAC in Deployments: A use case"
If you create ServiceAccount for your deployment and create appropriate Role/ClusterRole and Rolebinding/ClusterRoleBinding:
you can perform:
kubectl can-i get secrets --as=system:serviceaccount:[namespace]:[service_account_name] -n [target_namespace]
For testing please take a look also at Access Clusters Using the Kubernetes API.
This command will show you if particular ServiceAccount (defined by subject in Rolebinding/ClusterRolebinding) has permissions (defined by verbs in Role/ClusteRole) to get secrets (defined by resources Role/ClusterRole) in specified namespace.
Following this approach you can verify if your deployment has enough permissions to perform all required operation against Kubernetes API.
While working with RBAC in Kubernetes you should consider mentioned below topics:
Have multiple users with different properties, establishing a proper authentication mechanism.
Have full control over which operations each user or group of users can execute.
Have full control over which operations each process inside a pod can execute.
Limit the visibility of certain resources of namespaces.
I have bunch of users. Every user should be able to create/change/delete substances in namespaces like *-stage. Namespaces can be added or removed dynamically. I can create ServiceAccount in every namespace and grant privileges.
I created pod in k8s and install kubectl and ssh into it. So every user has access to this pod and can use kubectl. I know that I can mount ServiceAccount secrets to pod. As far as I have different ServiceAccounts for every namespace I don't know how to grant privileges to all *-stage namespaces for every user. I don't want to create cluster-admin ClusterRoleBinding for ServiceAccount, cause users should be able to modify only *-stage namespaces. Can you help me please?
I am posting a community wiki answer based on OP's solution for better visibility:
Actually, I have already solved problem. I create ["*"] role in every
*-stage namespace and bind it to ServiceAccount. Then I mount ServiceAccount to kubectl pod which is available over ssh. So every
user has unlimited access to *-stage namespaces.
Also I am adding links for the official docs regarding ServiceAccount and role-based access control as a supplement.
I found a lot of information on how to give helm permission to create resources in a particular namespace.
I am trying to see if I can create namespaces on the fly(with random names) and then use helm to install and delete resources inside the namespace.
My idea is to create a namespace with name such as Fixedsuffix-randomprefix and then allow helm to create all resources inside it. Is this possible ?
I can create a clusterrole and clusterrolebinding to allow tiller's serviceaccount to create namespaces, but I am not able to figure out how to have a serviceaccount that could create resources in the particular namespace( mainly because this serviceaccount to create resources cant would have to be created when the namespace is created and then assigned to tiller pod).
TIA
My question is why would you create sa, clusterrole and rolebinding to do that? Helm has it´s own resources which allow him to install and delete resources inside new namespace.
My idea is to create a namespace with name such as Fixedsuffix-randomprefix and then allow helm to create all resources inside it. Is this possible ?
Yes, you can create your new namespace and use helm to install everything in this namespace.Or even better you can just use helm install and it will create new namespace for you. For that purpose helm have helm install --namespace.
-n, --namespace string namespace scope for this request
For example you can install traefik chart in namespace tla.
helm install stable/traefik --namespace=tla
NAME: oily-beetle
LAST DEPLOYED: Tue Mar 24 07:33:03 2020
NAMESPACE: tla
STATUS: DEPLOYED
Another idea which came to my mind is you might want tiller not to use cluster-admin credentials, then this link could help.
Prometheus deployed using kube-prometheus can't scrape resources on namespaces other than default, monitoring and kube-system. I added additional namespaces on my jsonnet as described in kube-prometheus README but no success...
I also tried to create a new ServiceMonitor manually, but no success...
I appreciate any help.
Thanks.
If you used the pre-compiled manifests here you will only have your service account with 3 rolebindings allowing access to the namespaces you mentioned.
You can add more namespaces for example by applying the same roleBinding in more namespaces.
This is more secure as opposed to using a clusterRoleBinding since it allows for more finegrained permissions.
Is it possible to share a ServiceAccount between namespaces or somehow start a pod with a ServiceAccount from a different namespace?
We are looking to use vault to store common secret data between dynamic development environments. Following the very good walk though HERE we were able to authenticate and pull secrets for a single namespace. However in our use case we will be creating a new namespace for each development environment during it's lifetime.
If possible we would like to avoid having to also configure vault with a new auth backend for each namespace.
When you create the Vault role, you can configure bound_service_account_namespaces to be the special value *, and allow a fixed service account name from any namespace. To adapt the "create role" example from the documentation:
vault write auth/kubernetes/role/demo \
bound_service_account_names=vault-auth \
bound_service_account_namespaces='*' \
policies=default \
ttl=1h
You have to recreate the Kubernetes service account in every namespace, and it must have the exact name specified in the role. However, the Kubernetes service account is a single k8s object and it's not any harder than the Deployments, Services, ConfigMaps, and Secrets you already have; this pattern doesn't require any Vault reconfiguration.
(If you're using a templating tool like Helm, the service account can't follow a naming convention like {{ .Release.Name }}-{{ .Chart.Name }}: Vault doesn't appear to have any sort of pattern matching on this name.)
Service Accounts are namespaced therefore not shared , so you may copy the token from one account to another , but that is not the recommneded way.
C02W84XMHTD5:kubernetes-gitlab iahmad$ kubectl api-resources --namespaced | grep service
serviceaccounts sa true ServiceAccount
services svc true Service
C02W84XMHTD5:kubernetes-gitlab iahmad$
If you want to share a secret or account the way you are trying to do , then there is no need to use vault at all.
You may just need to automate this process , instead of sharing accounts.