I have installed a cluster on Google Kubernetes Engine.
And then, I created namespace "staging"
$ kubectl get namespaces
default Active 26m
kube-public Active 26m
kube-system Active 26m
staging Active 20m
Then, I switched to operate in the staging namespace
$ kubectl config use-context staging
$ kubectl config current-context
staging
And then, I installed postgresql using helm on staging namespace
helm install --name staging stable/postgresql
But I got:
Error: release staging failed: namespaces "staging" is forbidden: User "system:serviceaccount:kube-system:default" cannot get namespaces in the namespace "staging": Unknown user "system:serviceaccount:kube-system:default"
What does it mean..?? How to get it working..??
Thank youu..
As your cluster is RBAC enabled, seems like your tiller Pod do not have enough permission.
You are using default ServiceAccount which lacks enough RBAC permission, tiller requires.
All you need to create ClusterRole, ClusterRoleBinding and ServiceAccount. With them you can provide necessary permission to your Pod.
Follow this steps
_1. Create ClusterRole tiller
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: tiller
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
Note: I have used full permission here.
_2. Create ServiceAccount tiller in kube-system namespace
$ kubectl create sa tiller -n kube-system
_3. Create ClusterRoleBinding tiller
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: tiller
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
apiGroup: ""
roleRef:
kind: ClusterRole
name: tiller
apiGroup: rbac.authorization.k8s.io
Now you need to use this ServiceAccount in your tiller Deployment.
As you already have one, edit that
$ kubectl edit deployment -n kube-system tiller-deploy
Set serviceAccountName to tiller under PodSpec
Read more about RBAC
Try:
helm init --upgrade --service-account tiller
as suggested by Scott S in this comment.
Related
I have a problem in my Kubernetes cluster, that suddendly appeared two weeks ago. The ClusterRoles I create are not visible when RBAC for a given ServiceAccount are resolved. Here is a minimal set to reproduce the problem.
Create relevant ClusterRole, ClusterRoleBinding and a ServiceAccount in the default namespace to have the rights to see Endpoints with this SA.
# test.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-sa
namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: test-cr
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: test-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: test-cr
subjects:
- kind: ServiceAccount
name: test-sa
namespace: default
$ kubectl apply -f test.yaml
serviceaccount/test-sa created
clusterrole.rbac.authorization.k8s.io/test-cr created
clusterrolebinding.rbac.authorization.k8s.io/test-crb created
All objects, in particular the ClusterRole, are visible if requested directly.
$ kubectl get serviceaccount test-sa
NAME SECRETS AGE
test-sa 1 57s
$ kubectl get clusterrolebinding test-crb
NAME AGE
test-crb 115s
$ kubectl get clusterrole test-cr
NAME AGE
test-cr 2m19s
However, when I try to resolve the effective rights for this ServiceAccount, here the error I get back:
$ kubectl auth can-i get endpoints --as=system:serviceaccount:default:test-sa
no - RBAC: clusterrole.rbac.authorization.k8s.io "test-cr" not found
The RBAC rules created before the breakage are working properly. For instance, here for the ServiceAccount of my etcd-operator that I deployed with Helm several months ago:
$ kubectl auth can-i get endpoints --as=system:serviceaccount:etcd:etcd-etcd-operator-etcd-operator
yes
The version of Kubernetes in this cluster is the 1.17.0-0.
I am also seeing very slow deployements lately of new Pods, that can take up to 5 mins to start to be deployed after they have been created by a StatefulSet or a Deployment, if this can help.
Do you have any insight of what is going on, or even what I could do about it? Please note that my Kubernetes cluster is managed, so I do not have any control on the underlying system, I just have the cluster-admin privileges as a customer. But it would greatly help anyway if I could give any direction to the administrators.
Thanks in advance!
Thanks a lot for your answers!
It turned out that we will certainly never have the final world about what happen. The cluster provider just restarted the kube-apiserver, and this fixed the issue.
I suppose that something went wrong like caching or other transient failures, that can not be defined as a reproductible error.
To give a little more data for a future reader, the error occured on a Kubernetes cluster managed by OVH, and their specificity is to run the control plane itself as pods deployed in a master Kubernetes cluster on their side.
I have just created a GKE cluster on Google Cloud platform. I have installed in the cloud console helm :
$ helm version
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}
I have also created the necessary serviceaccount and clusterrolebinding objects:
$ cat helm-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
$ kubectl apply -f helm-rbac.yaml
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
However trying to initialise tiller gives me the following error:
$ helm init --service-account tiller --history-max 300
Error: unknown flag: --service-account
Why is that?
However trying to initialise tiller gives me the following error:
Error: unknown flag: --service-account
Why is that?
Helm 3 is a major upgrade. The Tiller component is now obsolete.
There is no command helm init therefore also the flag --service-account is removed.
The internal implementation of Helm 3 has changed considerably from Helm 2. The most apparent change is the removal of Tiller.
I have a Kubernetes cluster with various resources running fine. I am trying to get the Dashboard working but getting the following error when I launch the dashboard and enter the service-account token.
persistentvolumeclaims is forbidden: User
"system:serviceaccount:kube-system:kubernetes-dashboard" cannot list
resource "persistentvolumeclaims" in API group "" in the namespace
"default"
It does not allow the listing of any resources from my cluster (persistent volumes, pods, ingresses etc). My cluster has multiple namespaces.
This is my service-account yaml file:
apiVersion: v1
kind: ServiceAccount
metadata:
name: k8s-test # replace with your preferred username
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin # replace with your preferred username
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin # replace with your preferred username
namespace: kube-system
Any help is appreciated.
FIX: Create a Role Binding for the cluster role.
This should fix the problem:
kubectl delete clusterrole cluster-admin
kubectl delete clusterrolebinding kubernetes-dashboard
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard
The above command will create a role binding that gives all permissions to all resources.
Run the Proxy:
kubectl proxy
Check the DashBoard: Please check the URL and port provided by kubectl
http://127.0.0.1:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/persistentvolume?namespace=default
More info: Cluster role:
You can check out the 'cluster-admin' role by:
kubectl edit clusterrole cluster-admin
The problem here is that the serviceaccount 'kubernetes-dashboard' does not have 'list' permissions for the resource 'persistentVolumeClaims'.
I would recommend using Web UI (Dashboard) documentation from Kubernetes.
Deploying the Dashboard UI
The Dashboard UI is not deployed by default. To deploy it, run the following command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml
From your yaml I can see that you specified them for namespace kube-system but dashboard is trying to list resources from namespace default, at least that's what is says in your error message.
Also it seems your yaml is also incorrect for ServiceAccount name, as in the file you have k8s-test and error message says it's using kubernetes-dashboard.
I'm trying to install Prometheus on my K8S cluster
when I run command
kubectl get namespaces
I got the following namespace:
default Active 26h
kube-public Active 26h
kube-system Active 26h
monitoring Active 153m
prod Active 5h49m
Now I want to create the Prometheus via
helm install stable/prometheus --name prom -f k8s-values.yml
and I got error:
Error: release prom-demo failed: namespaces "default" is forbidden:
User "system:serviceaccount:kube-system:default" cannot get resource
"namespaces" in API group "" in the namespace "default"
even if I switch to monitoring ns I got the same error,
the k8s-values.yml look like following
rbac:
create: false
server:
name: server
service:
nodePort: 30002
type: NodePort
Any idea what could be missing here ?
You are getting this error because you are using RBAC without giving the right permissions.
Give the tiller permissions:
taken from https://github.com/helm/helm/blob/master/docs/rbac.md
Example: Service account with cluster-admin role
In rbac-config.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
Note: The cluster-admin role is created by default in a Kubernetes cluster, so you don't have to define it explicitly.
$ kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding "tiller" created
$ helm init --service-account tiller
Create a service account for prometheus:
Change the value of rbac.create to true:
rbac:
create: true
server:
name: server
service:
nodePort: 30002
type: NodePort
Look at prometheus operator to spin up all monitoring services from prometheus stack.
below link is helpful
https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus/manifests
all the manifests are listed there. go through those files and deploy whatever you need to monitor in your k8s cluster
I have installed helm 2.6.2 on the kubernetes 8 cluster. helm init worked fine. but when I run helm list it giving this error.
helm list
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
How to fix this RABC error message?
Once these commands:
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
helm init --service-account tiller --upgrade
were run, the issue has been solved.
More Secure Answer
The accepted answer gives full admin access to Helm which is not the best solution security wise. With a little more work, we can restrict Helm's access to a particular namespace. More details in the Helm documentation.
$ kubectl create namespace tiller-world
namespace "tiller-world" created
$ kubectl create serviceaccount tiller --namespace tiller-world
serviceaccount "tiller" created
Define a Role that allows Tiller to manage all resources in tiller-world like in role-tiller.yaml:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: tiller-manager
namespace: tiller-world
rules:
- apiGroups: ["", "batch", "extensions", "apps"]
resources: ["*"]
verbs: ["*"]
Then run:
$ kubectl create -f role-tiller.yaml
role "tiller-manager" created
In rolebinding-tiller.yaml,
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: tiller-binding
namespace: tiller-world
subjects:
- kind: ServiceAccount
name: tiller
namespace: tiller-world
roleRef:
kind: Role
name: tiller-manager
apiGroup: rbac.authorization.k8s.io
Then run:
$ kubectl create -f rolebinding-tiller.yaml
rolebinding "tiller-binding" created
Afterwards you can run helm init to install Tiller in the tiller-world namespace.
$ helm init --service-account tiller --tiller-namespace tiller-world
Now prefix all commands with --tiller-namespace tiller-world or set TILLER_NAMESPACE=tiller-world in your environment variables.
More Future Proof Answer
Stop using Tiller. Helm 3 removes the need for Tiller completely. If you are using Helm 2, you can use helm template to generate the yaml from your Helm chart and then run kubectl apply to apply the objects to your Kubernetes cluster.
helm template --name foo --namespace bar --output-dir ./output ./chart-template
kubectl apply --namespace bar --recursive --filename ./output -o yaml
Helm runs with "default" service account. You should provide permissions to it.
For read-only permissions:
kubectl create rolebinding default-view --clusterrole=view --serviceaccount=kube-system:default --namespace=kube-system
For admin access: Eg: to install packages.
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default
The default serviceaccount does not have API permissions. Helm likely needs to be assigned a service account, and that service account given API permissions. See the RBAC documentation for granting permissions to service accounts: https://kubernetes.io/docs/admin/authorization/rbac/#service-account-permissions
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
kubectl apply -f your-config-file-name.yaml
and then update helm instalation to use serviceAccount:
helm init --service-account tiller --upgrade
I got a this error while trying to install tiller in offline mode, I thought the 'tiller' service account didn't have enough rights but at it turns out that a network policy was blocking the communication between tiller and the api-server.
The solution was to create a network policy for tiller allowing all egress communication of tiller
export TILLER_NAMESPACE=<your-tiller-namespace> solved it for me, if <your-tiller-namespace> is not kube-system. This points the Helm client to the right Tiller namespace.
If you are using an EKS cluster from AWS and are facing the forbidden issue ( eg: forbidden: User ... cannot list resource "jobs" in API group "batch" in the namespace "default" then this worked for me:
Solution:
Ensure you have configured AWS
Ensure that configured user has the permission to access the cluster.