helm list : cannot list configmaps in the namespace "kube-system" - kubernetes

I have installed helm 2.6.2 on the kubernetes 8 cluster. helm init worked fine. but when I run helm list it giving this error.
helm list
Error: configmaps is forbidden: User "system:serviceaccount:kube-system:default" cannot list configmaps in the namespace "kube-system"
How to fix this RABC error message?

Once these commands:
kubectl create serviceaccount --namespace kube-system tiller
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
helm init --service-account tiller --upgrade
were run, the issue has been solved.

More Secure Answer
The accepted answer gives full admin access to Helm which is not the best solution security wise. With a little more work, we can restrict Helm's access to a particular namespace. More details in the Helm documentation.
$ kubectl create namespace tiller-world
namespace "tiller-world" created
$ kubectl create serviceaccount tiller --namespace tiller-world
serviceaccount "tiller" created
Define a Role that allows Tiller to manage all resources in tiller-world like in role-tiller.yaml:
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: tiller-manager
namespace: tiller-world
rules:
- apiGroups: ["", "batch", "extensions", "apps"]
resources: ["*"]
verbs: ["*"]
Then run:
$ kubectl create -f role-tiller.yaml
role "tiller-manager" created
In rolebinding-tiller.yaml,
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: tiller-binding
namespace: tiller-world
subjects:
- kind: ServiceAccount
name: tiller
namespace: tiller-world
roleRef:
kind: Role
name: tiller-manager
apiGroup: rbac.authorization.k8s.io
Then run:
$ kubectl create -f rolebinding-tiller.yaml
rolebinding "tiller-binding" created
Afterwards you can run helm init to install Tiller in the tiller-world namespace.
$ helm init --service-account tiller --tiller-namespace tiller-world
Now prefix all commands with --tiller-namespace tiller-world or set TILLER_NAMESPACE=tiller-world in your environment variables.
More Future Proof Answer
Stop using Tiller. Helm 3 removes the need for Tiller completely. If you are using Helm 2, you can use helm template to generate the yaml from your Helm chart and then run kubectl apply to apply the objects to your Kubernetes cluster.
helm template --name foo --namespace bar --output-dir ./output ./chart-template
kubectl apply --namespace bar --recursive --filename ./output -o yaml

Helm runs with "default" service account. You should provide permissions to it.
For read-only permissions:
kubectl create rolebinding default-view --clusterrole=view --serviceaccount=kube-system:default --namespace=kube-system
For admin access: Eg: to install packages.
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=kube-system:default

The default serviceaccount does not have API permissions. Helm likely needs to be assigned a service account, and that service account given API permissions. See the RBAC documentation for granting permissions to service accounts: https://kubernetes.io/docs/admin/authorization/rbac/#service-account-permissions

apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
kubectl apply -f your-config-file-name.yaml
and then update helm instalation to use serviceAccount:
helm init --service-account tiller --upgrade

I got a this error while trying to install tiller in offline mode, I thought the 'tiller' service account didn't have enough rights but at it turns out that a network policy was blocking the communication between tiller and the api-server.
The solution was to create a network policy for tiller allowing all egress communication of tiller

export TILLER_NAMESPACE=<your-tiller-namespace> solved it for me, if <your-tiller-namespace> is not kube-system. This points the Helm client to the right Tiller namespace.

If you are using an EKS cluster from AWS and are facing the forbidden issue ( eg: forbidden: User ... cannot list resource "jobs" in API group "batch" in the namespace "default" then this worked for me:
Solution:
Ensure you have configured AWS
Ensure that configured user has the permission to access the cluster.

Related

Kubernetes kube-apiserver to kubelet permissions

What controls the permissions when you call
kubectl logs pod-name ?
I've played around and tried calling the kubelet api from on of the controller nodes.
sudo curl -k --key /var/lib/kubernetes/cert-k8s-apiserver-key.pem --cert /var/lib/kubernetes/cert-k8s-apiserver.pem https://worker01:10250/pods
This fails with Forbidden (user=apiserver, verb=get, resource=nodes, subresource=proxy).
I've tried the same call using the admin key and cert and it succeeds and return a healthy blob of JOSN.
I'm guessing this is why kubectl logs pod-name doesn't work.
A little more reading suggests that the CN of the certificate determines the user that is authenticated and authorized.
What controls whether a user is authorized to access the kubelet API?
Background
I'm setting up a K8s cluster using the following instructions Kubernetes the not so hard way with Ansible
Short Answer
The short answer is that you need to grant the user apiserver access to the resource node by creating a ClusterRole and ClusterRoleBinding.
Longer Explanation
Kubernetes has a bunch of resources. The relevant ones here are:
Role
Node
ClusterRole
ClusterRoleBinding
Roles and ClusterRoles are similar, except ClusterRoles are not namespaced.
A ClsuterRole can be associated (bound) to a user with a ClusterRoleBinding object.
Kubelet provides some resources (maybe more)
nodes/proxy
nodes/stats
nodes/log
nodes/spec
nodes/metrics
To make this work, you need to create a ClusterRole that allow access to the resource and sub-resource on the Node.
cat <<EOF | kubectl apply -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: system:kube-apiserver-to-kubelet
rules:
- apiGroups:
- ""
resources:
- nodes/proxy
- nodes/stats
- nodes/log
- nodes/spec
- nodes/metrics
verbs:
- "*"
EOF
Then you associate this ClusteRole with a user. In my case, the kube-apiserver is using a certificate with CN=apiserver.
cat <<EOF | kubectl apply --kubeconfig admin.kubeconfig -f -
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: system:kube-apiserver
namespace: ""
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:kube-apiserver-to-kubelet
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: apiserver
EOF

Unable to initialize helm (tiller) on newly created GKE cluster

I have just created a GKE cluster on Google Cloud platform. I have installed in the cloud console helm :
$ helm version
version.BuildInfo{Version:"v3.0.0", GitCommit:"e29ce2a54e96cd02ccfce88bee4f58bb6e2a28b6", GitTreeState:"clean", GoVersion:"go1.13.4"}
I have also created the necessary serviceaccount and clusterrolebinding objects:
$ cat helm-rbac.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
$ kubectl apply -f helm-rbac.yaml
serviceaccount/tiller created
clusterrolebinding.rbac.authorization.k8s.io/tiller created
However trying to initialise tiller gives me the following error:
$ helm init --service-account tiller --history-max 300
Error: unknown flag: --service-account
Why is that?
However trying to initialise tiller gives me the following error:
Error: unknown flag: --service-account
Why is that?
Helm 3 is a major upgrade. The Tiller component is now obsolete.
There is no command helm init therefore also the flag --service-account is removed.
The internal implementation of Helm 3 has changed considerably from Helm 2. The most apparent change is the removal of Tiller.

Kubernetes dashboard error using service account token

I have a Kubernetes cluster with various resources running fine. I am trying to get the Dashboard working but getting the following error when I launch the dashboard and enter the service-account token.
persistentvolumeclaims is forbidden: User
"system:serviceaccount:kube-system:kubernetes-dashboard" cannot list
resource "persistentvolumeclaims" in API group "" in the namespace
"default"
It does not allow the listing of any resources from my cluster (persistent volumes, pods, ingresses etc). My cluster has multiple namespaces.
This is my service-account yaml file:
apiVersion: v1
kind: ServiceAccount
metadata:
name: k8s-test # replace with your preferred username
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: dashboard-admin # replace with your preferred username
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: dashboard-admin # replace with your preferred username
namespace: kube-system
Any help is appreciated.
FIX: Create a Role Binding for the cluster role.
This should fix the problem:
kubectl delete clusterrole cluster-admin
kubectl delete clusterrolebinding kubernetes-dashboard
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard
The above command will create a role binding that gives all permissions to all resources.
Run the Proxy:
kubectl proxy
Check the DashBoard: Please check the URL and port provided by kubectl
http://127.0.0.1:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/persistentvolume?namespace=default
More info: Cluster role:
You can check out the 'cluster-admin' role by:
kubectl edit clusterrole cluster-admin
The problem here is that the serviceaccount 'kubernetes-dashboard' does not have 'list' permissions for the resource 'persistentVolumeClaims'.
I would recommend using Web UI (Dashboard) documentation from Kubernetes.
Deploying the Dashboard UI
The Dashboard UI is not deployed by default. To deploy it, run the following command:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml
From your yaml I can see that you specified them for namespace kube-system but dashboard is trying to list resources from namespace default, at least that's what is says in your error message.
Also it seems your yaml is also incorrect for ServiceAccount name, as in the file you have k8s-test and error message says it's using kubernetes-dashboard.

Not able to create Prometheus in K8S cluster

I'm trying to install Prometheus on my K8S cluster
when I run command
kubectl get namespaces
I got the following namespace:
default Active 26h
kube-public Active 26h
kube-system Active 26h
monitoring Active 153m
prod Active 5h49m
Now I want to create the Prometheus via
helm install stable/prometheus --name prom -f k8s-values.yml
and I got error:
Error: release prom-demo failed: namespaces "default" is forbidden:
User "system:serviceaccount:kube-system:default" cannot get resource
"namespaces" in API group "" in the namespace "default"
even if I switch to monitoring ns I got the same error,
the k8s-values.yml look like following
rbac:
create: false
server:
name: server
service:
nodePort: 30002
type: NodePort
Any idea what could be missing here ?
You are getting this error because you are using RBAC without giving the right permissions.
Give the tiller permissions:
taken from https://github.com/helm/helm/blob/master/docs/rbac.md
Example: Service account with cluster-admin role
In rbac-config.yaml:
apiVersion: v1
kind: ServiceAccount
metadata:
name: tiller
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: tiller
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
Note: The cluster-admin role is created by default in a Kubernetes cluster, so you don't have to define it explicitly.
$ kubectl create -f rbac-config.yaml
serviceaccount "tiller" created
clusterrolebinding "tiller" created
$ helm init --service-account tiller
Create a service account for prometheus:
Change the value of rbac.create to true:
rbac:
create: true
server:
name: server
service:
nodePort: 30002
type: NodePort
Look at prometheus operator to spin up all monitoring services from prometheus stack.
below link is helpful
https://github.com/coreos/prometheus-operator/tree/master/contrib/kube-prometheus/manifests
all the manifests are listed there. go through those files and deploy whatever you need to monitor in your k8s cluster

Unable to install kubernetes charts on specified namespace

I have installed a cluster on Google Kubernetes Engine.
And then, I created namespace "staging"
$ kubectl get namespaces
default Active 26m
kube-public Active 26m
kube-system Active 26m
staging Active 20m
Then, I switched to operate in the staging namespace
$ kubectl config use-context staging
$ kubectl config current-context
staging
And then, I installed postgresql using helm on staging namespace
helm install --name staging stable/postgresql
But I got:
Error: release staging failed: namespaces "staging" is forbidden: User "system:serviceaccount:kube-system:default" cannot get namespaces in the namespace "staging": Unknown user "system:serviceaccount:kube-system:default"
What does it mean..?? How to get it working..??
Thank youu..
As your cluster is RBAC enabled, seems like your tiller Pod do not have enough permission.
You are using default ServiceAccount which lacks enough RBAC permission, tiller requires.
All you need to create ClusterRole, ClusterRoleBinding and ServiceAccount. With them you can provide necessary permission to your Pod.
Follow this steps
_1. Create ClusterRole tiller
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: tiller
rules:
- apiGroups: ["*"]
resources: ["*"]
verbs: ["*"]
Note: I have used full permission here.
_2. Create ServiceAccount tiller in kube-system namespace
$ kubectl create sa tiller -n kube-system
_3. Create ClusterRoleBinding tiller
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
name: tiller
subjects:
- kind: ServiceAccount
name: tiller
namespace: kube-system
apiGroup: ""
roleRef:
kind: ClusterRole
name: tiller
apiGroup: rbac.authorization.k8s.io
Now you need to use this ServiceAccount in your tiller Deployment.
As you already have one, edit that
$ kubectl edit deployment -n kube-system tiller-deploy
Set serviceAccountName to tiller under PodSpec
Read more about RBAC
Try:
helm init --upgrade --service-account tiller
as suggested by Scott S in this comment.