We're testing Shiny-proxy Kubernetes containers and each application spins it's own container, it's working fine until this part. We have made some changes to create a PVC/PV to persist user specific data for each container, noticed that serviceaccount is unable to create the PVC though I have following roles configured for the account. In general, are there any other steps for making sure that SA is able to access/create PVC?
The PV/PVC are accessible when testing from a normal container, but seem to be an issue with the serviceaccount roles/permissions that's used to create new containers.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: sp-ns
name: sp-sa
rules:
- apiGroups: [""]
resources: ["pods", "pods/log", "persistentvolumeclaims"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
I have verfied that the serviceaccount roles are set right as below commands returns 'yes'.
kubectl auth can-i create pvc --as=system:serviceaccount:sp-ns:sp-sa -n sp-ns
Error during container creation from the application:
at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: io.fabric8.kubernetes.client.KubernetesClientException: Failure executing: POST at: http://localhost:8001/api/v1/namespaces/sp-ns/pods. Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. pods "sp-pod-92e1efc0-0859-4a87-8b9b-04d6adaa11f5" is forbidden: user "system:serviceaccount:sp-ns:sp-sa" is not an admin and does not have permissions to use host bind mounts for resource .
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.requestFailure(OperationSupport.java:503)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.assertResponseCode(OperationSupport.java:440)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:406)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleResponse(OperationSupport.java:365)
at io.fabric8.kubernetes.client.dsl.base.OperationSupport.handleCreate(OperationSupport.java:234)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.handleCreate(BaseOperation.java:735)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:325)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.create(BaseOperation.java:321)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.lambda$createNew$0(BaseOperation.java:336)
at io.fabric8.kubernetes.api.model.DoneablePod.done(DoneablePod.java:26)
at eu.openanalytics.containerproxy.backend.kubernetes.KubernetesBackend.startContainer(KubernetesBackend.java:223)
at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.doStartProxy(AbstractContainerBackend.java:129)
at eu.openanalytics.containerproxy.backend.AbstractContainerBackend.startProxy(AbstractContainerBackend.java:110)
... 95 more
Container is not running as privileged. Use privileged: true in pod spec.
Service account don't have cluster-admin role. Use below to provide permission.
kubectl create clusterrolebinding add-on-cluster-admin --clusterrole=cluster-admin --serviceaccount=sp-ns:sp-sa
Related
I deleted my cluster-admin role via kubectl using:
kubectl delete clusterrole cluster-admin
Not sure what I expected, but now I don't have access to the cluster from my account. Any attempt to get or change resources using kubectl returns a 403, Forbidden.
Is there anything I can do to revert this change without blowing away the cluster and creating a new one? I have a managed cluster on Digital Ocean.
Not sure what I expected, but now I don't have access to the cluster from my account.
If none of the kubectl commands actually work, unfortunately you will not be able to create a new cluster role. The problem is that you won't be able to do anything without an admin role. You can try creating the cluster-admin role directly through the API (not using kubectl), but if that doesn't help you have to recreate the cluster.
Try applying this YAML to creaste the new Cluster role
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
annotations:
rbac.authorization.kubernetes.io/autoupdate: "true"
labels:
kubernetes.io/bootstrapping: rbac-defaults
name: cluster-admin
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- '*'
- nonResourceURLs:
- '*'
verbs:
- '*'
apply the YAML file changes
kubectl apply -f <filename>.yaml
1.I tried to get kubernetes cluster detail by entering command below .
kubectl describe service {NAME}
2.I got an error message below.
error from server (Forbidden): servicies “max-Object-detectoer” is forbidden:User “{username}” cannot get resource “services” in API group “” in the namespace “default”
Looks like the service account (user) you are using doesn't have access to that Service
You can create a ClusterRole like so:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: service-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["services"]
verbs: ["get", "watch", "list"]
Then you can create a clusterRoleBinding, giving your service account the above mentioned role, like so:
kubectl create clusterrolebinding service-reader-pod \
--clusterrole=service-reader \
--serviceaccount={name_of_service_account}
Let me know if this worked for you.
ERROR: Job failed (system failure): pods is forbidden: User "system:serviceaccount:gitlab:gitlab-admin" cannot create resource "pods" in API group "" in the namespace "gitlab"
From the looks of it, because you did not provided enough information I would say that your RBAC is incorrectly configure.
I would advice to read following Kubernetes documentation regarding Managing Service Accounts and Configure Service Accounts for Pods.
If I'm not mistaken this command should fix it:
kubectl create clusterrolebinding gitlab-cluster-admin --clusterrole=cluster-admin --group=system:serviceaccounts --namespace=gitlab
If not then you will need to edit your Role and ClusterRole with something like the following:
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: gitlab
name: gitlab-admin
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["pods"]
verbs: ["create", "get", "watch", "list"]
This is an example and you should make changes to better suit your needs.
If you provide more details I'll try to help you further.
Upon deploying a service with Spring Cloud Kubernetes Discovery Client, I get the KubernetesClientException stating that the user "default" is forbidden to access pods.
I have already added a Role and a Rolebinding as specified here
The guide states that a ClusterRole is necessary. But that is not an option for me, as we share the cluster with other departments. I only want the role to affect our project / namespace.
Is ClusterRole required or should Role be sufficient?
To allow a service account access to these one needs to create a role with the necessary permissions and assign it to the account.This is done with a cluster role, or a role, if one only wants it in
one namespace, and a role binding, which is specific to a namespace.
It says that you can use either Role or ClusterRole.
Just bear in mind when creating a Role a namespace should be defined. i.e.
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: mynamespace
name: service-discovery-client
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["services", "pods", "configmaps", "endpoints"]
verbs: ["get", "watch", "list"]
Forbidden!Configured service account doesn't have access. Service account may have been revoked. User "system:serviceaccount:default:default" cannot get services in the namespace "mycomp-services-process"
For the above issue I have created "mycomp-service-process" namespace and checked the issue.
But it shows again message like this:
Message: Forbidden!Configured service account doesn't have access. Service account may have been revoked. User "system:serviceaccount:mycomp-services-process:default" cannot get services in the namespace "mycomp-services-process"
Creating a namespace won't, of course, solve the issue, as that is not the problem at all.
In the first error the issue is that serviceaccount default in default namespace can not get services because it does not have access to list/get services. So what you need to do is assign a role to that user using clusterrolebinding.
Following the set of minimum privileges, you can first create a role which has access to list services:
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
namespace: default
name: service-reader
rules:
- apiGroups: [""] # "" indicates the core API group
resources: ["services"]
verbs: ["get", "watch", "list"]
What above snippet does is create a clusterrole which can list, get and watch services. (You will have to create a yaml file and apply above specs)
Now we can use this clusterrole to create a clusterrolebinding:
kubectl create clusterrolebinding service-reader-pod \
--clusterrole=service-reader \
--serviceaccount=default:default
In above command the service-reader-pod is name of clusterrolebinding and it is assigning the service-reader clusterrole to default serviceaccount in default namespace. Similar steps can be followed for the second error you are facing.
In this case I created clusterrole and clusterrolebinding but you might want to create a role and rolebinding instead. You can check the documentation in detail here
This is only for non prod clusters
You should bind service account system:serviceaccount:default:default (which is the default account bound to Pod) with role cluster-admin, just create a yaml (named like fabric8-rbac.yaml) with following contents:
# NOTE: The service account `default:default` already exists in k8s cluster.
# You can create a new account following like this:
#---
#apiVersion: v1
#kind: ServiceAccount
#metadata:
# name: <new-account-name>
# namespace: <namespace>
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: fabric8-rbac
subjects:
- kind: ServiceAccount
# Reference to upper's `metadata.name`
name: default
# Reference to upper's `metadata.namespace`
namespace: default
roleRef:
kind: ClusterRole
name: cluster-admin
apiGroup: rbac.authorization.k8s.io
Then, apply it by running kubectl apply -f fabric8-rbac.yaml.
If you want unbind them, just run kubectl delete -f fabric8-rbac.yaml.
Just to add.
This can also occur when you are redeploying an existing application to the wrong Kubernetes cluster that are similar.
Ensure you check to be sure that the Kubernetes cluster you're deploying to is the correct cluster.