I trying to create a service account with permissions to get information about endpoints pod IPs and getting back a permissions problem.
Creating the service account and give it the right permissions:
$ kubectl create role endpoints-reader --verb=get --verb=list --resource=endpoints
$ kubectl create serviceaccount endpoints-reader-sa
$ kubectl create rolebinding default-endpoints-reader --role=endpoints-reader --serviceaccount=endpoints-reader-sa:endpoints-reader-sa
Adding this sa to the deployment YAML file:
...
spec:
serviceAccountName: endpoints-reader-sa
containers:
- name: ...
I stated the pod and logged in into it (ssh). Now I want to run a REST call to pull the information:
$ TOKEN=$(cat /var/run/secrets/kubernetes.io/serviceaccount/token);
$ curl https://kubernetes.default.svc/api/v1/namespaces/XXX/endpoints --silent --header "Authorization: Bearer $TOKEN" --insecure
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "endpoints is forbidden: User \"system:serviceaccount:XXX:endpoints-reader-sa\" cannot list resource \"endpoints\" in API group \"\" in the namespace \"XXX\"",
"reason": "Forbidden",
"details": {
"kind": "endpoints"
},
"code": 403
}
What I'm doing wrong?
OK... Found the issue.
So, this line:
kubectl create rolebinding default-endpoints-reader --role=endpoints-reader --serviceaccount=endpoints-reader-sa:endpoints-reader-sa
should be changed to this:
kubectl create rolebinding default-endpoints-reader --role=endpoints-reader --serviceaccount=XXX:endpoints-reader-sa
Where XXX is the namespace name.
Similar problem you can find here.
Related
I have deploy a k8s cluster with kubeadm, I want to get controller manager's metrics with following command:
curl -k https://localhost:10257/metrics
but got the following error:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "forbidden: User \"system:anonymous\" cannot get path \"/metrics\"",
"reason": "Forbidden",
"details": {
},
"code": 403
}
So my question is, how to get k8s controller manager's metrics?
This is a forbidden error due to permission issues which need to be authenticated with a valid user. For this,You need to create a service account, then give that service account access permissions to the metrics Path through RBAC, then this will make that service account to get the metrics.
As per this Role and Cluster Binding doc, you need to allow metrics path(replace with /healthz) as below and give a try.
Allow GET and POST requests to the non-resource endpoint /healthz and all subpaths (must be in a ClusterRole bound with a ClusterRoleBinding to be effective):
rules:
- nonResourceURLs: ["/healthz", "/healthz/*"] # '*' in a nonResourceURL is a suffix glob match
verbs: ["get", "post"]
This question already has answers here:
Sharing secret across namespaces
(18 answers)
Closed 6 months ago.
In Azure Kubernetes I want have a pod with jenkins in defualt namespace, that needs read secret from my aplication workspace.
When I tried I get the next error:
Error from server (Forbidden): secrets "myapp-mongodb" is forbidden: User "system:serviceaccount:default:jenkinspod" cannot get resource "secrets" in API group "" in the namespace "myapp"
How I can bring access this jenkisn pod to read secrets in 'myapp' namespace
secret is a namespaced resource and can be accessed via proper rbac permissions. However any improper rbac permissions may lead to leakage.
You must role bind the pod's associated service account. Here is a complete example. I have created a new service account for role binding in this example. However, you can use the default service account if you want.
step-1: create a namespace called demo-namespace
kubectl create ns demo-namespace
step-2: create a secret in demo-namespace:
kubectl create secret generic other-secret -n demo-namespace --from-literal foo=bar
secret/other-secret created
step-2: Create a service account(my-custom-sa) in the default namespace.
kubectl create sa my-custom-sa
step-3: Validate that, by default, the service account you created in the last step has no access to the secrets present in demo-namespace.
kubectl auth can-i get secret -n demo-namespace --as system:serviceaccount:default:my-custom-sa
no
step-4: Create a cluster role with permissions of get and list secrets from demo-namespace namespace.
kubectl create clusterrole role-for-other-user --verb get,list --resource secret
clusterrole.rbac.authorization.k8s.io/role-for-other-user created
step-5: Create a rolebinding to bind the cluster role created in last step.
kubectl create rolebinding role-for-other-user -n demo-namespace --serviceaccount default:my-custom-sa --clusterrole role-for-other-user
rolebinding.rbac.authorization.k8s.io/role-for-other-user created
step-6: validate that the service account in the default ns now has access to the secrets of demo-namespace. (note the difference from step 3)
kubectl auth can-i get secret -n demo-namespace --as system:serviceaccount:default:my-custom-sa
yes
step-7: create a pod in default namsepace and mount the service account you created earlier.
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: my-pod
name: my-pod
spec:
serviceAccountName: my-custom-sa
containers:
- command:
- sleep
- infinity
image: bitnami/kubectl
name: my-pod
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
step-7: Validate that you can read the secret of demo-namespace from the pod in the default namespace.
curl -sSk -H "Authorization: Bearer $(cat /run/secrets/kubernetes.io/serviceaccount/token)" https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_PORT_443_TCP_PORT/api/v1/namespaces/demo-namespace/secrets
{
"kind": "SecretList",
"apiVersion": "v1",
"metadata": {
"resourceVersion": "668709"
},
"items": [
{
"metadata": {
"name": "other-secret",
"namespace": "demo-namespace",
"uid": "5b3b9dba-be5d-48cc-ab16-4e0ceb3d1d72",
"resourceVersion": "662043",
"creationTimestamp": "2022-08-19T14:51:15Z",
"managedFields": [
{
"manager": "kubectl-create",
"operation": "Update",
"apiVersion": "v1",
"time": "2022-08-19T14:51:15Z",
"fieldsType": "FieldsV1",
"fieldsV1": {
"f:data": {
".": {},
"f:foo": {}
},
"f:type": {}
}
}
]
},
"data": {
"foo": "YmFy"
},
"type": "Opaque"
}
]
}
I am trying to deploy applications on GKE using REST APIs. However, the GKE documentation is all mixed up and unclear as to how to enable the Kubernetes REST API access.
Does anyone here have a clear idea about how to create a Deployment on Kubernetes cluster on Google Cloud?
If yes, I would love to know the detailed steps for enabling the same. Currently, this is what I get.
https://xx.xx.xx.xx/apis/apps/v1/namespaces/default/deployments/nginx-1 GET call gives below JSON output despite valid authorization token
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "deployments.apps \"nginx-1\" is forbidden: User \"system:serviceaccount:default:default\" cannot get resource \"deployments\" in API group \"apps\" in the namespace \"default\"",
"reason": "Forbidden",
"details": {
"name": "nginx-1",
"group": "apps",
"kind": "deployments"
},
"code": 403
}
Administration APIs however seems to be enabled:
Following the instructions at this link and executing the below commands:
# Check all possible clusters, as your .KUBECONFIG may have multiple contexts:
kubectl config view -o jsonpath='{"Cluster name\tServer\n"}{range .clusters[*]}{.name}{"\t"}{.cluster.server}{"\n"}{end}'
# Select name of cluster you want to interact with from above output:
export CLUSTER_NAME="some_server_name"
# Point to the API server referring the cluster name
APISERVER=$(kubectl config view -o jsonpath="{.clusters[?(#.name==\"$CLUSTER_NAME\")].cluster.server}")
# Gets the token value
TOKEN=$(kubectl get secrets -o jsonpath="{.items[?(#.metadata.annotations['kubernetes\.io/service-account\.name']=='default')].data.token}"|base64 --decode)
# Explore the API with TOKEN
curl -X GET $APISERVER/api --header "Authorization: Bearer $TOKEN" --insecure
gives the desired output.
The service account default in default namespace does not have RBAC to perform get verb on deployment resource in default namespace.
Use below role and rolebinding to provide the necessary permission to the service account.
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
namespace: default
name: deployment-reader
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "watch", "list"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: read-deployment
namespace: default
subjects:
# You can specify more than one "subject"
- kind: ServiceAccount
name: default # "name" is case sensitive
namespace: default
roleRef:
# "roleRef" specifies the binding to a Role / ClusterRole
kind: Role #this must be Role or ClusterRole
name: deployment-reader # this must match the name of the Role or ClusterRole you wish to bind to
apiGroup: rbac.authorization.k8s.io
To verify the permission
kubectl auth can-i get deployments --as=system:serviceaccount:default:default -n default
yes
I am unable to get list of namespaces using rest api and rest end point is https://<localhost>:8001/api/v1/namespaces
Using this kubernetes document:
I am using postman. I will repeat the steps:
Created a user and given cluster admin privileges:
kubectl create serviceaccount exampleuser
Created a rolebinding for our user with cluster role cluster-admin:
kubectl create rolebinding <nameofrolebinding> --clusterrole cluster-admin
--serviceaccount default:exampleuser
Checked rolebinding using:
kubectl describe rolebinding <nameofrolebinding>
Now by using:
kubectl describe serviceaccount exampleuser
kubectl describe secret exampleuser-xxxx-xxxx
I will use token I got here to authenticate postman.
GET https://<ipofserver>:port/api/v1/namespace
AUTH using bearer token.
Expected result to list all namespaces in cluster. like
kubectl get namespaces. But got a warning as follows.
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {},
"status": "Failure",
"message": "namespaces is forbidden: User \"system:serviceaccount:default:exampleuser\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope",
"reason": "Forbidden",
"details": {
"kind": "namespaces"
},
"code": 403
}
I have used "cluster-admin" clusterrole for the user, still getting authentication related error.
please help.
You should use clusterrolebinding instead of rolebinding:
kubectl create clusterrolebinding <nameofrolebinding> --clusterrole cluster-admin --serviceaccount default:exampleuser
RoleBinding means permissions to a namespaced resources, but namespace is not a namespaced resources, you can check this by kubectl api-resouces.
More detail at rolebinding-and-clusterrolebinding:
Permissions can be granted within a namespace with a RoleBinding, or cluster-wide with a ClusterRoleBinding
so issue is instead of using rolebinding , i need to use clusterrolebinding check below
kubectl create rolebinding nameofrolebinding --clusterrole cluster-admin --serviceaccount default:exampleuser
kubectl create clusterrolebinding nameofrolebinding --clusterrole cluster-admin --serviceaccount default:exampleuser
rolebinding scope is upto a namespace and
clusterrolebinding scope is entire cluster.
To work with api/v1/namespaces we need to use clusterrolebinding
I created a cluster role "try-usr"
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: try-usr
rules:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch
While accessing the Web UI(Dashboard), it's throwing an error as follows:
{
"kind": "Status",
"apiVersion": "v1",
"metadata": {
},
"status": "Failure",
"message": "services \"https:kubernetes-dashboard:\" is forbidden: User \"xyz\" cannot get services/proxy in the namespace \"kube-system\"",
"reason": "Forbidden",
"details": {
"name": "https:kubernetes-dashboard:",
"kind": "services"
},
"code": 403
}
Depending on the kubernetes version, the dashboard will require different permissions according to the docs
v1.7
create and watch permissions for secrets in kube-system namespace required to - create and watch for changes of kubernetes-dashboard-key-holder secret.
get, update and delete permissions for secrets named kubernetes-dashboard-key-holder and kubernetes-dashboard-certs in kube-system namespace.
proxy permission to heapster service in kube-system namespace required to allow getting metrics from heapster.
v1.8
create permission for secrets in kube-system namespace required to create kubernetes-dashboard-key-holder secret.
get, update and delete permissions for secrets named kubernetes-dashboard-key-holder and kubernetes-dashboard-certs in kube-system namespace.
get and update permissions for config map named kubernetes-dashboard-settings in kube-system namespace.
proxy permission to heapster service in kube-system namespace required to allow getting metrics from heapster.