Currently, I have an ABAC policy that gives "system:autheticated" all access. K8s starts up fine when I have this defined, but if I remove it, K8s doesn't start up. I'm trying to find out what namespaces, service accounts, groups, users, etcs are being used on my K8s cluster so I can define a specific set of users/groups in the ABAC policy.
How can I get the groups and users in the K8s cluster? I'm using
"kubectl --namespace=kube-system get serviceaccounts"
to get the serviceaccounts... but where are the groups and users defined?
For Groups you might try (example for "system:masters"):
kubectl get clusterrolebindings -o json | jq -r '.items[] | select(.subjects[0].kind=="Group") | select(.subjects[0].name=="system:masters") | .metadata.name'
Also, you can read all the namespaces at once adding --all-namespaces=true inside the kubectl command.
You should also check all local files for policies that might be applied.
Here is Kubernetes documentation regarding Using ABAC Authorization
As for users, I was only able to find a way of checking if a particular user is able, for example, to create a deployment in a namespace:
$ kubectl auth can-i create deployments --namespace dev
yes
$ kubectl auth can-i create deployments --namespace prod
no
Related
Being absolutely new to openshift, i'm curious how i can check if any of the running containers are running in "privileged" mode (openshift v4.6). Digging in the documentation and searching the net i could only find information regarding SCCs, which is great and all, but i didnt find anything regarding this (apart from an older version of openshift, where the oc get pods (or something similar command) used to show if a pod was running with such privileges
By default pods use the Restricted SCC. The pod's SCC is determined by the User/ServiceAccount and/or Group. Then, you also have to consider that a SA may or may not be bound to a Role, which can set a list of available SCCs.
To find out what SCC a pod runs under:
oc get pod $POD_NAME -o yaml | grep openshift.io/scc
The following commands can also be useful:
# get pod's SA name
oc get pod $POD_NAME -o yaml | grep serviceAccount:
# list service accounts that can use a particular SCC
oc adm policy who-can use scc privileged
# list users added by the oc adm policy command
oc get scc privileged -o yaml
# check roles and role bindings of your SA
# you need to look at rules.apiGroups: security.openshift.io
oc get rolebindings -o wide
oc get role $ROLE_NAME -o yaml
An OpenShift project comes with 3 service accounts by default which are builder, default, deployer.
The containers you deploy to that namespace will be assigned to "default" service account and that is the one which has "restricted" scc role.
You can find more here: https://www.openshift.com/blog/managing-sccs-in-openshift
I'm trying to understand why on one cluster an operation is permitted but on the other i'm getting the following
Exception encountered setting up namespace watch from Kubernetes API v1 endpoint https://10.100.0.1:443/api: namespaces is forbidden: User \"system:serviceaccount:kube-system:default\" cannot list resource \"namespaces\" in API group \"\" at the cluster scope ({\"kind\":\"Status\",\"apiVersion\":\"v1\",\"metadata\":{},\"status\":\"Failure\",\"message\":\"namespaces is forbidden: User \\\"system:serviceaccount:kube-system:default\\\" cannot list resource \\\"namespaces\\\" in API group \\\"\\\" at the cluster scope\",\"reason\":\"Forbidden\",\"details\":{\"kind\":\"namespaces\"},\"code\":403}\n)"
I'm managing two Kubernetes clusters -
clusterA booted with Kops version v1.14.8
clusterB booted on AWS EKS version v1.14.9-eks-f459c0
So i've tried using the kubectl auth command to try figuring out and I do see that on one i'm allowed however on the second i'm not as you can see:
kubectl config use-context clusterA
Switched to context "clusterA".
kubectl auth can-i list pods --as=system:serviceaccount:kube-system:default -n kube-system
yes
kubectl config use-context clusterB
Switched to context "clusterB".
kubectl auth can-i list pods --as=system:serviceaccount:kube-system:default -n kube-system
no
Is there a way to understand what are these two decisions based on yes/no?
Thanks for helping out!
The decision yes/no is based on whether there is a clusterrole and a clusterrolebinding or rolebinding which permits the default serviceaccount in kube-system namespace to perform verb list on resource namespace.
The trick in case of namespace resource is that there needs to be a clusterrole instead of role because namespace is a cluster scoped resource.
You check what are the clusterrole,role, clusterrolebinding,rolebinding exists in a kubernetes cluster using below command
kubectl get clusterrole,clusterrolebinding
kubectl get role,rolebinding -n namespacename
For more details refer Kubernetes RBAC here
I'm following the guide to setup GCP's Workload Identity and have it working for a service account configured against the default Kubernetes namespace as below:
gcloud iam service-accounts add-iam-policy-binding \
--role roles/iam.workloadIdentityUser \
--member "serviceAccount:[PROJECT_ID].svc.id.goog[default/[KSA_NAME]]" \
[GSA_NAME]#[PROJECT_ID].iam.gserviceaccount.com
I'm wondering if there is a way to create the binding for all k8s namespaces; something like below, where I've replaced the default namespace with *:
--member "serviceAccount:[PROJECT_ID].svc.id.goog[*/[KSA_NAME]]"
Is there any way to do this?
There is no way to set all namespaces to a Kubernetes Service Account(KSA).
Maybe you would have to get each namespace and doing the binding for each namespace, an automation script could help in the process to do it n times (1 time for each namespace).
You can get only namespaces column name using the following kubectl command:
kubectl get ns --all-namespaces --no-headers -o custom-columns=":metadata.name"
Getting something like this:
default
kube-public
kube-system
Which can be used to iterate them using a binding automation script.
I am trying to invoke kubectl from within my CI system. I wish to use a google cloud service account for authentication. I have a secret management system in place that injects secrets into my CI system.
However, my CI system does not have gcloud installed, and I do not wish to install that. It only contains kubectl. Is there any way that I can use a credentials.json file containing a gcloud service account (not a kubernetes service account) directly with kubectl?
The easiest way to skip the gcloud CLI is to probably use the --token option. You can get a token with RBAC by creating a service account and tying it to a ClusterRole or Role with either a ClusterRoleBinding or RoleBinding.
Then from the command line:
$ kubectl --token <token-from-your-service-account> get pods
You still will need a context in your ~/.kube/config:
- context:
cluster: kubernetes
name: kubernetes-token
Otherwise, you will have to use:
$ kubectl --insecure-skip-tls-verify --token <token-from-your-service-account> -s https://<address-of-your-kube-api-server>:6443 get pods
Note that if you don't want the token to show up on the logs you can do something like this:
$ kubectl --token $(cat /path/to/a/file/where/the/token/is/stored) get pods
Also, note that this doesn't prevent you from someone running ps -Af on your box and grabbing the token from there, for the lifetime of the kubectl process (It's a good idea to rotate the tokens)
Edit:
You can use the --token-auth-file=/path/to/a/file/where/the/token/is/stored with kubectl to avoid passing it through $(cat /path/to/a/file/where/the/token/is/stored)
I'm trying to deploy an application in a GKE 1.6.2 cluster running ContainerOS but the instructions on the website / k8s are not accurate anymore.
The error that I'm getting is:
Error from server (Forbidden): User "circleci#gophers-slack-bot.iam.gserviceaccount.com"
cannot get deployments.extensions in the namespace "gopher-slack-bot".:
"No policy matched.\nRequired \"container.deployments.get\" permission."
(get deployments.extensions gopher-slack-bot)
The repository for the application is available here available here.
Thank you.
I had a few breaking changes in the past with using the gcloud tool to authenticate kubectl to a cluster, so I ended up figuring out how to auth kubectl to a specific namespace independent of GKE. Here's what works for me:
On CircleCI:
setup_kubectl() {
echo "$KUBE_CA_PEM" | base64 --decode > kube_ca.pem
kubectl config set-cluster default-cluster --server=$KUBE_URL --certificate-authority="$(pwd)/kube_ca.pem"
kubectl config set-credentials default-admin --token=$KUBE_TOKEN
kubectl config set-context default-system --cluster=default-cluster --user=default-admin --namespace default
kubectl config use-context default-system
}
And here's how I get each of those env vars from kubectl.
kubectl get serviceaccounts $namespace -o json
The service account will contain the name of it's secret. In my case, with the default namespace, it's
"secrets": [
{
"name": "default-token-655ls"
}
]
Using the name, I get the contents of the secret
kubectl get secrets $secret_name -o json
The secret will contain ca.crt and token fields, which match the $KUBE_CA_PEM and $KUBE_TOKEN in the shell script above.
Finally, use kubectl cluster-info to get the $KUBE_URL value.
Once you run setup_kubectl on CI, your kubectl utility will be authenticated to the namespace you're deploying to.
In Kubernetes 1.6 and GKE, we introduce role based cess control. The authors of your took need to give the service account the ability to get deployments (along with probably quite a few others) to its account creation.
https://kubernetes.io/docs/admin/authorization/rbac/