Is it possible to enable k8s basic auth in AWS EKS?
I need it to make Jenkins Kubernetes plugin work when Jenkins is deployed outside k8s.
You can use service account tokens.
Read more about it here: https://kubernetes.io/docs/reference/access-authn-authz/authentication/#service-account-tokens
You can use service account tokens (as Bearer Tokens).
Service account bearer tokens are perfectly valid to use outside the cluster and can be used to create identities for long standing jobs that wish to talk to the Kubernetes API. To manually create a service account, simply use the kubectl create serviceaccount (NAME) command. This creates a service account in the current namespace and an associated secret.
kubectl create serviceaccount jenkins
serviceaccount "jenkins" created
Check an associated secret:
kubectl get serviceaccounts jenkins -o yaml
apiVersion: v1
kind: ServiceAccount
metadata:
# ...
secrets:
- name: jenkins-token-1yvwg
The created secret holds the public CA of the API server and a signed JSON Web Token (JWT).
kubectl get secret jenkins-token-1yvwg -o yaml
apiVersion: v1
data:
ca.crt: (APISERVER'S CA BASE64 ENCODED)
namespace: ZGVmYXVsdA==
token: (BEARER TOKEN BASE64 ENCODED)
kind: Secret
metadata:
# ...
type: kubernetes.io/service-account-token
The signed JWT can be used as a bearer token to authenticate as the given service account.
If you need more control, install nginx-ingress and then tell it to enforce HTTP Basic authentication.
https://kubernetes.github.io/ingress-nginx/examples/auth/basic/
https://aws.amazon.com/blogs/opensource/network-load-balancer-nginx-ingress-controller-eks/
https://www.rfc-editor.org/rfc/rfc7617
Related
First, I have created a service account jenkins
> kubectl create serviceaccount jenkins
serviceaccount/jenkins created
Second, create a token for this service account.
> kubectl create token jenkins
eyJhbGc****************iQS-AVXfIzA
Then, I run kubectl describe serviceaccount jenkins command to check the tokens of newly created service account.
But the output shows None tokens.
> kubectl describe serviceaccount jenkins
Name: jenkins
Namespace: default
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: <none>
Tokens: <none> <===== look at this!
Events: <none>
Questions
The output shows "None" tokens, how do I know there is a token associated with this service account?
If I run kubectl create token jenkins command multiple times, will kubernetes create multiple tokens for this account? or the latest one will overwrite the previous one?
# first time
kubectl create token jenkins
# second time
kubectl create token jenkins
# third time
kubectl create token jenkins
`
What's the mapping relation between the service account and the token? is it 1:n ?
The mechanism of service accounts and tokens has been changed (Moved to stable in v1.22).
In the past, you have created a service account. Then in the first time you ran a pod with that service account, Kubernetes created a long-lived, never expired, token that resided in secret of type kubernetes.io/service-account-token. Kubernetes attached this token to the pod via volume mount.
Due to the unsecured nature of long-lived tokens this has been changed to something called Bound Service Account Token Volumes.
In short, Kubernetes no more creates a secret with a token for the service account but instead Kubelet injects a short-lived token with a default timespan of 1 hour to the pod and refresh it when it's expired.
When you ran kubectl describe serviceaccount jenkins you saw <none> in the section of the Tokens because it represents the 'old fashioned' static tokens that as mentioned are not created by default anymore.
You can manually create such a static token with
> cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Secret
type: kubernetes.io/service-account-token
metadata:
name: jenkins
annotations:
kubernetes.io/service-account.name: jenkins
EOF
and then when you run describe again and you will the new token
> kubectl describe serviceaccount jenkins
Name: jenkins
Namespace: jenkins-demo
Labels: <none>
Annotations: <none>
Image pull secrets: <none>
Mountable secrets: <none>
Tokens: jenkins
Events: <none>
You can create multiple tokens with different names and you will see all of them in the describe output.
BUT This is a bad practice to create these static tokens because they never expired. You should use short-lived token that you can create with the command you mentioned kubectl create token jenkins
You can control the duration with --duration <seconds>s and create a token with an expiration of up to 48h. The default is 1h.
The creation of new token doesn't overwrite the previous one. These tokens are JWTs - meaning they are signed and distributed, and are not kept on the server. If you want to see the content of a token you can paste the output
of kubectl create token jenkins in jwt.io.
Same as with the static token. You can run
kubectl get secret jenkins --output=jsonpath='{.data.token}' | base64 -d
and paste the output in jwt.io. You can notice this token doesn't have an expiration date.
Reference:
https://kubernetes.io/docs/reference/access-authn-authz/service-accounts-admin/
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#manually-create-an-api-token-for-a-serviceaccount
https://github.com/kubernetes/enhancements/tree/master/keps/sig-auth/1205-bound-service-account-tokens
https://github.com/kubernetes/enhancements/issues/542
I have used kubectl create serviceaccount sa1 to create service account. Then I used kubectl get serviceaccount sa1 -oyaml command to get service account info. But it returns as below.
apiVersion: v1
kind: ServiceAccount
metadata:
creationTimestamp: "2022-05-16T08:03:50Z"
name: sa1
namespace: default
resourceVersion: "19651"
uid: fdddacba-be9d-4e77-a849-95ca243781cc
I need to get,
secrets:
- name: <secret>
part. but it doesn't return secrets. How to fix it?
In Kubernetes 1.24, ServiceAccount token secrets are no longer automatically generated. See "Urgent Upgrade Notes" in the 1.24 changelog file:
The LegacyServiceAccountTokenNoAutoGeneration feature gate is beta, and enabled by default. When enabled, Secret API objects containing service account tokens are no longer auto-generated for every ServiceAccount. Use the TokenRequest API to acquire service account tokens, or if a non-expiring token is required, create a Secret API object for the token controller to populate with a service account token by following this guide. (#108309, #zshihang)
This means, in Kubernetes 1.24, you need to manually create the Secret; the token key in the data field will be automatically set for you.
apiVersion: v1
kind: Secret
metadata:
name: sa1-token
annotations:
kubernetes.io/service-account.name: sa1
type: kubernetes.io/service-account-token
Since you're manually creating the Secret, you know its name: and don't need to look it up in the ServiceAccount object.
This approach should work fine in earlier versions of Kubernetes too.
I am setting up a namespace for my application that has statefulsets, deployments, and secrets into that namespace. Using RBAC, I am defining specific roles and binding them to a service account that is used by the deployment/statefulset. This works as expected.
Now when I try to test if the secrets are secure by not assigning any service account to the deployment, it still pulls down the secrets. The default service account in the namespace is bound with the view clusterrole which should not have access to secrets.
Any clue what is happening here?
Thanks in advance.
I believe you need to assign a RoleBinding to the default service account on your namespace. For example:
kubectl create rolebinding myapp-view-binding --clusterrole=view --serviceaccount=default:default --namespace=default
The view role should prevent you from reading secrets.
Now when I try to test if the secrets are secure by not assigning any service account to the deployment...
If you don't assign a service account to your deployment, the default service account in the deployment's namespace will be used.
... it still pulls down the secrets
Try set the automountServiceAccountToken: false on the pod. That will ensure the service account token is not automatically mounted. So something like:
apiVersion: apps/v1
kind: Deployment
metadata:
name: my-pod
spec:
...
template:
...
spec:
serviceAccountName: default
automountServiceAccountToken: false
I have my Kubernetes cluster running in GKE I want to run an application outside the cluster and talk to the Kubernetes API.
By using password retrieved from running:
gcloud container clusters get-credentials cluster-2 --log-http
I am able to access the API with Basic authentication.
But I want to create multiple Kubernetes service accounts and configure them with required authorization and use appropriately.
So, I created service accounts and obtained the tokens using:
kubectl create serviceaccount sauser1
kubectl get serviceaccounts sauser1 -o yaml
kubectl get secret sauser1-token-<random-string-as-retrieved-from-previous-command> -o yaml
If I try to access the Kubernetes API with the obtained token using Bearer authentication then I get a 401 HTTP error. I thought that some permissions may have to be set for the service account, so based on the documentation here, I created below YAML file:
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
name: pod-reader
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "watch", "list"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pod-reader
subjects:
- kind: ServiceAccount
name: sauser1
namespace: default
roleRef:
kind: ClusterRole
name: pod-reader
apiGroup: rbac.authorization.k8s.io
and tried to apply it using the below command:
kubectl apply -f default-sa-rolebinding.yaml
I got the following error:
clusterrolebinding "pod-reader" created
Error from server (Forbidden): error when creating "default-sa-rolebinding.yaml"
: clusterroles.rbac.authorization.k8s.io "pod-reader" is forbidden: attempt to g
rant extra privileges: [PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["g
et"]} PolicyRule{Resources:["pods"], APIGroups:[""], Verbs:["watch"]} PolicyRule
{Resources:["pods"], APIGroups:[""], Verbs:["list"]}] user=&{xyz#gmail.
com [system:authenticated] map[authenticator:[GKE]]} ownerrules=[PolicyRule{Res
ources:["selfsubjectaccessreviews"], APIGroups:["authorization.k8s.io"], Verbs:[
"create"]} PolicyRule{NonResourceURLs:["/api" "/api/*" "/apis" "/apis/*" "/healt
hz" "/swagger-2.0.0.pb-v1" "/swagger.json" "/swaggerapi" "/swaggerapi/*" "/versi
on"], Verbs:["get"]}] ruleResolutionErrors=[]
I dont know how to proceed from here. Is my approach correct or am I missing something here?
UPDATE: As per the post referred by #JanosLenart in the comments, modified the kubectl command and the above error is not observed. But accessing the API, still gives 401 error. The curl command that I am using is:
curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v
#Janos pointed out the potential problem, however I think you will need an actual Cloud IAM Service Account as well, because you said:
I want to run an application outside the cluster [...]
If you're authenticating to GKE from outside, I believe you can only use the Google IAM identities. (I might be wrong, if so, please let me know.)
In this case, what you need to do:
Create an IAM service account and download a json key file of it.
set GOOGLE_APPLICATION_CREDENTIALS to that file.
either:
use RBAC like in your question to give permissions to the email address of the IAM Service Account
use IAM Roles to give the IAM Service Account on the GKE API (e.g. Container Developer role is usually sufficient)
Use kubectl command against the cluster (make sure you have a .kube/config file with the cluster's IP/CA cert beforehand) with the environment variable above, it should work.
YMMV
I managed to get it work without USING an actual Cloud IAM Service Account
First, I decided to use an shell inside GKE's k8s cluster by running
kubectl run curl-random --image=radial/busyboxplus:curl -i --tty --rm
Second, I made sure I decoded my token by copying the token and then running through
pbpaste | base64 -D
Third, I created the rolebinding for the serviceaccount, NOT the username.
kubectl create clusterrolebinding shaoserverless-cluster-admin-binding --clusterrole=cluster-admin --serviceaccount=default:shaoserverless
The third step was particularly tricky but I got the inspiration since the error message used to be
Unknown user \"system:serviceaccount:default:shaoserverless\"",
Lastly, then this works
curl -k -1 -H "Authorization: Bearer <token>" https://<ip-address>/api/v1/namespaces/default/pods -v
I am trying to create a service account with a known, fixed token used by Jenkins to deploy stuff into kubernetes. I manage to create the token all right with the following yaml:
apiVersion: v1
kind: Secret
metadata:
name: integration-secret
annotations:
kubernetes.io/service-account.name: integration
type: kubernetes.io/service-account-token
data:
token: YXNkCg== # yes this base64
Then I've attached the secret to 'integration' user and it's visible:
-> kubectl describe sa integration
Name: integration
Namespace: default
Labels: <none>
Annotations: <none>
Mountable secrets: integration-secret
integration-token-283k9
Tokens: integration-secret
integration-token-283k9
Image pull secrets: <none>
But the login fails. If i remove the data and data.token, the token get auto-created and login works. Is there something I'm missing? My goal is to have fixed token for CI so that I won't have to update it everywhere when creating a project (don't worry this is just dev environments). Is it possible for example to define username/password for service accounts for API access?
Is it possible for example to define username/password for service accounts for API access?
No, the tokens must be valid JWTs, signed by the service account token signing key.