I'm trying to run a deployment on a Kubernetes cluster at work through a GitLab CI/CD process (i.e. I don't control most of the configs). I'm also new to Kubernetes, so please forgive me if this is basic and obvious.
I have created my rolebindings:
kubectl create rolebinding [foo] --clusterrole=edit --serviceaccount=[bar]:default
And added my tokens and all settings to GitLab
When the deployment kicks off however, it will always fail at deployment with:
Error from server (Forbidden): error when creating "/builds/bar/baz/deployment.yml": service is forbidden: User "system:serviceaccount:bar:bar-service-account" cannot create services in namespace "bar"
I thought I should be working in system:serviceaccount:bar:default. why is :default being replaced with :bar-service-account and/or how do I fix this.
Many many thanks in advance
You are granting permissions to the default service account with the rolebinding you are creating. However, the deployment is not using that service account. If you look at the deployment manifest, it will have a serviceAccountName of bar-service-account.
Either change the deployment to use the default service account or change the rolebinding to grant permissions to the service account being used.
Related
I didn't get how I can restrict the access of custom resources in Openshift using RBAC
let's assume I have a custom api:
apiVersion: yyy.xxx.com/v1
kind: MyClass
metadata:
...
Is it possible to prevent some users to deploy resources where apiVersion=yyy.xxx.com/v1 and kind=MyClass?
Also can I grant access to other users to deploy resources where apiVersion=yyy.xxx.com/v1 and kind=MyOtherClass?
If this can be done using RBAC roles, how can I deploy RBAC roles in Openshift? only using CLI or I can create some yaml configuration files and deploy them with Yaml for example?
You can use cluster roles and RBAC roles:
oc adm policy add/remove-cluster-role-to-group oauth:system:authenticated
So the general idea is to remove the permission to deploy the resource to all the authenticated users.
The next step is to add the permission to deploy that resourse only to ServicesAccounts assigned to specific namepsaces
OpenShift/Kubernetes has Cluster Role/Binding and Local Role/Binding.
Here is the definitions in the docs. *1
Cluster role/binding: Roles and bindings that are applicable across all projects. Cluster roles exist cluster-wide, and cluster role bindings can reference only cluster roles.
Local role/binding: Roles and bindings that are scoped to a given project. While local roles exist only in a single project, local role bindings can reference both cluster and local roles.
If your Custom Resource is the resource existing in a single namespace. You can manage to give permission to others.
Whereas, if the Custom Resource is the cluster wide resource, cluster admin can only deploy the resource.
*1: https://docs.openshift.com/container-platform/4.11/authentication/using-rbac.html
I am trying to see which kubernetes user is creating the deployment and what type of authentication is used (basic auth, token, etc).
I try to do it using this:
kubectl describe deployment/my-workermole
but I am not finding that type of information in there.
Cluster is not managed by me and I am not able to find it in the deployment Jenkinsfile. Where and how can I find that type of information in my kubernetes deployment but after deployment?
I understand that I can copy my .kube/config to my CI/CD server, or just name the ServiceAccount to allow my CD pipeline to use HELM for deployment.
However, what if I want to allow deployment via Helm, but restrict a lot of other access, like:
reading data from pods or a deployed database
port-forward services
... so basically accessing all data in the cluster, except for stateless Docker containers deployed via Helm.
Would it be possible to create a new ClusterRole with limited rights? What verbs in a ClusterRole does Helm need at least to function properly?
What rights does Helm need at the least?
It comes down to what your Helm chart is doing to Kubernetes.
ClusterRoles can be bound to a particular namespace through reference in a RoleBinding. The admin, edit and view default ClusterRoles are commonly used in this manner. For more detailed info see this description. For example edit is a default ClusterRole which allows read/write access to most objects in a namespace. It does not allow viewing or modifying Roles or RoleBindings; and granting a user cluster-admin access at the namespace scope provides full control over every resource in the namespace, including the namespace itself.
You can also restrict a user's access to a particular namespace by using either the edit or the admin role. See this example.
The permissions strategy could also depend on what objects will be created by the installation. The user will need all access to those API objects that will be managed by helm installations. Using RBAC Authorization has this concept explained in more detail with several examples that you could use as a reference. Also, this source would be helpful.
I have a Gitlab runner using a K8s executor. But when running the pipeline I am getting below error
Checking for jobs... received job=552009999
repo_url=https://gitlab.com/deadbug/rns.git runner=ZuT1t3BJ
WARNING: Namespace is empty, therefore assuming 'default'. job=552009999 project=18763260
runner=ThT1t3BJ
ERROR: Job failed (system failure): secrets is forbidden: User "deadbug" cannot create resource
"secrets" in API group "" in the namespace "default" duration=548.0062ms job=552009999
From the error message, I undestand the namespace needs to be updated. I specified namespace in the Gitlab variables
But after this also, pipeline is failing with the above error message. How do I change the namespace for the runner ?
This seems to be linked to the permissions of the service account rather than the namespace directly. If you use GitLab's Kubernetes integration, you should not override the namespace, as GitLab will create one for you.
Make sure the service account you added to GitLab has the correct role. From https://docs.gitlab.com/ee/user/project/clusters/add_remove_clusters.html:
When GitLab creates the cluster, a gitlab service account with cluster-admin privileges is created in the default namespace to manage the newly created cluster
You may be having the same issue I was having. Instead of installing the Gitlab Runner into the existing Kubernetes cluster with helm install, I used helm template and another manager to install it (kapp). This breaks the logic in the Helm template that specifies the namespace as the one used in the helm install (See code). This led the runner to attempt to create the pods in the default namespace, instead of the namespace I created. I was able to specify it manually in my values.yml file though:
runners:
namespace: my-namespace
Is it possible to invoke a kubernetes Cron job inside a pod . Like I have to run this job from the application running in pod .
Do I have to use kubectl inside the pod to execute the job .
Appreciate your help
Use the Default Service Account to access the API server. When you
create a pod, if you do not specify a service account, it is
automatically assigned the default service account in the same
namespace. If you get the raw json or yaml for a pod you have created
(for example, kubectl get pods/ -o yaml), you can see the
spec.serviceAccountName field has been automatically set.
You can access the API from inside a pod using automatically mounted
service account credentials, as described in Accessing the Cluster.
The API permissions of the service account depend on the authorization
plugin and policy in use.
In version 1.6+, you can opt out of automounting API credentials for a
service account by setting automountServiceAccountToken: false on the
service account
https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/
So the First task is to either grant the permission of doing what you need to create to the default service account of the pod OR create a custom service account and use it inside the pod
Programatically access the API server using that service account to create the job you need
It could be just a simple curl POST to the API server from inside the pod with the json for the job creation
How do I access the Kubernetes api from within a pod container?
you can also use the application specific SDK , for example if you have a python application , you can import kubernetes and run the job.