When run kubectl get cs on centos 7 I got below error message.
No resources found.
Error from server (Forbidden): componentstatuses is forbidden:
User "system:node:<server-name>" cannot list componentstatuses at the cluster scope
I can confirm the api server is running kubectl cluster-info
Kubernetes master is running at https://<server-IP>:6443
KubeDNS is running at https://<server-IP>:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
Also I have below in ~/.bash_profile
export http_proxy=http://<proxy-server-IP>:3128
export https_proxy=http://<proxy-server-IP>:3128
export no_proxy=$no_proxy,127.0.0.1,localhost,<server-IP>,<server-name>
export KUBECONFIG=/etc/kubernetes/kubelet.conf
Not only kubectl get cs yield the error message, kubectl apply -f kubernetes-dashboard.yaml yield similar error message
Error from server (Forbidden): error when retrieving current configuration of:
Resource: "/v1, Resource=secrets", GroupVersionKind: "/v1, Kind=Secret"
Name: "kubernetes-dashboard-certs", Namespace: "kube-system"
Object: &{map["kind":"Secret" "metadata":map["labels":map["k8s-app":"kubernetes-dashboard"] "name":"kubernetes-dashboard-certs" "namespace":"kube-system" "annotations":map["kubectl.kubernetes.io/last-applied-configuration":""]] "type":"Opaque" "apiVersion":"v1"]}
from server for: "https://raw.githubusercontent.com/kubernetes/dashboard/master/src/deploy/recommended/kubernetes-dashboard.yaml":
secrets "kubernetes-dashboard-certs" is forbidden:
User "system:node:<server-name>" cannot get secrets in the namespace "kube-system":
no path found to object
export KUBECONFIG=/etc/kubernetes/kubelet.conf
Is completely incorrect; you are, as the error message is cheerfully trying to inform you, attempting to perform cluster operations as a Node, not as one of the Users or ServiceAccounts. RBAC is almost explicitly designed to stop you from doing exactly what you are currently doing. You would never want a Node to be able to read sensitive credentials nor create arbitrary Pods at cluster scope.
If you want to be all caviler about it, then ssh into a master Node and use the cluster-admin credentials usually found in /etc/kubernetes/admin.conf (or a similar file -- depending on how your cluster was provisioned). If you don't already have a cluster-admin credential, then create an X.509 certificate that is signed by a CA that the apiserver trusts with an Organization (O= in X.509 parlance) of cluster-admin and then create yourself a ServiceAccount (or whatever) with a ClusterRoleBinding of cluster-admin and go from there.
Try the below snippets
1) sudo su
2) kubectl get cs
after reinstall centos 7 and follow below steps i am able to bring up the master properly
install docker-ce and add proxy
install kubeadm,kubectl,kubelet
disable firewalld and turn off swap
export no_proxy in .bash_profile
export no_proxy=$no_proxy,127.0.0.1,localhost,<master-server-name>,<master-server-ip>,10.96.0.0/12,10.244.0.0/16
kubeadm init
kubeadm init --apiserver-advertise-address=<master-server-ip> --pod-network-cidr=10.244.0.0/16
mkdir -p $HOME/.kube
\cp -f /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
test with kubectl get cs
NAME STATUS MESSAGE ERROR
scheduler Healthy ok
controller-manager Healthy ok
etcd-0 Healthy {"health": "true"}
No need to manually install etcd nor export KUBECONFIG.
Related
I am new to Kubernetes. I just trying to create a tls secret using kubectl. My ultimate goal is deploy a keycloak cluster in kubernetes.
So I follow this youtube tutorial. But in this tutorial doesn't mention how to generate my own tls key and tls cert. So to do that I use this documentation (https://www.linode.com/docs/guides/create-a-self-signed-tls-certificate/).
Then I could generate MyCertTLS.crt and MyKeyTLS.key
gayan#Gayan:/srv$ cd certs
gayan#Gayan:/srv/certs$ ls
MyCertTLS.crt MyKeyTLS.key
To create secret key for the kubernetes, I ran this command
sudo kubectl create secret tls my-tls --key="MyKeyTLS.key" --cert="MyCertTLS.crt" -n keycloak-test
But It's not working, I got this error,
gayan#Gayan:/srv/certs$ sudo kubectl create secret tls my-tls --key="MyKeyTLS.key" --cert="MyCertTLS.crt" -n keycloak-test
[sudo] password for gayan:
error: failed to create secret Post "http://localhost:8080/api/v1/namespaces/keycloak-test/secrets?fieldManager=kubectl-create&fieldValidation=Strict": dial tcp 127.0.0.1:8080: connect: connection refused
Note:
MiniKube is Running...
And Ingress Addon also enabled...
I have created a namespace called keycloak-test.
gayan#Gayan:/srv/keycloak$ kubectl get namespaces
NAME STATUS AGE
default Active 3d19h
ingress-nginx Active 119m
keycloak-test Active 4m12s
kube-node-lease Active 3d19h
kube-public Active 3d19h
kube-system Active 3d19h
kubernetes-dashboard Active 3d19h
I am trying to fix this error. But I have no idea why I get this, looking for a solution from the genius community.
I figured this out! I posted this, because this may helpful for someone.
I am getting that error,
error: failed to create secret Post "http://localhost:8080/api/v1/namespaces/keycloak-test/secrets?fieldManager=kubectl-create&fieldValidation=Strict": dial tcp 127.0.0.1:8080: connect: connection refused
Because my kubernetes api-server is running on a different port.
You can view what port your kubernetes api-server is running by running this command,
kubectl config view
Then for example, if you can see server: localhost:40475 like that, It's mean your server running on port 40475.
And kubernetes default port is 8443
Then you should mention the correct port on your kubectl command to create the secret.
So, I add --server=https://localhost:40475 to my command.
kubectl create secret tls my-tls --key="tls.key" --cert="tls.crt" -n keycloak-test --server=https://localhost:40475
And another thing, if you getting error like permission denied
You have to change the ownership of your tls.key file and tls.crt file.
I did this by running these commands,
sudo chmod 666 tls.crt
sudo chmod 666 tls.key
Then you should run above kubectl command, without sudo! It works !!!!!!
If you run that command with sudo, It will ask username and passwords and it confused me and it did not work.
So, by doing this way, I solved this issue!
Hope this will help to someone!!! Thanks!
In your examples, kubectl get namespaces works, but sudo kubectl create secret doesn't.
You don't need sudo to work with Kubernetes. In particular, the connection information is stored in a $HOME/.kube/config file by default, but when you sudo kubectl ..., that changes the home directory and you can't find the connection information.
The standard Kubernetes assumption is that the cluster is remote, and so your local user ID doesn't really matter to it. All that does matter is the Kubernetes-specific permissions assigned to the user that's accessing the cluster.
After certification renewal in K8 Cluster, we are getting permission error while running some kubectl commands. For example; we get this error while running the commands "kubectl get deployments" and "kubectl get pv"
"Error from server (Forbidden): pods "<>" is forbidden: User "system:node:<>" cannot create resource "pods/exec" in API group "" in the namespace "......"
But, we can able to run commands such as "Kubectl get nodes" and "kubectl get pods" without any issues.
During cert renewal process, we ran the below command and manually updated kubelet.conf in /etc/kubernetes and config file in /root/.kube directory.
Is there any other files, we need to update the new certificate related details ? Kindly help us with remediation process/steps at the earliest possible.
• sudo kubeadm alpha kubeconfig user --org system:nodes --client-name system:node:$(hostname) > kubelet.conf
Our Prod Kubernetes Cluster Information:
Kubernetes Cluster Version - 1.13.x
Master Node - 1
Worker Nodes - 11
Recently, the below mentioned certifications got renewed recently.
apiserver.crt
apiserver-kubelet-client.crt
front-proxy-client.crt
apiserver-etcd-client.crt
Apparently, you are using the kubelet certificate instead of that for the kubectl (admin cert).
Try running this command and see if it works:
sudo kubectl get pv --kubeconfig /etc/kubernetes/admin.conf
If it does, do:
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl get pv
kubectl get deploy
I accidentally deleted the kubernetes svc:
service "kubernetes" deleted
using:
kubectl delete svc --all
what should I do? I was just trying to remove services so I could launch new ones.
A bit theory first ;)
Whenever you delete kubernetes svc, you also delete endpoint and this is where
Reconciler comes in. It is actually a controller manager for the core bootstrap Kubernetes controller loops, which manage creating the "kubernetes" service, the
"default", "kube-system" and "kube-public" namespaces, and provide the IP repair check on service IPs.
So, in healthy clusters default.kubernetes service should be automatically recreated by controller manager.
If it's not, I'd recommend to:
Check api-server logs
kubectl logs -f kube-apiserver-master -n kube-system
You should see something like:
Resetting endpoints for master service "kubernetes" to [10.156.0.3]
If you don't see it, try to manually remove etcd key for this service
Because the current state of the cluster is stored in etcd, it may happen that the key remain when you deleted a service:
a. exec to etcd-master pods
kubectl exec -it etcd-master -n kube-system sh
b. get the etcd key value
ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt get /registry/services/endpoints/default/kubernetes
c. if you get any value like:
v1 Endpointst
O
kubernetesdefault"*$eafc04cf-90f3-11e9-a75e-42010a9c00032����z!
10.156.0.3
https�2TCP"
just remove it by
ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/etcd/ca.crt --key=/etc/kubernetes/pki/etcd/server.key --cert=/etc/kubernetes/pki/etcd/server.crt rm /registry/services/endpoints/default/kubernetes
After you did it, check the api-server logs once again.
ubuntu#master:~$ kubectl delete all --all
service "kubernetes" deleted
ubuntu#master:~$ kubectl get svc
Command 'kubernetes' not found, but can be installed with:
sudo apt install kubernetes
ubuntu#master:~$ sudo apt install kubernetes
ran the installation and it was restored.
I'm trying to enable RBAC on my cluster and iadded those following line to the kube-apiserver.yml :
- --authorization-mode=RBAC
- --runtime-config=rbac.authorization.k8s.io/v1beta1
- --authorization-rbac-super-user=admin
and i did systemctl restart kubelet ;
the apiserver starts successfully but i'm not able to run kubectl command and i got this error :
kubectl get po
Error from server (Forbidden): pods is forbidden: User "kubectl" cannot list pods in the namespace "default"
where am I going wrong or i should create some roles to the kubectl user ? if so how that possible
Error from server (Forbidden): pods is forbidden: User "kubectl" cannot list pods in the namespace "default"
You are using user kubectl to access cluster by kubectl utility, but you set --authorization-rbac-super-user=admin, which means your super-user is admin.
To fix the issue, launch kube-apiserver with superuser "kubectl" instead of "admin."
Just update the value of the option: --authorization-rbac-super-user=kubectl.
Old question but for google searchers, you can use the insecure port:
If your API server runs with the insecure port enabled (--insecure-port), you can also make API calls via that port, which does not enforce authentication or authorization.
Source: https://kubernetes.io/docs/reference/access-authn-authz/rbac/#privilege-escalation-prevention-and-bootstrapping
So add --insecure-port=8080 to your kube-apiserver options and then restart it.
Then run:
kubectl create clusterrolebinding kubectl-cluster-admin-binding --clusterrole=cluster-admin --user=kubectl
Then turn the insecure-port off.
I'm trying to deploy an application in a GKE 1.6.2 cluster running ContainerOS but the instructions on the website / k8s are not accurate anymore.
The error that I'm getting is:
Error from server (Forbidden): User "circleci#gophers-slack-bot.iam.gserviceaccount.com"
cannot get deployments.extensions in the namespace "gopher-slack-bot".:
"No policy matched.\nRequired \"container.deployments.get\" permission."
(get deployments.extensions gopher-slack-bot)
The repository for the application is available here available here.
Thank you.
I had a few breaking changes in the past with using the gcloud tool to authenticate kubectl to a cluster, so I ended up figuring out how to auth kubectl to a specific namespace independent of GKE. Here's what works for me:
On CircleCI:
setup_kubectl() {
echo "$KUBE_CA_PEM" | base64 --decode > kube_ca.pem
kubectl config set-cluster default-cluster --server=$KUBE_URL --certificate-authority="$(pwd)/kube_ca.pem"
kubectl config set-credentials default-admin --token=$KUBE_TOKEN
kubectl config set-context default-system --cluster=default-cluster --user=default-admin --namespace default
kubectl config use-context default-system
}
And here's how I get each of those env vars from kubectl.
kubectl get serviceaccounts $namespace -o json
The service account will contain the name of it's secret. In my case, with the default namespace, it's
"secrets": [
{
"name": "default-token-655ls"
}
]
Using the name, I get the contents of the secret
kubectl get secrets $secret_name -o json
The secret will contain ca.crt and token fields, which match the $KUBE_CA_PEM and $KUBE_TOKEN in the shell script above.
Finally, use kubectl cluster-info to get the $KUBE_URL value.
Once you run setup_kubectl on CI, your kubectl utility will be authenticated to the namespace you're deploying to.
In Kubernetes 1.6 and GKE, we introduce role based cess control. The authors of your took need to give the service account the ability to get deployments (along with probably quite a few others) to its account creation.
https://kubernetes.io/docs/admin/authorization/rbac/