I'd like to find out the other pods' name running in the same single-host cluster. All pods are single-application containers. I have pod A (written in Java) that acts as a TCP/IP server and a few other pods (written in C++) connect to the server.
In pod A, I can get IP address of clients (other pods). How do I get their pods' name? I can't run kubectl commands because pod A has no kubectl installed.
Thanks,
You can directly call kube-apiserver with cURL.
First you need to have a serviceaccount binded to clusterrole to be able to send requests to apiserver.
kubectl create clusterrole listpods --verb=get,list,watch --resource=pods
kubectl create clusterrolebinding listpods --clusterrole=listpods --serviceaccount=default:default
Get a shell inside a container
kubectl exec -it deploy/YOUR_DEPLOYMENT -- sh
Define necessary parameters for your cURL, run below commands inside container
APISERVER=https://kubernetes.default.svc
SERVICEACCOUNT=/var/run/secrets/kubernetes.io/serviceaccount
NAMESPACE=$(cat ${SERVICEACCOUNT}/namespace)
TOKEN=$(cat ${SERVICEACCOUNT}/token)
CACERT=${SERVICEACCOUNT}/ca.crt
Send pods list request to apiserver
curl -s -k -H "Authorization: Bearer $TOKEN" -H 'Accept: application/json' $APISERVER/api/v1/pods | jq "[ .items[] | .metadata.name ]"
Done ! It will return you a json list of "kubectl get pods"
For more examples, you can check OpenShift RestAPI Reference. Also, if you are planning to do some programmatic stuff, I advice you to checkout official kubernetes-clients.
Credits for jq improvement to #moonkotte
Using minikube, when running the following command:
kubectl -v=11 --kubeconfig /dev/null --insecure-skip-tls-verify -s http://localhost:8001 --token "invalid" -n namespace get pods
I have an answer when I don't want one. And I don't know how it was authorized. Moreover, if I use a valid token with specific rights, these are not used.
kubectl --token=$TOKEN doesn't run with the permissions of the token doesn't answer my question as I specified to used /dev/null as a config file.
Any idea ?
I will try to summarize the answer I provided in the comments.
The question was: Why does running kubectl -s http://localhost:8001 --kubeconfig /dev/null --token <invalid_token> (where :8001 is a port opened by kubectl proxy) repoonds as if I was authorized, when it shouldn't beacause I set all possible authorization options to null or incorrect values?
The answer is that kubectl proxy opens a port and handles all authorization for you so you dont have to. Now to access REST api of kubernetes all you need to do is to use curl localhost:8001/.... No tokens and certificates.
Because you are already authorized with kubectl proxy, using kubectl and pointing it to localhost:8001 is causing that it won't need to authorize and you won't need any tokens to access k8s.
As an alternative you can check what happens when you run the same but instead of connecting through kubectl proxy you use kubernetes port directly.
You mentioned that you are using minikube so by default that would be port 8443
$ kubectl --kubeconfig /dev/null -s https://$(minikube ip):8443 --token "invalid" --insecure-skip-tls-verify get pods
error: You must be logged in to the server (Unauthorized)
As you see now it works as expected.
When I am using the command kubectl with the --token flag and specify a token, it still uses the administrator credentials from the kubeconfig file.
This is what I did:
NAMESPACE="default"
SERVICE_ACCOUNT_NAME="sa1"
kubectl create sa $SERVICE_ACCOUNT_NAME
kubectl create clusterrolebinding list-pod-clusterrolebinding \
--clusterrole=list-pod-clusterrole \
--serviceaccount="$NAMESPACE":"$SERVICE_ACCOUNT_NAME"
kubectl create clusterrole list-pod-clusterrole \
--verb=list \
--resource=pods
TOKEN=`kubectl get secrets $(kubectl get sa $SERVICE_ACCOUNT_NAME -o json | jq -r '.secrets[].name') -o json | jq -r '.data.token' | base64 -d`
# Expected it will fail but it doesn't because it uses the admin credentials
kubectl get secrets --token $TOKEN
The token have permissions to list pods, so I expect the kubectl get secrets --token $TOKEN to fail but it doesn't because it still uses the context of the administrator.
I don't create new context, I know kubectl have this ability to use bearer token and want to understand how to do it.
I also tried this kubectl get secrets --insecure-skip-tls-verify --server https://<master_ip>:6443 --token $TOKENand it also didn't return a Forbidden result.
If you test it you can use katacoda:
https://www.katacoda.com/courses/kubernetes/playground
EDIT:
I tried to create context with this:
NAMESPACE="default"
SERVICE_ACCOUNT_NAME="sa1"
CONTEXT_NAME="sa1-context"
USER_NAME="sa1-username"
CLUSTER_NAME="kubernetes"
kubectl create sa "$SERVICE_ACCOUNT_NAME" -n "$NAMESPACE"
SECRET_NAME=`kubectl get serviceaccounts $SERVICE_ACCOUNT_NAME -n $NAMESPACE -o json | jq -r '.secrets[].name'`
TOKEN=`kubectl get secrets $SECRET_NAME -n $NAMESPACE -o json | jq -r '.data | .token' | base64 -d`
# Create user with the JWT token of the service account
echo "[*] Setting credentials for user: $USER_NAME"
kubectl config set-credentials $USER_NAME --token=$TOKEN
# Makue sure the cluster name is correct !!!
echo "[*] Setting context: $CONTEXT_NAME"
kubectl config set-context $CONTEXT_NAME \
--cluster=$CLUSTER_NAME \
--namespace=$NAMESPACE \
--user=$USER_NAME
But when I tried kubectl get secrets --context $CONTEXT_NAME it still succeeded and was supposed fail because it doesn't have permissions for that.
Edit 2:
Option to run it correctly based on the kubectl API:
kubectl get pods --token `cat /home/natan/token` -s https://<ip>:8443 --certificate-authority /root/.minikube/ca.crt --all-namespaces
Or without TLS:
kubectl get pods --token `cat /home/natan/token` -s https://<ip>:8443 --insecure-skip-tls-verify --all-namespaces
This is tricky because if you are using client certificate for authenticating to kubernetes API server overriding token with kubectl is not going to work because the authentication with certificate happens early in the process during the TLS handshake.Even if you provide a token in kubectl it will be ignored.This is the reason why you are able to get secrets because the client certificate have permission to get secrets and the token is ignored.
So if you want to use kubectl token the kubeconfig file should not have client certificate and then you can override that token with --token flag in Kubectl. See the discussion in the question on how to create a kubeconfig file for a service account token.
Also you can view the bearer token being sent in kubectl command using command
kubectl get pods --v=10 2>&1 | grep -i bearer
This command will help you verify the authorization of the service account.
kubectl auth can-i <verb> <resources> --as=system:serviceaccount:<namespace>:<service account name>
kubectl auth can-i get pods --as=system:serviceaccount:default:default
I need to delete POD on my GCP kubernetes cluster. Actually in Kubernetes Engine API documentation I can find only REST api's for: projects.locations.clusters.nodePools, but nothing for PODs.
The GKE API is used to manage the cluster itself on an infrastructure level. To manage Kubernetes resources, you'd have to use the Kubernetes API. There are clients for various languages, but of course you can also directly call the API.
Deleting a Pod from within another or the same Pod:
PODNAME=ubuntu-xxxxxxxxxx-xxxx
curl https://kubernetes/api/v1/namespaces/default/pods/$PODNAME \
-X DELETE -k \
-H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
From outside, you'd have to use the public Kubernetes API server URL and a valid token. Here's how you get those using kubectl:
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
TOKEN=$(kubectl get secret $(kubectl get serviceaccount default -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode )
Here's more official information on accessing the Kubernetes API server.
What is correct way to kubernetes cluster setup using minikube through the kubernetes api ?
At the moment, I can't find a port through which the kubernetes cluster can be accessed.
The easiest way to access the Kubernetes API with when running minikube is to use
kubectl proxy --port=8080
You can then access the API with
curl http://localhost:8080/api/
This also allows you to browse the API in your browser. Start minikube using
minikube start --extra-config=apiserver.Features.EnableSwaggerUI=true
then start kubectl proxy, and navigate to http://localhost:8080/swagger-ui/ in your browser.
You can access the Kubernetes API with curl directly using
curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/client.crt --key ~/.minikube/client.key https://`minikube ip`:8443/api/
but usually there is no advantage in doing so. Common browsers are not happy with the certificates minikube generates, so if you want to access the API with your browser you need to use kubectl proxy.
Running minikube start will automatically configure kubectl.
You can run minikube ip to get the IP that your minikube is on. The API server runs on 8443 by default.
Update: To access the API server directly, you'll need to use the custom SSL certs that have been generated. by minikube. The client certificate and key are typically stored at: ~/.minikube/apiserver.crt and ~/.minikube/apiserver.key. You'll have to load them into your HTTPS client when you make requests.
If you're using curl use the --cert and the --key options to use the cert and key file. Check the docs for more details.
Update2: The client certificate and key are typically stored at: ~/.minikube/profiles/minikube directory when you use the version >= 0.19 (more informations). You probably need to set the --insecure options to the curl client because of the self-signed certificate.
I went through lots of answers, but lots of them are wrong.
Before we do, we need IP and token.
How to get IP: minikube ip
How to generate Token:
$export secret=kubectl get serviceaccount default -o json | jq -r '.secrets[].name'
$kubectl get secret $secret -o yaml | grep "token:" | awk {'print $2'} | base64 -D > token
Note: base64 uses -D for mac, but -d for Linux.
Then, the correct command is:
#curl -v -k -H --cacert ~/.minikube/ca.crt -H "Authorization: Bearer $(cat ~/YOUR_TOKEN)" "https://{YOUR_IP}:8443/api/v1/pods"
User Sven Marnach got me in the right direction however to get the correct server ip, crt and key location I ran kubectl config view.
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/user/.minikube/ca.crt
server: https://127.0.0.1:32792
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/user/.minikube/profiles/minikube/client.crt
client-key: /Users/user/.minikube/profiles/minikube/client.key
$ curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "172.17.0.2:8443"
}
]
}
$ curl -s --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/v1/pods | jq .items[].metadata | jq '"\(.name), \(.namespace), \(.selfLink)"'
"shell-demo, default, /api/v1/namespaces/default/pods/shell-demo"
"coredns-f9fd979d6-6b2nx, kube-system, /api/v1/namespaces/kube-system/pods/coredns-f9fd979d6-6b2nx"
"etcd-minikube, kube-system, /api/v1/namespaces/kube-system/pods/etcd-minikube"
"kube-apiserver-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-apiserver-minikube"
"kube-controller-manager-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-controller-manager-minikube"
"kube-proxy-bbck9, kube-system, /api/v1/namespaces/kube-system/pods/kube-proxy-bbck9"
"kube-scheduler-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-scheduler-minikube"
"storage-provisioner, kube-system, /api/v1/namespaces/kube-system/pods/storage-provisioner"
Readers may also be interested in link.
For windows users, here is an alternative to the much simpler kubectl proxy command:
mount your local host's .minikube folder using "minikube mount [path-to-folder]:/host . This way, you will be able to access the certificates from within the node.If you don't know the exact path to this folder, you can get it by looking at the kubectl config view response.
On a different command prompt, take note of the IP of your kube api server. this can be done running from your host ( windows ) minikube ip. Note that this is the virtual IP within your minikube container.
Start a bash within the minikube container. docker exec -it {your-container-id} bash
Access to the folder you mounted on point 1). Now, simply curl to the Kubectl api server through its virtual ip from 2.):
curl https://{your-ip-from-2}:8443/api --key ./ca.key --cert ./ca.crt
Here we are passing the certs to be used. Notice how I am not using the proxy-client ones.
That's it. For learning purposes I think this is a more interesting method that directly proxying.
These instructions worked for me https://github.com/jenkinsci/kubernetes-plugin#configuration-on-minikube
Needed to generate & upload pfx file, along with the other steps mentioned there.
Most of the above answers are right in their own sense.
I will give my version of the answer:
1) What is the correct way to Kubernetes cluster setup using minikube through the Kubernetes API ?
Ans: I think this is pretty straight forward. Follow the installation steps mentions in the official k8s documentation for minikube installation
2) At the moment, I can't find a port through which the kubernetes cluster can be accessed.
Ans: This is too has a straight forward answer. You have to check your Kube config file. You can find it in your home directory ~/.kube/config. View this file and it will have the details.
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/username/.minikube/ca.crt
server: https://192.168.64.2:8443
name: minikube
contexts:
- context:
cluster: minikube
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/username/.minikube/client.crt
client-key: /Users/username/.minikube/client.key
The server detail mentioned here is your api-server endpoint to hit.
You can view this information using the kubectl command as well like this kubectl config view
Use below curl to hit the api-server using curl
curl https://192.168.64.2:8443/api/v1/pod --key /Users/sanjay/.minikube/client.key --cert /Users/sanjay/.minikube/client.crt --cacert /Users/sanjay/.minikube/ca.crt
Note: replace the ip port and the path as per your config file in above command.
Based on xichen's and Seba's answers above, this is how to acquire a token from a terminal:
$ function get_token() { secret=$(kubectl get serviceaccount "$1" -o jsonpath='{.secrets[0].name}') && kubectl get secret "$secret" -o jsonpath='{.data.token}' | base64 --decode; }
$ get_token target_account
I hope this would be useful for those who must use kubectl below 1.24 due to minikube issue with enabling ingress as stated in this question.