What is correct way to kubernetes cluster setup using minikube through the kubernetes api ?
At the moment, I can't find a port through which the kubernetes cluster can be accessed.
The easiest way to access the Kubernetes API with when running minikube is to use
kubectl proxy --port=8080
You can then access the API with
curl http://localhost:8080/api/
This also allows you to browse the API in your browser. Start minikube using
minikube start --extra-config=apiserver.Features.EnableSwaggerUI=true
then start kubectl proxy, and navigate to http://localhost:8080/swagger-ui/ in your browser.
You can access the Kubernetes API with curl directly using
curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/client.crt --key ~/.minikube/client.key https://`minikube ip`:8443/api/
but usually there is no advantage in doing so. Common browsers are not happy with the certificates minikube generates, so if you want to access the API with your browser you need to use kubectl proxy.
Running minikube start will automatically configure kubectl.
You can run minikube ip to get the IP that your minikube is on. The API server runs on 8443 by default.
Update: To access the API server directly, you'll need to use the custom SSL certs that have been generated. by minikube. The client certificate and key are typically stored at: ~/.minikube/apiserver.crt and ~/.minikube/apiserver.key. You'll have to load them into your HTTPS client when you make requests.
If you're using curl use the --cert and the --key options to use the cert and key file. Check the docs for more details.
Update2: The client certificate and key are typically stored at: ~/.minikube/profiles/minikube directory when you use the version >= 0.19 (more informations). You probably need to set the --insecure options to the curl client because of the self-signed certificate.
I went through lots of answers, but lots of them are wrong.
Before we do, we need IP and token.
How to get IP: minikube ip
How to generate Token:
$export secret=kubectl get serviceaccount default -o json | jq -r '.secrets[].name'
$kubectl get secret $secret -o yaml | grep "token:" | awk {'print $2'} | base64 -D > token
Note: base64 uses -D for mac, but -d for Linux.
Then, the correct command is:
#curl -v -k -H --cacert ~/.minikube/ca.crt -H "Authorization: Bearer $(cat ~/YOUR_TOKEN)" "https://{YOUR_IP}:8443/api/v1/pods"
User Sven Marnach got me in the right direction however to get the correct server ip, crt and key location I ran kubectl config view.
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/user/.minikube/ca.crt
server: https://127.0.0.1:32792
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/user/.minikube/profiles/minikube/client.crt
client-key: /Users/user/.minikube/profiles/minikube/client.key
$ curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "172.17.0.2:8443"
}
]
}
$ curl -s --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/v1/pods | jq .items[].metadata | jq '"\(.name), \(.namespace), \(.selfLink)"'
"shell-demo, default, /api/v1/namespaces/default/pods/shell-demo"
"coredns-f9fd979d6-6b2nx, kube-system, /api/v1/namespaces/kube-system/pods/coredns-f9fd979d6-6b2nx"
"etcd-minikube, kube-system, /api/v1/namespaces/kube-system/pods/etcd-minikube"
"kube-apiserver-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-apiserver-minikube"
"kube-controller-manager-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-controller-manager-minikube"
"kube-proxy-bbck9, kube-system, /api/v1/namespaces/kube-system/pods/kube-proxy-bbck9"
"kube-scheduler-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-scheduler-minikube"
"storage-provisioner, kube-system, /api/v1/namespaces/kube-system/pods/storage-provisioner"
Readers may also be interested in link.
For windows users, here is an alternative to the much simpler kubectl proxy command:
mount your local host's .minikube folder using "minikube mount [path-to-folder]:/host . This way, you will be able to access the certificates from within the node.If you don't know the exact path to this folder, you can get it by looking at the kubectl config view response.
On a different command prompt, take note of the IP of your kube api server. this can be done running from your host ( windows ) minikube ip. Note that this is the virtual IP within your minikube container.
Start a bash within the minikube container. docker exec -it {your-container-id} bash
Access to the folder you mounted on point 1). Now, simply curl to the Kubectl api server through its virtual ip from 2.):
curl https://{your-ip-from-2}:8443/api --key ./ca.key --cert ./ca.crt
Here we are passing the certs to be used. Notice how I am not using the proxy-client ones.
That's it. For learning purposes I think this is a more interesting method that directly proxying.
These instructions worked for me https://github.com/jenkinsci/kubernetes-plugin#configuration-on-minikube
Needed to generate & upload pfx file, along with the other steps mentioned there.
Most of the above answers are right in their own sense.
I will give my version of the answer:
1) What is the correct way to Kubernetes cluster setup using minikube through the Kubernetes API ?
Ans: I think this is pretty straight forward. Follow the installation steps mentions in the official k8s documentation for minikube installation
2) At the moment, I can't find a port through which the kubernetes cluster can be accessed.
Ans: This is too has a straight forward answer. You have to check your Kube config file. You can find it in your home directory ~/.kube/config. View this file and it will have the details.
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/username/.minikube/ca.crt
server: https://192.168.64.2:8443
name: minikube
contexts:
- context:
cluster: minikube
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/username/.minikube/client.crt
client-key: /Users/username/.minikube/client.key
The server detail mentioned here is your api-server endpoint to hit.
You can view this information using the kubectl command as well like this kubectl config view
Use below curl to hit the api-server using curl
curl https://192.168.64.2:8443/api/v1/pod --key /Users/sanjay/.minikube/client.key --cert /Users/sanjay/.minikube/client.crt --cacert /Users/sanjay/.minikube/ca.crt
Note: replace the ip port and the path as per your config file in above command.
Based on xichen's and Seba's answers above, this is how to acquire a token from a terminal:
$ function get_token() { secret=$(kubectl get serviceaccount "$1" -o jsonpath='{.secrets[0].name}') && kubectl get secret "$secret" -o jsonpath='{.data.token}' | base64 --decode; }
$ get_token target_account
I hope this would be useful for those who must use kubectl below 1.24 due to minikube issue with enabling ingress as stated in this question.
Related
I've got no access to kube-apiserver pod directly, but I do have an access to kube-proxy pod. Can I run curl https://localhost:6443/healthz as a healthness probe to kube-apiserver or something?
All the Pods can have access to the API Server: this is granted by the Service Account secret mounted in /var/run/secrets/kubernetes.io/serviceaccount/token.
Before proceeding, you have to ensure your Pod is allowed to reach the API Server, thus not blocked by a NetworkPolicy: this requirement hasn't been declared on your question, so giving this is not the case.
The said token is used to performs actions against the API Server, such as CRUD ops for those resources protected by RBAC.
If you just need to check the healthiness of the API Server, you can cURL the API Server using the mounted CA public certificate in /var/run/secrets/kubernetes.io/serviceaccount/ca.pem and the environment variable KUBERNETES_SERVICE_HOST already injected at runtime in your Pod pointing to the API Server, as well with the KUBERNETES_SERVICE_PORT although the 443 should be the default value.
Example
# kubectl run -it --image curlimages/curl curl --command -- sh
If you don't see a command prompt, try pressing enter.
/ $ env | grep -i kubernetes
KUBERNETES_SERVICE_PORT=443
KUBERNETES_PORT=tcp://10.96.0.1:443
KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1
KUBERNETES_PORT_443_TCP_PORT=443
KUBERNETES_PORT_443_TCP_PROTO=tcp
KUBERNETES_SERVICE_PORT_HTTPS=443
KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443
KUBERNETES_SERVICE_HOST=10.96.0.1
/ $ curl --cacert /var/run/secrets/kubernetes.io/serviceaccount/ca.crt https://$KUBERNETES_SERVICE_HOST:$KUBERNETES_SERVICE_PORT/api
-H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "172.20.0.2:6443"
}
]
}
Added from a form which is quay.io/keycloak/keycloak
Changed from Loadbalancer to a Nodeport
Can visit that but ip:port
Showing error to add a user from localhost:8080 or use add-user-keycloak script
Please follow the docs Keycloak on Kubernetes.
You can find instructions there, how deploy keycloak inside minikube.
However you can download this deployment files and modify the settings according to your needs.
F.E. you can change service type from Loadbalancer to NodePort.
In addition please consider making a changes in other settings like: KEYCLOAK_USER, KEYCLOAK_PASSWORD:
wget -q -O - https://raw.githubusercontent.com/keycloak/keycloak-quickstarts/latest/kubernetes-examples/keycloak.yaml | \
sed "s/LoadBalancer/NodePort/" | \
kubectl create -f -
In order to access your keycloak instance you should change:
minikube ip ( to your externalIp address associated with your vm or use nodeIP from inside the vm)
verify your service NodePort by running.
kubectl get services/keycloak -o go-template='{{(index .spec.ports 0).nodePort}}'
kubectl get svc -o wide
By default NodePort should be in the range (30000-32767)
Using minikube, when running the following command:
kubectl -v=11 --kubeconfig /dev/null --insecure-skip-tls-verify -s http://localhost:8001 --token "invalid" -n namespace get pods
I have an answer when I don't want one. And I don't know how it was authorized. Moreover, if I use a valid token with specific rights, these are not used.
kubectl --token=$TOKEN doesn't run with the permissions of the token doesn't answer my question as I specified to used /dev/null as a config file.
Any idea ?
I will try to summarize the answer I provided in the comments.
The question was: Why does running kubectl -s http://localhost:8001 --kubeconfig /dev/null --token <invalid_token> (where :8001 is a port opened by kubectl proxy) repoonds as if I was authorized, when it shouldn't beacause I set all possible authorization options to null or incorrect values?
The answer is that kubectl proxy opens a port and handles all authorization for you so you dont have to. Now to access REST api of kubernetes all you need to do is to use curl localhost:8001/.... No tokens and certificates.
Because you are already authorized with kubectl proxy, using kubectl and pointing it to localhost:8001 is causing that it won't need to authorize and you won't need any tokens to access k8s.
As an alternative you can check what happens when you run the same but instead of connecting through kubectl proxy you use kubernetes port directly.
You mentioned that you are using minikube so by default that would be port 8443
$ kubectl --kubeconfig /dev/null -s https://$(minikube ip):8443 --token "invalid" --insecure-skip-tls-verify get pods
error: You must be logged in to the server (Unauthorized)
As you see now it works as expected.
I have a k8s cluster mounted in a Amazon EC2 instance, and i want configure the CI with GitLab. To do that, GitLab requested me the Kubernetes API URL.
I ran kubectl cluster-info to get the requested information and i can see 3 rows:
Kubernetes master https://10.10.1.253:6443
coredns https://10.10.1.253:6443
kubernetes-dashboard https://10.10.1.253:6443
I suppose that need the Kubernetes master URL but, is a private IP. How i can expose the API correctly ?
Any ideas ?
For better security keep the IPs of the kubernetes master nodes private and use LoadBalancer provided by AWS to expose the Kubernetes API Server. You could also configure TLS termination at the LoadBalancer.
use the kubectl config view to get the server address, it will looks like server: https://172.26.2.101:6443.
First you need to define your public ip of the master node or the load balancer if any as a DNS Alternative. You can do this by,
remove current apiserver certificates
sudo rm /etc/kubernetes/pki/apiserver.*
generate new certificates
sudo kubeadm init phase certs apiserver --apiserver-cert-extra-sans=<public_ip>
Then, you have to capture your admin key, cert and the ca cert from the .kube/config file
client-key-data:
echo -n "LS0...Cg==" | base64 -d > admin.key
client-certificate-data:
echo -n "LS0...Cg==" | base64 -d > admin.crt
certificate-authority-data:
echo -n "LS0...Cg==" | base64 -d > ca.crt
Now you can request your api through curl, example below to request pods info
curl https://<public_ip>:6443/api/v1/pods \
--key admin.key \
--cert admin.crt \
--cacert ca.crt
And of course make sure you allowed required ports
I have been able to successfully setup kubernetes on my Centos 7 server.
On trying to get the dashboard working after following the documentation, running 'kubectl proxy' it
attempts to run using 127.0.0.1:9001 and not my server ip. Do this mean I cannot access kubernetes dashboard outside the server?
I need help on getting the dashboard running using my public ip
You can specify on which address you want to run kubectl proxy, i.e.
kubectl proxy --address <EXTERNAL-IP> -p 9001
Starting to serve on 100.105.***.***:9001
You can also use port forwarding to access the dashboard.
kubectl port-forward --address 0.0.0.0 pod/dashboard 8888:80
This will listen port 8888 on all addresses and route traffic directly to your pod.
For instance:
rsha:~$ kubectl port-forward --address 0.0.0.0 deploy/webserver 8888:80
Forwarding from 0.0.0.0:8888 -> 80
In another terminal running
rsha:~$ curl 100.105.***.***:8888
<html><body><h1>It works!</h1></body></html>
As I understand, you would like to access the dashboard from your laptop. What you should do is create an admin account called k8s-admin:
$ kubectl --namespace kube-system create serviceaccount k8s-admin
$ kubectl create clusterrolebinding k8s-admin --serviceaccount=kube-system:k8s-admin --clusterrole=cluster-admin
Then setup kubectl on your laptop, e.g. for macOS it looks like this (see documentation):
$ brew install kubernetes-cli
Setup a proxy to your workstation. Create a ~/.kube directory on your laptop and then scp the ~/.kube/config file from the k8s (Kubernetes) master to your ~/.kube directory.
Then get the authentication token you need to connect to the dashboard:
$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep k8s-admin | awk '{print $1}')
Now start the proxy:
$ kubectl proxy
Now open the dashboard by going to:
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
You should see the Token option and then copy-paste the token from the prior step and Sign-In.
You can follow this tutorial.