I have a k8s cluster mounted in a Amazon EC2 instance, and i want configure the CI with GitLab. To do that, GitLab requested me the Kubernetes API URL.
I ran kubectl cluster-info to get the requested information and i can see 3 rows:
Kubernetes master https://10.10.1.253:6443
coredns https://10.10.1.253:6443
kubernetes-dashboard https://10.10.1.253:6443
I suppose that need the Kubernetes master URL but, is a private IP. How i can expose the API correctly ?
Any ideas ?
For better security keep the IPs of the kubernetes master nodes private and use LoadBalancer provided by AWS to expose the Kubernetes API Server. You could also configure TLS termination at the LoadBalancer.
use the kubectl config view to get the server address, it will looks like server: https://172.26.2.101:6443.
First you need to define your public ip of the master node or the load balancer if any as a DNS Alternative. You can do this by,
remove current apiserver certificates
sudo rm /etc/kubernetes/pki/apiserver.*
generate new certificates
sudo kubeadm init phase certs apiserver --apiserver-cert-extra-sans=<public_ip>
Then, you have to capture your admin key, cert and the ca cert from the .kube/config file
client-key-data:
echo -n "LS0...Cg==" | base64 -d > admin.key
client-certificate-data:
echo -n "LS0...Cg==" | base64 -d > admin.crt
certificate-authority-data:
echo -n "LS0...Cg==" | base64 -d > ca.crt
Now you can request your api through curl, example below to request pods info
curl https://<public_ip>:6443/api/v1/pods \
--key admin.key \
--cert admin.crt \
--cacert ca.crt
And of course make sure you allowed required ports
Related
Why does the following error occur when I install Linkerd 2.x on a private cluster in GKE?
Error: could not get apiVersions from Kubernetes: unable to retrieve the complete list of server APIs: tap.linkerd.io/v1alpha1: the server is currently unable to handle the request
The default firewall rules of a private cluster on GKE only permit traffic on ports 443 and 10250. This allows communication to the kube-apiserver and kubelet, respectively.
Linkerd uses ports 8443 and 8089 for communication between the control and the proxies deployed to the data plane.
The tap component uses port 8089 to handle requests to its apiserver.
The proxy injector and service profile validator components, both of which are types of admission controllers, use port 8443 to handle requests.
The Linkerd 2 docs include instructions for configuring your firewall on a GKE private cluster: https://linkerd.io/2/reference/cluster-configuration/
They are included below:
Get the cluster name:
CLUSTER_NAME=your-cluster-name
gcloud config set compute/zone your-zone-or-region
Get the cluster MASTER_IPV4_CIDR:
MASTER_IPV4_CIDR=$(gcloud container clusters describe $CLUSTER_NAME \
| grep "masterIpv4CidrBlock: " \
| awk '{print $2}')
Get the cluster NETWORK:
NETWORK=$(gcloud container clusters describe $CLUSTER_NAME \
| grep "^network: " \
| awk '{print $2}')
Get the cluster auto-generated NETWORK_TARGET_TAG:
NETWORK_TARGET_TAG=$(gcloud compute firewall-rules list \
--filter network=$NETWORK --format json \
| jq ".[] | select(.name | contains(\"$CLUSTER_NAME\"))" \
| jq -r '.targetTags[0]' | head -1)
Verify the values:
echo $MASTER_IPV4_CIDR $NETWORK $NETWORK_TARGET_TAG
# example output
10.0.0.0/28 foo-network gke-foo-cluster-c1ecba83-node
Create the firewall rules for proxy-injector and tap:
gcloud compute firewall-rules create gke-to-linkerd-control-plane \
--network "$NETWORK" \
--allow "tcp:8443,tcp:8089" \
--source-ranges "$MASTER_IPV4_CIDR" \
--target-tags "$NETWORK_TARGET_TAG" \
--priority 1000 \
--description "Allow traffic on ports 8843, 8089 for linkerd control-plane components"
Finally, verify that the firewall is created:
gcloud compute firewall-rules describe gke-to-linkerd-control-plane
In my case, it was related to linkerd/linkerd2#3497, when the Linkerd service had some internal problems and couldn't respond back to the API service requests. Fixed by restarting its pods.
Solution:
The steps I followed are:
kubectl get apiservices : If linkered apiservice is down with the error CrashLoopBackOff try to follow the step 2 otherwise just try to restart the linkered service using kubectl delete apiservice/"service_name". For me it was v1alpha1.tap.linkerd.io.
kubectl get pods -n kube-system and found out that pods like metrics-server, linkered, kubernetes-dashboard are down because of the main coreDNS pod was down.
For me it was:
NAME READY STATUS RESTARTS AGE
pod/coredns-85577b65b-zj2x2 0/1 CrashLoopBackOff 7 13m
Use kubectl describe pod/"pod_name" to check the error in coreDNS pod and if it is down because of /etc/coredns/Corefile:10 - Error during parsing: Unknown directive proxy, then we need to use forward instead of proxy in the yaml file where coreDNS config is there. Because CoreDNS version 1.5x used by the image does not support the proxy keyword anymore.
This was a linkerd issue for me. To diagnose any linkerd related issues, you can use the linkerd CLI and run linkerd check this should show you if there is an issue with linkerd and links on instructions to fix it.
For me, the issue was that linkerd root certs had expired. In my case, linkerd was experimental in a dev cluster so I removed it. However, if you need to update your certificates you can follow the instructions at the following link.
https://linkerd.io/2.11/tasks/replacing_expired_certificates/
Thanks to https://stackoverflow.com/a/59644120/1212371 I was put on the right path.
I have been able to successfully setup kubernetes on my Centos 7 server.
On trying to get the dashboard working after following the documentation, running 'kubectl proxy' it
attempts to run using 127.0.0.1:9001 and not my server ip. Do this mean I cannot access kubernetes dashboard outside the server?
I need help on getting the dashboard running using my public ip
You can specify on which address you want to run kubectl proxy, i.e.
kubectl proxy --address <EXTERNAL-IP> -p 9001
Starting to serve on 100.105.***.***:9001
You can also use port forwarding to access the dashboard.
kubectl port-forward --address 0.0.0.0 pod/dashboard 8888:80
This will listen port 8888 on all addresses and route traffic directly to your pod.
For instance:
rsha:~$ kubectl port-forward --address 0.0.0.0 deploy/webserver 8888:80
Forwarding from 0.0.0.0:8888 -> 80
In another terminal running
rsha:~$ curl 100.105.***.***:8888
<html><body><h1>It works!</h1></body></html>
As I understand, you would like to access the dashboard from your laptop. What you should do is create an admin account called k8s-admin:
$ kubectl --namespace kube-system create serviceaccount k8s-admin
$ kubectl create clusterrolebinding k8s-admin --serviceaccount=kube-system:k8s-admin --clusterrole=cluster-admin
Then setup kubectl on your laptop, e.g. for macOS it looks like this (see documentation):
$ brew install kubernetes-cli
Setup a proxy to your workstation. Create a ~/.kube directory on your laptop and then scp the ~/.kube/config file from the k8s (Kubernetes) master to your ~/.kube directory.
Then get the authentication token you need to connect to the dashboard:
$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep k8s-admin | awk '{print $1}')
Now start the proxy:
$ kubectl proxy
Now open the dashboard by going to:
http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
You should see the Token option and then copy-paste the token from the prior step and Sign-In.
You can follow this tutorial.
I need to delete POD on my GCP kubernetes cluster. Actually in Kubernetes Engine API documentation I can find only REST api's for: projects.locations.clusters.nodePools, but nothing for PODs.
The GKE API is used to manage the cluster itself on an infrastructure level. To manage Kubernetes resources, you'd have to use the Kubernetes API. There are clients for various languages, but of course you can also directly call the API.
Deleting a Pod from within another or the same Pod:
PODNAME=ubuntu-xxxxxxxxxx-xxxx
curl https://kubernetes/api/v1/namespaces/default/pods/$PODNAME \
-X DELETE -k \
-H "Authorization: Bearer $(cat /var/run/secrets/kubernetes.io/serviceaccount/token)"
From outside, you'd have to use the public Kubernetes API server URL and a valid token. Here's how you get those using kubectl:
APISERVER=$(kubectl config view --minify -o jsonpath='{.clusters[0].cluster.server}')
TOKEN=$(kubectl get secret $(kubectl get serviceaccount default -o jsonpath='{.secrets[0].name}') -o jsonpath='{.data.token}' | base64 --decode )
Here's more official information on accessing the Kubernetes API server.
I did create a Master Cluster with the following command:
kubeadm init --pod-network-cidr $CALICO_NETWORK
Now it is listening in the internal IP 10.3.8.23:6443, which is ok because I want that the master uses the internal IP to communicate with Nodes.
Now I want to access the cluster using the public IP and I get the following error:
http: proxy error: x509: certificate is valid for 10.96.0.1, 10.3.8.23, not for 18.230.*.*.
How can I generate an additional certificate for the publicIP?
I need to use the public IP in order to access the dashboard using the browser.
I install it using: https://github.com/kubernetes/dashboard
If you don't want to recreate your cluster you can also do what's described here: Invalid x509 certificate for kubernetes master
For K8s 1.7 and earlier:
rm /etc/kubernetes/pki/apiserver.*
kubeadm alpha phase certs selfsign \
--apiserver-advertise-address=0.0.0.0 \
--cert-altnames=10.96.0.1 \
--cert-altnames=10.3.8.23 \
--cert-altnames=18.230.x.x # <== Public IP
docker rm `docker ps -q -f 'name=k8s_kube-apiserver*'`
systemctl restart kubelet
For K8s 1.8 an newer:
rm /etc/kubernetes/pki/apiserver.*
kubeadm alpha phase certs all \
--apiserver-advertise-address=0.0.0.0 \
--apiserver-cert-extra-sans=10.96.0.1,10.3.8.23,18.230.x.x # <== Public IP
docker rm -f `docker ps -q -f 'name=k8s_kube-apiserver*'`
systemctl restart kubelet
And you can also add DNS name with the --apiserver-cert-extra-sans option.
If you want to get access to your cluster using public IP, you can pass the IP with kubeadm init command. Like:
kubeadm init --apiserver-cert-extra-sans=private-ip,public-ip \
--pod-network-cidr $CALICO_NETWORK \
--apiserver-advertise-address=private-ip
What is correct way to kubernetes cluster setup using minikube through the kubernetes api ?
At the moment, I can't find a port through which the kubernetes cluster can be accessed.
The easiest way to access the Kubernetes API with when running minikube is to use
kubectl proxy --port=8080
You can then access the API with
curl http://localhost:8080/api/
This also allows you to browse the API in your browser. Start minikube using
minikube start --extra-config=apiserver.Features.EnableSwaggerUI=true
then start kubectl proxy, and navigate to http://localhost:8080/swagger-ui/ in your browser.
You can access the Kubernetes API with curl directly using
curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/client.crt --key ~/.minikube/client.key https://`minikube ip`:8443/api/
but usually there is no advantage in doing so. Common browsers are not happy with the certificates minikube generates, so if you want to access the API with your browser you need to use kubectl proxy.
Running minikube start will automatically configure kubectl.
You can run minikube ip to get the IP that your minikube is on. The API server runs on 8443 by default.
Update: To access the API server directly, you'll need to use the custom SSL certs that have been generated. by minikube. The client certificate and key are typically stored at: ~/.minikube/apiserver.crt and ~/.minikube/apiserver.key. You'll have to load them into your HTTPS client when you make requests.
If you're using curl use the --cert and the --key options to use the cert and key file. Check the docs for more details.
Update2: The client certificate and key are typically stored at: ~/.minikube/profiles/minikube directory when you use the version >= 0.19 (more informations). You probably need to set the --insecure options to the curl client because of the self-signed certificate.
I went through lots of answers, but lots of them are wrong.
Before we do, we need IP and token.
How to get IP: minikube ip
How to generate Token:
$export secret=kubectl get serviceaccount default -o json | jq -r '.secrets[].name'
$kubectl get secret $secret -o yaml | grep "token:" | awk {'print $2'} | base64 -D > token
Note: base64 uses -D for mac, but -d for Linux.
Then, the correct command is:
#curl -v -k -H --cacert ~/.minikube/ca.crt -H "Authorization: Bearer $(cat ~/YOUR_TOKEN)" "https://{YOUR_IP}:8443/api/v1/pods"
User Sven Marnach got me in the right direction however to get the correct server ip, crt and key location I ran kubectl config view.
$ kubectl config view
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/user/.minikube/ca.crt
server: https://127.0.0.1:32792
name: minikube
contexts:
- context:
cluster: minikube
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/user/.minikube/profiles/minikube/client.crt
client-key: /Users/user/.minikube/profiles/minikube/client.key
$ curl --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/
{
"kind": "APIVersions",
"versions": [
"v1"
],
"serverAddressByClientCIDRs": [
{
"clientCIDR": "0.0.0.0/0",
"serverAddress": "172.17.0.2:8443"
}
]
}
$ curl -s --cacert ~/.minikube/ca.crt --cert ~/.minikube/profiles/minikube/client.crt --key ~/.minikube/profiles/minikube/client.key https://127.0.0.1:32792/api/v1/pods | jq .items[].metadata | jq '"\(.name), \(.namespace), \(.selfLink)"'
"shell-demo, default, /api/v1/namespaces/default/pods/shell-demo"
"coredns-f9fd979d6-6b2nx, kube-system, /api/v1/namespaces/kube-system/pods/coredns-f9fd979d6-6b2nx"
"etcd-minikube, kube-system, /api/v1/namespaces/kube-system/pods/etcd-minikube"
"kube-apiserver-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-apiserver-minikube"
"kube-controller-manager-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-controller-manager-minikube"
"kube-proxy-bbck9, kube-system, /api/v1/namespaces/kube-system/pods/kube-proxy-bbck9"
"kube-scheduler-minikube, kube-system, /api/v1/namespaces/kube-system/pods/kube-scheduler-minikube"
"storage-provisioner, kube-system, /api/v1/namespaces/kube-system/pods/storage-provisioner"
Readers may also be interested in link.
For windows users, here is an alternative to the much simpler kubectl proxy command:
mount your local host's .minikube folder using "minikube mount [path-to-folder]:/host . This way, you will be able to access the certificates from within the node.If you don't know the exact path to this folder, you can get it by looking at the kubectl config view response.
On a different command prompt, take note of the IP of your kube api server. this can be done running from your host ( windows ) minikube ip. Note that this is the virtual IP within your minikube container.
Start a bash within the minikube container. docker exec -it {your-container-id} bash
Access to the folder you mounted on point 1). Now, simply curl to the Kubectl api server through its virtual ip from 2.):
curl https://{your-ip-from-2}:8443/api --key ./ca.key --cert ./ca.crt
Here we are passing the certs to be used. Notice how I am not using the proxy-client ones.
That's it. For learning purposes I think this is a more interesting method that directly proxying.
These instructions worked for me https://github.com/jenkinsci/kubernetes-plugin#configuration-on-minikube
Needed to generate & upload pfx file, along with the other steps mentioned there.
Most of the above answers are right in their own sense.
I will give my version of the answer:
1) What is the correct way to Kubernetes cluster setup using minikube through the Kubernetes API ?
Ans: I think this is pretty straight forward. Follow the installation steps mentions in the official k8s documentation for minikube installation
2) At the moment, I can't find a port through which the kubernetes cluster can be accessed.
Ans: This is too has a straight forward answer. You have to check your Kube config file. You can find it in your home directory ~/.kube/config. View this file and it will have the details.
apiVersion: v1
clusters:
- cluster:
certificate-authority: /Users/username/.minikube/ca.crt
server: https://192.168.64.2:8443
name: minikube
contexts:
- context:
cluster: minikube
namespace: default
user: minikube
name: minikube
current-context: minikube
kind: Config
preferences: {}
users:
- name: minikube
user:
client-certificate: /Users/username/.minikube/client.crt
client-key: /Users/username/.minikube/client.key
The server detail mentioned here is your api-server endpoint to hit.
You can view this information using the kubectl command as well like this kubectl config view
Use below curl to hit the api-server using curl
curl https://192.168.64.2:8443/api/v1/pod --key /Users/sanjay/.minikube/client.key --cert /Users/sanjay/.minikube/client.crt --cacert /Users/sanjay/.minikube/ca.crt
Note: replace the ip port and the path as per your config file in above command.
Based on xichen's and Seba's answers above, this is how to acquire a token from a terminal:
$ function get_token() { secret=$(kubectl get serviceaccount "$1" -o jsonpath='{.secrets[0].name}') && kubectl get secret "$secret" -o jsonpath='{.data.token}' | base64 --decode; }
$ get_token target_account
I hope this would be useful for those who must use kubectl below 1.24 due to minikube issue with enabling ingress as stated in this question.