I need a way to get service cluster ip range (as CIDR) that works accross all Kubernetes clusters.
I tried the following, which works fine for clusters created with kubeadm as it greps arguments of apiserver pod:
$ kubectl cluster-info dump | grep service-cluster-ip-range
"--service-cluster-ip-range=10.96.0.0/12",
This does not work on all Kubernetes clusters, i.e. gcloud
So the question is, what is the best way to get service ip range programatically?
I don't think there is a way to access such information through K8s Api, there is an open issue to address lack of this functionality: https://github.com/kubernetes/kubernetes/issues/25533 . If you have access to the etcd of the k8s cluster in question, then there is a key with information about service cidr range: /registry/ranges/serviceips . You can get the value, by using etcdctl (assuming, you have the proper permissions): etcdctl --enpoints=<your etcd> --<any authentication flags> get "/registry/ranges/serviceips" --prefix=true.
Related
I have a kubernetes cluster installed using 5 Virtual machines. 1 for the kubernetes master, and the other for the workers.
I tried using RBAC method, I created a kubeconfig file to give access to only one namespace, when I test it using (kubectl --kubeconfig developper.kubeconfig get pods), it works but if this user didn't use the flag --kubeconfig, he can simply use: kubectl get all --all-namespaces( for axample ) and he will be able the access to all the cluster resources, So this is not the efficient method that I need.
Do you have any other solutions please?
I need to use a domain for GKE cluster to access ingress into the cluster and applications, similar like azure AKS http add-on which gives a generic-created domain(not a custom domain)
https://learn.microsoft.com/en-us/azure/aks/http-application-routing
Is there any solution on Google cloud as well?
Our GKE creating/deleting process is a part of IaC tooling and we are automating cluster and our app deployment for dev/test/staging. And the generic domain creation and binding managed dns zone to the cluster resources gives us great flexibility. Otherwise we have to create custom domain and managed dns zone which will be static and bring unnecessary complexity to the provisioning tooling.
GCP has not implemented resources like that, however this operation could be automated using one of the available Cloud DNS APIs 1, as for example the ResourceRecordSets 2 to configure A records to the ManagedZone you want to assign the host, scripting this configuration after the Ingress controller creation.
Example, retrieving the IP address allocated to the ingress controller issuing the command like kubectl describe ing <ingress-name> |grep “Address:” |awk ‘{print $2}’ than using the IP information to construct the API body request 3.
There is not generic domain options in gke so I have to purchase a domain and update NS according to created managed dns zone NS and they will be automated sync when I update ingress in gke by external-dns
I can say I solve this problem with this steps,
1- Create a managed zone which has domain name belongs own and be sure it has permission to access domain from dns zones which you create. Mean is giving access the google project which your dns zone exist
Note: when you create the cluster be sure giving scopes for readwrite perm for managed dns zone
gcloud container clusters create “external-dns” \
—num-nodes 1 \
—scopes “https://www.googleapis.com/auth/ndev.clouddns.readwrite
Create a DNS zone which will contain the managed DNS records.
$ gcloud dns managed-zones create “xxx.test-dev” \
—dns-name “xxx.test.dev.” \
—description “Automatically managed zone by kubernetes.io/external-dns test.dev domain name”
2- Please deploy the resources to gke which name is external-dns
https://github.com/kubernetes-sigs/external-dns/blob/master/docs/tutorials/gke.md#deploy-externaldns
And check the logs with
kubectl logs $(kubectl get pods --template '{{range .items}}{{.metadata.name}}{{"\n"}}{{end}}' | grep dns)
Or
kubectl logs $(kubectl get pods --no-headers -o custom-columns=":metadata.name" | grep dns)
And if you see something like everything is going smoothly
time="2021-01-20T11:37:46Z" level=info msg="Add records: xxx.test.dev. A [34.89.xx.xx] 300"
time="2021-01-20T11:37:46Z" level=info msg="Add records: xxx.test.dev. TXT [\"heritage=external-dns,external-dns/owner=my-identifier,external-dns/resource=ingress/default/ingress-test\"] 300"
time="2021-01-20T11:38:47Z" level=info msg="All records are already up to date"
Note created TXT record alongside A record. TXT record signifies that the corresponding A record is managed by ExternalDNS. This makes ExternalDNS safe for running in environments where there are other records managed via other means.
Let’s check that we can resolve this DNS name. We’ll ask the nameservers assigned to your zone first.
$ dig +short #ns-cloud-e1.googledomains.com. xxx.test.dev.
104.155.xx.xx
And you can check the ip of the domain is correct or has a problem
host https://xxx.test.dev/
Host https://xxx.test.dev/ not found: 3(NXDOMAIN)
It can be complained bed domain for a while but then you will get the correct response
host xxx.test.dev
xxx.test.dev has address 35.197.xx.xx
When I created a given k8 cluster I didn't specify anything specific for service-cluster-ip-range. Now when I create new loadBalancer services k8 is assigning values that walk on existing ip addresses within the network.
Checking the allowed range via kubectl cluster-info dump | grep service-cluster-ip-range gives me:
"--service-cluster-ip-range=10.96.0.0/12"
which (oddly enough) isn't where the assigned values are coming from. New values seem to have started at 10.95.96.235 and incremented from there.
Attempts to preset a valid ip in a service descriptor via spec.loadBalancerIP gives me errors from kubelet:
Failed to allocate IP for "newservice": "10.95.96.233" is not allowed in config
My questions are:
is it possible to change service-cluster-ip-range without rebuilding the entire cluster?
if not, do I have any other options for (pre)setting loadBalancerIP ?
In Kubernetes(kubectl) 1.10 or Istio(istioctl) 1.0
I know I can list services with kubectl get svc, and I can figure out the namespace and from there add .svc.cluster.local.
Is it possible to list internal dns?
Thanks
Unfortunately you would have to list the DNS of each pod manually as there is no command available for both Istio or Kubernetes. You can however look at this following link which will guide you into creating a shell in the running container so that you can list the internal DNS manually
Newbie setup :
Created First project in GCP
Created cluster with default, 3 nodes. Node version 1.7.6. cluster master version 1.7.6-gke.1.
Deployed aan application in a pod, per example.
Able to access "hello world" and the hostname, using the external-ip and the port.
In GCP / GKE webpage of my cloud console, clicked "discovery and loadbalancing", I was able to see the "kubernetes-dashboard" process in green-tick, but cannot access throught the IP listed. tried 8001,9090, /ui and nothing worked.
not using any cloud shell or gcloud commands on my local laptop. Everything is done on console.
Questions :
How can anyone access the kubernetes-dashboard of the cluster created in console?
docs are unclear, are the dashboard components incorporated in the console itself? Are the docs out of sync with GCP-GKE screens?
tutorial says run "kubectl proxy" and then to open
"http://localhost:8001/ui", but it doesnt work, why?
If you create a cluster with with version 1.9.x or greater, then u can access using tokens.
get secret.
kubectl -n kube-system describe secrets `kubectl -n kube-system get secrets | awk '/clusterrole-aggregation-controller/ {print $1}'` | awk '/token:/ {print $2}'
Copy secret.
kubectl proxy.
Open UI using 127.0.0.1:8001/ui. This will redirect to login page.
there will be two options to login, kubeconfig and token.
Select token and paste the secret copied earlier.
hope this helps
It seems to be an issue with the internal Kubernetes DNS service starting at version 1.7.6 on Google Cloud.
The solution is to access the dashboard at this endpoint instead:
http://localhost:8001/api/v1/proxy/namespaces/kube-system/services/kubernetes-dashboard
Github Issue links:
https://github.com/kubernetes/dashboard/issues/2368
https://github.com/kubernetes/kubernetes/issues/52729
The address of the dashboard service is only accessible from inside of the cluster. If you ssh into a node in your cluster, you should be able to connect to the dashboard. You can verify this by noticing that the address is within the services CIDR range for your cluster.
The dashboard in running as a pod inside of your cluster with an associated service. If you open the Workloads view you will see the kubernetes-dashboard deployment and can see the pod that was created by the deployment. I'm not sure which docs you are referring to, since you didn't provide a link.
When you run kubectl proxy it creates a secure connection from your local machine into your cluster. It works by connecting to your master and then running through a proxy on the master to the pod/service/host that you are connecting to via an ssh tunnel. It's possible that it isn't working because the ssh tunnels are not running; you should verify that your project has newly created ssh rules allowing access from the cluster endpoint IP address. Otherwise, if you could explain more about how it fails, that would be useful for debugging.
First :
gcloud container clusters get-credentials cluster-1 --zone my-zone --project my-project
Then find your kubernetes dashboard endpoint doing :
kubectl cluster-info
It will be something like https://42.42.42.42/api/v1/namespaces/kube-system/services/kubernetes-dashboard/proxy
Install kube-dashboard
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml
Run:
$ kubectl proxy
Access:
http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login