I am trying to deploy a Traefik Ingress controller in my minikube environment by following this:
helm install stable/traefik --name-template traefik --set dashboard.enabled=true,dashboard.domain=dashboard.traefik,rbac.enabled=true --namespace kube-system
Even after half an hour I still see that External IP is pending:
pascals#pascals:~$ kubectl get svc -l app=traefik -n kube-system
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik LoadBalancer 10.96.172.128 <pending> 443:30812/TCP,80:31078/TCP 20m
traefik-dashboard ClusterIP 10.96.56.105 <none> 80/TCP 20m
Ideally I would like to reach http://dashboard.traefik but I am not able to do so.
I tried to assign an External Ip using the kubectl patch Api:
kubectl patch svc traefik -n kube-system -p '{"spec":{"externalIPs":["192.168.99.107"]}}'
where, 192.168.99.107 is the minikube ip. This however still did not solve my problem.
Appreciate any nudge in the right direction!
The external IP is assigned by the ServiceController if any cloud provider used in the cluster, usually in managed clusters.
In a minikube cluster, LoadBalance-typed Service will never have an external IP. You can access Services through minikubeip:nodeport, or running minikube service. For the Service traefik-dashboard, it should be a NodePort-typed Service first.
You should install some Kubernetes bare-metal load balancer, like MetalLB
Related
In minikube, how to expose a service using nodeport ?
For example, I start a kubernetes cluster using the following command and create and expose a port like this:
$ minikube start
$ kubectl run hello-minikube --image=gcr.io/google_containers/echoserver:1.4 --port=8080
$ kubectl expose deployment hello-minikube --type=NodePort
$ curl $(minikube service hello-minikube --url)
CLIENT VALUES:
client_address=192.168.99.1
command=GET
real path=/ ....
Now how to access the exposed service from the host? I guess the minikube node needs to be configured to expose this port as well.
I am not exactly sure what you are asking as it seems you already know about the minikube service <SERVICE_NAME> --url command which will give you a url where you can access the service. In order to open the exposed service, the minikube service <SERVICE_NAME> command can be used:
$ kubectl run hello-minikube --image=gcr.io/google_containers/echoserver:1.4 --port=8080
deployment "hello-minikube" created
$ kubectl expose deployment hello-minikube --type=NodePort
service "hello-minikube" exposed
$ kubectl get svc
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-minikube 10.0.0.102 <nodes> 8080/TCP 7s
kubernetes 10.0.0.1 <none> 443/TCP 13m
$ minikube service hello-minikube
Opening kubernetes service default/hello-minikube in default browser...
This command will open the specified service in your default browser.
There is also a --url option for printing the url of the service which is what gets opened in the browser:
$ minikube service hello-minikube --url
http://192.168.99.100:31167
As minikube is exposing access via nodeIP:nodePort and not on localhost:nodePort, you can get this working by using kubectl's port forwarding capability. For example, if you are running mongodb service:
kubectl port-forward svc/mongo 27017:27017
This would expose the service on localhost:27017, FWIW. Furthermore, you might want to figure out how to run this in background.
minikube runs on something like 192.168.99.100. So you should be able to access it on the NodePort you exposed your service at. For eg, say your NodePort is 30080, then your service will be accessible as 192.168.99.100:30080.
To get the minikube ip, run the command minikube ip.
Update Sep 14 2017:
Here's a small example that works with minikube v0.16.0.
1) Run the commands below to create an nginx running on 8080 and a NodePort svc forwarding to it:
$ kubectl run hello-minikube --image=gcr.io/google_containers/echoserver:1.4 --port=8080
deployment "hello-minikube" created
$ kubectl expose deployment hello-minikube --type=NodePort
service "hello-minikube" exposed
2) Find the nodeport used by the svc:
$ kubectl get svc hello-minikube
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-minikube 10.0.0.76 <nodes> 8080:30341/TCP 4m
3) Find the minikube ip:
$ minikube ip
192.168.99.100
4) Talk to it with curl:
$ curl 192.168.99.100:30341
CLIENT VALUES:
client_address=172.17.0.1
command=GET
real path=/
...
I ran into a similar issue in 2022. Here are the commands I ran:
kubectl create deployment deploymentName --image=dockerHubUsername/imageTag:imageVersion
kubectl expose deployment deploymentName --type=LoadBalancer --port=8080
minikube tunnel
kubectl get services deploymentName this provides the external ip address needed to access the application. I access the app with 127.0.0.1:8080
Source
Just a note for anyone looking for connection refused answers: If your minikube does not run on "something like 192.168.99.100" you probably runned with another vm-driver like "none". In that case delete your minikube cluster and rebuild using the default. it 'll work....ish... I do not seem to be able to get the tunnel working...
I tried deploying on EKS, and my config.yaml follows this suggested format:
botfront:
app:
# The complete external host of the Botfront application (eg. botfront.yoursite.com). It must be set even if running on a private or local DNS (it populates the ROOT_URL).
host: botfront.yoursite.com
mongodb:
enabled: true # disable to use an external mongoDB host
# Username of the MongoDB user that will have read-write access to the Botfront database. This is not the root user
mongodbUsername: username
# Password of the MongoDB user that will have read-write access to the Botfront database. This is not the root user
mongodbPassword: password
# MongoDB root password
mongodbRootPassword: rootpassword
And I ran this command:
helm install -f config.yaml -n botfront --namespace botfront botfront/botfront
and the deployment appeared successful with all pods listed as running.
But botfront.yoursite.com goes nowhere. I checked the ingress and it matches, but there are no external ip addresses or anything. I don't know how to actually access my botfront site once deployed on kubernetes.
What am I missing?
EDIT:
With nginx lb installed kubectl get ingresses -n botfront now
returns:
NAME CLASS HOSTS ADDRESS PORTS AGE
botfront-app-ingress <none> botfront.cream.com a182b0b24e4fb4a0f8bd6300b440e5fa-423aebd224ce20ac.elb.us-east-2.amazonaws.com 80 4d1h
and
kubectl get svc -n botfront returns:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
botfront-api-service NodePort 10.100.207.27 <none> 80:31723/TCP 4d1h
botfront-app-service NodePort 10.100.26.173 <none> 80:30873/TCP 4d1h
botfront-duckling-service NodePort 10.100.75.248 <none> 80:31989/TCP 4d1h
botfront-mongodb-service NodePort 10.100.155.11 <none> 27017:30358/TCP 4d1h
If you run kubectl get svc -n botfront, it will show you all the Services that expose your botfront
$ kubectl get svc -n botfront
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
botfront-api-service NodePort 10.3.252.32 <none> 80:32077/TCP 63s
botfront-app-service NodePort 10.3.249.247 <none> 80:31201/TCP 63s
botfront-duckling-service NodePort 10.3.248.75 <none> 80:31209/TCP 63s
botfront-mongodb-service NodePort 10.3.252.26 <none> 27017:31939/TCP 64s
Each of them is of type NodePort, which means it exposes your app on the external IP address of each of your EKS cluster nodes on a specific port.
So if you your node1 ip happens to be 1.2.3.4 you can acess botfront-api-service on 1.2.3.4:32077. Don't forget to allow access to this port on firewall/security groups. If you have any registered domain e.g. yoursite.com you can configure for it a subdomain botfront.yoursite.com and point it to one of your EKS nodes. Then you'll be able to access it using your domain. This is the simplest way.
To be able to access it in a more effective way than by using specific node's IP and non-standard port, you may want to expose it via Ingress which will create an external load balancer, making your NodePort services available under one external IP adress and standard http port.
Update: I see that this chart already comes with ingress that exposes your app:
$ kubectl get ingresses -n botfront
NAME HOSTS ADDRESS PORTS AGE
botfront-app-ingress botfront.yoursite.com 80 70m
If you retrieve its yaml definition by:
$ kubectl get ingresses -n botfront -o yaml
you'll see that it uses the following annotation:
kubernetes.io/ingress.class: nginx
which means you need nginx-ingress controller installed on your EKS cluster. This might be one reason why it fails. As you can see in my example, this ingress doesn't get any external IP. That's because nginx-ingress wasn't installed on my GKE cluster. Not sure about EKS but as far as I know it doesn't come with nginx-ingress preinstalled.
One more thing: I assume that in your config.yaml you put some real domain name that you have registered instead of botfront.yoursite.com. Suppose your domain is yoursite.com and you successfully created subdomain botfront.yoursite.com, you should redirected it to the IP of your load balancer (the one used by your ingress).
If you run kubectl get ingresses -n botfront but the ADDRESS is empty, you probably don't have nginx-ingress installed and the underlying load balancer cannot be created. If you have here some external IP address, then redirect your registered domain to this address.
I Installed K8S with Helm Charts on EKS but the Loadbalancer EXTERNAL IP is in pending state , I see that EKS does support the service Type : LoadBalancer now.
Is it something I will have to check at the network outgoing traffic level ? Please share your experience if any.
Tx,
The Loadbalancer usually takes some seconds or a few minutes to provision you an IP.
If after 5 minutes the IP isn't provisioned:
- run kubectl get svc <SVC_NAME> -o yaml and if there is any different annotation set.
By default services with Type:LoadBalancer are provisioned with Classic Load Balancers automatically. Learn more here.
If you wish to use Network load Balancers you have to use the annotation:
service.beta.kubernetes.io/aws-load-balancer-type: nlb
The process is really automatic, you don't have to check for network traffic.
You can check if there is any issue with the Helm Chart you are deploying by manually creating a service with loadbalancer type and check if it gets provisioned:
$ kubectl run --generator=run-pod/v1 nginx --image=nginx --port=80
pod/nginx created
$ kubectl get pod nginx
NAME READY STATUS RESTARTS AGE
nginx 1/1 Running 0 34s
$ kubectl expose pod nginx --type=LoadBalancer
service/nginx exposed
$ kubectl get svc nginx -w
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx LoadBalancer 10.1.63.178 <pending> 80:32522/TCP 7s
nginx LoadBalancer 10.1.63.178 35.238.146.136 80:32522/TCP 42s
In this example the LoadBalancer took 42s to be provisioned. This way you can verify if the issue is on the Helm Chart or something else.
If Kubernetes is running in an environment that doesn't support LoadBalancer services, the load balancer will not be provisioned, but the service will still behave like a NodePort service, your cloud/K8 engine should support LoadBalancer Service.
In that case, if you manage to add EIP or VIP to your node then you can attach to the EXTERNAL-IP of your TYPE=LoadBalancer in the K8 cluster, for example attaching the EIP/VIP address to the node 172.16.2.13.
kubectl patch svc ServiceName -p '{"spec": {"type": "LoadBalancer", "externalIPs":["172.16.2.13"]}}'
https://github.com/kubernetes-sigs/kind - version 0.4.0
Create kubernetes from kubernetes-sigs/kind
kind create cluster
Creating cluster "kind" ...
✓ Ensuring node image (kindest/node:v1.15.0) 🖼
kubectl create serviceaccount
kubectl create serviceaccount --namespace kube-system tiller
serviceaccount/tiller created
kubectl create clusterrolebinding
kubectl create clusterrolebinding tiller-cluster-rule --clusterrole=cluster-admin --serviceaccount=kube-system:tiller
clusterrolebinding.rbac.authorization.k8s.io/tiller-cluster-rule created
kubectl patch deploy
kubectl patch deploy --namespace kube-system tiller-deploy -p '{"spec":{"template":{"spec":{"serviceAccount":"tiller"}}}}'
deployment.extensions/tiller-deploy patched
helm init
helm install stable/nginx-ingress
helm install --name grafana stable/grafana --set=ingress.enabled=True,ingress.hosts={grafana.domain.com} --namespace demo --set rbac.create=true
kubectl logs loping-wallaby-nginx-ingress-controller-76d574f8b7-5m6n5
W0629 17:13:59.709497 6 controller.go:797] Service "demo/grafana" does not have any active Endpoint.
[29/Jun/2019:17:14:03 +0000]TCP200000.000
I0629 17:14:45.223234 6 status.go:295] updating Ingress demo/grafana status from [] to [{ }]
I0629 17:14:45.226343 6 event.go:209] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"demo", Name:"grafana", UID:"228cde81-cb97-4313-ad86-90a273b2206d", APIVersion:"extensions/v1beta1", ResourceVersion:"1938", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress demo/grafana
kubectl get ingress --all-namespaces
NAMESPACE NAME HOSTS ADDRESS PORTS AGE
demo grafana grafana.domain.com 80 3m58s
kubectl get svc --all-namespaces -l app=grafana
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default grafana ClusterIP 10.104.203.243 <none> 80/TCP 24m
kubectl get endpoints
NAME ENDPOINTS AGE
grafana 10.244.0.10:3000 21m
kubernetes 172.17.0.2:6443 56m
loping-wallaby-nginx-ingress-controller 10.244.0.8:80,10.244.0.8:443 48m
loping-wallaby-nginx-ingress-default-backend 10.244.0.7:8080 48m
Thanks!
A few concerns about your current scenario:
You have to check installed nginx-ingress helm chart in order to find out why grafana service resides in separate namespace default and not in demo namespace as per helm deploy parameter --namespace demo.
Since you have not specified in helm install command controller.service.type parameter, Nginx Ingress Controller would be implemented with LoadBalancer type of relevant service, in this case Ingress Controller expects to receive external IP address using cloud provider’s load balancer and I assume that your current kubernetes provisioner kubernetes-sigs/kind is not a good choice to adopt outward access to the Kubernetes cluster. Therefore, I would suggest to use NodePort service for Nginx Ingress controller in order to expose 80 and 443 port on some specific port in the host machine.
helm install --name grafana stable/grafana --set=ingress.enabled=True,ingress.hosts={grafana.domain.com} --namespace demo --set rbac.create=true --controller.service.type=NodePort
Issue that you mentioned is more like harmless and doesn't significantly affect the Nginx Ingress Controller's functionality, because it means that for some short period of time Liveness probe for Grafana Pod has not been initiated and target enpoint has not been released during Grafana Helm chart deploy. You can even re-spawn Nginx Ingress controller Pod to justify my assumption.
you are using type of service as "ClusterIp" , so you will not get external Ip address.
Change the Service type to "Loadbalancer" then you get Ip address which you can browse through internet.
I do exactly with the kubernetes url.
But the last step I can't get the external ip with the cmd: kubectl get svc nginx.
root#XXX:~# kubectl expose rc nginx --port=80
service "nginx" exposed
root#in28-051:~# kubectl get svc nginx
NAME CLUSTER_IP EXTERNAL_IP PORT(S) SELECTOR AGE
nginx 10.0.0.109 <none> 80/TCP run=nginx 5m
https://github.com/kubernetes/kubernetes/blob/master/docs/user-guide/services.md#type-loadbalancer
starts with:
"On cloud providers which support external load balancers..."
I agree we should better document which cloud providers do support this. I'll file an issue.