We have deployed an application on to Kubernetes Cluster configured on local VMWare servers in the On-prem. I have created a default ingress rule, and however, still, I'm not able to access the service from other machines. I can access locally using "curl" command.
I have re-installed Nginx Ingress controller and configured default ingress resource but not able to access from the outside
[root#uat-amk8smaster01 ~]# kubectl -n stackstorm get svc dd-stackstorm-st2web
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dd-stackstorm-st2web NodePort 10.101.23.22 <none> 80:32714/TCP 16h
[root#uat-amk8smaster01 ~]#
[root#uat-amk8smaster01 ~]# kubectl -n stackstorm get ingress
NAME HOSTS ADDRESS PORTS AGE
st2-ingress-default * 80 15h
[root#uat-amk8smaster01 ~]#
# cat st2-default-ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
labels:
name: st2-ingress-default
name: st2-ingress-default
namespace: stackstorm
spec:
backend:
serviceName: dd-stackstorm-st2web
servicePort: 80
#
The webpage should open when we try to open using IP:32714.
My advice is to check the status of the ingress using kubectl describe ingress st2-ingress-default and see if it has some events, normally a bad livenessprobe and readinessprobe causes to not be able to connect.
Also, you can review the nginx controller pod logs and see if your traffic is going inside the cluster.
If kubernetes is running on premises, then you have to implement your own ingress controller.
In order for the Ingress resource to work, the cluster must have an ingress controller running.
Check this page :
https://kubernetes.io/docs/concepts/services-networking/ingress-controllers/
If you want to rout traffic to your service via ingress, the flow should be the following :
Ingress --> Ingress controller service --> Ingress controller --> dd-stackstorm-st2web service --> dd-stackstorm-st2web pod
And, apparently, you are trying to expose your dd-stackstorm-st2web service via NodePort and reach it omitting ingress.
My assumption is that you don't have ingress-controller service exposed.
Still, if you want to access service directly through the NodePort
curl http://<node-external-ip>:32714
to find node external ip
kubectl get nodes -o wide
Related
I'm trying to get an ingress controller working in Minikube and am following the steps in the K8s documentation here, but am seeing a different result in that the IP address for the ingress controller is different than that for Minikube (the example seems to indicate they should be the same):
$ kubectl get ingress
NAME HOSTS ADDRESS PORTS AGE
example-ingress hello-world.info 10.0.2.15 80 12m
$ minikube ip
192.168.99.101
When I try to connect to the Minikube IP address (using the address directly vs. adding it to my local hosts file), I'm getting a "Not found" response from NGINX:
$ curl http://`minikube ip`/
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>openresty/1.15.8.1</center>
</body>
</html>
When I try to connect to the IP address associated with the ingress controller, it just hangs.
Should I expect the addresses to be the same as the K8s doc indicates?
Some additional information:
$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
minikube Ready master 2d23h v1.16.0 10.0.2.15 <none> Buildroot 2018.05.3 4.15.0 docker://18.9.9
$ kubectl get ingresses example-ingress -o yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"networking.k8s.io/v1beta1","kind":"Ingress","metadata":{"annotations":{"nginx.ingress.kubernetes.io/rewrite-target":"/$1"},"name":"example-ingress","namespace":"default"},"spec":{"rules":[{"host":"hello-world.info","http":{"paths":[{"backend":{"serviceName":"web","servicePort":8080},"path":"/"}]}}]}}
nginx.ingress.kubernetes.io/rewrite-target: /$1
creationTimestamp: "2019-10-28T15:36:57Z"
generation: 1
name: example-ingress
namespace: default
resourceVersion: "25609"
selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/example-ingress
uid: 5e96c378-fbb1-4e8f-9738-3693cbce7d9b
spec:
rules:
- host: hello-world.info
http:
paths:
- backend:
serviceName: web
servicePort: 8080
path: /
status:
loadBalancer:
ingress:
- ip: 10.0.2.15
Here’s what worked for me:
minikube start
minikube addons enable ingress
minikube addons enable ingress-dns
Wait until you see the ingress-nginx-controller-XXXX is up and running using Kubectl get pods -n ingress-nginx
Create an ingress using the K8s example yaml file
Update the service section to point to the NodePort Service that you already created
Append
127.0.0.1 hello-world.info
to your /etc/hosts file on MacOS (NOTE:
Do NOT use the Minikube IP)
Run minikube tunnel ( Keep the window open. After you entered the password there will be no more messages, and the cursor just blinks)
Hit the hello-world.info ( or whatever host you configured in the yaml file) in a browser and it should work
I've reproduced your scenario in a Linux environment (on GCP) and I also have different IPs:
user#bf:~$ minikube ip
192.168.39.144
user#bf:~$ kubectl get ingresses
NAME HOSTS ADDRESS PORTS AGE
example-ingress * 192.168.122.173 80 30m
Your problem is not related to the fact you have different IPs. The guide instructs us to create an ingress with the following rule:
spec:
rules:
- host: hello-world.info
This rule is telling the ingress service that a DNS record with hello-world.info name is expected.
If you follow the guide a bit further, it instructs you to create an entry on your hosts file pointing to your ingress IP or Minikube IP.
Note: If you are running Minikube locally, use minikube ip to get the external IP. The IP address displayed within the ingress list
will be the internal IP.
Source: Set up Ingress on Minikube with the NGINX Ingress Controller
(if you want to curl the IP instead of DNS name, you need to remove the host rule from your ingress)
It should look like this:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: web
servicePort: 8080
Apply your changes:
user#bf:~$ kubectl apply -f example-ingress.yaml
And curl the IP using -Lk options to surpass problems related to secure connections.
user#bf:~$ curl -Lk 192.168.39.144
Hello, world!
Version: 1.0.0
Hostname: web-9bbd7b488-l5gc9
In addition to the accepted answer, minikube now has a tunnel command which allows you generate external ip addresses for your services which can be accessed directly on your host machine without using the general minikube ip.
Run minikube tunnel in a separate terminal. This runs in the foreground as a daemon.
In a different terminal, execute your kubectl apply -f <file_name> command to deploy your desired service. It should generate an ip address for you that is routed directly to your service and available on port 80 on that address.
More here on the minikube documentation: https://minikube.sigs.k8s.io/docs/tasks/loadbalancer/
I got Minikube on Windows 11 to work for me
minikube start --vm-driver=hyperv
Install minikube Ingress Controller
minikube addons enable ingress
minikube addons enable ingress-dns
Deploy Helm Chart
helm install ...
Get Kubernetes IP Address
nslookup <host-found-in-ingress> $(minikube ip)
Add to etc/host
<minikube-ip> <domain-url>
Live!
curl <domain-url>
I built my own 1 host kubernetes cluster (1 host, 1 node, many namespaces, many pods and services) on a virtual machine, running on a always-on server.
The applications running on the cluster are working fine (basically, a NodeJS backend and HTML frontend).
So far, I have a NodePort Service, which is exposing Port 30000:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
traefik-ingress-service NodePort 10.109.211.16 <none> 443:30000/TCP 147d
So, now I can access the web interface by typing https://<server-alias>:30000 in my browser adress bar.
But I would like to access it without giving the port, by only typing https://<server-alias>.
I know, this can be done with the kubectl port-forwarding command:
kubectl -n kube-system port-forward --address 0.0.0.0 svc/traefik-ingress-service 443:443
This works. But it does not seem to be a very professional thing to do.
Port forwarding also seems to keep disconnecting from time to time. Sometimes, it throws an error and quits, but leaves the process open, which leaves the port open - have to kill the process manually.
So, is there a way to do that access-my-application stuff professionally? How do the cluster provider (AWS, GCP...) do that?
Thank you!
Using Ingress Nginx you can access to you website with the name server:
Step 1: Install Nginx ingress in you cluster you can flow this link
After the installation is completed you will have a new pod
NAME READY STATUS
nginx-ingress-xxxxx 1/1 Running
And a new Service
NAME TYPE CLUSTER-IP EXTERNAL-IP
nginx-ingress LoadBalancer 10.109.x.y a.b.c.d
Step 2 : Create new deployment for you application but be sure that you are using the same name space for nginx ingress svc/pod and you application and you set the svc type to ClusterIP
Step 3: Create Kubernetes Ingress Object
Now you have to create the ingress object
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-ingress
namespace: **Same Name Space**
spec:
rules:
- host: your DNS <server-alias>
http:
paths:
- backend:
serviceName: svc Name
servicePort: svc Port
Now you can access to your website using the .
To create a DNS for free you can use freenom or you can use /etc/hosts
update it with :
server-alias a.b.c.d
Since the Type of your Traefik Ingress Service is NodePort, you get to access to the port provided which will have a value from 30000-32000.
You can also configure it to be of type LoadBalancer and interface with a cloud-based Load Balancer.
Ref: https://kubernetes.io/docs/tasks/access-application-cluster/create-external-load-balancer/
Here's a very related question: Should I use NodePort in my Traefik deployment on Kubernetes?
In my k8s cluster the ingress does not work on the k8s cluster of digital ocean. I don't get an external ip and so it is not available. Locally there seems to be no problem.
I already searched a lot and tried some tutorials, f.e. https://www.digitalocean.com/community/tutorials/how-to-set-up-an-nginx-ingress-with-cert-manager-on-digitalocean-kubernetes.
But it seems that it is an older version and the solution (and even the links) does not work anymore.
The nginx-ingress should call the service of a website backend which is on port 8080.
I stripped down my ingress code to the following one:
kind: Ingress
metadata:
name: test-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: website
servicePort: 8080
With kubectl get ing I see the ingress, but it has no address. It looks like this
NAME HOSTS ADDRESS PORTS AGE
test-ingress * 80 50s
Can anyone help me out and tell me what I have to do to get my k8s cluster running?
Thanks
peter
Firstly, if you are using Nginx Ingress Controller, you don't need to see ingress address.
When you install Nginx Ingress Controller to your k8s cluster, it creates Load Balancer to handle all incoming requests. Make sure that below part completed as explained in Step 2 of guide you posted and you are able to see LoadBalancer External ip address.
$ kubectl get svc --namespace=ingress-nginx
You should see an external IP address, corresponding to the IP address of the DigitalOcean Load Balancer:
Output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx LoadBalancer 10.245.247.67 203.0.113.0 80:32486/TCP,443:32096/TCP 20h
In above case, after deploying your ingress resource, if you hit http://203.0.113.0 you will get your website:8080 backend service.
Hope it helps!
I have deployed Kong-Ingress-controller using helm
And I have Kubernetes's Cluster v1.10 On centos 7
I am using dedicated Server From OVH Provider
When I create Ingress
cat ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: jenkins
spec:
backend:
serviceName: jenkins
servicePort: 8080
kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
jenkins * 80 3s
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins ClusterIP 10.254.104.80 <none> 8080/TCP 1d
Now I Can not access this Ingress from Out Side because I am using OVH Server.
Is there a solution?
OVH is not officially supported by Kubernetes. It was supported then generally you would create a Service jenkins of the type LoadBalancer and that would be your externally facing endpoint with a public IP.
Since it's not supported the next best thing is to create a NodePort service. That will create a service that listens on a specific port on all the Kubernetes nodes and forwards the requests to your Pods (only where they are running). So, in this case, you will have to create an OVH Load Balancer with a public IP and point the backend of that load balancer to the NodePort of the service where your Ingress is listening on.
Base OS : CentOS (1 master 2 minions)
K8S version : 1.9.5 (deployed using KubeSpray)
I am new to Kubernetes Ingress and am setting up 2 different services, each reachable with its own path.
I have created 2 deployments :
kubectl run nginx --image=nginx --port=80
kubectl run echoserver --image=gcr.io/google_containers/echoserver:1.4 --port=8080
I have also created their corresponding services :
kubectl expose deployment nginx --target-port=80 --type=NodePort
kubectl expose deployment echoserver --target-port=8080 --type=NodePort
My svc are :
[root#node1 kubernetes]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
echoserver NodePort 10.233.48.121 <none> 8080:31250/TCP 47m
nginx NodePort 10.233.44.54 <none> 80:32018/TCP 1h
My NodeIP address is 172.16.16.2 and I can access both pods using
http://172.16.16.2:31250 &
http://172.16.16.2:32018
Now on top of this I want to deploy an Ingress so that I can reach both pods not using 2 IPs and 2 different ports BUT 1 IP address with different paths.
So my Ingress file is :
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: fanout-nginx-ingress
spec:
rules:
- http:
paths:
- path: /nginx
backend:
serviceName: nginx
servicePort: 80
- path: /echo
backend:
serviceName: echoserver
servicePort: 8080
This yields :
[root#node1 kubernetes]# kubectl describe ing fanout-nginx-ingress
Name: fanout-nginx-ingress
Namespace: development
Address:
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
*
/nginx nginx:80 (<none>)
/echo echoserver:8080 (<none>)
Annotations:
Events: <none>
Now when I try accessing the Pods using the NodeIP address (172.16.16.2), I get nothing.
http://172.16.16.2/echo
http://172.16.16.2/nginx
Is there something I have missed in my configs ?
I had the same issue on my bare metal installation - or rather something close to that (kubernetes virtual cluster - set of virtual machines connected via Host-Only-Adapter). Here is link to my kubernetes vlab.
First of all make sure that you have ingress controller installed. Currently there are two ingress controller worth trying kubernetes nginx ingress controller and nginx kubernetes ingress controller -I installed first one.
Installation
Go to installation instructions and execute first step
# prerequisite-generic-deployment-command
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml
Next get IP addresses of cluster nodes.
$ kubectl get nodes -o wide
NAME STATUS ROLES ... INTERNAL-IP
master Ready master ... 192.168.121.110
node01 Ready <none> ... 192.168.121.111
node02 Ready <none> ... 192.168.121.112
Further, crate ingress-nginx service of type LoadBalancer. I do it by downloading NodePort template service from installation tutorial and making following adjustments in svc-ingress-nginx-lb.yaml file.
$ curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml > svc-ingress-nginx-lb.yaml
# my changes svc-ingress-nginx-lb.yaml
type: LoadBalancer
externalIPs:
- 192.168.121.110
- 192.168.121.111
- 192.168.121.112
externalTrafficPolicy: Local
# create ingress- service
$ kubectl apply -f svc-ingress-nginx-lb.yaml
Verification
Check that ingress-nginx service was created.
$ kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx LoadBalancer 10.110.127.9 192.168.121.110,192.168.121.111,192.168.121.112 80:30284/TCP,443:31684/TCP 70m
Check that nginx-ingress-controller deployment was created.
$ kubectl get deploy -n ingress-nginx
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
nginx-ingress-controller 1 1 1 1 73m
Check that nginx-ingress pod is running.
$ kubectl get pods --all-namespaces -l
app.kubernetes.io/name=ingress-nginx
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx nginx-ingress-controller-5cd796c58c-lg6d4 1/1 Running 0 75m
Finally, check ingress controller version. Don't forget to change pod name!
$ kubectl exec -it nginx-ingress-controller-5cd796c58c-lg6d4 -n ingress-nginx -- /nginx-ingress-controller --version
-------------------------------------------------------------------------------
NGINX Ingress controller
Release: 0.21.0
Build: git-b65b85cd9
Repository: https://github.com/aledbf/ingress-nginx
-------------------------------------------------------------------------------
Testing
Test that ingress controller is working by executing steps in this tutorial -of course, you will omit minikube part.
Successful, execution of all steps will create ingress controler resource that should look like this.
$ kubectl get ing
NAME HOSTS ADDRESS PORTS AGE
ingress-tutorial myminikube.info,cheeses.all 192.168.121.110,192.168.121.111,192.168.121.112 80 91m
And pods that looks like this.
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
cheddar-cheese-6f94c9dbfd-cll4z 1/1 Running 0 110m
echoserver-55dcfbf8c6-dwl6s 1/1 Running 0 104m
stilton-cheese-5f6bbdd7dd-8s8bf 1/1 Running 0 110m
Finally, test that request to myminikube.info propagates via ingress load balancer.
$ curl myminikube.info
CLIENT VALUES:
client_address=10.44.0.7
command=GET
real path=/
query=nil
request_version=1.1
request_uri=http://myminikube.info:8080/
SERVER VALUES:
server_version=nginx: 1.10.0 - lua: 10001
HEADERS RECEIVED:
accept=*/*
host=myminikube.info
user-agent=curl/7.29.0
x-forwarded-for=10.32.0.1
x-forwarded-host=myminikube.info
x-forwarded-port=80
x-forwarded-proto=http
x-original-uri=/
x-real-ip=10.32.0.1
x-request-id=b2fb3ee219507bfa12472c7d481d4b72
x-scheme=http
BODY:
It was a long journey to make ingress working on bear metal like environment.Thus, i will include relevant links that helped me along.
reproducable tutorial
installation of minikube on ubuntu
ingress I
ingress II
digging
reverse engineering on ingress in kubernetes
Check if you have an ingress controller in your cluster:
$ kubectl get po --all-namespaces
You should see something like:
kube-system nginx-ingress-controller-gwts0 1/1 Running 0 18d
It's only possible to create an ingress to address services inside the namespace in which the Ingress resides.
Cross-namespace ingresses are not implemented for security reasons.
It seems that your cluster is missing Ingress controller.
In general, Ingress controller works as follows:
1. search for a certain type of objects (ingress,"nginx") in a cluster
2. parse that object and create configuration section for a specific ingress pod.
3. update that pod object (restart it with updated configuration)
That particular pod is responsible for processing traffic from incoming ports (usually a couple of dedicated ports on nodes) to configured traffic destination in cluster.
You can choose from two supported and maintained controllers - Nginx and GCE
The ingress controller consists of several components that you create during installation.
Here is installation part from Nginx Ingress documentation:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/namespace.yaml | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/default-backend.yaml | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/configmap.yaml | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/tcp-services-configmap.yaml | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/udp-services-configmap.yaml | kubectl apply -f -
If you have RBAC authorization configured in your cluster:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/rbac.yaml | kubectl apply -f -
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/with-rbac.yaml | kubectl apply -f -
If no RBAC configured:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/without-rbac.yaml | kubectl apply -f -
In case you create cluster from scratch:
curl https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml | kubectl apply -f -
Verify your installation:
kubectl get pods --all-namespaces -l app=ingress-nginx --watch
You should see something like:
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx nginx-ingress-controller-699cdf846-nj2rw 1/1 Running 0 1h
Check available services and their parameters:
kubectl get services --all-namespaces
If you are using custom service provider deployment (minikube, AWS, Azure, GKE), follow Nginx Ingress documentation for installation details.
See official Kubernetes Ingress documentation for details about Ingress.
I was using microk8s default nginx ingress controller on a later version of k8s (> 1.18) and I noticed this specific annotation was causing me an issue:
kubernetes.io/ingress.class: "nginx"
It's present in a lot of older documentation and examples but it's apparently deprecated (see https://kubernetes.io/docs/concepts/services-networking/ingress/) and I had also defined an ingressClassName of "public" using the newer annotation. I'm not sure if it was the conflict between the two that caused the issue but once I removed the deprecated annotation my address appeared.
For working your ingress resources (fanout-nginx-ingress), you need to first deploy the ingress controller which is by-default not come in your local kubernetes cluster. You need to deploy it yourself.
There are many solution out there and you can use any of them, but nginx ingress controller is fine.
For detail information you can refer a great video on ingress by Mumshad Mannambeth here:
https://www.youtube.com/watch?v=GhZi4DxaxxE