I am using nginx ingress for path based routing for multiple services .
My services output
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S)
ingress-nginx-controller LoadBalancer X.X.X.33 X.X.X.112 80:30853/TCP,443:31386/TCP
ingress-nginx-controller-admission ClusterIP X.X.X.139 <none> 443/TCP
test1 ClusterIP X.X.X.197 <none> 9080/TCP
test2 ClusterIP X.X.X.30 <none> 9995/TCP
My ingress.yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-resource
annotations:
nginx.ingress.kubernetes.io/rewrite-target: "/"
spec:
rules:
- http:
paths:
- path: /test2
backend:
serviceName: test2
servicePort: 9995
- path: /test1
backend:
serviceName: test1
servicePort: 9080
and kubectl ingress output
NAME HOSTS ADDRESS PORTS AGE
ingress-resource * 80 45m
When i am trying to access service from outside cluster -
I am getting 404
-> http:// X.X.X.112/test1 return 404
-> http:// X.X.X.112/test2 return 404
Where X.X.X.112 is external ip of ingress-nginx-controller
-> When I create ingress to services running on port 8080, it seems to be working fine.
What the issue ?
I figured out the issue. I was missing
annotations:
kubernetes.io/ingress.class: nginx
Now I have other issue.
When I use
spec:
rules:
- http:
paths:
- path: /
backend:
serviceName: test2
servicePort: 9995
My http:// X.X.X.112/ works fine and it points to serviceName - test2
Similarly it works for other service.
but when I use
spec:
rules:
- http:
paths:
- path: /test2
backend:
serviceName: test2
servicePort: 9995
- path: /test1
backend:
serviceName: test1
servicePort: 9080
and I get 405 method not allowed, when doing
-> http:// X.X.X.112/test1 return 405
-> http:// X.X.X.112/test2 return 405
I tired adding few more annotations .
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS, DELETE, PATCH
nginx.ingress.kubernetes.io/enable-cors: "true"
I am missing something else ?
Ive tried your example on GKE (with changed deployment as you didnt provide which one you have used).
As I mentioned in comments, to force Nginx Ingress on GKE you must use annotation:
annotations:
kubernetes.io/ingress.class: nginx
Otherwise GKE will use default gce ingress which do not support rewrite annotation.
apiVersion: apps/v1
kind: Deployment
metadata:
name: test1
spec:
replicas: 1
selector:
matchLabels:
key: test1
template:
metadata:
labels:
key: test1
spec:
containers:
- name: hello1
image: gcr.io/google-samples/hello-app:1.0
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
---
apiVersion: v1
kind: Service
metadata:
name: test1
spec:
selector:
key: test1
ports:
- port: 80
targetPort: 8080
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: test2
labels:
app: mycha-app
spec:
replicas: 1
selector:
matchLabels:
app: test2
template:
metadata:
labels:
app: test2
spec:
containers:
- name: nginx2
image: nginx
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: test2
labels:
app: test2
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
selector:
app: test2
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /test1
backend:
serviceName: test1
servicePort: 80
- path: /test2
backend:
serviceName: test2
servicePort: 80
$ kubectl get svc -A
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.0.0.1 <none> 443/TCP 7h59m
default test1 ClusterIP 10.0.14.220 <none> 80/TCP 9m18s
default test2 ClusterIP 10.0.0.243 <none> 80/TCP 9m16s
ingress-nginx ingress-nginx-controller LoadBalancer 10.0.7.75 35.228.2XX.XX 80:32643/TCP,443:32733/TCP 18m
$ curl 35.228.2XX.XX/test1
Hello, world!
Version: 1.0.0
Hostname: test1-5844dc9688-n95tj
$ curl 35.228.2XX.XX/test2
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
body {
width: 35em;
...
If this example will not work on your cluster, you will need to create Firewall rules to allow traffic.
If this example will work it means that you have configuration mismatch in your Deployment/Service.
It might be selector misconfigure between service and deployment (in first one it was key: test1 and in second deployment app: test2).
Application is listening on different port that was set in containerPort. If you set only port in service, the same value will be set in containerPort. In test1 deployment it would not work as hello-app:1.0 listening only on port 8080.
Hello pod
# netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 :::8080 :::* LISTEN 1/hello-app
Nginx pod
# netstat -plnt
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 1/nginx: master pro
Please comment if you will still have issue.
Related
I have Kubernetes cluster v1.19.16 set up in bare metal Ubuntu-18.04 server and currently i want to connect cluster jenkins service through http://jenkins.company.com. Haproxy server side frontend & backend already been configured.
My service.yaml file content as follows,
apiVersion: v1
kind: Service
metadata:
name: jenkins-svc
namespace: jenkins
annotations:
prometheus.io/scrape: 'true'
prometheus.io/path: /
prometheus.io/port: '8080'
spec:
selector:
app: jenkins-server
type: ClusterIP
ports:
- protocol: TCP
port: 8080
targetPort: 80
ingress-resource.yaml file content as follows,
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: jenkins-ingress
namespace: jenkins
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: "jenkins.company.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: jenkins-svc
servicePort: 8080
# kubectl get service -n jenkins
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
jenkins-svc ClusterIP 10.96.136.255 <none> 8080/TCP 20m
# kubectl get ing jenkins-ingress
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAME CLASS HOSTS ADDRESS PORTS AGE
jenkins-ingress <none> jenkins.company.com 80 5h42m
# kubectl describe ingress -n jenkins
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
Name: jenkins-ingress
Namespace: jenkins
Address:
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
jenkins.dpi.com
/ jenkins-svc:8080 (10.244.0.16:80)
Annotations: ingress.kubernetes.io/rewrite-target: /
kubernetes.io/ingress.class: nginx
Events: <none>
When i tried to access http://jenkins.company.com it shows below error message on browser.
Please let me know what i'm missing here?
The issue with the service port and container port. Jenkins default port is 8080, so I assume your service port is 80
ports:
- protocol: TCP
port: 80
targetPort: 8080
and ingress should be
spec:
rules:
- host: "jenkins.company.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
serviceName: jenkins-svc
servicePort: 80
port: The port of this service
targetPort The target port on the pod(s) to forward traffic to
Difference between targetPort and port in Kubernetes Service definition
I have 2 services. I want to create ingress fanout for them. 1 service runs properly, the other one says
Error: Server Error
The server encountered a temporary error and could not complete your request.
Please try again in 30 seconds.
I opened all firewall settings. here are some details
bahaddin#b k get ingress -n ingress-nginx
NAME CLASS HOSTS ADDRESS PORTS AGE
fanout-ingress <none> * 35.201.67.49 80 2m57s
bahaddin#bahaddin-ThinkPad-E15-Gen-2:~/projects/personal/exposer/k8s-ingress$ k get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.12.1.185 34.134.200.199 80:30803/TCP,443:32306/TCP 10m
ingress-nginx-controller-admission ClusterIP 10.12.0.83 <none> 443/TCP 10m
web NodePort 10.12.12.129 <none> 8080:30702/TCP 7m43s
web2 NodePort 10.12.5.55 <none> 8080:30160/TCP 7m42s
Here you see that external IP of ingress-nginx-controller (34.134.200.199) is different than fanout-ingress address. but when firstly the fanout-ingress has created , it was the same IP as ingress-nginx-controller (34.134.200.199). I cannot understand why, after some seconds it has gained the newer IP (35.201.67.49). I would be happy to get answer this as well.
The main question is , when I curl http://35.201.67.49/v2/ I get the result properly. But when I curl http://35.201.67.49/web1/hello - which is another service and defined on service, ingress and deployment files below - is not reachable. When I curl 35.201.67.49/web1/hello i got an error
Error: Server Error The server encountered a temporary error and could
not complete your request.
Please try again in 30 seconds.
fanput-ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: fanout-ingress
namespace: ingress-nginx
spec:
rules:
- http:
paths:
- path: /web1/*
backend:
serviceName: web
servicePort: 8080
- path: /v2/*
backend:
serviceName: web2
servicePort: 8080
deployment and services
apiVersion: apps/v1
kind: Deployment
metadata:
name: web
namespace: ingress-nginx
spec:
selector:
matchLabels:
run: web
template:
metadata:
labels:
run: web
spec:
containers:
- image: bago1/web1:latest
imagePullPolicy: IfNotPresent
name: web
ports:
- containerPort: 8080
protocol: TCP
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: web2
namespace: ingress-nginx
spec:
selector:
matchLabels:
run: web2
template:
metadata:
labels:
run: web2
spec:
containers:
- image: gcr.io/google-samples/hello-app:2.0
imagePullPolicy: IfNotPresent
name: web2
ports:
- containerPort: 8080
protocol: TCP
---
apiVersion: v1
kind: Service
metadata:
name: web
namespace: ingress-nginx
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
run: web
type: NodePort
---
apiVersion: v1
kind: Service
metadata:
name: web2
namespace: ingress-nginx
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
selector:
run: web2
type: NodePort
Here
Running microk8s v1.23.3 on Ubuntu 20.04.4 LTS. I have set up a minimal pod+service:
kubectl create deployment whoami --image=containous/whoami --namespace=default
This works as expected, curl 10.1.76.4:80 gives the proper reply from whoami.
I have a service configured, see content of service-whoami.yaml:
apiVersion: v1
kind: Service
metadata:
name: whoami
namespace: default
spec:
selector:
app: whoami
ports:
- protocol: TCP
port: 80
targetPort: 80
This also works as expected, the pod can be reached through the clusterIP on curl 10.152.183.220:80.
Now I want to expose the service using the ingress-whoami.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: whoami-ingress
namespace: default
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
defaultBackend:
service:
name: whoami
port:
number: 80
rules:
- http:
paths:
- path: /whoami
pathType: Prefix
backend:
service:
name: whoami
port:
number: 80
ingress addon is enabled.
microk8s is running
high-availability: no
datastore master nodes: 127.0.0.1:19001
datastore standby nodes: none
addons:
enabled:
ha-cluster # Configure high availability on the current node
ingress # Ingress controller for external access
ingress seems to point to the correct pod and port. kubectl describe ingress gives
Name: whoami-ingress
Labels: <none>
Namespace: default
Address:
Default backend: whoami:80 (10.1.76.12:80)
Rules:
Host Path Backends
---- ---- --------
*
/whoami whoami:80 (10.1.76.12:80)
Annotations: <none>
Events: <none>
Trying to reach the pod from outside with curl 127.0.0.1/whoami gives a 404:
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
Where did I go wrong? This setup worked a few weeks ago.
Ok, figured it out.
I had forgotten to specify the ingress.class in the annotations-block.
I updated ingress-whoami.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: whoami-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: public
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /whoami
pathType: Prefix
backend:
service:
name: whoami
port:
number: 80
Now everything is working.
I have set up a baremetal k8 cluster ( 1 master node - intel NUC & 2 worker nodes on Raspberry pi). I managed to set up a metal-lb load balance and nginx ingress controller. I have launched two applications, ghost (listens on default port 2368) and nextcloud ( listens on default port 80) . I'm trying to access the applications from public ip myhomeserver.io ( to access the ghost application) and nextcloud.myhomeserver.io ( to access the next cloud application). I can access the ghost application but I can't seem to access nextcloud.Given below are the yaml files for ingress and services. Not sure where am I going wrong.
kubectl get services --all-namespaces
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
default kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 98d
ghost ghost-service ClusterIP 10.107.116.108 <none> 2368/TCP 7h37m
ingress-nginx ingress-nginx LoadBalancer 10.109.177.223 192.168.178.200 80:31619/TCP,443:30365/TCP 7d23h
kube-system kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 98d
nextcloud nextcloud-service ClusterIP 10.105.24.162 <none> 8080/TCP 137m
=============================================================================================================================
NAMESPACE NAME HOSTS ADDRESS PORTS AGE
ghost ingress-ghost myhomeserver.io 192.168.178.200 80 7d22h
nextcloud ingress-nextcloud nextcloud.myhomeserver.io 192.168.178.200 80 140m
=============================================================================================================================
cat ingress-object-ghost.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
spec:
rules:
- host: myhomeserver.io
http:
paths:
- backend:
serviceName: ghost-service
servicePort: 2368
=============================================================================================================================
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nextcloud
namespace: nextcloud
spec:
rules:
- host: nextcloud.myhomeserver.io
http:
paths:
- backend:
serviceName: nextcloud-service
servicePort: 8080
================================================================================================================================
cat ingress-object-nextcloud.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nextcloud
namespace: nextcloud
spec:
rules:
- host: nextcloud.myhomeserver.io
http:
paths:
- backend:
serviceName: nextcloud-service
servicePort: 8080
===================================================================================
apiVersion: apps/v1
kind: Deployment
metadata:
name:
deployment-nextcloud
namespace: nextcloud
labels:
env: prod
app: nextcloud-app
spec:
template:
metadata:
name: nextcloud-app-pod
labels:
app: nextcloud-app
env: production
spec:
containers:
- name: nextcloud
image: arm32v7/nextcloud
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8080
volumeMounts:
- mountPath: /var/www/html
name: nextcloud-data
securityContext:
privileged: True
volumes:
- name: nextcloud-data
persistentVolumeClaim:
claimName: pvc-nextcloud
nodeSelector:
kubernetes.io/arch: arm
replicas: 2
selector:
matchLabels:
app: nextcloud-app
================================================================================================================
apiVersion: v1
kind: Service
metadata:
name: nextcloud-service
namespace: nextcloud
labels:
app: nextcloud-app
spec:
type: ClusterIP
selector:
app: nextcloud-app
ports:
- port: 8080
targetPort: 8080
protocol: TCP
Note your nginx ingress controller is running in the ghost namespace so it only knows about the ghost service. You need to have another ingress controller for your nextcloud namespace if you want to have an ingress there. If you don't want another ingress controller then you can resolve the nextcloud service by targeting its dns in the following way servicename.namespacename.svc.cluster.local
On a side, there is not really a point in dividing your applications that much. Kubernetes already gives you enough privacy among applications in the same namespace.
UPDATE
Ingress that works for you given you have only 1 INGRESS CONTROLLER. Since there are two services I have added a path rule which will be rewritten to / so each service will receive a clean URI. Use myhomeserver.io/ghost to reach ghost and myhomeserver.io/nextcloud to reach nextcloud.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myhomeserver.io
http:
paths:
- path: /ghost
backend:
serviceName: ghost-service
servicePort: 2368
- path: /nextcloud
backend:
serviceName: nextcloud-service.nextcloud.svc.cluster.local
servicePort: 8080
UPDATE 2
So your ingress controller is running in the ghost namespace. Thus, your ingress has to be deployed in the ghost namespace. Note the http rules for each host.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-ghost
namespace: ghost
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: myhomeserver.io
http:
paths:
- path: /
backend:
serviceName: ghost-service
servicePort: 2368
- host: nextcloud.myhomeserver.io
http:
- path: /
backend:
serviceName: nextcloud-service.nextcloud.svc.cluster.local
servicePort: 8080
I used this config file to deploy a app , but I can't visit the url(I had add this domain to my /etc/hosts):
http://gogs.app, and it has a response with the 307 status of https://gogs.app
I am using docker for mac 18.09.2
And k8s is installed by docker for mac.
Problem:
I used this config file to deploy a app , but I can't visit the url(I had add this domain to my /etc/hosts):
http://gogs.app, and it has a response with the 307 status of https://gogs.app
☁ gogs-k8s kubectl get pods
NAME READY STATUS RESTARTS AGE
gogs-64cbd8c6f7-9qjbh 1/1 Running 0 32m
gogs-64cbd8c6f7-prcnf 1/1 Running 0 32m
gogs-64cbd8c6f7-rgcpn 1/1 Running 0 32m
☁ gogs-k8s kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx LoadBalancer 10.110.146.125 localhost 80:31255/TCP,443:30359/TCP 10d
☁ gogs-k8s kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
gogs-services LoadBalancer 10.109.151.183 <pending> 80:31053/TCP,2222:30591/TCP 36m
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 10d
this is my gogs.yaml:
apiVersion: apps/v1
kind: Deployment
metadata:
name: gogs
spec:
replicas: 3
selector:
matchLabels:
app: gogs
template:
metadata:
labels:
app: gogs
spec:
containers:
- name: gogs
image: gogs/gogs:latest
imagePullPolicy: IfNotPresent
resources:
limits:
memory: "200Mi"
cpu: "500m"
ports:
- containerPort: 3000
name: http
- containerPort: 22
name: ssh
volumeMounts:
- mountPath: /data
name: gogs-repo
volumes:
- name: gogs-repo
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
app: gogs
name: gogs-services
spec:
ports:
- name: http
port: 80
targetPort: 3000
- name: ssh
port: 2222
targetPort: 22
selector:
app: gogs
type: LoadBalancer
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gogs
spec:
rules:
- host: gogs.app
http:
paths:
- path: /
backend:
serviceName: gogs
servicePort: 80
visit the url: http://gogs.app correct.
If you don't want redirect to HTTPS then try using this manifest for ingress resource:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gogs
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- host: gogs.app
http:
paths:
- path: /
backend:
serviceName: gogs
servicePort: 80
Since you deployed your k8s with Docker for Mac, I am assuming you are attempting to get your application working on a local machine. The problem is that you have defined host: gogs.app in your nginx ingress. You should change your ingress definition to:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: gogs
spec:
rules:
- host: localhost
http:
paths:
- path: /
backend:
serviceName: gogs
servicePort: 80
With Docker for Mac, it seems like your ingress-nginx is exposed externally on the localhost interface; thus it would not allow access externally. If you wish to use your domain name, consider deploying this in a cloud environment with static public IP addressing, and create the appropriate DNS records for it.