Problem Sub Path Ingress Controller for Backend Service - kubernetes

I have problem setting path ingress controller for backend service. For example i want setup :
frontend app with angular (Path :/)
backend service with NodeJs (Path :/webservice).
NodeJS : Index.js
const express = require('express')
const app = express()
const port = 4000
app.get('/', (req, res) => res.send('Welcome to myApp!'))
app.use('/data/office', require('./roffice'));
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
Another Route:roffice.js
var express = require('express')
var router = express.Router()
router.get('/getOffice', async function (req, res) {
res.send('Get Data Office')
});
module.exports = router
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-stack
spec:
selector:
matchLabels:
run: ws-stack
replicas: 2
template:
metadata:
labels:
run: ws-stack
spec:
containers:
- name: ws-stack
image: wsstack/node/img
imagePullPolicy: IfNotPresent
ports:
- containerPort: 4000
Service.yaml
apiVersion: v1
kind: Service
metadata:
name: service-wsstack
labels:
run: service-wsstack
spec:
type: NodePort
ports:
- port: 80
protocol: TCP
nodePort: 30009
targetPort: 4000
selector:
run: ws-stack
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice
backend:
serviceName: service-wsstack --> backend
servicePort: 80
i setup deployment, service and ingress successfully. but when i called with curl
curl http://<minikubeip>/webservice --> Welcome to myApp! => Correct
curl http://<minikubeip>/webservice/data/office/getOffice --> Welcome to myApp! => Not correct
if i called another route, the result is the same 'Welcome to myApp'. But if i used Nodeport
curl http://<minikubeip>:30009/data/office/getOffice => 'Get Data Office', working properly.
What is the problem? any solution? Thank you

TL;DR
nginx.ingress.kubernetes.io/rewrite-target: /$2
path: /webservice($|/)(.*)
Explanation
The problem is from that line in your ingress:
nginx.ingress.kubernetes.io/rewrite-target: /
You're telling nginx to rewrite your url to / whatever it matched.
/webservice => /
/webservice/data/office/getOffice => /
To do what you're trying to do use regex, here is a simple example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice($|/)(.*)
backend:
serviceName: service-wsstack --> backend
servicePort: 80
This way you're asking nginx to rewrite your url with the second matching group.
Finally it gives you:
/webservice => /
/webservice/data/office/getOffice => /data/office/getOffice

Related

Error: connect ECONNREFUSED 127.0.0.1:443 http://ingress-nginx-controller.ingress-nginx.svc.cluster.local

kubectl get namespace
default Active 3h33m
ingress-nginx Active 3h11m
kube-node-lease Active 3h33m
kube-public Active 3h33m
kube-system Active 3h33m
kubectl get services -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.102.205.190 localhost 80:31378/TCP,443:31888/TCP 3h12m
ingress-nginx-controller-admission ClusterIP 10.103.97.209 <none> 443/TCP 3h12m
When I am making the request from nextjs getInitialProps http://ingress-nginx-controller.ingress-nginx.svc.cluster.local/api/users/currentuser then its throwing an error Error: connect ECONNREFUSED 127.0.0.1:443.
LandingPage.getInitialProps = async () => {
if (typeof window === "undefined") {
const { data } = await axios.get(
"http://ingress-nginx-controller.ingress-nginx.svc.cluster.local/api/users/currentuser",
{
headers: {
Host: "ticketing.dev",
},
}
);
return data;
} else {
const { data } = await axios.get("/api/users/currentuser");
return data;
}
};
My auth.deply.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: sajeebxn/auth
env:
- name: MONGO_URI
value: 'mongodb://tickets-mongo-srv:27017/auth'
- name: JWT_KEY
valueFrom:
secretKeyRef:
name: jwt-secret
key: JWT_KEY
---
apiVersion: v1
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
And my ingress-srv.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
tls:
- hosts:
- ticketing.dev
# secretName: e-ticket-secret
rules:
- host: ticketing.dev
http:
paths:
- path: /api/users/?(.*)
pathType: Prefix
backend:
service:
name: auth-srv
port:
number: 3000
- path: /?(.*)
pathType: Prefix
backend:
service:
name: client-srv
port:
number: 3000
Try using http://ingress-nginx-controller.ingress-nginx/api/users/currentuser.
This worked for me

error validating data: [ValidationError(Ingress.spec.rules[0].http.paths[0].backend): eks v1.22

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
alb.ingress.kubernetes.io/certificate-arn: "arn:aws:acm:us-west-1:0987654322:certificate/ee21bffb-ba4a-45a6-95a6-00551c1cfa32"
alb.ingress.kubernetes.io/healthcheck-interval-seconds: "15"
alb.ingress.kubernetes.io/healthcheck-port: traffic-port
alb.ingress.kubernetes.io/healthcheck-protocol: HTTP
alb.ingress.kubernetes.io/healthcheck-timeout-seconds: "5"
alb.ingress.kubernetes.io/healthy-threshold-count: "2"
alb.ingress.kubernetes.io/listen-ports: "[{"HTTPS":443}, {"HTTP":80}]"
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-TLS-1-1-2017-01
alb.ingress.kubernetes.io/ssl-redirect: "443"
alb.ingress.kubernetes.io/success-codes: "200"
alb.ingress.kubernetes.io/target-group-attributes: "stickiness.enabled=true,stickiness.lb_cookie.duration_seconds=60"
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/unhealthy-threshold-count: "2"
kubernetes.io/ingress.class: alb
name: alb-alb
namespace: deploy
spec:
rules:
-
http:
paths:
-
backend:
serviceName: service-svc
servicePort: 80
path: /*
Try:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
...
name: alb-alb
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: service-svc
port: # <-- Correction to your spec starts here
number: 80

Kubernetes: Serve paths from different service

I have two services one for serving static files and other for serving apis. I have created a single ingress controller for these.
I want to serve / from service1 and /api from service2. My services are running fine.
but I am getting 404 for /api path.
Below is my kubernetes yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: "myapp"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "service2"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
spec:
replicas: 2
selector:
matchLabels:
project: "myapp"
run: "service2"
matchExpressions:
- {key: project, operator: In, values: ["myapp"]}
template:
metadata:
labels:
project: "myapp"
env: "prod"
run: "service2"
spec:
securityContext:
sysctls:
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
imagePullSecrets:
- name: tildtr
containers:
- name: "node-container"
image: "images2"
imagePullPolicy: Always
ports:
- containerPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "service1"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
spec:
replicas: 2
selector:
matchLabels:
project: "myapp"
run: "service1"
matchExpressions:
- {key: project, operator: In, values: ["myapp"]}
template:
metadata:
labels:
project: "myapp"
env: "prod"
run: "service1"
spec:
securityContext:
sysctls:
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
imagePullSecrets:
- name: tildtr
containers:
- name: "nginx-container"
image: "image1"
imagePullPolicy: Always
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: "service1"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
run: "service1"
spec:
selector:
project: "myapp"
type: ClusterIP
ports:
- name: "service1"
port: 80
targetPort: 80
selector:
run: "service1"
---
apiVersion: v1
kind: Service
metadata:
name: "service2"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
run: "service2"
spec:
selector:
project: "myapp"
type: ClusterIP
ports:
- name: "service2"
port: 80
targetPort: 3000
selector:
run: "service2"
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: "myapp"
namespace: "myapp"
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-alias: "*.xyz.in"
nginx.ingress.kubernetes.io/server-snippet: |
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 50;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 20;
send_timeout 2;
types_hash_max_size 2048;
client_max_body_size 20M;
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/json;
gzip_disable "MSIE [1-6]\.";
spec:
rules:
- host: "myhost.in"
http:
paths:
- path: /api
backend:
serviceName: "service2"
servicePort: 80
- path: /
backend:
serviceName: "service1"
servicePort: 80
And this is my ingress desc.
Name: myapp
Namespace: myapp
Address: 10.100.160.106
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
ds-vaccination.timesinternet.in
/api service2:3000 (10.0.1.113:3000,10.0.2.123:3000)
/ service1:80 (10.0.1.37:80,10.0.2.59:80)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-alias: *.xyz.in
nginx.ingress.kubernetes.io/server-snippet:
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 50;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 20;
send_timeout 2;
types_hash_max_size 2048;
client_max_body_size 20M;
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/json;
gzip_disable "MSIE [1-6]\.";
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 6s (x5 over 31m) nginx-ingress-controller Scheduled for sync
Normal Sync 6s (x5 over 31m) nginx-ingress-controller Scheduled for sync
remove this annotation and try
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
if your cluster is supporting old API : extensions/v1beta1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: service-ingress
spec:
rules:
- host: service1.example.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: service2.example.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80

Kubernetes Websockets using Socket.io, ExpressJS and Nginx Ingress

I want to connect a React Native application using Socket.io to a server that is inside a Kubernetes Cluster hosted on Google Cloud Platform (GKE).
There seems to be an issue with the Nginx Ingress Controller declaration but I cannot find it.
I have tried adding nginx.org/websocket-services; rewriting my backend code so that it uses a separate NodeJS server (a simple HTTP server) on port 3004, then exposing it via the Ingress Controller under a different path than the one on port 3003; and multiple other suggestions from other SO questions and Github issues.
Information that might be useful:
Cluster master version: 1.15.11-gke.15
I use a Load Balancer managed with Helm (stable/nginx-ingress) with RBAC enabled
All deployments and services are within the namespace gitlab-managed-apps
The error I receive when trying to connect to socket.io is: Error: websocket error
For the front-end part, the code is as follows:
App.js
const socket = io('https://example.com/app-sockets/socketns', {
reconnect: true,
secure: true,
transports: ['websocket', 'polling']
});
I expect the above to connect me to a socket.io namespace called socketdns.
The backend code is:
app.js
const express = require('express');
const app = express();
const server = require('http').createServer(app);
const io = require('socket.io')(server);
const redis = require('socket.io-redis');
io.set('transports', ['websocket', 'polling']);
io.adapter(redis({
host: process.env.NODE_ENV === 'development' ? 'localhost' : 'redis-cluster-ip-service.gitlab-managed-apps.svc.cluster.local',
port: 6379
}));
io.of('/').adapter.on('error', function(err) { console.log('Redis Adapter error! ', err); });
const nsp = io.of('/socketns');
nsp.on('connection', function(socket) {
console.log('connected!');
});
server.listen(3003, () => {
console.log('App listening to 3003');
});
The ingress service is:
ingress-service.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
nginx.ingress.kubernetes.io/proxy-body-size: "100m"
certmanager.k8s.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/proxy-connect-timeout: "7200"
nginx.ingress.kubernetes.io/proxy-read-timeout: "7200"
nginx.ingress.kubernetes.io/proxy-send-timeout: "7200"
nginx.org/websocket-services: "app-sockets-cluster-ip-service"
name: ingress-service
namespace: gitlab-managed-apps
spec:
tls:
- hosts:
- example.com
secretName: letsencrypt-prod
rules:
- host: example.com
http:
paths:
- backend:
serviceName: app-cms-cluster-ip-service
servicePort: 3000
path: /?(.*)
- backend:
serviceName: app-users-cluster-ip-service
servicePort: 3001
path: /app-users/?(.*)
- backend:
serviceName: app-sockets-cluster-ip-service
servicePort: 3003
path: /app-sockets/?(.*)
- backend:
serviceName: app-sockets-cluster-ip-service
servicePort: 3003
path: /app-sockets/socketns/?(.*)
The solution is to remove the nginx.ingress.kubernetes.io/rewrite-target: /$1 annotation.
Here is a working configuration: (please note that apiVersion has changed since the question has been asked)
Ingress configuration
ingress-service.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
nginx.ingress.kubernetes.io/proxy-body-size: "64m"
cert-manager.io/cluster-issuer: "letsencrypt-prod"
name: ingress-service
namespace: default
spec:
tls:
- hosts:
- example.com
secretName: letsencrypt-prod
rules:
- host: example.com
http:
paths:
- backend:
service:
name: app-sockets-cluster-ip-service
port:
number: 3003
path: /app-sockets/?(.*)
pathType: Prefix
On the service (Express.js):
app.js
const redisAdapter = require('socket.io-redis');
const io = require('socket.io')(server, {
path: `${ global.NODE_ENV === 'development' ? '' : '/app-sockets' }/sockets/`,
cors: {
origin: '*',
methods: ['GET', 'POST'],
},
});
io.adapter(redisAdapter({
host: global.REDIS_HOST,
port: 6379,
}));
io.of('/').adapter.on('error', err => console.log('Redis Adapter error! ', err));
io.on('connection', () => {
//...
});
The global.NODE_ENV === 'development' ? '' : '/app-sockets' bit is related to an issue in development. If you change it here, you must also change it in the snippet below.
In development the service is under http://localhost:3003 (sockets endpoint is http://localhost:3003/sockets).
In production the service is under https://example.com/app-sockets (sockets endpoint is https://example.com/app-sockets/sockets).
On frontend
connectToWebsocketsService.js
/**
* Connect to a websockets service
* #param tokens {Object}
* #param successCallback {Function}
* #param failureCallback {Function}
*/
export const connectToWebsocketsService = (tokens, successCallback, failureCallback) => {
//SOCKETS_URL = NODE_ENV === 'development' ? 'http://localhost:3003' : 'https://example.com/app-sockets'
const socket = io(`${ SOCKETS_URL.replace('/app-sockets', '') }`, {
path: `${ NODE_ENV === 'development' ? '' : '/app-sockets' }/sockets/`,
reconnect: true,
secure: true,
transports: ['polling', 'websocket'], //required
query: {
// optional
},
auth: {
...generateAuthorizationHeaders(tokens), //optional
},
});
socket.on('connect', successCallback(socket));
socket.on('reconnect', successCallback(socket));
socket.on('connect_error', failureCallback);
};
Note: I wasn't able to do it on the project mentioned in the question, but I have on another project which is hosted on EKS, not GKE. Feel free to confirm if this works for you on GKE as well.
Just change annotations to
nginx.ingress.kubernetes.io/websocket-services: "app-sockets-cluster-ip-service"
instead of
nginx.org/websocket-services: "app-sockets-cluster-ip-service"
Mostly it will resolve your issue.

Tarefik Ingress does not return HTTP/HTTPS response.

My website doesn't generate any HTTP certs using LetsEncrypt, doesn't even return a response. This is for test purposes so is using a basic NGINX image that I want to serve over HTTPS.
I have an ingress type that looks like this:
apiVersion: v1
kind: Service
metadata:
name: nginxtest
labels:
app: nginx
spec:
selector:
k8s-app: traefik-ingress-lb
ports:
- name: http
port: 80
targetPort: 80
protocol: TCP
selector:
app: nginx
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: nginx-ingress
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: nginx.example.com
http:
paths:
- path: /
backend:
serviceName: nginxtest
servicePort: http
My TOML Config Map looks like the below:
apiVersion: v1
kind: ConfigMap
metadata:
name: traefik-conf
namespace: kube-system
data:
traefik.toml: |
defaultEntryPoints = ["http","https"]
debug = false
logLevel = "ERROR"
#Config to redirect http to https
[entryPoints]
[entryPoints.http]
address = ":80"
compress = true
[entryPoints.http.redirect]
entryPoint = "https"
[entryPoints.https]
address = ":443"
compress = true
[entryPoints.https.tls]
[api]
[api.statistics]
recentErrors = 10
[kubernetes]
[ping]
entryPoint = "http"
[accessLog]
[acme]
email = "a#b.com"
storage = "certs/acme.json"
acmeLogging = true
entryPoint = "https"
OnHostRule = true
#caServer = "https://acme-staging-v02.api.letsencrypt.org/directory"
[acme.httpChallenge]
entryPoint="http"
Any help is appreciated, please point out any errors.