I have two services one for serving static files and other for serving apis. I have created a single ingress controller for these.
I want to serve / from service1 and /api from service2. My services are running fine.
but I am getting 404 for /api path.
Below is my kubernetes yaml
---
apiVersion: v1
kind: Namespace
metadata:
name: "myapp"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "service2"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
spec:
replicas: 2
selector:
matchLabels:
project: "myapp"
run: "service2"
matchExpressions:
- {key: project, operator: In, values: ["myapp"]}
template:
metadata:
labels:
project: "myapp"
env: "prod"
run: "service2"
spec:
securityContext:
sysctls:
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
imagePullSecrets:
- name: tildtr
containers:
- name: "node-container"
image: "images2"
imagePullPolicy: Always
ports:
- containerPort: 3000
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: "service1"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
spec:
replicas: 2
selector:
matchLabels:
project: "myapp"
run: "service1"
matchExpressions:
- {key: project, operator: In, values: ["myapp"]}
template:
metadata:
labels:
project: "myapp"
env: "prod"
run: "service1"
spec:
securityContext:
sysctls:
- name: net.ipv4.ip_local_port_range
value: "1024 65535"
imagePullSecrets:
- name: tildtr
containers:
- name: "nginx-container"
image: "image1"
imagePullPolicy: Always
ports:
- containerPort: 3000
---
apiVersion: v1
kind: Service
metadata:
name: "service1"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
run: "service1"
spec:
selector:
project: "myapp"
type: ClusterIP
ports:
- name: "service1"
port: 80
targetPort: 80
selector:
run: "service1"
---
apiVersion: v1
kind: Service
metadata:
name: "service2"
namespace: "myapp"
labels:
project: "myapp"
env: "prod"
run: "service2"
spec:
selector:
project: "myapp"
type: ClusterIP
ports:
- name: "service2"
port: 80
targetPort: 3000
selector:
run: "service2"
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: "myapp"
namespace: "myapp"
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-alias: "*.xyz.in"
nginx.ingress.kubernetes.io/server-snippet: |
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 50;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 20;
send_timeout 2;
types_hash_max_size 2048;
client_max_body_size 20M;
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/json;
gzip_disable "MSIE [1-6]\.";
spec:
rules:
- host: "myhost.in"
http:
paths:
- path: /api
backend:
serviceName: "service2"
servicePort: 80
- path: /
backend:
serviceName: "service1"
servicePort: 80
And this is my ingress desc.
Name: myapp
Namespace: myapp
Address: 10.100.160.106
Default backend: default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
Rules:
Host Path Backends
---- ---- --------
ds-vaccination.timesinternet.in
/api service2:3000 (10.0.1.113:3000,10.0.2.123:3000)
/ service1:80 (10.0.1.37:80,10.0.2.59:80)
Annotations: nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/server-alias: *.xyz.in
nginx.ingress.kubernetes.io/server-snippet:
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 50;
keepalive_requests 100000;
reset_timedout_connection on;
client_body_timeout 20;
send_timeout 2;
types_hash_max_size 2048;
client_max_body_size 20M;
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/xml application/json;
gzip_disable "MSIE [1-6]\.";
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Sync 6s (x5 over 31m) nginx-ingress-controller Scheduled for sync
Normal Sync 6s (x5 over 31m) nginx-ingress-controller Scheduled for sync
remove this annotation and try
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
if your cluster is supporting old API : extensions/v1beta1
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: service-ingress
spec:
rules:
- host: service1.example.com
http:
paths:
- backend:
serviceName: service1
servicePort: 80
- host: service2.example.com
http:
paths:
- backend:
serviceName: service2
servicePort: 80
Related
kubectl get namespace
default Active 3h33m
ingress-nginx Active 3h11m
kube-node-lease Active 3h33m
kube-public Active 3h33m
kube-system Active 3h33m
kubectl get services -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP
PORT(S) AGE
ingress-nginx-controller LoadBalancer 10.102.205.190 localhost 80:31378/TCP,443:31888/TCP 3h12m
ingress-nginx-controller-admission ClusterIP 10.103.97.209 <none> 443/TCP 3h12m
When I am making the request from nextjs getInitialProps http://ingress-nginx-controller.ingress-nginx.svc.cluster.local/api/users/currentuser then its throwing an error Error: connect ECONNREFUSED 127.0.0.1:443.
LandingPage.getInitialProps = async () => {
if (typeof window === "undefined") {
const { data } = await axios.get(
"http://ingress-nginx-controller.ingress-nginx.svc.cluster.local/api/users/currentuser",
{
headers: {
Host: "ticketing.dev",
},
}
);
return data;
} else {
const { data } = await axios.get("/api/users/currentuser");
return data;
}
};
My auth.deply.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: auth-depl
spec:
replicas: 1
selector:
matchLabels:
app: auth
template:
metadata:
labels:
app: auth
spec:
containers:
- name: auth
image: sajeebxn/auth
env:
- name: MONGO_URI
value: 'mongodb://tickets-mongo-srv:27017/auth'
- name: JWT_KEY
valueFrom:
secretKeyRef:
name: jwt-secret
key: JWT_KEY
---
apiVersion: v1
kind: Service
metadata:
name: auth-srv
spec:
selector:
app: auth
ports:
- name: auth
protocol: TCP
port: 3000
targetPort: 3000
And my ingress-srv.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
tls:
- hosts:
- ticketing.dev
# secretName: e-ticket-secret
rules:
- host: ticketing.dev
http:
paths:
- path: /api/users/?(.*)
pathType: Prefix
backend:
service:
name: auth-srv
port:
number: 3000
- path: /?(.*)
pathType: Prefix
backend:
service:
name: client-srv
port:
number: 3000
Try using http://ingress-nginx-controller.ingress-nginx/api/users/currentuser.
This worked for me
I have two containers, one running phpfpm and the other running nginx. I'd like the container running nginx to access the webroot files which has css and javascript files. Right now, nginx successfully passes off php requests to phpfpm, but no styles are showing up for instance when the webpage is rendered.
This is running off of minikube on a linux system.
kind: ConfigMap
apiVersion: v1
metadata:
name: php-ini
namespace: mixerapi-docker
data:
php.ini: |
extension=intl.so
extension=pdo_mysql.so
extension=sodium
extension=zip.so
zend_extension=opcache.so
[php]
session.auto_start = Off
short_open_tag = Off
opcache.interned_strings_buffer = 16
opcache.max_accelerated_files = 20000
opcache.memory_consumption = 256
realpath_cache_size = 4096K
realpath_cache_ttl = 600
expose_php = off
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-conf
namespace: mixerapi-docker
data:
default.conf: |-
server {
listen 80;
root /srv/app/webroot/;
index index.php;
location / {
try_files $uri /index.php?$args;
}
location ~ \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
include fastcgi_params;
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: php-nginx-deployment
namespace: mixerapi-docker
labels:
app: php-nginx
spec:
replicas: 1
selector:
matchLabels:
app: php-nginx
template:
metadata:
labels:
app: php-nginx
spec:
containers:
- name: php
image: mixerapidev/demo:latest
imagePullPolicy: Always
ports:
- containerPort: 9000
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: php-secret
key: database-url
- name: SECURITY_SALT
valueFrom:
secretKeyRef:
name: php-secret
key: cakephp-salt
volumeMounts:
- name: php-ini
mountPath: /usr/local/etc/php/conf.d
- name: php-application
mountPath: /application
- name: nginx
image: nginx:1.19-alpine
ports:
- containerPort: 80
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: php-application
mountPath: /application
volumes:
- name: php-ini
configMap:
name: php-ini
- name: php-application
persistentVolumeClaim:
claimName: php-application-pv-claim
- name: nginx-conf
configMap:
name: nginx-conf
items:
- key: default.conf
path: default.conf
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: php-application-pv-claim
namespace: mixerapi-docker
labels:
app: php-nginx
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: mixerapi-docker
spec:
selector:
app: php-nginx
type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30002
---
apiVersion: v1
kind: Service
metadata:
name: php
namespace: mixerapi-docker
spec:
selector:
app: php-nginx
ports:
- protocol: TCP
port: 9000
targetPort: 9000
Not sure if this is the best approach, but I based it off of this:
How to share files between containers in the same kubernetes pod
kind: ConfigMap
apiVersion: v1
metadata:
name: php-ini
namespace: mixerapi-docker
data:
php.ini: |
extension=intl.so
extension=pdo_mysql.so
extension=sodium
extension=zip.so
zend_extension=opcache.so
[php]
session.auto_start = Off
short_open_tag = Off
opcache.interned_strings_buffer = 16
opcache.max_accelerated_files = 20000
opcache.memory_consumption = 256
realpath_cache_size = 4096K
realpath_cache_ttl = 600
expose_php = off
---
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-conf
namespace: mixerapi-docker
data:
default.conf: |-
server {
listen 80;
root /application/webroot/;
index index.php;
location / {
try_files $uri /index.php?$args;
}
location ~ \.php$ {
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_buffers 16 16k;
fastcgi_buffer_size 32k;
include fastcgi_params;
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: php-nginx-deployment
namespace: mixerapi-docker
labels:
app: php-nginx
spec:
replicas: 1
selector:
matchLabels:
app: php-nginx
template:
metadata:
labels:
app: php-nginx
spec:
containers:
- name: php
image: mixerapidev/demo:latest
imagePullPolicy: Always
ports:
- containerPort: 9000
env:
- name: DATABASE_URL
valueFrom:
secretKeyRef:
name: php-secret
key: database-url
- name: SECURITY_SALT
valueFrom:
secretKeyRef:
name: php-secret
key: cakephp-salt
volumeMounts:
- name: php-ini
mountPath: /usr/local/etc/php/conf.d
- name: application
mountPath: /application
lifecycle:
postStart:
exec:
command:
- "/bin/sh"
- "-c"
- >
cp -r /srv/app/. /application/.
- name: nginx
image: nginx:1.19-alpine
ports:
- containerPort: 80
volumeMounts:
- name: nginx-conf
mountPath: /etc/nginx/conf.d
- name: application
mountPath: /application
volumes:
- name: php-ini
configMap:
name: php-ini
- name: nginx-conf
configMap:
name: nginx-conf
items:
- key: default.conf
path: default.conf
- name: application
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
name: nginx
namespace: mixerapi-docker
spec:
selector:
app: php-nginx
type: LoadBalancer
ports:
- protocol: TCP
port: 80
targetPort: 80
nodePort: 30002
---
apiVersion: v1
kind: Service
metadata:
name: php
namespace: mixerapi-docker
spec:
selector:
app: php-nginx
ports:
- protocol: TCP
port: 9000
targetPort: 9000
I had composer install running in a docker entrypoint, so I needed to move that into the Dockerfile for this to work. I updated my Dockerfile with this and removed install from my entrypoint if the ENV is prod:
RUN if [[ "$ENV" = "prod" ]]; then \
composer install --prefer-dist --no-interaction --no-dev; \
fi
I have everything working using a classic load balancer. I would now like to update my Kubernetes environment to use an application load balancer instead of the classic load balancer. I have tried a few tutorials but no luck so far. I keep getting 503 errors after I deploy.
I brought my cluster up with eksctl then installed and ran the sample application in this tutorial https://docs.aws.amazon.com/eks/latest/userguide/alb-ingress.html. I did get an ALB up and all worked as it should with the sample application outlined in the tutorial. I tried modifying the YAML for my environment to use an ALB and keep getting 503 errors. I am not sure what to try next.
I suspect my issue might be that I have the Nginx and my application in the same container(which I would like to keep if possible).
Here is the YAML for my application that I updated to try to get the ALB working:
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
data:
nginx.conf: |
events {
}
http {
include /etc/nginx/mime.types;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name test-ggg.com www.test-ggg.com;
if ($http_x_forwarded_proto = "http") {
return 301 https://$server_name$request_uri;
}
root /var/www/html;
index index.php index.html;
location static {
alias /var/www/html;
}
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
location / {
try_files $uri $uri/ /index.php?$query_string;
gzip_static on;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
labels:
name: deployment
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 2
selector:
matchLabels:
name: templated-pod
template:
metadata:
name: deployment-template
labels:
name: templated-pod
spec:
volumes:
- name: app-files
emptyDir: {}
- name: nginx-config-volume
configMap:
name: nginx-config
containers:
- image: xxxxxxx.dkr.ecr.us-east-2.amazonaws.com/test:4713
name: app
volumeMounts:
- name: app-files
mountPath: /var/www/html
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "cp -r /var/www/public/. /var/www/html"]
resources:
limits:
cpu: 100m
requests:
cpu: 50m
- image: nginx:alpine
name: nginx
volumeMounts:
- name: app-files
mountPath: /var/www/html
- name: nginx-config-volume
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
resources:
limits:
cpu: 100m
requests:
cpu: 50m
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: "service-alb"
namespace: default
annotations:
alb.ingress.kubernetes.io/target-group-attributes: slow_start.duration_seconds=45
alb.ingress.kubernetes.io/healthcheck-interval-seconds: '5'
alb.ingress.kubernetes.io/healthcheck-timeout-seconds: '2'
alb.ingress.kubernetes.io/healthy-threshold-count: '2'
alb.ingress.kubernetes.io/unhealthy-threshold-count: '3'
spec:
ports:
- port: 80
targetPort: 80
name: http
type: NodePort
selector:
app: templated-pod
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: app-ingress
namespace: default
annotations:
kubernetes.io/ingress.class: alb
alb.ingress.kubernetes.io/target-type: ip
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:dddddddd:certificate/f61c2837-484c-ddddddddd-bab7c4d4452c
alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}]'
alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
labels:
app: app-ingress
spec:
rules:
- host: test-ggg.com
http:
paths:
- backend:
serviceName: "service-alb"
servicePort: 80
path: /*
Here is the yaml with the classic load balancer. Everything works when I use this :
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
data:
nginx.conf: |
events {
}
http {
include /etc/nginx/mime.types;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name test-ggg.com www.test-ggg.com;
if ($http_x_forwarded_proto = "http") {
return 301 https://$server_name$request_uri;
}
root /var/www/html;
index index.php index.html;
location static {
alias /var/www/html;
}
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_param PATH_INFO $fastcgi_path_info;
}
location / {
try_files $uri $uri/ /index.php?$query_string;
gzip_static on;
}
}
}
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: deployment
labels:
name: deployment
spec:
replicas: 1
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 2
selector:
matchLabels:
name: templated-pod
template:
metadata:
name: deployment-template
labels:
name: templated-pod
spec:
volumes:
- name: app-files
emptyDir: {}
- name: nginx-config-volume
configMap:
name: nginx-config
containers:
- image: 99ddddddddd.dkr.ecr.us-east-2.amazonaws.com/test:4713
name: app
volumeMounts:
- name: app-files
mountPath: /var/www/html
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "cp -r /var/www/public/. /var/www/html"]
resources:
limits:
cpu: 100m
requests:
cpu: 50m
- image: nginx:alpine
name: nginx
volumeMounts:
- name: app-files
mountPath: /var/www/html
- name: nginx-config-volume
mountPath: /etc/nginx/nginx.conf
subPath: nginx.conf
resources:
limits:
cpu: 100m
requests:
cpu: 50m
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: service-loadbalancer
annotations:
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:us-east-2:dddddddd:certificate/f61c2837-484c-4fac-a26c-dddddddd4452c
spec:
selector:
name: templated-pod
ports:
- name: http
port: 80
targetPort: 80
- name: https
port: 443
targetPort: 80
type: LoadBalancer
After some tutorial help I learned more about services, selectors and pod naming!
(Great tutorial - https://www.youtube.com/watch?v=sGZx3OjMPQI)
I had the pod named - "name: templated-pod"
I had the selector in the service looking for:
selector:
app: templated-pod
It could not make the connection!
I changed the selector to the following and it worked :
selector:
name: templated-pod
Hope this helps others!!
I have problem setting path ingress controller for backend service. For example i want setup :
frontend app with angular (Path :/)
backend service with NodeJs (Path :/webservice).
NodeJS : Index.js
const express = require('express')
const app = express()
const port = 4000
app.get('/', (req, res) => res.send('Welcome to myApp!'))
app.use('/data/office', require('./roffice'));
app.listen(port, () => console.log(`Example app listening on port ${port}!`))
Another Route:roffice.js
var express = require('express')
var router = express.Router()
router.get('/getOffice', async function (req, res) {
res.send('Get Data Office')
});
module.exports = router
Deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: ws-stack
spec:
selector:
matchLabels:
run: ws-stack
replicas: 2
template:
metadata:
labels:
run: ws-stack
spec:
containers:
- name: ws-stack
image: wsstack/node/img
imagePullPolicy: IfNotPresent
ports:
- containerPort: 4000
Service.yaml
apiVersion: v1
kind: Service
metadata:
name: service-wsstack
labels:
run: service-wsstack
spec:
type: NodePort
ports:
- port: 80
protocol: TCP
nodePort: 30009
targetPort: 4000
selector:
run: ws-stack
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice
backend:
serviceName: service-wsstack --> backend
servicePort: 80
i setup deployment, service and ingress successfully. but when i called with curl
curl http://<minikubeip>/webservice --> Welcome to myApp! => Correct
curl http://<minikubeip>/webservice/data/office/getOffice --> Welcome to myApp! => Not correct
if i called another route, the result is the same 'Welcome to myApp'. But if i used Nodeport
curl http://<minikubeip>:30009/data/office/getOffice => 'Get Data Office', working properly.
What is the problem? any solution? Thank you
TL;DR
nginx.ingress.kubernetes.io/rewrite-target: /$2
path: /webservice($|/)(.*)
Explanation
The problem is from that line in your ingress:
nginx.ingress.kubernetes.io/rewrite-target: /
You're telling nginx to rewrite your url to / whatever it matched.
/webservice => /
/webservice/data/office/getOffice => /
To do what you're trying to do use regex, here is a simple example:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: stack-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/use-regex: "true"
spec:
rules:
- host: hello-world.info
- http:
paths:
- path: /
backend:
serviceName: service-ngstack --> frondend
servicePort: 80
- path: /webservice($|/)(.*)
backend:
serviceName: service-wsstack --> backend
servicePort: 80
This way you're asking nginx to rewrite your url with the second matching group.
Finally it gives you:
/webservice => /
/webservice/data/office/getOffice => /data/office/getOffice
I was able to achieve load-balancing with sample istio applications
https://github.com/piomin/sample-istio-services
https://istio.io/docs/guides/bookinfo/
But was not able to get istio load-balancing working with single private service having 2 versions. Example: 2 consul servers with different versions .
Service and pod definition :
apiVersion: v1
kind: Service
metadata:
name: consul-test
labels:
app: test
spec:
ports:
- port: 8500
name: http
selector:
app: test
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: consul-test-v1
spec:
replicas: 1
template:
metadata:
labels:
app: test
version: v1
spec:
containers:
- name: consul-test-v1
image: consul:latest
ports:
- containerPort: 8500
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: consul-test-v2
spec:
replicas: 1
template:
metadata:
labels:
app: test
version: v2
spec:
containers:
- name: consul-test-v2
image: consul:1.1.0
ports:
- containerPort: 8500
Gateway definition:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: http-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: con-gateway
spec:
hosts:
- "*"
gateways:
- http-gateway
http:
- match:
- uri:
exact: /catalog
route:
- destination:
host: consul-test
port:
number: 8500
Routing rules in virtual service:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: consul-test
spec:
hosts:
- consul-test
gateways:
- con-gateway
- mesh
http:
- route:
- destination:
host: consul-test
subset: v1
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: consul-test
spec:
host: consul-test
subsets:
- name: v1
labels:
version: v1
- name: v2
labels:
version: v2
Though I route all traffic ( http requests ) to consul server version v1, my http requests on consul-service lands on v1 and v2 alternately i.e, it follows Round-Robin rule .
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
consul-test ClusterIP 10.97.200.140 <none> 8500/TCP 9m
$ curl -L http://10.97.200.140:8500/v1/catalog/nodes
[
{
"ID": "ebfa341b-4557-a392-9f8a-8ee307113faa",
"Node": "consul-test-v1-765dd566dd-6cmj9",
"Address": "127.0.0.1",
"Datacenter": "dc1",
"TaggedAddresses": {
"lan": "127.0.0.1",
"wan": "127.0.0.1"
},
"Meta": {
"consul-network-segment": ""
},
"CreateIndex": 9,
"ModifyIndex": 10
}
]
$ curl -L http://10.97.200.140:8500/v1/catalog/nodes
[
{
"ID": "1b60a5bd-9a17-ff18-3a65-0ff95b3a836a",
"Node": "consul-test-v2-fffd475bc-st4mv",
"Address": "127.0.0.1",
"Datacenter": "dc1",
"TaggedAddresses": {
"lan": "127.0.0.1",
"wan": "127.0.0.1"
},
"Meta": {
"consul-network-segment": ""
},
"CreateIndex": 5,
"ModifyIndex": 6
}
]
I have the above mentioned issue when curl is done on the service ClusterIP:ClusterPort
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
consul-test ClusterIP 10.97.200.140 <none> 8500/TCP 9m
$ curl -L http://10.97.200.140:8500/v1/catalog/nodes
But LoadBalancing works as expected when curl is done on INGRESS_HOST and INGRESS_PORT ( determining INGRESS_HOST and INGRESS_PORT present here )
$ curl -L http://$INGRESS_HOST:$INGRESS_PORT/v1/catalog/nodes --- WORKS