We have set up Istio service mesh on a Kubernetes cluster. Over this, we are attempting to deploy a spring boot REST application that connects to an external database (MySQL).
When we try to use the Istio Egress gateway, we get an UnknownHostException, whereas using just the service entry works fine.
Sharing the YAML file for Egress gateway approach. Let us know if anything is missing.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: mysql
namespace: istio-system
spec:
hosts:
- dbhost.svc.local
addresses:
- X.X.X.X/32
ports:
- number: 3306
name: tcp
protocol: TCP
location: MESH_EXTERNAL
resolution: STATIC
endpoints:
- address: X.X.X.X
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
namespace: istio-system
spec:
selector:
istio: egressgateway
servers:
- port:
number: 777
name: tcp
protocol: TCP
hosts:
- dbhost.svc.local
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-mysql
namespace: istio-system
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: mysql
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: mysql
namespace: istio-system
spec:
host: dbhost.svc.local
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-mysql-through-egress-gateway
namespace: istio-system
spec:
hosts:
- dbhost.svc.local
gateways:
- mesh
- istio-egressgateway
tcp:
- match:
- gateways:
- mesh
destinationSubnets:
- X.X.X.X/32
port: 3306
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: mysql
port:
number: 777
- match:
- gateways:
- istio-egressgateway
port: 777
route:
- destination:
host: dbhost.svc.local
port:
number: 3306
weight: 100
We can ping the DB DNS from cluster VM, but it's not getting resolved at pod level. Because of this we are using the DB hostname (dbhost.svc.local:3306) in application.properties. Below is the error we encounter with this configuration:
java.net.UnknownHostException: dbhost.svc.local
Egress gateway details are as:
kubectl get svc istio-egressgateway -n istio-system -o wide
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
istio-egressgateway ClusterIP X.X.X.X <none> 80/TCP,443/TCP,777/TCP 26d app=istio-egressgateway,istio=egressgateway
Related
I have a local website.
The website was created by a docker-compose and it is listening on a localhost port 3000.
When I try:
curl 127.0.0.1:3000
I can see the response.
What I did:
From my domain provider I edited the DNS to point to my server, then I changed nginx-ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: virtual-host-ingress
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
cert-manager.io/cluster-issuer: "letsencrypt-pp"
spec:
tls:
- hosts:
- nextformulainvesting.com
secretName: ***
rules:
- host: "nextformulainvesting.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: e-frontend-saleor
port:
number: 80
and I created the service:
apiVersion: v1
kind: Service
metadata:
name: e-frontend-saleor
spec:
ports:
- protocol: TCP
port: 80
targetPort: 3000
But with the service or without the service I receive the error 503 Service Temporarily Unavailable.
How can I use nginx-ingress to point to my local TCP service?
To clarify the issue I am posting a community wiki answer.
The answer that helped to resolve this issue is available at this link. Based on that - the clue of the case is to create manually a Service and an Endpoint objects for external server.
After that one can create an Ingress object that will point to Service external-ip with adequate port .
Here are the examples of objects provided in similar question.
Service and an Endpoint objects:
apiVersion: v1
kind: Service
metadata:
name: external-ip
spec:
ports:
- name: app
port: 80
protocol: TCP
targetPort: 5678
clusterIP: None
type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
name: external-ip
subsets:
- addresses:
- ip: 10.0.40.1
ports:
- name: app
port: 5678
protocol: TCP
Ingress object:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: external-service
spec:
rules:
- host: service.example.com
http:
paths:
- backend:
serviceName: external-ip
servicePort: 80
path: /
See also this reference.
Your service that you have created is for forwarding the traffic to deployments
As your service is running out side of Kubernetes cluster you should be using the Endpoint in this case
apiVersion: v1
kind: Endpoints
metadata:
name: my-service
subsets:
- addresses:
- IP: <External IP>
ports:
- port: 3000
and you can use this Endpoint to ingress so that it will route the traffic.
Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: virtual-host-ingress
namespace: ingress-basic
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "true"
cert-manager.io/cluster-issuer: "letsencrypt-pp"
spec:
tls:
- hosts:
- nextformulainvesting.com
secretName: ***
rules:
- host: "nextformulainvesting.com"
http:
paths:
- pathType: Prefix
path: "/"
backend:
service:
name: my-service
port:
number: 3000
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: test-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: test-virtualservice
spec:
hosts:
- "*"
gateways:
- test-gateway
http:
- match:
- uri:
exact: /test-charts
route:
- destination:
host: test-charts
port:
number: 9080
Note: But i could able to access the application with LB ip
ubuntu#ip-172-31-100-164:~/istio-1.9.0$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
test-charts LoadBalancer 10.0.101.148 20.62.141.231 80:31480/TCP 6h19m
I'm using redis with k8s 1.15.0, istio 1.4.3, it works well inside the network.
However when I tryed to use the istio gateway and sidecar to expose it to outside network, it failed.
Then I removed the istio sidecar and just started the redis server in k8s, it worked.
After searching I added DestinationRule to the config, but it didn't help.
So what's the problem of it? Thanks for any tips!
Here is my redis.yml
apiVersion: apps/v1
kind: Deployment
metadata:
name: redis
spec:
replicas: 1
selector:
matchLabels:
app: redis
template:
metadata:
labels:
app: redis
spec:
containers:
- name: redis
image: docker.io/redis:5.0.5-alpine
imagePullPolicy: IfNotPresent
ports:
- containerPort: 16379
protocol: TCP
name: redis-port
volumeMounts:
- name: redis-data
mountPath: /data
- name: redis-conf
mountPath: /etc/redis
command:
- "redis-server"
args:
- "/etc/redis/redis.conf"
- "--protected-mode"
- "no"
volumes:
- name: redis-conf
configMap:
name: redis-conf
items:
- key: redis.conf
path: redis.conf
- name: redis-data
nfs:
path: /data/redis
server: 172.16.8.34
---
apiVersion: v1
kind: Service
metadata:
name: redis-svc
labels:
app: redis-svc
spec:
type: ClusterIP
ports:
- name: redis-port
port: 16379
protocol: TCP
selector:
app: redis
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: redis-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: tcp
protocol: TCP
hosts:
- "redis.basic.svc.cluster.local"
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: redis-svc
spec:
host: redis-svc
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: redis-vs
spec:
hosts:
- "redis.basic.svc.cluster.local"
gateways:
- redis-gateway
tcp:
- route:
- destination:
host: redis-svc.basic.svc.cluster.local
port:
number: 16379
Update:
This is how I request
[root]# redis-cli -h redis.basic.svc.cluster.local -p 80
redis.basic.svc.cluster.local:80> get Test
Error: Protocol error, got "H" as reply type byte
There are few thing that need to be different in case of exposing TCP application with istio.
The hosts: needs to be "*" as TCP protocol works only with IP:PORT. There are no headers in L4.
There needs to be TCP port match Your VirtualService that matches GateWay. I suggest to name it in a unique way and match Deployment port name.
I suggest avoiding using port 80 as it is already used in default ingress configuration and it could result in port conflict, so i changed it to 11337.
So Your GateWay should look something like this:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: redis-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 11337
name: redis-port
protocol: TCP
hosts:
- "*"
And Your VirtualService like this:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: redis-vs
spec:
hosts:
- "*"
gateways:
- redis-gateway
tcp:
- match:
- port: 11337
route:
- destination:
host: redis-svc
port:
number: 16379
Note that I removed namespaces for clarity.
Then add our custom port to default ingress gateway use the following command:
kubectl edit svc istio-ingressgateway -n istio-system
And add following next other port definitions:
- name: redis-port
nodePort: 31402
port: 11337
protocol: TCP
targetPort: 16379
To access the exposed application use istio gateway external IP and port that we
just set up.
To get Your gateway external IP you can use:
export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
redis-cli -h $INGRESS_HOST -p 11337
If Your istio-ingressgateway does not have external IP assigned, use one of Your nodes IP address and port 31402.
Hope this helps.
Thanks for suren's answer.
But i think redis.basic.svc.cluster.local is outside DNS host to match by VirtualService, and VirtualService.host is route to service redis-svc with full namespace path.
Maybe not for that reason.
I'm new to istio, and I want to access my app through istio ingress gateway, but I do not know why it does not work.
This is my kubenetes_deploy.yaml file content:
apiVersion: v1
kind: Service
metadata:
name: batman
labels:
run: batman
spec:
#type: NodePort
ports:
- port: 8000
#nodePort: 32000
targetPort: 7000
#protocol: TCP
name: batman
selector:
run: batman
#version: v1
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: batman-v1
spec:
replicas: 1
selector:
matchLabels:
run: batman
template:
metadata:
labels:
run: batman
version: v1
spec:
containers:
- name: batman
image: leowu/batman:v1
ports:
- containerPort: 7000
env:
- name: MONGODB_URL
value: mongodb://localhost:27017/articles_demo_dev
- name: mongo
image: mongo
And here is my istio ingress_gateway.yaml config file:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: batman-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 15000
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: batman
spec:
hosts:
- "*"
gateways:
- batman-gateway
http:
- match:
route:
- destination:
host: batman
port:
number: 7000
I created the ingress gateway from example, and it looks well but when I run kubectl get svc istio-ingressgateway -n istio-system I can't see the listening port 15000 in the output。I donot know way.
Is there anyone can help me? Thanks.
First of all, as #Abhyudit Jain mentioned you need to correct port in VirtualService to 8000
And then you just add another port to your istio-ingressgateway service
kubectl edit svc istio-ingressgateway -n istio-system
add section:
ports:
- name: http
nodePort: 30001
port: 15000
protocol: TCP
targetPort: 80
This will accept HTTP traffic on port 15000 and rout it to your destination service on port 8000
simple schema as follows:
incoming traffic --> istio-gateway service --> istio-gateway --> virtual service --> service --> pod
You batman service listens on port 8000 and forwards traffic to container's port 7000.
The istio traffic works like this:
ingress-gateway -> virtual-service -> destination-rule [optional] -> service
So your virtual service should be like:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: batman
spec:
hosts:
- "*"
gateways:
- batman-gateway
http:
- match:
route:
- destination:
host: batman
port:
number: 8000 <--- change
I have a web service running on a port on my local network exposed at port 6003. I also have a Kubernetes Cluster running on a different machine on the same network that uses and Nginx Ingress to proxy to all the services in the cluster. How can I set up an ingress to proxy to the machine? I had a set up that worked. But now, I am either getting DNS errors on the nginx pod or the response times out in the browser and nothing happens.
Here is the manifest I have been using.
apiVersion: v1
kind: Service
metadata:
name: myservice-service
spec:
type: ExternalName
externalName: 192.xxx.xx.x
ports:
- name: myservice
port: 80
protocol: TCP
targetPort: 6003
---
apiVersion: v1
kind: Endpoints
metadata:
name: myservice-ip
subsets:
- addresses:
# list all external ips for this service
- ip: 192.xxx.xx.x
ports:
- name: myservice
port: 6003
protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: service.example.com
annotations:
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
spec:
rules:
- host: service.example.com
http:
paths:
- backend:
serviceName: myservice-service
servicePort: 80
path: /
tls:
- secretName: secret-prod-tls
hosts:
- service.example.com
Edit for more information:
This manifest does work. What I realized is that you must specify https even though the ingress has a tls block. This still is showing Lua DNS errors in the Nginx-ingress pod though.
You MUST specify nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" in your ingress resource if upstream listening for HTTPS requests. So, this is related to backend, not ingress itself.
TLS configuration is for Ingress (frontend), and not for backend application.
You don't need ExternalName here. Usual headless service will do the job:
apiVersion: v1
kind: Service
metadata:
name: external-ip
spec:
ports:
- name: http
port: 80
clusterIP: None
type: ClusterIP
---
apiVersion: v1
kind: Endpoints
metadata:
name: external-ip
subsets:
- addresses:
- ip: 172.17.0.5
ports:
- name: http
port: 80