My app deployed in openshift cluster needs to connect to 2 external kafka brokers. Since the application is on the istio mesh, all outbound traffic must go through the egress gateway. The connection to kafka is via the log4j2 appender over SSL.
I made the following istio config:
kind: ServiceEntry
metadata:
name: se-kafka
spec:
hosts:
- kafka1.host.com
- kafka2.host.com
addresses:
- 10.200.200.1
- 10.200.200.2
ports:
- name: kafka-port
number: 9093
protocol: TCP
location: MESH_EXTERNAL
resolution: NONE
exportTo:
- .
=====================
kind: DestinationRule
metadata:
name: dr-kafka
spec:
host: egressgateway #name egressgateway deployment
subnets:
- name: se-kafka
=====================
kind: Gateway
metadata:
name: gw-kafka
spec:
servers:
- hosts:
- kafka1.host.com
port:
name: kafka1-egress-port
number: 16001
protocol: TCP
- hosts:
- kafka2.host.com
port:
name: kafka2-egress-port
number: 16002
protocol: TCP
selector:
istio: egressgateway
=======================
kind: VirtualService
metadata:
name: vs-kafka
spec:
hosts:
- kafka1.host.com
- kafka2.host.com
gateways:
- mesh
- gw-kafka
tls:
- match:
- gateways:
- mesh
port: 9093
sniHosts:
- kafka1.host.com
route:
- destination:
host: egressgateway
port:
number: 16001
- match:
- gateways:
- mesh
port: 9093
sniHosts:
- kafka2.host.com
route:
- destination:
host: egressgateway
port:
number: 16002
- match:
- gateways:
- gw-kafka
port: 16001
sniHosts:
- kafka1.host.com
route:
- destination:
host: kafka1.host.com
port:
number: 9093
- match:
- gateways:
- gw-kafka
port: 16002
sniHosts:
- kafka2.host.com
route:
- destination:
host: kafka2.host.com
port:
number: 9093
========================
It works. But I think that traffic bypasses the istio egressgateway. There is no connection in kiali between ServiceEntry and Egressgateway. And if you look at the egressgateway logs, you can see the following warning:
gRPC config for envoy.api.v2.ClusterLoadAssigment rejected: malformed IP address: kafka1.host.com. Consider setting resolver_name or setting cluster type to 'STRICT_DNS' or 'LOGICAL_DNS'
What is the problem and how to properly configure the egress gateway?
Related
I am trying to configure egress gateway with tls origination using service type ExternalName.
I followed this example: https://istio.io/latest/docs/tasks/traffic-management/egress/egress-tls-origination/
But I am having Host problems with the request because of the SNI, I have to change the host because of the SSL from example-test to example.com, I tried to set header host and rewrite authority on virtual service but didn't work, and now I am trying to use envoy filter but I got nothing so far:
curl example-test
cause a 502 and 400 on the istio-proxy:
GET / HTTP/1.1" 502 UPE upstream_reset_before_response_started{protocol_error} - "-" 0 87 6 - "-" "curl/7.75.0" "example.com" outbound|80|example-test|egressgateway-internal.istio-system.svc.cluster.local service_entry_match
Gateway:
HTTP/1.1" 400 DPE http1.codec_error - "-" 0 11 0 - "-" "-" "-" "-" "-" - - example.com -
Destination rule configuration:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: egressgateway-for-example
spec:
host: egressgateway-internal.istio-system.svc.cluster.local
subsets:
- name: example-test
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 80
tls:
mode: ISTIO_MUTUAL
sni: 'example.com'
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-tls-for-example
spec:
host: 'example.com'
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode:
sni: 'example.com'
Service Entry and service type externalName configuration:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: example-test
spec:
hosts:
- 'example.com'
ports:
- number: 80
name: http-port
protocol: HTTP
- number: 443
name: https-port
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
---
kind: Service
apiVersion: v1
metadata:
name: example-test
namespace: default
spec:
type: ExternalName
externalName: example.com
ports:
- protocol: TCP
port: 443
targetPort: 443
name: tcp
Virtual service ( I tried the rewrite on destination to authority and set Host to example.com and did not work)
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: direct-example-through-egress-gateway
spec:
hosts:
- 'example.com'
gateways:
- egressgateway-internal
- mesh
http:
- name: "service_entry_match"
match:
- gateways:
- mesh
route:
- destination:
host: egressgateway-internal.istio-system.svc.cluster.local
subset: example-test
port:
number: 80
weight: 100
- name: "tls_destination"
match:
- gateways:
- egressgateway-internal
port: 80
route:
- destination:
host: 'example.com'
port:
number: 443
weight: 100
And the gateway:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: egressgateway-internal
namespace: istio-system
spec:
selector:
istio: egressgateway-internal
servers:
- port:
number: 80
name: https-port-for-tls-origination
protocol: HTTPS
hosts:
- 'example.com'
tls:
mode: ISTIO_MUTUAL
Envoy filter:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: test-add-example-host
namespace: default
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
listener:
portNumber: 80
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
subFilter:
name: "envoy.filters.http.router"
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
function envoy_on_request(request_handle)
if (request_handle:headers():get("Host") == "example-test")
then
request_handle:logInfo("[INFO] Add Host to request...")
request_handle:headers():replace(":authority",'example.com')
else
request_handle:logInfo("[INFO]debug..")
end
end
I created a cluster in GKE - (Google Kubernetes Engine)
Folowed the instructions here https://cloud.google.com/service-mesh/docs/quickstart-asm
but used my own deployment files.
I deployed these service and gateway file.
Partial Service
---
apiVersion: v1
kind: Service
metadata:
name: node-microservice-service
spec:
selector:
app: node-microservice
# type: LoadBalancer
ports:
- name: tcp-node
protocol: TCP
port: 8080
targetPort: 8080
# nodePort: 30000
---
Gateway
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: backend-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "backend.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: backend-ingress
spec:
hosts:
- "backend.com"
gateways:
- backend-gateway
http:
- match:
- uri:
prefix: "/node"
route:
- destination:
port:
number: 8080
host: node-microservice-service
- match:
- uri:
prefix: "/java"
route:
- destination:
port:
number: 8080
host: java-microservice-service
- match:
- uri:
prefix: "/golang"
route:
- destination:
port:
number: 8080
host: golang-microservice-service
- match:
- uri:
prefix: "/python"
route:
- destination:
port:
number: 8080
host: python-microservice-service
- route:
- destination:
port:
number: 8080
host: python-microservice-service
I am using the proper IP address and host together still unable to reach the microservices through the istio ingress gateway service.
I deploy rabbit mq on cluster, so far running well on port 15672 : http://test.website.com/
but there need open some other ports (25672, 15672, 15674). I has defined in yaml like this :
apiVersion: v1
kind: Service
metadata:
name: rabbitmq
spec:
selector:
name: rabbitmq
ports:
- port: 80
name: http
targetPort: 15672
protocol: TCP
- port: 443
name: https
targetPort: 15672
protocol: TCP
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: rabbitmq
spec:
selector:
matchLabels:
app: rabbitmq
strategy:
type: RollingUpdate
template:
metadata:
name: rabbitmq
spec:
containers:
- name: rabbitmq
image: rabbitmq:latest
ports:
- containerPort: 15672
name: http
protocol: TCP
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: rabbitmq
spec:
hosts:
- “test.website.com”
gateways:
- gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
port:
number: 80
host: rabbitmq
How do I setup in yaml file to open some other ports ?
Assuming that Istio Gateway is serving TCP network connections, you might be able to combine one Gateway configuration for two external ports.
Here is an example:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: port1
protocol: TCP
hosts:
- example.myhost.com
- port:
number: 443
name: port2
protocol: TCP
hosts:
- example.myhost.com
Field hosts identifies here a list of target addresses that have to be exposed by this Gateway.
In order to make appropriate network routing to the nested Pods specify VirtualService with the matching set for the ports:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: rabbitmq-virtual-service
spec:
hosts:
- example.myhost.com
gateways:
- gateway
tcp:
- match:
- port: 80
route:
- destination:
host: app.example.svc.cluster.local
port:
number: 15672
- match:
- port: 443
route:
- destination:
host: app.example.svc.cluster.local
port:
number: 15674
Above VirtualService defines the rules to route network traffic coming on 80 and 443 ports for test.website.com to the rabbitmq service ports 15672, 15674 respectively.
You can adjust these files to your needs to open some other ports.
Take a look: virtualservice-for-a-service-which-exposes-multiple-ports.
Istio allows you to route a http request in a VirtualService to an external host provided a ServiceEntry exists. For example:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.domain.co
gateways:
- public-gateway.istio-system.svc.cluster.local
- mesh
http:
- match:
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 443
host: httpbin.domain.co
route:
- destination:
host: httpbin.org
port:
number: 80
However this only allows for a HTTP endpoint - how do I configure the external endpoint to be TLS/HTTPS?
This took me hours to work out - so worth sharing I feel.
In order to terminate this service as a TLS, a Destination Rule is required. My final config:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.domain.co
gateways:
- public-gateway.istio-system.svc.cluster.local
- mesh
http:
- match:
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 443
host: httpbin.domain.co
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 80
host: httpbin.domain.co
route:
- destination:
host: httpbin.org
port:
number: 443
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin-org
spec:
host: httpbin.org
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE
I'm trying to set some route rules and I have a problem in the following case. If I send a HTTP request to the port, let's say, 5000 of the service "service-a" I want my route rule to forward the request to the port 5001 of the service "service-b". Is it possible?
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-route
spec:
hosts:
- service-a:5000
- service-a
- service-a.service:5000
- service-a.service
- service-a.service.consul:5000
- service-a.service.consul
gateways:
- my-gateway
http:
- match:
- uri:
prefix: "/v1/service-a"
rewrite:
uri: "/v1/ser-a"
route:
- destination:
port:
number: 5001
name: service-b
The gateway was defined as follows:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: my-gateway
spec:
selector:
app: my-gatweway-controller
servers:
- port:
number: 5000
name: http
protocol: HTTP
hosts:
- localhost
- port:
number: 5001
name: http
protocol: HTTP
hosts:
- localhost
The result is an internal error 500 and the request lands on port 5000 instead on 5001. Does anybody know how to solve the problem?
Best regards
Martin
The port rewriting was fixed three days ago by this PR - https://github.com/istio/istio/pull/5543, it should appear in the next release of Istio.