I’m working on setting up an Egress Gateway. I have a simple one that handles traffic for one host configured based on the Istio docs, so that part is fine. But when I look at how to handle multiple hosts, I find this verbiage:
To direct multiple hosts through an egress gateway, you can include a
list of hosts, or use * to match all, in the Gateway. The subset
field in the DestinationRule should be reused for the additional
hosts.
On this Istio docs page.
Unfortunately I have no idea what the second part of that actually means: "The subset field in the DestinationRule should be reused for the additional hosts." To be honest, I don't entirely understand how DR's work with EgressGateway's in general. I'm more used to a DR that specifies a set of pods as it's target in the "subset" field. So I don't quite get how the DR connections a specific host to the Gateway. I tried the following, but this doesn't work:
egress-gateway.yml
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: istio-egressgateway
namespace: env-specific-egress-experiments
spec:
selector:
istio: egressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- news.ycombinator.com
- www.slashdot.org
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: env-specific-egressgateway-dr
namespace: env-specific-egress-experiments
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: hn
- name: slashdot
service-entries.yml
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: news-ycombinator-com
namespace: env-specific-egress-experiments
spec:
hosts:
- news.ycombinator.com
ports:
- number: 80
name: http-port-hn
protocol: HTTP
- number: 443
name: https-port-hn
protocol: HTTPS
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: www-slashdot-org
namespace: env-specific-egress-experiments
spec:
hosts:
- www.slashdot.org
ports:
- number: 80
name: http-port-slashdot
protocol: HTTP
- number: 443
name: https-port-slashdot
protocol: HTTPS
resolution: DNS
virtual-service.yml
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: external-service-vs-hn
namespace: env-specific-egress-experiments
spec:
hosts:
- news.ycombinator.com
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
- port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: hn
port:
number: 80
rewrite:
authority: news.ycombinator.com
- match:
- gateways:
- istio-egressgateway
port: 80
route:
- destination:
host: news.ycombinator.com
port:
number: 80
weight: 100
rewrite:
authority: news.ycombinator.com
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: external-service-vs-slash
namespace: env-specific-egress-experiments
spec:
hosts:
- www.slashdot.org
gateways:
- istio-egressgateway
- mesh
http:
- match:
- gateways:
- mesh
- port: 80
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: slashdot
port:
number: 80
rewrite:
authority: www.slashdot.org
- match:
- gateways:
- istio-egressgateway
port: 80
route:
- destination:
host: www.slashdot.org
port:
number: 80
weight: 100
rewrite:
authority: www.slashdot.org
If I try to curl one of the hosts with this configuration, I get
/ # curl http://news.ycombinator.com
/ # curl -SsL -D - http://news.ycombinator.com
HTTP/1.1 503 Service Unavailable
date: Tue, 23 Feb 2021 19:27:49 GMT
server: envoy
x-envoy-upstream-service-time: 45
content-length: 0
Also, just as an FYI, a few additional points:
This is just a simulation of part of what I'm trying to accomplish. Obviously I don't really care about connecting to HN or /. Those are just stand-ins for external sites in general
Long-term, the goal is to put an abstract name in front of a external service that can vary in terms of the actual host, based on the deployment. Eg, if I deploy qa-deployment.yml, then "mongodatabase" actually points to mongo.qa.mycompany.com, and if I deploy int-deployment.yml, "mongodatabase" points to mongo.int.mycompany.com, and so on. I think I can accomplish this part with an ExternalName service, and/or a ClusterIP that "floats" and doesn't point to any pods, but is just there for DNS resolution so the VirtualService can trigger.
Ultimately I want all outgoing traffic to go through an EgressGateway. But there will be multiple "things" that need to go through it, hence the particular point I'm stuck on now, which is getting the EgressGateway to work with multiple hosts.
Related
I have a k8s cluster with latest istio installed. I also have the outbound traffic policy set to only allow k8s services and things defined by service entries. I have read https://istio.io/latest/docs/tasks/traffic-management/egress/http-proxy/ and implemented it. But I only want to allow some traffic to the proxy. To be specific, I want allow traffic to https://www.amazon.com when the environment variable HTTPS_PROXY is set to my proxy, but not to https://www.google.com. I don't want to rely on the proxy owner to keep the most restrictive allowlist of urls. I am pretty sure I could achieve this using the CIDR range for the ips for amazon.com, but I don't want to be responsible for trying to keep track of amazon's ip addresses. Is there a way to achieve this using any combination of virtualservices, egressgateway, detinationrule, and serviceentry? So far all of the many attempts to achieve such results have failed. The most promising, I thought, was a virtualservice that used an HTTPRoute to match traffic based on the header. The problem seemed to be that the HTTPRoute never seems to be used. I tried with an HTTPRoute and a TCPRoute at the same time since the serviceentry needs to be TCP for the proxy traffic to open a tunnel (seems likely that only one set of routing rules can be applied at a time, but I tried anyway). I know in the above documentation it says that istio is blind to the destination of proxied traffic, but it should be able to see the headers and potentially make routing decisions based on them. I have seen other questions asking similar questions, but no real answers and none asking exactly what I'm asking. I'll paste an example below of what I hoped would work, but did not (allows all traffic -- doesn't seem to apply the http rule -- probably because there is no serviceentry for proxy with http protocol --- tcp tunnel needed for proxy to work).
---
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: proxy
spec:
hosts:
- some-external-proxy.com
ports:
- number: 3128
name: tcp
protocol: TCP
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
name: istio-egressgateway
spec:
selector:
istio: egressgateway
servers:
- port:
number: 3128
name: proxy-tcp
protocol: HTTP
hosts:
- "some-external-proxy.com"
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: dr-for-proxy
spec:
host: istio-egressgateway.istio-system.svc.cluster.local
subsets:
- name: proxy-se
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-amazon-through-egressgateway
spec:
hosts:
- "some-external-proxy.com"
gateways:
- mesh
- istio-egressgateway
http:
- match:
- gateways:
- mesh
headers:
authority:
exact: "www.amazon.com"
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: proxy-se
port:
number: 3128
- match:
- gateways:
- istio-egressgateway
headers:
authority:
exact: "www.amazon.com"
route:
- destination:
host: "some-external-proxy.com"
port:
number: 3128
weight: 100
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-proxy-through-egressgateway
spec:
hosts:
- "some-external-proxy.com"
gateways:
- mesh
- istio-egressgateway
tcp:
- match:
- gateways:
- mesh
port: 3128
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: proxy-se
port:
number: 3128
- match:
- gateways:
- istio-egressgateway
port: 3128
route:
- destination:
host: "some-external-proxy.com"
port:
number: 3128
weight: 100
My current solution is to use a separate proxy, route traffic to that proxy and add my own whitelist to it to only allow traffic to amazon, then forward that traffic on to the external proxy. (FYI, I have tried some things in conjunction with networkpolicies, but of course that only operates at layer 3/4, so I'd be back to CIDR blocks instead of url filtering)
I've been stuck on this for quite some time, it seems possible to do what I want, but I can't seem to get the configuration correct. I'd greatly appreciate any assistance. Thanks.
I think I found the solution for this. I haven't actually tested it yet, I am building my stack with this change now. I'll test and if it works I'll update the comments and mark this solved. Bottom line: I set the HTTPS_PROXY variable to use a different port, created a port rule in the ServiceEntry for that port, then in the VirtualService match the traffic on that port and route it to the EgressGateway and change to the true proxy port (3128 usually). Then a TCP rule (since there is still a TCP service entry for that port/proxy service) matching the traffic from the EgressGateway and routing it to the proxy on the TCP port (3128).
Example (substitute the ServiceEntry and VirtualServices with below -- keep the other things):
---
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: proxy
spec:
hosts:
- some-external-proxy.com
ports:
- number: 3128
name: tcp
protocol: TCP
- number: 3134
name: http
protocol: HTTP
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: direct-proxy-through-egressgateway
spec:
hosts:
- "some-external-proxy.com"
gateways:
- mesh
- istio-egressgateway
http:
- match:
- gateways:
- mesh
headers:
authority:
exact: "www.amazon.com"
port: 3134
route:
- destination:
host: istio-egressgateway.istio-system.svc.cluster.local
subset: proxy-se
port:
number: 3128
tcp:
- match:
- gateways:
- istio-egressgateway
port: 3128
route:
- destination:
host: "some-external-proxy.com"
port:
number: 3128
weight: 100
Istio allows you to route a http request in a VirtualService to an external host provided a ServiceEntry exists. For example:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.domain.co
gateways:
- public-gateway.istio-system.svc.cluster.local
- mesh
http:
- match:
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 443
host: httpbin.domain.co
route:
- destination:
host: httpbin.org
port:
number: 80
However this only allows for a HTTP endpoint - how do I configure the external endpoint to be TLS/HTTPS?
This took me hours to work out - so worth sharing I feel.
In order to terminate this service as a TLS, a Destination Rule is required. My final config:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin
spec:
hosts:
- httpbin.domain.co
gateways:
- public-gateway.istio-system.svc.cluster.local
- mesh
http:
- match:
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 443
host: httpbin.domain.co
- gateways:
- public-gateway.istio-system.svc.cluster.local
port: 80
host: httpbin.domain.co
route:
- destination:
host: httpbin.org
port:
number: 443
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: httpbin-org
spec:
host: httpbin.org
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE
I want that all ingress traffic for specific service is served by version 8 of this service.
Service is reachable from outside with following host:
my.ciro.it
If I call my.ciro.it I can see that rules works fine.Always respondes with 8 version. If I call my.ciro.it from other service (my2.ciro.it) rules doesn't works and service respondes alternative with version 8 or 9.
This is my virtual service and destination rules:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: ciro-service2-destination-rules
spec:
host: ciro-service2.myns.svc.cluster.local
subsets:
- labels:
version: v8
name: v8
- labels:
version: v9
name: v9
trafficPolicy:
tls:
mode: DISABLE
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: ciro-service2-virtual-service
spec:
hosts:
- my.ciro.it
gateways:
- ciro-service2-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
host: ciro-service2.myns.svc.cluster.local
subset: v8
port:
number: 8082
weight: 100
- destination:
host: ciro-service2.myns.svc.cluster.local
subset: v9
port:
number: 8082
weight: 0
add gateway
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: ciro-service2-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "my.ciro.it"
---
The hosts in Your yaml files is my.ciro.it. Which does not match with my2.ciro.it. If Your plan is to use these rules on both sub-domains hosts You can use wildcard hosts:
...
spec:
hosts:
- *.ciro.it
...
Otherwise You should have:
...
spec:
hosts:
- my2.ciro.it
...
The solution is very simple.
gateways:
- ciro-service2-gateway
- mesh
Add mesh gateway, the rules also applied on internal calls.
I am currently trying to configure Control Egress Traffic to be able to access external services in https specifically with hashicorp Vault which runs on port 8200.
Below I attach the detail of my virtual service and service entry
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: vault-se
spec:
hosts:
- vault.x.com
ports:
- number: 8200
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vault-vs
spec:
hosts:
- vault.x.com
tls:
- match:
- port: 8200
sni_hosts:
- vault.x.com
route:
- destination:
host: vault.x.com
port:
number: 8200
weight: 100
Any idea where my fault is?
I'm trying to set some route rules and I have a problem in the following case. If I send a HTTP request to the port, let's say, 5000 of the service "service-a" I want my route rule to forward the request to the port 5001 of the service "service-b". Is it possible?
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: my-route
spec:
hosts:
- service-a:5000
- service-a
- service-a.service:5000
- service-a.service
- service-a.service.consul:5000
- service-a.service.consul
gateways:
- my-gateway
http:
- match:
- uri:
prefix: "/v1/service-a"
rewrite:
uri: "/v1/ser-a"
route:
- destination:
port:
number: 5001
name: service-b
The gateway was defined as follows:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: my-gateway
spec:
selector:
app: my-gatweway-controller
servers:
- port:
number: 5000
name: http
protocol: HTTP
hosts:
- localhost
- port:
number: 5001
name: http
protocol: HTTP
hosts:
- localhost
The result is an internal error 500 and the request lands on port 5000 instead on 5001. Does anybody know how to solve the problem?
Best regards
Martin
The port rewriting was fixed three days ago by this PR - https://github.com/istio/istio/pull/5543, it should appear in the next release of Istio.