istio - tracing egress traffic - kubernetes

I installed Istio with
gateways.istio-egressgateway.enabled = true
I have a service that consumes external services, so I define the following egress rule.
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-service1
spec:
hosts:
- external-service1.com
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
But using Jaeger I can not see the traffic to the external service, and thus be able to detect problems in the network.
I'm forwarding the appropriate headers to the external service (x-request-id, x-b3-traceid, x-b3-spanid, b3-parentspanid, x-b3-sampled, x-b3-flags, x-ot-span-context)
Is this the correct behavior?
what is happening?
Can I only have statistics of internal calls?
How can I have statistics for egress traffic?

Assuming that your services are defined in Istio’s internal service registry. If not please configure it according to instruction service-defining.
In HTTPS all the HTTP-related information like method, URL path, response code, is encrypted so Istio cannot see and cannot monitor that information for HTTPS.
If you need to monitor HTTP-related information in access to external HTTPS services, you may want to let your applications issue HTTP requests and configure Istio to perform TLS origination.
First you have to redefine your ServiceEntry and create VirtualService to rewrite the HTTP request port and add a DestinationRule to perform TLS origination.
kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-service1
spec:
hosts:
- external-service1.com
ports:
- number: 80
name: http-port
protocol: HTTP
- number: 443
name: http-port-for-tls-origination
protocol: HTTP
resolution: DNS
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: external-service1
spec:
hosts:
- external-service1.com
http:
- match:
- port: 80
route:
- destination:
host: external-service1.com
port:
number: 443
---
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: external-service1
spec:
host: external-service1.com
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 443
tls:
mode: SIMPLE # initiates HTTPS when accessing external-service1.com
EOF
The VirtualService redirects HTTP requests on port 80 to port 443 where the corresponding DestinationRule then performs the TLS origination. Unlike the previous ServiceEntry, this time the protocol on port 443 is HTTP, instead of HTTPS, because clients will only send HTTP requests and Istio will upgrade the connection to HTTPS.
I hope it helps.

Related

How to route external HTTPS request to internal http mock service using Istio

I am using Istio with mTLS enabled.
I am trying to mock some external requests using a mock service deployed inside my cluster.
For example, I want all requests to https://httpbin.org redirect to my internal service http://mock.default.svc.cluster.local
My mock service is a simple webservice serving on 80 port.
I was able to do this using service entry & virtualservices for http requests, but I also want to be able to do the same thing for https requests.
Here's my setup:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: httpbin-ext
spec:
hosts:
- httpbin.org
ports:
- number: 80
name: http
protocol: HTTP
- number: 443
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: httpbin-mock
spec:
hosts:
- httpbin.org
http:
- route:
- destination:
host: mock.default.svc.cluster.local
tls:
- match:
- port: 443
sniHosts:
- httpbin.org
route:
- destination:
host: mock.default.svc.cluster.local
port:
number: 80
But when I curl the https://httpbin.org, I got the following error:
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
I am guessing it's because on my mock service 80 port, no one is handling the https protocol.
What's the best approach to do this? A self-signed nginx and destination rule with certificates? Or there's a better way?

Use https protocol for endpoints in Kubernetes Services

Tried creating a Kubernetes endpoints service to invoke resource hosted outside the cluster via static IP's over HTTPS protocol.
Below is the endpoint code
kind: Service
apiVersion: v1
metadata:
name: serviceRequest
spec:
ports:
- port: 8081
targetPort: 8094
---
kind: Endpoints
apiVersion: v1
metadata:
name: serviceRequest
subsets:
- addresses:
- ip: XX.XX.XX.XX // **external IP which is accessible as https://XX.XX.XX.XX:8094**
ports:
- port: 8094
But the above configuration is giving 400 Bad Request with message as "This combination of host and port requires TLS."
Same is working for http not for https exposed "ip".Could someone please guide how to achieve this.
##Update1
This is how the flow is configured.
Ingress->service->endpoints
This is the error message your get when calling a https endpoint with http. Are you sure that whoever is calling your service, is calling it with https:// at the beginning?
Kubernetes Service is no more than a set of forwarding rules in iptables (most often), and it knows nothing about TLS.
If you want to enforce https redirection you might use ingress controller for this. All major ingress controllers have this capability.
For example, check for nginx-ingress.
https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/#server-side-https-enforcement-through-redirect.
Basically, all you need is to add this annotation to your ingress rule.
nginx.ingress.kubernetes.io/ssl-redirect: "true"
Easypeasy, just add port 443 to the Service that will make the request TLS/https:
kind: Service
apiVersion: v1
metadata:
name: serviceRequest
spec:
ports:
- port: 443 # <-- this is the way
targetPort: 8094
---
kind: Endpoints
apiVersion: v1
metadata:
name: serviceRequest
subsets:
- addresses:
- ip: XX.XX.XX.XX # **external IP which is accessible as https://XX.XX.XX.XX:8094**
ports:
- port: 8094
So you can reach your serviceRequest from your containers on https://serviceRequest url.
Also keep in mind that in yaml the # character is the comment sing not //

Istio Ingress Gateway with TLS termination returning 503 service unavailable

We want to to route https traffic to an https endpoint using Istio Ingress Gateway.
We terminate the TLS traffic at the Ingress Gateway, but our backend service uses https as well.
I have the following manifests:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: datalake-dsodis-istio-gateway
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- "gw-hdfs-spark.dsodis.domain"
- "spark-history.dsodis.domain"
port:
name: https-wildcard
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
privateKey: /etc/istio/ingressgateway-certs/tls.key
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: gw-spark-history-istio-vs
spec:
gateways:
- default/datalake-dsodis-istio-gateway
hosts:
- "spark-history.dsodis.domain"
http:
- match:
- uri:
prefix: /
route:
- destination:
host: gateway-svc-clusterip.our_application_namespace.svc.cluster.local
port:
number: 8443
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: originate-tls-for-spark-history
spec:
host: gateway-svc-clusterip.our_application_namespace.svc.cluster.local
trafficPolicy:
loadBalancer:
simple: ROUND_ROBIN
portLevelSettings:
- port:
number: 8443
tls:
mode: SIMPLE
The problem is most likely, that we are sending TLS terminated traffic, (so to say) HTTP traffic, to the HTTPS backend. Therefore we might get 503 Service Unavailable when accessing the service through Istio.
The command accessing it is:
curl -vvvv -H"Host: spark-history.dsodis.domain" --resolve "spark-history.dsodis.domain:31390:IP" https://spark-history.dsodis.domain:31390/gateway/default/sparkhistory -k
My question is, how can I tell Istio to route traffic to the backend service using https?
Thanks in advance.
Best regards,
rforberger
As RonnyForberger mentioned in his comment this can be achieved by creating DestinationRule that tells the traffic to the destination service to be TLS connection.
So in this scenario:
HTTPS request gets TLS terminated at GateWay to HTTP.
Then the HTTP request is translated to TLS with DestinationRule to HTTPS.
HTTPS request reaches HTTPS backend.

Cannot allow external traffic through ISTIO

I am trying to setup Istio and I need to whitelist few ports for allowing non mTLS traffic from outside world coming in through specfic port for few pods runnings in local k8s.
I am unable to find a successful way of doing it.
Tried Service entry, policy and destination rule and didnt succeed.
Helps is highly appreciated.
version.BuildInfo{Version:"1.1.2", GitRevision:"2b1331886076df103179e3da5dc9077fed59c989", User:"root", Host:"35adf5bb-5570-11e9-b00d-0a580a2c0205", GolangVersion:"go1.10.4", DockerHub:"docker.io/istio", BuildStatus:"Clean", GitTag:"1.1.1"}```
Service Entry
```apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: external-traffic
namespace: cloud-infra
spec:
hosts:
- "*.cluster.local"
ports:
- number: 50506
name: grpc-xxx
protocol: TCP
location: MESH_EXTERNAL
resolution: NONE```
You need to add a DestinationRule and a Policy :
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: destinationrule-test
spec:
host: service-name
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
portLevelSettings:
- port:
number: 8080
tls:
mode: DISABLE
---
apiVersion: authentication.istio.io/v1alpha1
kind: Policy
metadata:
name: policy-test
spec:
targets:
- name: service-name
ports:
- number: 8080
peers:
This has been tested with istio 1.0, but it will probably work for istio 1.1. It is heavily inspired by the documentation https://istio.io/help/ops/setup/app-health-check/
From your question, I understood that you want to control your ingress traffic allow some ports to your services that functioning in your mesh/cluster from outside, but your configuration is for egress traffic.
In order to control and allow ports to your services from outside, you can follow these steps.
1.Make sure that containerPort included to your deployment/pod configuration.
For more info
2.You have to have service pointing to your backends/pods. For more info about Kubernetes Services.
3.Then in your Istio enabled cluster, you have to create Gateway similar to below configuration:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: your-service-gateway
namespace: foo-namespace # Use same namespace with backend service
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: HTTP
protocol: HTTP
hosts:
- "*"
4.Then configure route to your service for traffic entering via the this gateway by creating VirtualService:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: your-service
namespace: foo-namespace # Use same namespace with backend service
spec:
hosts:
- "*"
gateways:
- your-service-gateway # define gateway name
http:
- match:
- uri:
prefix: "/"
route:
- destination:
port:
number: 3000 # Backend service port
host: your-service # Backend service name
Hope it helps.

egress istio can't access to external service

I am currently trying to configure Control Egress Traffic to be able to access external services in https specifically with hashicorp Vault which runs on port 8200.
Below I attach the detail of my virtual service and service entry
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: vault-se
spec:
hosts:
- vault.x.com
ports:
- number: 8200
name: https
protocol: HTTPS
resolution: DNS
location: MESH_EXTERNAL
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: vault-vs
spec:
hosts:
- vault.x.com
tls:
- match:
- port: 8200
sni_hosts:
- vault.x.com
route:
- destination:
host: vault.x.com
port:
number: 8200
weight: 100
Any idea where my fault is?