I'm studying how Istio works and in particular I'm looking at timeouts.
I followed this guide provided by Istio Docs, where the following VirtualService gets defined:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews
spec:
hosts:
- reviews
http:
- route:
- destination:
host: reviews
subset: v2
timeout: 0.5s
If I understood well, every call to ratings service get a 0.5s timeout. I was wondering if it's possible to define timeout only in the communication beetween (for example) a pod and a service, and not for every call received by the service
I find that the Rewrite feature of my Virtual Service is not working very well. Here is my Virtual Service and DestinationRule yaml file:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: leads-http
namespace: seldon
spec:
gateways:
- istio-system/seldon-gateway
hosts:
- '*'
http:
- match:
- uri:
prefix: /seldon/seldon/leads/
rewrite:
uri: /
route:
- destination:
host: leads-leads
port:
number: 8000
subset: leads
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: leads-leads
namespace: seldon
spec:
host: leads-leads
subsets:
- labels:
version: leads
name: leads
trafficPolicy:
connectionPool:
http:
idleTimeout: 60s
When I send an http request：
curl --location --request POST 'http://localhost/seldon/seldon/leads/v2/models/leads-lgb/versions/v0.1.0/infer'
I find that the istio-proxy service prints 404 not found in the logs:
"POST /seldon/seldon/leads/v2/models/leads-lgb/versions/v0.1.0/infer HTTP/1.1" 404
even though I expect:
POST /v2/models/leads-lgb/versions/v0.1.0/infer HTTP/1.1
I am not sure what's happening. Does anyone have any idea? Thanks!
I think your issue is incorrectly configured DestinationRule or service name coneintion.
DestinationRule:
These rules specify configuration for load balancing, connection pool size from the sidecar, and outlier detection settings to detect and evict unhealthy hosts from the load balancing pool.
Version specific policies can be specified by defining a named subset and overriding the settings specified at the service level.
Note: Policies specified for subsets will not take effect until a route rule explicitly sends traffic to this subset.
DestinationRule-Subset:
It seems to me that name should go first in the structure. At least I havent seen/met another examples.
So in your case correct(at least I hope) DR is:
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
name: leads-leads
namespace: seldon
spec:
host: leads-leads
subsets:
- name: leads
labels:
version: leads
However, if that wont help - I encourage you to check this self-resolved question:
Dont you have the same situation with named service port? I mean as per Explicit protocol selection you should add sufix in service name...
name: <protocol>[-<suffix>]
In a project I'm currently working on, I'd like to create a DNS alias for a Kubernetes service located in another namespace. To do so, I created an ExternalName service such as the following:
kind: Service
apiVersion: v1
metadata:
name: connector
namespace: test
spec:
type: ExternalName
externalName: gateway.eventing.svc.cluster.local
So far, so good. When I request the 'connector' DNS, I successfully hit the external name, i.e. gateway.eventing.svc.cluster.local.
Now, I would like to add headers to all http requests sent to the connector ExternalName service, so I created an Istio VirtualService to do so:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: connector
namespace: test
spec:
hosts:
- connector
- connector.test.svc.cluster.local
http:
- match:
- uri:
prefix: /
route:
- destination:
host: connector
port:
number: 80
#headers config ignored for brevity
The problem is that the VirtualService is never called. It seems it does not intercepts request made to the connector DNS or to its fully qualified name, i.e. connector.test.svc.cluster.local.
I figured, after reading the documentation, that this happens because the Istio VirtualService checks the service registry, and the ExternalName service is not part of it, it's just some kind of DNS alias.
I therefore attempted to create an Istio ServiceEntry such as the following:
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: connector
namespace: test
spec:
hosts:
- connector
endpoints:
- address: gateway.eventing.svc.cluster.local
ports:
- number: 80
name: http
protocol: HTTP
location: MESH_INTERNAL
resolution: DNS
It works, and I can see in Kiali that instead of calling the PassthroughCluster when requesting connector, it is the connector ServiceEntry which is called, which is to my understanding what should be happening.
However, my connector VirtualService is still not called. Why is that? Is there a way to make it happen?
If not, what can I do to alias in a given namespace (i.e. test) a service located in another (i.e. eventing) and proxy http request thanks to an Istio VirtualService?
Thanks in advance for your help!
EDIT:
Sidecar injection is enabled namespace-wide (i.e. test)
So, it turns out that all that was missing to make it work was to both specify and name the port on the ExternalName service.
Here's the updated yaml:
kind: Service
apiVersion: v1
metadata:
name: connector
namespace: test
spec:
type: ExternalName
externalName: gateway.eventing.svc.cluster.local
ports:
- name: http
port: 80
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: connector
namespace: test
spec:
hosts:
- connector
- connector.test.svc.cluster.local
http:
- match:
- uri:
prefix: /
route:
- destination:
host: connector
port:
number: 80
#headers config ignored for brevity
Naming the port is absolutely required, as it lets Istio know of the application protocol to use, as defined by the VirtualService.
No need to add a ServiceEntry, it will work with the BYON host specified in the VirtualService.
Note that the answer supplied by #Christoph Raab works as well, but is unhappilly too verbose to be marked as my prefered answer.
Update
I didn't see that the ports list was missing and am not sure how you could apply the yml, because the list should be required.
Anyways, I leave my answer. Maybe it will help someone else in the further.
Original Post (slightly modified)
Docs are not clear, but I think the header manipulation be could done by the receiving sidecar. As far as I understand your setup, the resource behind the ServiceEntry does not have a sidecar, so if that would be true, the manipulation wouldn't work.
In order to add custom headers you can use a EnvoyFilter of type lua that is applied to the sender's sidecar and can manipulate the traffic on the fly.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: add-custom-header-filter
namespace: test
spec:
configPatches:
- applyTo: CLUSTER
match:
context: SIDECAR_OUTBOUND
cluster:
service: connector
patch:
operation: INSERT_BEFORE
value:
name: envoy.lua
typed_config:
"#type": "type.googleapis.com/envoy.config.filter.http.lua.v2.Lua"
inlineCode: |
function envoy_on_request(request_handle)
response_handle:logInfo("adding custom headers..."");
response_handle:headers():add("X-User-Header", "worked");
end
This filter is applied to every request to the service entry connector by every sidecar in the namespace test on outbound and adds a custom header before any other action done.
I'm trying to implement some sort of traffic routing using Istio in a Kubernetes cluster.
The situattion is the following one:
(customer service) => (preference service) => (recommendation service) which has two versions: v1 and v2.
I want to use a custom header, for example X-Svc-Env from an Istio VirtualService and to specify through this header the version of the Recommendation Service which I want to hit.
The configuration for the VirtualService is the following one:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: recommendation
namespace: online
spec:
hosts:
- recommendation
http:
- match:
- headers:
x-svc-env:
regex: v2
route:
- destination:
host: recommendation
subset: version-v2
- route:
- destination:
host: recommendation
subset: version-v1
Also, the DestinationRule that it's being used is the following one:
kind: DestinationRule
metadata:
name: recommendation
namespace: online
spec:
host: recommendation
subsets:
- labels:
version: v1
name: version-v1
- labels:
version: v2
name: version-v2
Well... this is not working because somehow, my custom header is not propagated through Envoy Proxy (I suppose).
I should mention that if I am using a well-known HTTP header, eg: baggage-user-agent (which is the User-Agent header from the OpenTracing specification, all the things are going pretty well).
Tx!
I have a confusion between Virtual Service and Destinationrule on which one is executed first?
Let’s say I have below configs,
Destinationrule -
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: movies
namespace: aio
spec:
host: movies
subsets:
- labels:
version: v1
name: version-v1
- labels:
version: v2
name: version-v2
---
VirtualService
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: movies
namespace: aio
spec:
hosts:
- movies
http:
- route:
- destination:
host: movies
subset: version-v1
weight: 10
- destination:
host: movies
subset: version-v2
weight: 90
---
I read somewhere that,
A VirtualService defines a set of traffic routing rules to apply when a host is addressed.
DestinationRule defines policies that apply to traffic intended for service after routing has occurred.
Does this mean Destinationrules are invoked after Virtualservices?
I have a small diagram, is my understanding correct?
Yes,
According to istio documentation about DestinationRule:
DestinationRule defines policies that apply to traffic intended for a service after routing has occurred.
And for VirtualService:
A VirtualService defines a set of traffic routing rules to apply when a host is addressed.
There is an youtube video: Life of a Packet through Istio it explains in detail the order of processes that are applied to a packet going through the istio mesh.