I am interested in using Istio in a use case where I spin up a pod based on some event (user starting a game for example), and allow that user to connect to the specific pod through a unqiue URL. Then when this game is over, I can spin the pod down.
I am trying to configure Istio in such a way that either the subdomain or URL can indicate a specific pod to send that request.
Maybe some way to dynamically control label matching? Like <pod_label>.api.com routes to pod matching label label_name: <pod_label> without having to update VirtualServices and DestinationRoutes every time a pod is created?
Edit:
A psuedo config would look similar to:
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: dynamic-route
spec:
hosts:
- r.prod.svc.cluster.local
http:
- match:
uri:
prefix: "/pod/{pod_name}"
ignoreUriCase: true
route:
- destination:
host: {pod_name}.prod.svc.cluster.local
(not sure if this would belong in the virtual service or the destination route)
Related
I have a number of restful services within our system
Some are our within the kubernetes cluster
Others are on legacy infrasture and are hosted on VM's
Many of our restful services make synchronous calls to each other (so not asynchronously using message queues)
We also have a number of UI's (fat clients or web apps) that make use of these services
We might define a simple k8s manifest file like this
Pod
Service
Ingress
apiVersion: v1
kind: Pod
metadata:
name: "orderManager"
spec:
containers:
- name: "orderManager"
image: "gitlab-prem.com:5050/image-repo/orderManager:orderManager_1.10.22"
---
apiVersion: v1
kind: Service
metadata:
name: "orderManager-service"
spec:
type: NodePort
selector:
app: "orderManager"
ports:
- protocol: TCP
port: 50588
targetPort: 50588
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: orderManager-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /orders
pathType: Prefix
backend:
service:
name: "orderManager-service"
port:
number: 50588
I am really not sure what the best way for restful services on the cluster to talk to each other.
It seems like there is only one good route for callers outside the cluster which is use the url built by the ingress rule
Two options within the cluster
This might illustrate it further with an example
Caller
Receiver
Example Url
UI
On Cluster
http://clusterip/orders
The UI would use the cluster ip and the ingress rule to reach the order manager
Service off cluster
On Cluster
http://clusterip/orders
Just like the UI
On Cluster
On Cluster
http://clusterip/orders
Could use ingress rule like the above approach
On Cluster
On Cluster
http://orderManager-service:50588/
Could use the service name and port directly
I write cluster ip a few times above but in real life we put something top so there is a friendly name like http://mycluster/orders
So when caller and reciever are both on cluster is it either
Use the ingress rule which is also used by services and apps outside the cluster
Use the nodeport service name which is used in the ingress rule
Or perhaps something else!
One benefit of using nodeport service name is that you do not have to change your base URL.
The ingress rule appends an extra elements to the route (in the above case orders)
When I move a restful service from legacy to k8s cluster it will increase the complexity
It depends on whether you want requests to be routed through your ingress controller or not.
Requests sent to the full URL configured in your Ingress resource will be processed by your ingress controller. The controller itself — NGINX in this case — will proxy the request to the Service. The request will then be routed to a Pod.
Sending the request directly to the Service’s URL simply skips your ingress controller. The request is directly routed to a Pod.
The trade offs between the two options depend on your setup.
Sending requests through your ingress controller will increase request latency and resource consumption. If your ingress controller does nothing other than route requests, I would recommend sending requests directly to the Service.
However, if you use your ingress controller for other purposes, like authentication, monitoring, logging, or tracing, then you may prefer that the controller process internal requests.
For example, on some of my clusters I use the NGINX ingress controller to measure request latency and track HTTP response statuses. I route requests between apps running in the same cluster through the ingress controller in order to have that information available. I pay the cost of increased latency and resource usage in order to have improved observability.
Whether the trade offs are worth it in your case depends on you. If your ingress controller does nothing more that basic routing, then my recommendation is to skip it entirely. If it does more, then you need to weigh the pros and cons of routing requests through it.
I'm currently working on a case when we need to dynamically create services and provide access to them via URI subpaths of the main gateway.
I'm planning to use virtual services for traffic routing for them. Virtual Service for a particular service should look like:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: subpaths-routes
spec:
hosts:
- mainservice.prod.svc.cluster.local
http:
- name: "subpath-redirection"
match:
- uri:
prefix: "/bservices/svc-2345-6789"
route:
- destination:
host: svc-2345-6789.prod.svc.cluster.local
But there may be a huge number of such services (like thousands). All follow the same pattern of routing.
I would like to know if Istio has a mechanism to specify VirtualService with variables/parameters like the following:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: subpaths-routes
spec:
hosts:
- mainservice.prod.svc.cluster.local
http:
- name: "subpath-redirection"
match:
- uri:
prefix: "/bservices/"{{ variable }}
route:
- destination:
host: {{ variable }}.prod.svc.cluster.local
In Nginx, one can do a similar thing by specifying something like this:
location ~ /service/(?<variable>[0-9a-zA-Z\_\-]+)/ {
proxy_pass http://$variable:8080;
}
Is there a way in Istio to accomplish that?
And if there is not, how would thousands of VSs impact the performance of request processing? Is It expensive to keep them in terms of CPU and RAM being consumed?
Thank you in advance!
How to use variables in Istio VirtualService?
As far as I know there is no such option in istio to specify a variable in prefix and host, if it was only a prefix then you could try with regex instead of prefix.
If you would like to automate it in some way, I mean create a variable and put in in both, prefix and host, then you could try do to it with helm.
There are few examples for virtual service in helm.
https://github.com/streamsets/helm-charts/blob/master/incubating/control-hub/templates/istio-gateway-virtualservice.yaml
https://github.com/salesforce/helm-starter-istio/blob/master/templates/virtualService.yaml
how would thousands of VSs impact the performance of request processing?
There is github issue about that, as #lanceliuu mentioned there
When we create ~1k virtualservices in a single cluster, the ingress gateway is picking up new virtualservice slowly.
So that might be one of the issues with thousands of Virtual Services.
Is It expensive to keep them in terms of CPU and RAM being consumed?
I would say it would require testing. I checked in above github issue and they mentioned that there is no mem/cpu pressure for istio components, but I can't say how expensive is that.
In theory you could create 1 big virtual service instead of thousands, but as mentioned in documentation you should rather Split large virtual services into multiple resources.
Additional resources:
https://engineering.hellofresh.com/everything-we-learned-running-istio-in-production-part-2-ff4c26844bfb
https://istio.io/latest/docs/ops/deployment/performance-and-scalability/
https://perf.dashboard.istio.io/
I’ve been testing istio (1.6) authorization policies and would like to confirm the following:
Can I use k8s service names as shown below where httpbin.bar is the service name for deployment/workload httpbin:
- to:
- operation:
hosts: ["httpbin.bar"]
I have the following rule; only ALLOW access to the httpbin.bar service from service account sleep in foo namespace.
---
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: whitelist-httpbin-bar
namespace: bar
action: ALLOW
rules:
- from:
- source:
principals: ["cluster.local/ns/foo/sa/sleep"]
- to:
- operation:
hosts: ["httpbin.bar"]
I setup 2 services; httpbin.bar and privatehttpbin.bar. My assumption was that it would block access to privatehttpbin.bar but this is not the case. On a side note, I deliberately avoided adding selector.matchLabels because as far as I can tell the rule should only succeed for httpbin.bar.
The docs state:
A match occurs when at least one source, operation and condition matches the request.
as per here.
I interpreted that AND logic will apply to the source and operation.
Would appreciate if I can find out why this may not be working or if my understanding needs to be corrected.
With your AuthorizationPolicy object, you have two rules in the namespace bar:
Allow any request coming from foo namespace; with service account sleep to any service.
Allow any request to httpbin service; from any namespace, with any service account.
So it is an OR, you are applying.
If you want and AND to be applied; meaning allow any request from the namespace foo with service account sleep to talk to the service httpbin, in the namespace bar, you need to apply the following rule:
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
name: whitelist-httpbin-bar
namespace: bar
rules:
- from:
- source:
principals: ["cluster.local/ns/foo/sa/sleep"]
to: # <- remove the dash (-) from here
- operation:
hosts: ["httpbin.bar"]
On the first point You can specify the host name by k8s service name.Therefore httpbin.bar is acceptable for the host field.
On the second point,
As per here ,
Authorization Policy scope (target) is determined by
“metadata/namespace” and an optional “selector”.
“metadata/namespace” tells which namespace the policy applies. If set
to root namespace, the policy applies to all namespaces in a mesh.
So the authorization policy whitelist-httpbin-bar applies to workloads in the namespace foo.But the services httpbin and privatehttpbin you want to authorize lies in bar namespace.So your authorization policy does not restrict access to these services.
If there are no ALLOW policies for the workload, allow the request.
The above criteria makes the request a valid one.
Hope this helps.
We have a cluster with Istio, but there is this one condition, I can't find how to fulfill.
We need one of the services to have certain restrictions within the mesh as well, and to talk to one external endpoint. Through Sidecar object, I should be able to set the restrictions internally, but I don't know how to restrict to one external endpoint.
I can set the external endpoint in the Sidecar object as well, but I have to create a ServiceEntry anyways, in which case all the services can talk to that external endpoint.
It seems that what I need is to set a ServiceEntry for one specific service, but this is not possible. Is there any other way to achieve this?
I asked this question on GitHub; to Istio team, and the only way to achieve this is putting the service in a different namespace, and make the ServiceEntry to apply to the workloads only in that namespace through exportTo parameter.
The ServiceEntry would look like this:
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: se-demo
spec:
exportTo:
- . # with ".", we are saying the ServiceEntry to only apply to the workloads in the same namespace.
hosts:
- www.google.com
location: MESH_EXTERNAL
ports:
- name: https
number: 443
protocol: HTTPS
resolution: DNS
I have a RESTful service within a spring boot application. This spring boot app is deployed inside a kubernetes cluser and we have Istio as a service mesh attached to the sidecar of each container pod in the cluster. Every request to my service first hits the service mesh i.e Istio and then gets routed accordingly.
I need to put a validation for a request header and if that header is not present then randomly generate a unique value and set it as a header to the request. I know that there is Headers.HeaderOperations which i can use in the destination rule but how can i generate a unique value every time the header is missing? I dont want to write the logic inside my application as this is a general rule to apply for all the applications inside the cluster
There is important information that needs to be said in this subject. And it looks to me like You are trying to make a workaround tracing for an applications that does not forward/propagate headers in Your cluster. So I am going to mention few problems that can be encountered with this solution (just in case).
As mentioned in answer from Yuri G. You can configure unique x-request-id headers but they will not be very useful in terms of tracing if the requests are passing trough applications that do not propagate those x-request-id headers.
This is because tracing entire request paths needs to have unique x-request-id though out its entire trace. If the x-request-id value is different in various parts of the path the request takes, how are We going to put together the entire trace path?
In a scenario where two requests are received in application pod at the same time even if they had unique x-request-id headers, only application is able to tell which inbound request matches with which outbound connection. One of the requests could take longer to process and without forwarded trace header we can't tell which one is which.
Anyway for applications that do support forwarding/propagating x-request-id headers I suggest following guide from istio documentation.
Hope it helps.
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: enable-envoy-xrequestid-in-response
namespace: istio-system
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.http_connection_manager"
patch:
operation: MERGE
value:
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
always_set_request_id_in_response: true
From reading the documentation of istio and envoy it seems like this is not supported by istio/envoy out of the box. As a workaround you have 2 options
Option 1: To set the x-envoy-force-trace header in virtual service
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: reviews-route
spec:
hosts:
- reviews.prod.svc.cluster.local
http:
- headers:
request:
set:
x-envoy-force-trace: true
It will generate a header x-request-id if it is missing. But it seems like abuse of tracing mechanism.
Option 2: To use consistentHash balancing based on header, e.g:
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: bookinfo-ratings
spec:
host: ratings.prod.svc.cluster.local
trafficPolicy:
loadBalancer:
consistentHash:
httpHeaderName:
name: x-custom-request-id
It will generate the header x-custom-request-id for any request that doesn't have this header. In this case the requests with same x-custom-request-id value will go always to the same pod that can cause uneven balancing.
The answer above works well! I have updated it for the latest istio (filter name is in full):
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: enable-envoy-xrequestid-in-response
namespace: istio-system
spec:
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: "envoy.filters.network.http_connection_manager"
patch:
operation: MERGE
value:
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager"
always_set_request_id_in_response: true