I am trying to mirror traffic to two copies of the same service in different namespaces. I can access both services by curling their FQDN from a pod running in the default namespace but when i apply the following virtual service nothing gets mirrored. What am i doing wrong?
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: testservice-virtual-service
spec:
hosts:
- testservice.default.svc.cluster.local
gateways:
- istio-system/my-gateway
http:
- route:
- destination:
host: testservice.default.svc.cluster.local
weight: 100
mirror:
host: testservice.staging.svc.cluster.local
mirrorPercentage:
value: 100.0
EDIT - I have tried adding just the service from the default namespace using the FQDN to the hosts field as well as adding both the default and staging namespace services using the FQDN and still do not see any traffic being mirrored to the staging service.
I have posted community wiki answer for better visibility.
As OP has mentioned in the comment, the problem is resolved:
I was doing something extremely stupid and did not have istio enabled in the default namespace.
I had the similar issue while following the istio official document.
I resolved it by adding the port for the host
mirror:
host: <service_name>.<namespace>.svc.cluster.local
port: 80
mirrorPercentage:
value: 100
Related
I am fairly new to Kubernetes and have just deployed my first cluster to IBM Cloud. When I created the cluster, I get a dedicated ingress subdomain, which I will be referring to as <long-k8subdomain>.cloud for the scope of this post. Now, this subdomain works for my app. For example: <long-k8subdomain>.cloud/ping works from my browser/curl just fine- I get the expected JSON response back. But, if I add this subdomain to a CNAME record on my domain provider's DNS settings (I have used Bluehost and IBM Cloud's Internet Services), I get a 404 response back from all routes. However this response is the default nginx 404 response (it says "nginx" under "404 Not Found"). I believe this means that this means the ingress load balancer is being reached, but the request does not get routed right. I am using Kubernetes version 1.20.12_1561 on VPC gen 2 and this is my ingress-config.yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host: <long-k8subdomain>.cloud";
spec:
rules:
- host: <long-k8subdomain>.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
I am pretty sure this problem is due to the annotations. Maybe I am using the wrong ones or I do not have enough. Ideally, I would like something like this: api..com/ to route correctly. I have also read a little bit about default backends, but I have not dove too much into that just yet. Any help would be greatly appreciated, as I have spent multiple hours trying to fix this.
Some sources I have used:
https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning
https://cloud.ibm.com/docs/containers?topic=containers-ingress-types
https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotations#annotations
Note: The reason why I have the second annotation is because for some reason, requests without that header were not being routed directly. So that was part of my debugging process and I just ended up leaving it as I am not sure if that annotation solves that, so I left it for now.
For the NGINX ingress controller to route requests for your own domain's CNAME record to the service instead of the IBM Cloud one, you need a rule in the ingress where the host identifies your domain.
For instance, if your domain's DNS entry is api.example.com, then change the resource YAML to:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
spec:
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
You should not need the second annotation for this to work.
If you want both of the hosts to work, then you could add a second rule instead of replacing host in the existing one.
I have two services, say svcA and svcB that may sit in different namespaces or even in different k8s clusters. I want to configure the services so that svcA can refer to svcB using some constant address, then deploy an Istio Service Entry object depending on the environment to route the request. I will use Helm to do the deployment, so using a condition to choose the object to deploy is easy.
If svcB is in a completely different cluster, it is just like any external server and is easy to configure.
But when it is in a different namespace on the same cluster, I just could not get the Service Entry work. Maybe I don't understand all the options it provides.
Istio objects
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: demo-gateway
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: svcB-se
spec:
hosts:
- svcB.alias
ports:
- number: 80
name: http
protocol: HTTP2
location: MESH_INTERNAL
resolution: svcB.svcb-ns.svc.cluster.local
Update
After doing some random/crazy test, I found that the alias domain name must ends with well know suffix, like .com, .org, arbitrary suffix, like .svc, .alias, won't work.
If I update the above ServiceEntry object to like this. My application works.
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: svcB-se
spec:
hosts:
- svcB.com
ports:
- number: 80
name: http
protocol: HTTP2
location: MESH_INTERNAL
resolution: svcB.svcb-ns.svc.cluster.local
I searched for a while and checked the Istio documentations, but could not find any reference about domain name suffix restrictions.
Is it implicit knowledge that only domain names like .com and .org are valid? I have left school for a long time.
I have posted community wiki answer to summarize the topic and paste explanation of the problem:
After doing some random/crazy test, I found that the alias domain name must ends with well know suffix, like .com, .org, arbitrary suffix, like .svc, .alias, won't work.
If I update the above ServiceEntry object to like this. My application works.
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: svcB-se
spec:
hosts:
- svcB.com
ports:
- number: 80
name: http
protocol: HTTP2
location: MESH_INTERNAL
resolution: svcB.svcb-ns.svc.cluster.local
I searched for a while and checked the Istio documentations, but could not find any reference about domain name suffix restrictions.
Is it implicit knowledge that only domain names like .com and .org are valid? I have left school for a long time.
Explanation:
You can find ServiceEntry requirements in the offical documentation. You can find description how you can properly set this value:
The hosts associated with the ServiceEntry. Could be a DNS name with wildcard prefix.
The hosts field is used to select matching hosts in VirtualServices and DestinationRules.
For HTTP traffic the HTTP Host/Authority header will be matched against the hosts field.
For HTTPs or TLS traffic containing Server Name Indication (SNI), the SNI value will be matched against the hosts field.
NOTE 1: When resolution is set to type DNS and no endpoints are specified, the host field will be used as the DNS name of the endpoint to route traffic to.
NOTE 2: If the hostname matches with the name of a service from another service registry such as Kubernetes that also supplies its own set of endpoints, the ServiceEntry will be treated as a decorator of the existing Kubernetes service. Properties in the service entry will be added to the Kubernetes service if applicable. Currently, the only the following additional properties will be considered by istiod:
subjectAltNames: In addition to verifying the SANs of the service accounts associated with the pods of the service, the SANs specified here will also be verified.
Based on this issue you don't have to use FQDN in your hosts field, but you need to set proper value to select matching hosts in VirtualServices and DestinationRules.
I try to deploy 2 applications (behind 2 separates Deployments objects). I have 1 Service per Deployment, with type NodePort.
application1_service.yaml
apiVersion: v1
kind: Service
metadata:
name: application1-service
namespace: default
spec:
ports:
- port: 80
protocol: TCP
targetPort: 80
selector:
run: application1
type: NodePort
application2_service.yaml is the exact same (except for name and run)
I use an Ingress to make the 2 services available,
ingress.yaml
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: "my-static-ip"
networking.gke.io/managed-certificates: "my-certificate"
kubernetes.io/ingress.class: "gce"
spec:
rules:
- host: "my.host.com"
http:
paths:
- path: /*
backend:
serviceName: application1-service
servicePort: 80
- path: /application2/*
backend:
serviceName: application2-service
servicePort: 80
I also create a ManagedCertificate object, to be able to handle HTTPS requests.
managed_certificate.yaml
apiVersion: networking.gke.io/v1beta1
kind: ManagedCertificate
metadata:
name: my-certificate
spec:
domains:
- my.host.com
The weird thing here is that curl https://my.host.com/ works fine and I can access my service, but when I try curl https://my.host.com/application2/, I keep getting 404 Not Found.
Why is the root working and not the other ?
Additional info:
The ManagedCertificate is valid and works fine with /.
application1 and application2 are the exact same app and if I swap them in the ingress, the output is the same.
Thanks for your help !
EDIT:
Here is the 404 I get when I try to access application2
Don't know if it can help but here is also the part of the Ingress access logs showing the 404
i think you can't use the same port for 2 different applications because this port is used on every node to route to one app.
From docs:
NodePort: Exposes the Service on each Node's IP at a static port (the NodePort). A ClusterIP Service, to which the NodePort Service routes, is automatically created. You'll be able to contact the NodePort Service, from outside the cluster, by requesting :.
So in your case one app is already using port 80, you could try to use a different one for application 2
I have replicated the problem without a domain and managed a
Certificate. For me it's working fine using the Load Balancer IP
address.
(ingress-316204)$ curl http://<LB IP address>/v2/
Hello, world!
Version: 2.0.0
Hostname: web2-XXX
(ingress-316204)$ curl http://<LB IP address>/
Hello, world!
Version: 1.0.0
Hostname: web-XXX
Ingress configuration seems to be correct as per doc. Check if curl https://LB IP address/application2/ is working or not, if it's working then there might be some issue with the host name.
Check if you have updated the host file (/etc/hosts) with line LB IP
address my.host.com.
Check if host, path and backend are configured correctly in Load
Balancer configuration.
If still having the problem then check Port,Nodeport and Targetport configured correctly or else share the output of ‘kubectl describe ing my-ingress’ for further investigation.
Answering my own question:
After searching for days, I ultimately found the reason of the problem.
Everything was fine with the cluster and the configs, my problem was from my Flask API.
All the URLs was like this one:
#app.route("/my_function")
So it was working fine on root path with my.host.com/my_function, but when I was typing my.host.com/application1/my_function it wasn't working...
I just changed my app to
#app.route("/application1/my_function")
Everything works fine now :) Hope it will help !
Ingress gateway is located behind AWS ELB(classic) using nodeport and I want to route TCP traffic in Virtual Service based on client ip.
Of course Proxy Protocol of ELB is enabled.
When I use HTTP, it works. The configuration is below.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: app-vservice
namespace: test
spec:
hosts:
- "app-service"
http:
- match:
- headers:
x-forwarded-for:
exact: 123.123.123.123
route:
- destination:
host: app-service
subset: v2
- route:
- destination:
host: app-service
subset: v1
But I can't find headers field of TCP route in official documents.
Is it impossible?
Thank you.
According to docs yes there is no field to pass headers in TCPRoute in Istio. Also to answer your question every header manipulation should be done using envoy filters because Istio, built on envoy supports that and also decreases the complexity.
Using envoy and lua filters as stated in Istio docs. It can be achieved. Please follow envoy docs.
Checkout the Istio Discussion for headers in Virtual Service.
For implementation of the same using Lua. And a blog showing an example how to implement filters on envoy.
I want to config the route strategy for *-pim.webapi.com and pim.webapi.com. When apply the yaml to k8s, it says the two hosts are duplicated. But I don't think they are duplicated because *-pim.webapi.com can not match pim.webapi.com. Is this a bug of istio or my mistake?
The VirtualService yaml is:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: public-ingress
namespace: netops
spec:
hosts:
- "*-pim.webapi.com"
- "pim.webapi.com"
gateways:
- public-ingress
http:
- route:
- destination:
host: pim.default.svc.cluster.local
The error message is
admission webhook "pilot.validation.istio.io" denied the request: configuration is invalid: duplicate hosts in virtual service: *-pim.webapi.com & pim.webapi.com
The source code for this error message is here https://github.com/istio/istio/blob/master/pilot/pkg/model/validation.go#L1602. But there is no help to understand and solve my problem.
This looks like a bug in Istio. The code you referenced calls this function to decide if the hosts match, and it's clearly doing the wrong thing in your case.
I would try using one host *pim.webapi.com, and see if that works. I'm not sure it will because I thought that wildcards could only be used for the full value of the left most component of a domain name, i.e., *.webapi.com, but not *pim.webapi.com.