Filter request logs in the istio sidecar proxy - kubernetes

Question

I have azure front door sitting on front of a aks cluster which has istio and proxy sidecars injected into each pod.
Azure front door has health probes which hit a request at least once a second due to the number of azure front door endpoints. The number of requests the apps are getting is very high to the point I want to slow the interval It hits with the affect of losing the benefits of front door.
Microsoft suggest to code a telemetry initialiser in dotnet to mark requests as synthetic, however this seems like a massive problem that I would need to get multiple teams to buy into. As well as replicate to multiple languages.
Instead I would like to use an envoy filter to look at the header of the requests and if it matches the front door agent "Edge Health Probe" I would like to completely ignore it.
This would mean I am in control of what logs get sent to the app insights, can roll out a one fix fits all and would not need to involve the Devs.
I have looked to envoy filter but can't really understand how it would work.
Is this possible with envoy filter or does anyone know of a better method?
Thanks
Kevin

Answer 1

You can do that with an EnvoyFilter. This example recognizes the header on the ingress gateway and simple sends a 200 response without sending it to the workloads:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: custom-ms-header
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: NETWORK_FILTER
match:
context: GATEWAY
listener:
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.network.http_connection_manager
patch:
operation: INSERT_BEFORE
value:
name: custom.ms-header
typed_config:
"#type": type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua
inlineCode: |
function envoy_on_request(request_handle)
val = request_handle:headers():get("some-header")
if (val and val ~= "some-value") then
request_handle:respond({[":status"] = "200"}, "ok")
end
end
Alternatively you can apply it to match certain workloads
[...]
namespace: my-namespace
spec:
workloadSelector:
labels:
istio: my-worklload
configPatches:
- applyTo: NETWORK_FILTER
match:
context: SIDECAR_INBOUND
listener:
filterChain:
[...]
Tough this requires you to apply it to every workload.

Answer 2

am I misunderstanding this but if you want to ignore the request why not simply turn off the health probes?
Or change the interval of the probes from the default 30 seconds to 255 which decreases the requests, also of course the default health probes are HEAD requests so you can easily filter them out.