k8s, Istio: remove transfer-encoding header - kubernetes

In application's responses we see doubled transfer-encoding headers.
Suppose, because of that we get 503 in UI, but at the same time application returns 201 in pod's logs.
Except http code: 201 there are transfer-encoding=chunked and Transfer-Encoding=chunked headers in logs, so that could be a reason of 503.
We've tried to remove transfer-encoding via Istio virtual service or envoy filter, but no luck..
Here are samples we tried:
VS definition:
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: test
namespace: my-ns
spec:
hosts:
- my-service
http:
- route:
- destination:
host: my-service
headers:
response:
remove:
- transfer-encoding
---
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: test
namespace: istio-system
spec:
gateways:
- wildcard-api-gateway
hosts:
- my-ns_domain
http:
- match:
- uri:
prefix: /operator/api/my-service
rewrite:
uri: /my-service
route:
- destination:
host: >-
my-service.my-ns.svc.cluster.local
port:
number: 8080
headers:
response:
remove:
- transfer-encoding
EnvoyFilter definition:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: test
namespace: istio-system
spec:
configPatches:
- applyTo: HTTP_FILTER
match:
context: SIDECAR_OUTBOUND
patch:
operation: ADD
value:
name: envoy.filters.http.lua
typed_config:
"#type": "type.googleapis.com/envoy.extensions.filters.http.lua.v3.Lua"
inlineCode: |
function envoy_on_response(response_handle)
response_handle:headers():remove("transfer-encoding")
end
In older envoy versions I see envoy.reloadable_features.reject_unsupported_transfer_encodings=false was a workaround. Unfortunately, it was deprecated.
Please advise what is wrong with VS/filter or is there any alternative to reject_unsupported_transfer_encodings option?
Istio v1.8.2
Envoy v1.16.1

Decision so far: created requirement for dev team to remove the duplication of chunked encoding

Related

using backtrack regex in istio

We are migrating from ingress-nginx to istio. While migrating existing ingress definitions to istio VirtualServices, we came across nginx style rewriting and wanted to achieve the same in istio. When researched it was found that istio doesn't support backtrack replacement. There is an open bug regarding the same in istio. People suggest to handle this via enjoy filters. Since I'm new to istio I've tried creating an Envoy filter but still, the URL returns 404.
Here is the sample ingress-nginx definition that we want to convert
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$2
nginx.ingress.kubernetes.io/configuration-snippet: |
rewrite ^(/sample)$ $1/ permanent;
nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
name: sample-ingress
spec:
tls:
- hosts:
- '*.example.com'
secretName: icog-ssl
rules:
- host: abc.example.com
http:
paths:
- backend:
service:
name: sample-ingress
port:
number: 80
path: /sample(/|$)(.*)
pathType: ImplementationSpecific
Here is the Envoy filter that was created to handle Reference
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: sample-filter
spec:
configPatches:
- applyTo: HTTP_ROUTE
match:
routeConfiguration:
vhost:
name: "inbound|http|80"
patch:
operation: MERGE
value:
route:
regex_rewrite:
pattern:
google_re2:
max_program_size: 100
regex: "^/sample(/|$)(.*)$"
substitution: "/\\2"
workloadSelector:
labels:
app: sample
we also tried the following as well Reference
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: sample-filter
spec:
configPatches:
- applyTo: HTTP_ROUTE
match:
context: ANY
patch:
operation: MERGE
value:
route:
regex_rewrite:
pattern:
google_re2:
max_program_size: 100
regex: "^/sample(/|$)(.*)$"
substitution: "/\\2"
workloadSelector:
labels:
app: sample
Here is the Virtual Service:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sample-vs
namespace: default
spec:
hosts:
- "*.xyz.com"
gateways:
- sample-gateway
http:
- name: sample
match:
- uri:
regex: /sample(/|$)(.*)
rewrite:
uri: /$2
route:
- destination:
host: sample
port:
number: 80
Gateway
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: sample-gateway
namespace: default
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
tls:
httpsRedirect: true
hosts:
- "*.xyz.com"
- port:
number: 443
name: https
protocol: HTTPS
tls:
mode: SIMPLE
credentialName: sample-ssl
hosts:
- "*.xyz.com"
Api REquests with Both the Envoyfilter returns 404. Not sure how to make this work with istio.
Finally, I was able to crack it down. It's actually simple. We can just use rewrite along with match in virtual service and there is no need to complicate it using filter. Here is the virtual service.
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: sample-vs
namespace: default
spec:
hosts:
- "*.xyz.com"
gateways:
- sample-gateway
http:
- name: sample-trailing
match:
- uri:
prefix: /sample
redirect:
uri: /sample/ # This ensures that the trailing slash is added to the path. same as **rewrite ^(/sample)$ $1/ permanent;**
- name: sample
match:
- uri:
prefix: /sample/
rewrite:
uri: / #This ensures that internally it gets routed to **/$2**
route:
- destination:
host: sample
port:
number: 80
We are misguided by rewrite here we think rewrite rewrites the HTTP URL in the browser whereas it actually rewrites and forwards the request to the respective destination.

istio VirtualService is not working for my configuration

I have a digitalocean kubernetes cluster with istio installed on it. I get 404 not found error on accessing the apis on the postman application. Here's my vs-gateway yaml file:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: ut-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: utbuild1
spec:
hosts:
- "*"
gateways:
- ut-gateway
http:
- match:
- uri:
prefix: /v3/api/products/search
rewrite:
uri: /api/products/search # or /
route:
- destination:
host: utbuild1 # pods/service created using istio-k8s-single-deployment.yaml (this service has 3 pods)
- match:
- uri:
prefix: /v4admin/api
route:
- destination:
host: utbuild1-admin # pod/service created using istio-k8s-single-deployment.yaml

Istio - redirect request to external url

I'm trying to set up a proxy service in the Kubernetes cluster using istio. I have created two different domains. If the domain is foo.com it should be redirected to an external URL else it should be routed to an app server. I have configured this using virtual service and service entry. But when I hit foo.com it is skipping the Authorization header. I need an Authorization header to process the request. Is there any way to fix this issue? Thanks in advance.
VirtualService.yaml
apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
name: external-svc-https
spec:
hosts:
- foo.com
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: TLS
resolution: DNS
---
kind: VirtualService
apiVersion: networking.istio.io/v1alpha3
metadata:
name: redirect
namespace: default
labels:
app: foo
env: staging
spec:
hosts:
- foo.com
gateways:
- istio-system/gateway
http:
- match:
- uri:
prefix: /
redirect:
authority: bar.com
if to redirect when foo.com domain get hit
apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: github
spec:
hosts:
- "raw.githubusercontent.com"
location: MESH_EXTERNAL
ports:
- number: 443
name: https
protocol: TLS
resolution: DNS
and
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: webserver
spec:
hosts:
- foo.com
http:
- match:
- uri:
regex: ".*"
rewrite:
uri: "/mcasperson/NodejsProxy/master/externalservice1.txt"
authority: raw.githubusercontent.com
route:
- destination:
host: raw.githubusercontent.com
port:
number: 443
rule
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: github
spec:
host: "raw.githubusercontent.com"
trafficPolicy:
tls:
mode: SIMPLE
read more at : https://octopus.com/blog/istio/istio-serviceentry

EnvoyFilter to exclude specific hosts

I need to exclude specific host from the EnvoyFilter that looks like this:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: authn-filter
spec:
workloadLabels:
istio: ingressgateway
filters:
- filterConfig:
httpService:
serverUri:
uri: http://authservice.$(namespace).svc.cluster.local
cluster: outbound|8080||authservice.$(namespace).svc.cluster.local
failureModeAllow: false
timeout: 10s
authorizationRequest:
allowedHeaders:
patterns:
- exact: "cookie"
- exact: "X-Auth-Token"
authorizationResponse:
allowedUpstreamHeaders:
patterns:
- exact: "kubeflow-userid"
statusOnError:
code: GatewayTimeout
filterName: envoy.ext_authz
filterType: HTTP
insertPosition:
index: FIRST
listenerMatch:
listenerType: GATEWAY
The problem is that the filter applies to the default istio ingress gateway which affects all traffic that is coming through that gateway, i would like to have some hosts that could be excluded / whitelisted from the filter.
I found my answer here. The question asks to exclude some paths, but I was successful with hosts as well. This is what I used:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: bypass-authn
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingressgateway
configPatches:
- applyTo: HTTP_ROUTE
match:
routeConfiguration:
vhost:
name: subdomain.example.org:80 # <== your host goes here
patch:
operation: MERGE
value:
name: envoy.ext_authz_disabled
typed_per_filter_config:
envoy.ext_authz:
"#type": type.googleapis.com/envoy.extensions.filters.http.ext_authz.v3.ExtAuthzPerRoute
disabled: true
More information in Istio documentation. Specifically, the documentation specifies that you should also put into the name: field the port, but I think it should work without it as well.

Why I can't expose the grafana that comes from istio with Istio Gateway?

I am using helm to install istio-1.0.0 version with --set grafana.enabled=true.
To access the grafana dashboard, I have to do port forwarding using kubectl command. It works okay. However, i want to access it using public ip, hence I am using this gateway yaml file
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: grafana-gateway
namespace: agung-ns
spec:
selector:
istio: ingressgateway # use Istio default gateway implementation
servers:
- port:
number: 15031
name: http-grafana
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: grafana-global-route
namespace: agung-ns
spec:
hosts:
- "grafana.domain"
gateways:
- grafana-gateway
- mesh
http:
- route:
- destination:
host: "grafana.istio-system"
port:
number: 3000
weight: 100
I tried to curl it, but it returns 404 status, which means something wrong with routing logic and/or my configuration above.
curl -HHost:grafana.domain http://<my-istioingressgateway-publicip>:15031 -I
HTTP/1.1 503 Service Unavailable
date: Tue, 14 Aug 2018 13:04:27 GMT
server: envoy
transfer-encoding: chunked
Any idea?
I think the problem is that you refer service in different namespace. You need to add FQDN (grafana.istio-system.svc.cluster.local).
If you need istio, grafana, prometheus and jaeger integrated, exposed through gateway and with enabled security you can check the project I am working on:
https://github.com/kyma-project/kyma
I did expose it like this:
grafana.yml
---
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: grafana-gateway
namespace: istio-system
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "my.dns.com"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: grafana-vts
namespace: istio-system
spec:
hosts:
- "my.dns.com"
gateways:
- grafana-gateway
http:
- match:
- uri:
prefix: /
route:
- destination:
host: grafana
port:
number: 3000
then:
kubectl apply grafana.yml