Http 400 from envoy on BAN request - kubernetes

Question

Istio Newbie here,
I’m doing my first tests with Istio (on version 1.3.0). Most things run nice without much effort.
What I’m having an issue is a service that talks with varnish to clean up the cache. This service makes a HTTP request to every pod behind a headless service and its failing with a HTTP 400 (Bad Request) error. This request uses the HTTP Method “BAN” which I believe is the source of the problem since other request method reach varnish without problems.
As a temporary workaround I changed the port name from http to varnish and everything start working again
I installed istio using the helm chart for 1.3.0:
helm install istio install/kubernetes/helm/istio --set kiali.enabled=true --set global.proxy.accessLogFile="/dev/stdout" --namespace istio-system --version 1.3.0
Running on GKE 1.13.9-gke.3 and Varnish is version 6.2

Answer 1

I was able to get it working using Istio without mTLS using the following definitions:
ConfigMap
Just allowing the pod and service CIDRs for BAN requests and expecting them to come from the Varnish service FQDN.
apiVersion: v1
kind: ConfigMap
metadata:
name: varnish-configuration
data:
default.vcl: |
vcl 4.0;
import std;
backend default {
.host = "varnish-service";
.port = "80";
}
acl purge {
"localhost";
"10.x.0.0"/14; #Pod CIDR
"10.x.0.0"/16; #Service CIDR
}
sub vcl_recv {
# this code below allows PURGE from localhost and x.x.x.x
if (req.method == "BAN") {
if (!client.ip ~ purge) {
return(synth(405,"Not allowed."));
}
return (purge);
}
}
Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: varnish
spec:
replicas: 1
selector:
matchLabels:
app: varnish
template:
metadata:
labels:
app: varnish
spec:
containers:
- name: varnish
image: varnish:6.3
ports:
- containerPort: 80
name: varnish-port
imagePullPolicy: IfNotPresent
volumeMounts:
- name: varnish-conf
mountPath: /etc/varnish
volumes:
- name: varnish-conf
configMap:
name: varnish-configuration
Service
apiVersion: v1
kind: Service
metadata:
name: varnish-service
labels:
workload: varnish
spec:
selector:
app: varnish
ports:
- name: varnish-port
protocol: TCP
port: 80
targetPort: 80
After deploying these, you can run a cURL enabled pod:
kubectl run bb-$RANDOM --rm -i --image=yauritux/busybox-curl --restart=Never --tty -- /bin/sh
And then, from the tty try curling it:
curl -v -X BAN http://varnish-service
From here, either you'll get 200 purged or 405 Not allowed. Either way, you've hit the Varnish pod across the mesh.
Your issue might be related to mTLS in your cluster. You can check if it's enabled by issuing this command*:
istioctl authn tls-check $(k get pod -l app=varnish -o jsonpath={.items..metadata.name}) varnish-service.default.svc.cluster.local
*The command assumess that you're using the definitions shared in this post. If not, you can adjust accordingly.
I tested this running GKE twice: One with open source Istio via Helm install and another using the Google managed Istio installation (in permissive mode).