Istio tls port 443 gives 503 Service Unavailable - kubernetes

I am running a service on kubernetes Azure AKS Cluster.
Istio-version: 1.3.2
My service is listening to both port 80 and 443:
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
hello-kubernetes ClusterIP 10.0.43.233 <none> 80/TCP,443/TCP 28h
Also istio-gateway.yaml file looks like below:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: my-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
#tls:
#httpsRedirect: true
- port:
number: 443
name: https
protocol: HTTPS
hosts:
- "*"
tls:
mode: SIMPLE
credentialName: "mycert" # must be the same as secret
privateKey: sds
serverCertificate: sds
#serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
#privateKey: /etc/istio/ingressgateway-certs/tls.key
The secret is created by below command- I have a custom certificate that I have uploaded on the cluster:
kubectl create -n istio-system secret generic mycert \
--from-file=key=/home/user/istio-1.3.2/ssl/myprivate.key \
--from-file=cert=/home/user/istio-1.3.2/ssl/mycert.pem
mycert.pem file includes both certificate key and intermediate key.
The VirtualService file is like:
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hello-kubernetes
spec:
hosts:
- "mydomain.com"
gateways:
- my-gateway
http:
- match:
- uri:
prefix: /hello-k8s
route:
- destination:
host: hello-kubernetes
If I curl it with http, it give me 200 OK response however when I curl it with https port, it gives HTTP/1.1 503 Service Unavailable.
Error message on the browser is:
NET::ERR_CERT_AUTHORITY_INVALID
Any idea of what it is missing?

The error is fixed by adding:
port:
number: 80
in destination part of virtual service file.

Related

I have deployed sample nginx app in k8s and configured istio but using istio gateway ip I could not able to access the application outside

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: test-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: test-virtualservice
spec:
hosts:
- "*"
gateways:
- test-gateway
http:
- match:
- uri:
exact: /test-charts
route:
- destination:
host: test-charts
port:
number: 9080
Note: But i could able to access the application with LB ip
ubuntu#ip-172-31-100-164:~/istio-1.9.0$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
test-charts LoadBalancer 10.0.101.148 20.62.141.231 80:31480/TCP 6h19m

How do I create a URL frontend to my keycloak instance after connecting it up to istio

I have istio installed and can see it on Rancher. I have keycloak installed as well. I am trying to connect the two and have a gateway setup so I can access keycloak front-end through a URL.
In my keycloak manifest I have
# Source: keycloak/templates/statefulset.yaml
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: keycloak
.
. #Many other lines here
.
ports:
- name: http
containerPort: 8080
protocol: TCP
I then setup a gateway with command -
kubectl apply -f networking/custom-gateway.yaml
And in my custom-gateway.yaml file I have -
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: keycloak-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: keycloak
spec:
hosts:
- "*"
gateways:
- keycloak-gateway
http:
- match:
- uri:
exact: /keycloak
rewrite:
uri: "/" # Non context aware backend
route:
- destination:
host: keycloak
port:
number: 80
websocketUpgrade: true
Now when I try to access the URL with http://node_ip_address:port/keycloak, I find that I am not able to access the front end. I have verified that keycloak is installed and the pod is up and running on Rancher.
I also have my istio instance connected to the bookinfo application and am able to run the bookinfo-gateway and connect to http://node_ip_address:port/productpage with a gateway that looks like the one described here. I am trying to setup the same gateway only for keycloak.
What am I doing wrong in my yaml files. How do I fix this? Any help is appreciated. Do I have the ports connected correctly?
As far as I can see, you should fix your Virtual Service.
I prepared small example with helm and keycloak helm chart.
Save this as keycloak.yaml, you can configure your keycloak password here.
keycloak:
service:
type: ClusterIP
password: mykeycloakadminpasswd
persistence:
deployPostgres: true
dbVendor: postgres
Install keycloak with helm and values prepared above.
helm upgrade --install keycloak stable/keycloak -f keycloak.yml
Create gateway and virtual service
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: keycloak-gateway
spec:
selector:
istio: ingressgateway # use istio default controller
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: keycloak
spec:
hosts:
- "*"
gateways:
- keycloak-gateway
http:
- match:
- uri:
prefix: /auth
- uri:
prefix: /keycloak
rewrite:
uri: /auth
route:
- destination:
host: keycloak-http
port:
number: 80
virtual service route.host is name of kubernetes keycloak pod service.
kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
keycloak-http ClusterIP 10.0.14.36 <none> 80/TCP 22m
You should be able to connect to keycloak via your ingress_gateway_ip/keycloak or ingress_gateway_ip/auth and login with keycloak credentials, in my example it's login: keycloak and password: mykeycloakadminpasswd.
Note that you need to add prefix for /auth as it's default keycloak web to do everything. Keycloak prefix just rewrite to /auth here.

Istio Gateway/VirtualService does not work (local ip works)

I just set up istio for the first time on a service, and i cannot get the gateway/vs working.
Here is my configuration, it is according with the docs:
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: dragon-gateway
spec:
selector:
# use Istio default gateway implementation
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- "*"
---
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: dragon
spec:
hosts:
- "vtest.westus.cloudapp.azure.com"
gateways:
- dragon-gateway
http:
- match:
- uri:
prefix: /
- uri:
prefix: /status
- uri:
prefix: /delay
- uri:
prefix: /api/values
route:
- destination:
host: dragon
port:
number: 80
The kubectl describe looks fine:
Name: dragon-gateway
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"networking.istio.io/v1alpha3","kind":"Gateway","metadata":{"annotations":{},"name":"dragon-gateway","namespace":"default"},...
API Version: networking.istio.io/v1alpha3
Kind: Gateway
Metadata:
Creation Timestamp: 2019-09-22T22:54:31Z
Generation: 1
Resource Version: 723889
Self Link: /apis/networking.istio.io/v1alpha3/namespaces/default/gateways/dragon-gateway
UID: f0738082-dd8b-11e9-b099-e259debf6109
Spec:
Selector:
Istio: ingressgateway
Servers:
Hosts:
*
Port:
Name: http
Number: 80
Protocol: HTTP
Name: dragon
Namespace: default
Labels: <none>
Annotations: kubectl.kubernetes.io/last-applied-configuration:
{"apiVersion":"networking.istio.io/v1alpha3","kind":"VirtualService","metadata":{"annotations":{},"name":"dragon","namespace":"default"},"...
API Version: networking.istio.io/v1alpha3
Kind: VirtualService
Metadata:
Creation Timestamp: 2019-09-22T22:54:31Z
Generation: 1
Resource Version: 723891
Self Link: /apis/networking.istio.io/v1alpha3/namespaces/default/virtualservices/dragon
UID: f0988c3c-dd8b-11e9-b099-e259debf6109
Spec:
Gateways:
dragon-gateway
Hosts:
vtest.westus.cloudapp.azure.com
Http:
Match:
Uri:
Prefix: /
Uri:
Prefix: /status
Uri:
Prefix: /delay
Uri:
Prefix: /api/values
Route:
Destination:
Host: dragon
Port:
Number: 80
The service has the configuration as follow:
apiVersion: v1
kind: Service
metadata:
namespace: flight
name: dragon
labels:
app: dragon
release: r1
version: 1.0.0
spec:
type: ClusterIP
ports:
- name: http
port: 80
targetPort: 80
- name: https
port: 443
targetPort: 80
selector:
app: dragon
release: r1
The docker file is quite simple:
FROM microsoft/dotnet:latest AS runtime
# ports
EXPOSE 80
EXPOSE 443
WORKDIR /
COPY /publish /app
RUN dir /app
WORKDIR /app
FROM runtime AS final
ENTRYPOINT ["dotnet", "dragon.dll"]
Please let me know if you have any idea. I tried to curl from another pod, and it works. The problem is using the external IP or using the internal IP that's assigned to the gateway. None of these work.
Thanks in advance for any clue.
Edit:
Adding more info about the curl
curl 40.118.228.111/api/values -v
* Trying 40.118.228.111...
* TCP_NODELAY set
* Connected to 40.118.228.111 (40.118.228.111) port 80 (#0)
> GET /api/values HTTP/1.1
> Host: 40.118.228.111
> User-Agent: curl/7.55.1
> Accept: */*
>
< HTTP/1.1 404 Not Found
< date: Sun, 22 Sep 2019 23:27:54 GMT
< server: istio-envoy
< content-length: 0
<
* Connection #0 to host 40.118.228.111 left intact
Adding proxy status as well:
NAME CDS LDS EDS RDS PILOT VERSION
dragon-dc789456b-g9fxb.flight SYNCED SYNCED SYNCED (50%) SYNCED istio-pilot-689d75bc8-j7j8m 1.1.3
istio-ingressgateway-5c4f9f859d-nj9sq.istio-system SYNCED SYNCED SYNCED (100%) SYNCED istio-pilot-689d75bc8-j7j8m 1.1.3
Looks like you put the dragon VirtualService and the dragon-gateway in the default namespace?
Because service names rely on dns and typically a pod's resolv.conf search paths only include the local namespace, the service name dragon will only resolve properly within the same namespace. Instead, use the fqdn for the dragon service:
...
route:
- destination:
host: dragon.flight.svc.cluster.local
port:
number: 80
You have configured istio to route based on hostname but your curl command is using the ip address. Either configure DNS with an A record like this - vtest.westus.cloudapp.azure.com -> 40.118.228.111, or force curl to send the correct host header:
curl http://vtest.westus.cloudapp.azure.com/api/values --resolve vtest.westus.cloudapp.azure.com:80:40.118.228.111
Hi I am not expert about istio but after invsetigation it looks like working with host and istio gateway, virtualnetworkservices you should use Host prefix in order to pass host HTTP Header,
like this:
curl -I -HHost:httpbin.example.com http://$INGRESS_HOST:$INGRESS_PORT/
This is needed because your ingress Gateway is configured to handle “httpbin.example.com”, but in your test environment you have no DNS binding for that host and are simply sending your request to the ingress IP.
From another point of view this setting must match Vitualservice:
A VirtualService must be bound to the gateway and must have one or more hosts that match the hosts specified in a server.
Specifying '*' bound all hostnames.
Also you can restrict Virtualservices or specify multiple rules for servers like hosts/hosts using this approach.
More advanced examples you can find here- Istio Server:
Hope this help.

Istio Gateway and VirtualService issue with Kubernetes Dashboard

I'm using Istio 1.1.7 and tried the following Gateway and VirtualService to the Kubernetes Dashboard.
But I'm seeing the following:
kubectl -n kube-system logs -f kubernetes-dashboard-7cf9657c74-5fsmq
2019/05/31 15:53:04 http: TLS handshake error from 10.244.3.11:54392: tls: first record does not look like a TLS handshake
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: default-ingressgateway
namespace: default
spec:
selector:
istio: ingressgateway
servers:
- hosts:
- "*"
port:
name: http
number: 80
protocol: HTTP
- hosts:
- "*"
port:
name: https
number: 443
protocol: HTTPS
tls:
mode: SIMPLE
privateKey: /etc/istio/ingressgateway-certs/tls.key
serverCertificate: /etc/istio/ingressgateway-certs/tls.crt
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: dashboard-virtualservice
namespace: kube-system
spec:
hosts:
- "*"
gateways:
- default/default-ingressgateway
http:
- match:
- uri:
prefix: "/dashboard"
rewrite:
uri: "/"
route:
- destination:
host: kubernetes-dashboard
port:
number: 443
NOTE: I've also tried change the Gateway Server-TLSOptions-TLSmode from mode: SIMPLE to mode mode: PASSTHROUGH, but PASSTHROUGH I just get We can’t connect to the server in my Browser.
I think PASSTHROUGH should work, if you configure it properly. Here is an example that does something similar. https://istio.io/docs/examples/advanced-gateways/ingress-sni-passthrough/

Why is my ISTIO policy configuration not applied?

I am using Istio-1.0.6 to implement Authentication/Authorization. I am attempting to use Jason Web Tokens (JWT). I followed most of the examples from the documentation but I am not getting the expected outcome. Here are my settings:
Service
kubectl describe services hello
Name: hello
Namespace: agud
Selector: app=hello
Type: ClusterIP
IP: 10.247.173.177
Port: <unset> 8080/TCP
TargetPort: 8080/TCP
Endpoints: 172.16.0.193:8080
Session Affinity: None
Gateway
kubectl describe gateway
Name: hello-gateway
Namespace: agud
Kind: Gateway
Metadata:
Cluster Name:
Creation Timestamp: 2019-03-15T13:40:43Z
Resource Version: 1374497
Self Link:
/apis/networking.istio.io/v1alpha3/namespaces/agud/gateways/hello-gateway
UID: ee483065-4727-11e9-a712-fa163ee249a9
Spec:
Selector:
Istio: ingressgateway
Servers:
Hosts:
*
Port:
Name: http
Number: 80
Protocol: HTTP
Virtual Service
kubectl describe virtualservices
Name: hello
Namespace: agud
API Version: networking.istio.io/v1alpha3
Kind: VirtualService
Metadata:
Cluster Name:
Creation Timestamp: 2019-03-18T07:38:52Z
Generation: 0
Resource Version: 2329507
Self Link:
/apis/networking.istio.io/v1alpha3/namespaces/agud/virtualservices/hello
UID: e099b560-4950-11e9-82a1-fa163ee249a9
Spec:
Gateways:
hello-gateway
Hosts:
*
Http:
Match:
Uri:
Exact: /hello
Uri:
Exact: /secured
Route:
Destination:
Host: hello.agud.svc.cluster.local
Port:
Number: 8080
Policy
kubectl describe policies
Name: jwt-hello
Namespace: agud
API Version: authentication.istio.io/v1alpha1
Kind: Policy
Metadata:
Cluster Name:
Creation Timestamp: 2019-03-18T07:45:33Z
Generation: 0
Resource Version: 2331381
Self Link:
/apis/authentication.istio.io/v1alpha1/namespaces/agud/policies/jwt-hello
UID: cf9ed2aa-4951-11e9-9f64-fa163e804eca
Spec:
Origins:
Jwt:
Audiences:
hello
Issuer: testing#secure.istio.io
Jwks Uri: https://raw.githubusercontent.com/istio/istio/release-1.0/security/tools/jwt/samples/jwks.json
Principal Binding: USE_ORIGIN
Targets:
Name: hello.agud.svc.cluster.local
RESULT
I am expecting to get a 401 error but I am getting a 200. What is wrong with my configuration and how do I fix this?
curl $INGRESS_HOST/hello -s -o /dev/null -w "%{http_code}\n"
200
You have:
Port: <unset> 8080/TCP
For Istio routing and security, you must set the port name to http or http-<something>.
I tried with Istio 1.1. I got a 503 rather than a 401.