I am running a react project on Kubernetes and I am having both frontend and backend as services in Kubernetes. The backend is running nestjs and graphql.
When I tried to spin it up online by using the following url https://my_backend.mydomain everything went as expected, though I realized that with this architecture the backend will be publicly available.
So I decided to use the internal url of Kubernetes such as http://my_service:my_port, and I got an error: Blocked loading mixed active content. I realized it was for http instead of https, re-configured the url and set up the connection for https://my-service:my_port. Though now I get a: Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource.
My ingress file looks like the following:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"
nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization"
nginx.ingress.kubernetes.io/cors-allow-origin: "$http_origin"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Content-Security-Policy: https://cdn.jsdelivr.net:* https://*.rsms.me:* https://rsms.me:* always";
more_set_headers "Access-Control-Allow-Origin: *";
spec:
...
Am I mistaken somewhere? I could not find that much regarding Kubernets+NestJs+CORS. I keep ending up in documentation that says to add "Access-Control-Allow-Origin: *". But apparently that does not much with my configuration.
Related
The following configuration has been taken out from here:
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
name: cors-example
spec:
virtualhost:
fqdn: www.example.com
corsPolicy:
allowCredentials: true
allowOrigin:
- "*" # allows any origin
allowMethods:
- GET
- POST
- OPTIONS
allowHeaders:
- authorization
- cache-control
exposeHeaders:
- Content-Length
- Content-Range
maxAge: "10m" # preflight requests can be cached for 10 minutes.
routes:
- conditions:
- prefix: /
services:
- name: cors-example
port: 80
My understanding is that entrance to the cluster is allowed only through www.example.com. Any other external url won't even hit the HTTPProxy.
Hence, I really do not get the role of corsPolicy. What exactly does? What does allows any origin mean? The only origin HTTPProxy allows, is www.example.com. Correct?
In general, are there any CORS restrictions inside K8s cluster (pod to pod)? My understanding again is no.
P.S. Please do not explain what CORS is. I know very well.This is not my question
I guess your overconfidence in knowing what CORS means is clouding your reasoning. Lets imagine the following scenario:
You are hosting a REST API at www.example.com
I am a developer of www.somewebsite.com and I want to use your API.
My website tries to fetch data from www.example.com.
The above policy will tell the browser to allow me to fetch your data since I will get a response from your server with a header that allowed origins are *.
If you don't include this configuration, the browser will not let me consume your API since the domain of my website [www.somewebsite.com] is not allowed to call this API and is not the same with the one where you host the API.
You see I am still trying to fetch data from your domain, www.example.com, and the HTTP Proxy will hit your pods, but the browser is the one that will prevent me from getting the data, unless you have the above configuration.
Is there a way to force an SSL upgrade for incoming connections on the ingress load-balancer? Or if that is not possible with, can I disable port :80? I haven't found a good documentation pages that outlines such an option in the YAML file. Thanks a lot in advance!
https://github.com/kubernetes/ingress-gce#frontend-https
You can block HTTP through the annotation kubernetes.io/ingress.allow-http: "false" or redirect HTTP to HTTPS by specifying a custom backend. Unfortunately GCE doesn't handle redirection or rewriting at the L7 layer directly for you, yet. (see https://github.com/kubernetes/ingress-gce#ingress-cannot-redirect-http-to-https)
Update: GCP now handles redirection rules for load balancers, including HTTP to HTTPS. There doesn't appear to be a method to create these through Kubernetes YAML yet.
This was already correctly answered by a comment on the accepted answer. But since the comment is buried I missed it several times.
As of GKE version 1.18.10-gke.600 you can add a k8s frontend config to redirect from http to https.
https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#https_redirect
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
name: ssl-redirect
spec:
redirectToHttps:
enabled: true
# add below to ingress
# metadata:
# annotations:
# networking.gke.io/v1beta1.FrontendConfig: ssl-redirect
The annotation has changed:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
kubernetes.io/ingress.allow-http: "false"
spec:
...
Here is the annotation change PR:
https://github.com/kubernetes/contrib/pull/1462/files
If you are not bound to the GCLB Ingress Controller you could have a look at the Nginx Ingress Controller. This controller is different to the builtin one in multiple ways. First and foremost you need to deploy and manage one by yourself. But if you are willing to do so, you get the benefit of not depending on the GCE LB (20$/month) and getting support for IPv6/websockets.
The documentation states:
By default the controller redirects (301) to HTTPS if TLS is enabled for that ingress . If you want to disable that behaviour globally, you
can use ssl-redirect: "false" in the NGINX config map.
The recently released 0.9.0-beta.3 comes with an additional annotation for explicitly enforcing this redirect:
Force redirect to SSL using the annotation ingress.kubernetes.io/force-ssl-redirect
Google has responded to our requests and is testing HTTP->HTTPS SSL redirection on their load balancers. Their latest answer said it should be in Alpha sometime before the end of January 2020.
Their comment:
Thank you for your patience on this issue. The feature is currently in testing and we expect to enter Alpha phase before the end of January. Our PM team will have an announcement with more details as we get closer to the Alpha launch.
My fingers are crossed that we'll have a straightforward solution to this very common feature in the near future.
UPDATE (April 2020):
HTTP(S) rewrites is now a Generally Available feature. It's still a bit rough around the edges and does not work out-of-the-box with the GCE Ingress Controller unfortunately. But time will tell and hopefully a native solution will appear.
A quick update. Here
Now a FrontEndConfig can be make to configure the ingress. Hopes it helps.
Example:
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
name: my-frontend-config
spec:
redirectToHttps:
enabled: true
responseCodeName: 301
You'll need to make sure that your load balancer supports HTTP and HTTPS
Worked on this for a long time. In case anyone isn't clear on the post above. You would rebuild your ingress with annotation -- kubernetes.io/ingress.allow-http: "false” --
Then delete your ingress and redeploy. The annotation will have the ingress only create a LB for 443, instead of both 443 and 80.
Then you do a compute HTTP LB, not one for GKE.
Gui directions:
Create a load balancer and choose HTTP(S) Load Balancing -- Start configuration.
choose - From Internet to my VMs and continue
Choose a name for the LB
leave the backend configuration blank.
Under Host and path rules, select Advanced host and path rules with the action set to
Redirect the client to different host/path.
Leave the Host redirect field blank.
Select Prefix Redirect and leave the Path value blank.
Chose the redirect response code as 308.
Tick the Enable box for HTTPS redirect.
For the Frontend configuration, leave http and port 80, for ip address select the static
IP address being used for your GKE ingress.
Create this LB.
You will now have all http traffic go to this and 308 redirect to your https ingress for GKE. Super simple config setup and works well.
Note: If you just try to delete the port 80 LB that GKE makes (not doing the annotation change and rebuilding the ingress) and then adding the new redirect compute LB it does work, but you will start to see error messages on your Ingress saying error 400 invalid value for field 'resource.ipAddress " " is in use and would result in a conflict, invalid. It is trying to spin up the port 80 LB and can't because you already have an LB on port 80 using the same IP. It does work but the error is annoying and GKE keeps trying to build it (I think).
Thanks to the comment of #Andrej Palicka and according to the page he provided: https://cloud.google.com/kubernetes-engine/docs/how-to/ingress-features#https_redirect now I have an updated and working solution.
First we need to define a FrontendConfig resource and then we need to tell the Ingress resource to use this FrontendConfig.
Example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: myapp-app-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: myapp-prd
networking.gke.io/managed-certificates: managed-cert
kubernetes.io/ingress.class: "gce"
networking.gke.io/v1beta1.FrontendConfig: myapp-frontend-config
spec:
defaultBackend:
service:
name: myapp-app-service
port:
number: 80
---
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
name: myapp-frontend-config
spec:
redirectToHttps:
enabled: true
responseCodeName: MOVED_PERMANENTLY_DEFAULT
You can disable HTTP on your cluster (note that you'll need to recreate your cluster for this change to be applied on the load balancer) and then set HTTP-to-HTTPS redirect by creating an additional load balancer on the same IP address.
I spend couple of hours on the same question, and ended up doing what I've just described. It works perfectly.
Redirecting to HTTPS in Kubernetes is somewhat complicated. In my experience, you'll probably want to use an ingress controller such as Ambassador or ingress-nginx to control routing to your services, as opposed to having your load balancer route directly to your services.
Assuming you're using an ingress controller, then:
If you're terminating TLS at the external load balancer and the LB is running in L7 mode (i.e., HTTP/HTTPS), then your ingress controller needs to use X-Forwarded-Proto, and issue a redirect accordingly.
If you're terminating TLS at the external load balancer and the LB is running in TCP/L4 mode, then your ingress controller needs to use the PROXY protocol to do the redirect.
You can also terminate TLS directly in your ingress controller, in which case it has all the necessary information to do the redirect.
Here's a tutorial on how to do this in Ambassador.
Adding the annotation:
annotations:
nginx.ingress.kubernetes.io/auth-url: http://my-auth-service.my-api.svc.cluster.local:8080
...to my ingress rule causes a 500 response from the ingress controller (the ingress works without it).
The service exists and I can ssh into the ingress controller and CURL it, getting a response:
curl http://my-auth-service.my-api.svc.cluster.local:8080 Produces a 200 response.
I checked the ingress controller logs but it says that the service returned a 404. If I can CURL to the same URL why would it return a 404?
2019/07/01 20:26:11 [error] 558#558: *443367 auth request unexpected status: 404 while sending to client, client: 192.168.65.3, server: localhost, request: "GET /mocks HTTP/1.1", host: "localhost"
I'm not sure what to check to deterine the problem.
FWIW, for future readers - I ran into the same problem, and after looking at my auth service logs, noticed nginx ingress' requests were appending a /_external-auth-xxxxxx path to the request url.
Here's where the ingress controller does it, in the source:
https://github.com/kubernetes/ingress-nginx/blob/master/internal/ingress/controller/template/template.go#L428
And how I'm handling it in my own auth service (a Elixir/Phoenix route):
get "/_external-auth*encoded_nginx_auth_url", TokenController, :index
Here are the options you should check:
Global External Authentication
According to this documentation:
By default the controller redirects all requests to an existing
service that provides authentication if global-auth-url is set in the
NGINX ConfigMap. If you want to disable this behavior for that
ingress, you can use enable-global-auth: "false" in the NGINX
ConfigMap. nginx.ingress.kubernetes.io/enable-global-auth: indicates
if GlobalExternalAuth configuration should be applied or not to this
Ingress rule. Default values is set to "true".
Server Name Indication
Check your proxy_ssl_server_name setting in nginx. It indicates if HTTPS uses SNI or not and it is set to false by default.
Please let me know if that helped.
I have two frontend applications that access the same API gateway:
example.com and
admin.example.com
I'm using Traefik as my ingress controller. I currently have the following annotation in my k8s ingress configuration:
ingress.kubernetes.io/custom-response-headers: Access-Control-Allow-Origin:https://example.com || Access-Control-Allow-Methods:POST, GET, HEAD, OPTIONS, PUT, DELETE
I also would like to handle https://admin.example.com within the same block. Is there a way that I can setup a conditional expression here dependant on the origin URL that the traffic is originating from? Access-Control-Allow-Origin: * is not acceptable for my usecase (browsers complain).
I need to allow requests from multiple origins: http://localhost:4200, http://localhost:4242, etc., on nginx-ingress version 1.7.1. But I'm not able to do that for multiple origins, because nginx.ingress.kubernetes.io/cors-allow-credentials: true will not work with nginx.ingress.kubernetes.io/cors-allow-origin: "*". It causes the browser to generate CORS error. Maybe someone has a solution for avoiding this error?
this is my config
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/cors-allow-origin: "*"
nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS, DELETE"
nginx.ingress.kubernetes.io/cors-allow-headers: "DNT,X-CustomHeader,X-LANG,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,X-Api-Key,X-Device-Id,Access-Control-Allow-Origin"
Access to XMLHttpRequest at 'https://stage.site.com/api/session' from
origin 'http://localhost:4200' has been blocked by CORS policy: The
value of the 'Access-Control-Allow-Origin' header in the response must
not be the wildcard '*' when the request's credentials mode is
'include'. The credentials mode of requests initiated by the
XMLHttpRequest is controlled by the withCredentials attribute.
Add the annotation to enable CORS:
nginx.ingress.kubernetes.io/enable-cors: "true"
Be aware that the string "*" cannot be used for a resource that supports credentials (https://www.w3.org/TR/cors/#resource-requests), try with your domain list (comma separated) instead of *
You can create a second Ingress, with a different domain and cors origin, directing to the same destination. Not the best solution but it works.
Or:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Access-Control-Allow-Origin: $http_origin";
nginx.ingress.kubernetes.io/cors-allow-credentials: "true"
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST,
OPTIONS, DELETE, PATCH
nginx.ingress.kubernetes.io/enable-cors: "true"
But attention $http_origin is allowing every origin!
This is a fairly requested feature: https://github.com/kubernetes/ingress-nginx/issues/5496
As a current workaround you can use the following snippet to define more than one domain for CORS: https://github.com/kubernetes/ingress-nginx/issues/5496#issuecomment-662798662
A PR has already been submitted and waits for completion. So this should roll out natively during one of the coming releases: https://github.com/kubernetes/ingress-nginx/pull/7134
Updated 2023:
You can now add multiple origins as a comma separated value in cors-allow-origin
Example:
nginx.ingress.kubernetes.io/cors-allow-origin: "https://example.com, https://another.com, http://localhost:8000"
Source: Cors Allow Multiple Origin
You can add following to your config to match your two origins against http_origin which ingress received in header and return add *-allow-origin only if pattern has been matched:
nginx.ingress.kubernetes.io/configuration-snippet: |
if ($http_origin ~* "^http://localhost\:(4200|4242)$") {
add_header Access-Control-Allow-Origin "$http_origin";
}
As Nicola Ben answered above, but this work for me: (Add the annotation to enable CORS)
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: mybackend-ingress
namespace: my-backend-namespace
annotations:
nginx.ingress.kubernetes.io/cors-allow-headers: Content-Type, authorization
nginx.ingress.kubernetes.io/cors-allow-methods: PUT, GET, POST, OPTIONS
nginx.ingress.kubernetes.io/cors-allow-origin: https://backend.your.url
nginx.ingress.kubernetes.io/enable-cors: 'true'