I'm trying to use an Ingress and ExternalName Service in Kubernetes to route traffic to an external storage service (DigitalOcean Spaces) - but no matter what I try, I get some form of http error.
Things I've tried:
https://github.com/kubernetes/ingress-nginx/pull/629#issue-116679227 (Error: 404 Not Found, nginx)
https://github.com/kubernetes/ingress-nginx/issues/1809 (Error: 502 Bad Gateway, nginx)
A fair bit of other tinkering which has been lost to time.
How do I configure a K8s Ingress/Service to direct ingress requests from example.com/static to a storage bucket (e.g. <zone>.digitaloceanspaces.com/<bucket-name>/<path>/<object>)?
It looks like some of the resources I was able to find were simply outdated. The following solution works as of Kubernetes v1.21.4.
Important Notes:
All Ingress annotations are required:
kubernetes.io/ingress.class: nginx - necessary to engage Nginx ingress controller.
nginx.ingress.kubernetes.io/backend-protocol: HTTPS - necessary to maintain HTTPS traffic to service (this replaces /secure-backends in older versions).
nginx.ingress.kubernetes.io/upstream-vhost - must match service externalName, removes hostname from request path (e.g. if this is missing and being tested through localhost, will likely encounter error: "No such bucket: localhost").
nginx.ingress.kubernetes.io/rewrite-target - passes matched asset URL path through to service.
The path.service.port.number in the Ingress definition must match whatever port the ExternalName service expects (443 in the case of our HTTPS traffic).
apiVersion: v1
kind: Service
metadata:
name: do-bucket-service
spec:
type: ExternalName
externalName: <zone>.digitaloceanspaces.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: do-bucket-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/rewrite-target: /<bucket>/$2
nginx.ingress.kubernetes.io/upstream-vhost: <zone>.digitaloceanspaces.com
spec:
rules:
- http:
paths:
- path: /path/to/static/assets(/|$)(.*)
pathType: Prefix
backend:
service:
name: do-bucket-service
port:
number: 443
Related
I have the following need:
There is API that may be accessed only from allowlisted IPs. I'd like to make this API available publicly.
I thought about the following solution:
Create a service of type ServiceName:
kind: Service
apiVersion: v1
metadata:
name: my-svc
spec:
type: ExternalName
externalName: restricted-api.com
Create an ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- mysite.com
secretName: mysite-tls
rules:
- host: example.com
http:
paths:
- path: /api(/|$)(.*)
pathType: Prefix
backend:
service:
name: my-svc
port:
name: https
Is my understanding correct that with such a setup when I call https://example.com/request on K8s level the request will be sent to https://restricted-api.com/request? The caller would not know that there is communication with restricted-api.com. Since the clients' IPs are dynamic the restricted-api.com would not allow them to call it.
The k8s IP is static and I could allowlist it.
Ok, if these are just your thoughts, I would recommend to look into this annotation:
externalTrafficPolicy=local is an annotation on the Kubernetes service resource that can be set to preserve the client source IP. When this value is set, the actual IP address of a client (e.g., a browser or mobile application) is propagated to the Kubernetes service instead of the IP address of the node.
For more information you can find here or in official Kubernetes docs.
Feel free to reach me out again, if you start to realize your thoughts and will face any issue with this.
i'm trying to reverse proxy using nginx-ingress.
but i cannot find a way to apply reverse proxy in only certain paths
for example, i want apply reverse proxy http://myservice.com/about/* from CDN static resources
and other paths serve my service (in example, it means 'my-service-web' service)
maybe in terms of k8s, CDN means "public external service"
in result,
http://myservice.com/about/* -> reverse proxy from CDN (external service)
http://myservice.com/* -> my-service-web (internal service)
here is my ingress.yaml file
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: my-service-web
namespace: my-service
annotations:
kubernetes.io/ingress.class: nginx-ingress
nginx.ingress.kubernetes.io/server-snippet: |
location ~ /about/(.*) {
proxy_pass https://CDN_URL/$1${is_args}${args};
......and other proxy settings
}
spec:
rules:
- host: myservice.com
http:
paths:
- path: /about
........how do i configuration this?
- path: /*
backend:
serviceName: my-service-web
servicePort: 80
how do i set rules and annotations?
You can create a service with externalName type that will point to your external service (CDN) and it's well explained in this blog post, for example:
apiVersion: v1
kind: Service
metadata:
name: my-service
namespace: my-service
spec:
type: ExternalName
externalName: FQDN
and then use it in your ingress rules by referring to the service name.
I am trying to setup Prometheus in k8 cluster, able to run using helm. Accessing dashboard when i expose prometheus-server as LoadBalancer service using external ip.
Same does not work when I try to configure this service as ClusterIP and making it as backend using ingress controller. Receiving 404 error, any thoughts on how to troubleshoot this?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
with above ingress definition in place, url “http://<>/prometheus/ getting redirected to http://<>/graph/ and then 404 error page getting rendered. When url adjusted to http://<>/prometheus/graph some of webcontrols gets rendered with lots of errors on browser console.
Prometheus might be expecting to have control over the root path (/).
Please change the Ingress to prometheus.example.com and it should work fine. (Changing it to a subdomain)
Please change your Ingress configuration file, add host field:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: prometheus.example.com
http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
then apply changes executing command:
$ kubectl aply -f your_ingress_congifguration_file.yaml
The host header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.
Please take a look here: hosts-header.
Ingress definition: ingress.
Useful information: helm-prometheus.
Useful documentation: ingress-path-matching.
We have an existing website, lets say example.com, which is a CNAME for where.my.server.really.is.com.
We're now developing new services using Kubernetes. Our first service /login is ready to be deployed. Using a mock HTML server I've been able to deploy two pods with seperate services that map to example.com and example.com/login.
What I would like to do is get rid of my mock HTML server, and provide a service inside of the cluster, that points to our full website outside of the server. Then I can change the DNS for example.com to point to our kubernetes cluster and people will still get the main site from where.my.server.really.is.com.
We are using Traefik for ingress, and these are the changes I've made to the config for the website:
---
kind: Service
apiVersion: v1
metadata:
name: wordpress
spec:
type: ExternalName
externalName: where.my.server.really.is.com
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: wordpress
annotations:
kubernetes.io/ingress.class: traefik
spec:
backend:
serviceName: wordpress
servicePort: 80
rules:
- host: example.com
http:
paths:
- backend:
serviceName: wordpress
servicePort: 80
Unfortunately, when I visit example.com, rather than getting where.my.server.really.is.com, I get a 503 with the body "Service Unavailable". example.com/login works as expected
What have I missed?
Following traefik documentation on using ExternalName
When specifying an ExternalName, Træfik will forward requests to the given host accordingly and use HTTPS when the Service port matches 443.
This still requires setting up a proper port mapping on the Service from the Ingress port to the (external) Service port.
I believe you are missing the ports configuration of the Service. Something like
apiVersion: v1
kind: Service
metadata:
name: wordpress
spec:
ports:
- name: http
port: 80
type: ExternalName
externalName: where.my.server.really.is.com
You can see a full example in the docs.
I want to expose a HTTP service running in Google Container Engine over HTTPS only load balancer.
How to define in ingress object that I want HTTPS only load balancer instead of default HTTP?
Or is there a way to permanently drop HTTP protocol from created load balancer? When I add HTTPS protocol and then drop HTTP protocol, HTTP is recreated after few minutes by the platform.
Ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: myapp-ingress
spec:
backend:
serviceName: myapp-service
servicePort: 8080
In order to have HTTPs service exposed only, you can block traffic on port 80 as mentioned on this link:
You can block traffic on :80 through an annotation. You might want to do this if all your clients are only going to hit the loadbalancer through https and you don't want to waste the extra GCE forwarding rule, eg:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
kubernetes.io/ingress.allow-http: "false"
spec:
tls:
# This assumes tls-secret exists.
# To generate it run the make in this directory.
- secretName: tls-secret
backend:
serviceName: echoheaders-https
servicePort: 80
You could also use FrontendConfig
HTTP to HTTPS redirects are configured using the redirectToHttps field
in a FrontendConfig custom resource. Redirects are enabled for the
entire Ingress resource so all services referenced by the Ingress will
have HTTPS redirects enabled.
The following FrontendConfig manifest enables HTTP to HTTPS redirects.
Set the spec.redirectToHttps.enabled field to true to enable HTTPS
redirects. The spec.responseCodeName field is optional. If it's
omitted a 301 Moved Permanently redirect is used.
For example
apiVersion: networking.gke.io/v1beta1
kind: FrontendConfig
metadata:
name: your-frontend-config-name
spec:
redirectToHttps:
enabled: true
responseCodeName: MOVED_PERMANENTLY_DEFAULT
MOVED_PERMANENTLY_DEFAULT is on of the available RESPONSE_CODE field value, to return a 301 redirect response code (default if responseCodeName is unspecified).
You can find more options here: HTTP to HTTPS redirects
Then you have to link your FrontendConfig to the Ingress, like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: your-ingress-name
annotations:
networking.gke.io/v1beta1.FrontendConfig: your-frontend-config-name
spec:
tls:
...