I have the following need:
There is API that may be accessed only from allowlisted IPs. I'd like to make this API available publicly.
I thought about the following solution:
Create a service of type ServiceName:
kind: Service
apiVersion: v1
metadata:
name: my-svc
spec:
type: ExternalName
externalName: restricted-api.com
Create an ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- mysite.com
secretName: mysite-tls
rules:
- host: example.com
http:
paths:
- path: /api(/|$)(.*)
pathType: Prefix
backend:
service:
name: my-svc
port:
name: https
Is my understanding correct that with such a setup when I call https://example.com/request on K8s level the request will be sent to https://restricted-api.com/request? The caller would not know that there is communication with restricted-api.com. Since the clients' IPs are dynamic the restricted-api.com would not allow them to call it.
The k8s IP is static and I could allowlist it.
Ok, if these are just your thoughts, I would recommend to look into this annotation:
externalTrafficPolicy=local is an annotation on the Kubernetes service resource that can be set to preserve the client source IP. When this value is set, the actual IP address of a client (e.g., a browser or mobile application) is propagated to the Kubernetes service instead of the IP address of the node.
For more information you can find here or in official Kubernetes docs.
Feel free to reach me out again, if you start to realize your thoughts and will face any issue with this.
Related
My modern application hosted within kubernetes works based on the host and the path app.dev.sandbox.com/ (root / path). I need access external legacy application hosted on EC2/VM with host name app-legacy.dev.sandbox.com via the kubernetes ingress. I want to be able to access the legacy sites pages with the same host name of application deployed on kubernetes like this app.dev.sandbox.com/nbs/login or app.dev.sandbox.com/nbs/homepage.do should actually request app-legacy.dev.sandbox.com/abc/login or app-legacy.dev.sandbox.com/nbs/homepage.do but preserve the modern hostname of app.dev.sandbox.com when displaying the page.
This is needed because, we are using strangler fig pattern to navigate between legacy and modern pages seamlessly. The request needs to be captured by the kubernetes ingress when the user clicks on certain links on the legacy pages. I am currently using the following external service and ingress resource.
apiVersion: v1
kind: Service
metadata:
name: nbs-legacy
spec:
type: ExternalName
externalName: app-legacy.dev.sandbox.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: "nginx"
cert-manager.io/cluster-issuer: "letsencrypt-production"
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/preserve-host: "true"
spec:
ingressClassName: nginx
tls:
- secretName: app.dev.sandbox.com
hosts:
- app.dev.sandbox.com
rules:
- host: app.dev.sandbox.com
http:
paths:
- path: /nbs #I would like the path to be appened to the nbs-legacy request. For example /nbs/login should be appened while requesting app-legacy.dev.sandbox.com/nbs/login
pathType: Exact
backend:
service:
name: nbs-legacy
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: modern-service
port:
number: 80
I am using nginx ingress. I tried with the above manifests but did not work.
I am expecting the following:
Navigate to external legacy application seamlessly while preserving the host name of the kubernetes.
Capture path from the request and pass that to legacy host name request. For example, /nbs/login or /nbs/dashboard should be passed when requesting legacy application pages.
Avoid routing of anything that starts with /nbs(eg: /nbs/abc) to the default root / path.
I'm trying to use an Ingress and ExternalName Service in Kubernetes to route traffic to an external storage service (DigitalOcean Spaces) - but no matter what I try, I get some form of http error.
Things I've tried:
https://github.com/kubernetes/ingress-nginx/pull/629#issue-116679227 (Error: 404 Not Found, nginx)
https://github.com/kubernetes/ingress-nginx/issues/1809 (Error: 502 Bad Gateway, nginx)
A fair bit of other tinkering which has been lost to time.
How do I configure a K8s Ingress/Service to direct ingress requests from example.com/static to a storage bucket (e.g. <zone>.digitaloceanspaces.com/<bucket-name>/<path>/<object>)?
It looks like some of the resources I was able to find were simply outdated. The following solution works as of Kubernetes v1.21.4.
Important Notes:
All Ingress annotations are required:
kubernetes.io/ingress.class: nginx - necessary to engage Nginx ingress controller.
nginx.ingress.kubernetes.io/backend-protocol: HTTPS - necessary to maintain HTTPS traffic to service (this replaces /secure-backends in older versions).
nginx.ingress.kubernetes.io/upstream-vhost - must match service externalName, removes hostname from request path (e.g. if this is missing and being tested through localhost, will likely encounter error: "No such bucket: localhost").
nginx.ingress.kubernetes.io/rewrite-target - passes matched asset URL path through to service.
The path.service.port.number in the Ingress definition must match whatever port the ExternalName service expects (443 in the case of our HTTPS traffic).
apiVersion: v1
kind: Service
metadata:
name: do-bucket-service
spec:
type: ExternalName
externalName: <zone>.digitaloceanspaces.com
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: do-bucket-ingress
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/backend-protocol: HTTPS
nginx.ingress.kubernetes.io/rewrite-target: /<bucket>/$2
nginx.ingress.kubernetes.io/upstream-vhost: <zone>.digitaloceanspaces.com
spec:
rules:
- http:
paths:
- path: /path/to/static/assets(/|$)(.*)
pathType: Prefix
backend:
service:
name: do-bucket-service
port:
number: 443
We are just getting started with k8s (bare metal on Ubuntu 20.04). Is it possible for ingress traffic arriving at a host for a load balanced service to go to a pod running on that host (if one is available)?
We have some apps that use client side consistent hashing (using customer ID) to select a service instance to call. The service instances are stateless but maintain in memory ML models for each customer. So it is useful (but not essential) to have repeated requests for a given customer go to the same service. Then we can just use antiAffinity to have one pod per host.
Our existing service discovery mechanism lets the clients find all the instances of the service and the nodes they are running on. All our k8s nodes are running the Nginx ingress controller.
I finally got this figured out. This was way harder than it should be IMO! Update: It's not working. Traffic frequently goes to the wrong pod.
The service needs externalTrafficPolicy: Local (see docs).
apiVersion: v1
kind: Service
metadata:
name: starterservice
spec:
type: LoadBalancer
selector:
app: starterservice
ports:
- port: 8168
externalTrafficPolicy: Local
The Ingress needs nginx.ingress.kubernetes.io/service-upstream: "true" (service-upstream docs).
The nginx.ingress.kubernetes.io/server-alias: "~^starterservice-[a-z0-9]+\\.example\\.com" bit is because our service discovery updates DNS so each instance of the service includes the name of the host it is running on in its DNS name.
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: starterservice
namespace: default
annotations:
nginx.ingress.kubernetes.io/server-alias: "~^starterservice-[a-z0-9]+\\.example\\.com"
nginx.ingress.kubernetes.io/service-upstream: "true"
spec:
rules:
- host: starterservice.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: starterservice
port:
number: 8168
So now a call https://starterservice-foo.example.com will go to the instance running on k8s host foo.
I believe Sticky Sessions is what you are looking for. Ingress does not communicate directly with pods, but with services. Sticky sessions try to bind requests from the same client to the same pod by setting an affinity cookie.
This is used for example with SignalR sessions, where the negotiation request has to be on the same host as the following websocket connection.
Example:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: minimal-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/affinity: cookie
nginx.ingress.kubernetes.io/affinity-mode: persistent
spec:
rules:
- http:
paths:
- path: /testpath
pathType: Prefix
backend:
service:
name: test
port:
number: 80
Affinity mode "balanced" is the default. If your pod count does not change, part of your clients will lose the session. Use "persistent" to have users connect to the same pod always (unless it dies of course). Further reading: https://github.com/kubernetes/ingress-nginx/issues/5944
I want to route 70% percentage of my traffic coming to service A to an external end point and append the URL.
To achieve this I created an externalName type service which points to external endpoint and then use treafik ingress controller to divide the weight in percentage.
My service definition looks something like this:
---
apiVersion: v1
kind: Service
metadata:
name: wensleydale
spec:
ports:
- name: http
targetPort: 80
port: 80
selector:
app: cheese
task: wensleydale
---
kind: Service
apiVersion: v1
metadata:
name: test-service
spec:
type: ExternalName
externalName: www.google.com
ports:
- name: http
targetPort: 80
port: 80
selector:
app: cheese
task: test-service
Ingress.yaml:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
traefik.ingress.kubernetes.io/service-weights: |
test-service: 70%
wensleydale: 30%
name: cheese
spec:
rules:
- http:
paths:
- backend:
serviceName: test-service
servicePort: 80
path: /
- backend:
serviceName: wensleydale
servicePort: 80
path: /
What I want in addition is when traffic goes to test-service, I want to append path.
In my test-service I want the URL to be something like www.google.com/something
I'm open to use other tools to achieve this.
You can do the following:
Use Istio Ingress Gateway instead of a traefik gateway. Istio Ingress Gateway is the recommended way for Ingress control in Istio. See https://istio.io/docs/tasks/traffic-management/ingress/
In the corresponding Virtual Service, use HTTPRewrite directive https://istio.io/docs/reference/config/istio.networking.v1alpha3/#HTTPRewrite :
rewrite:
uri: /something
Unfortunately you are hitting a limitation. The traefik ingress docs state this condition on weighting - "The associated service backends must share the same path and host". (https://docs.traefik.io/user-guide/kubernetes/#traffic-splitting) So you can't rewrite the path just for one of the weighted targets. The limitation comes from https://github.com/kubernetes/kubernetes/issues/25485 so you can see the suggestions there, many of which mention istio. (See also https://github.com/zalando/skipper/issues/324)
A simple solution might be to deploy another proxy into the cluster and use that to rewrite the target to the internal service that you can't change. Then your Ingress would be able to use the same path for both.
Another way would be to look at configuring a proxy using a conf file rather than ingress annotations. Configuration snippets may be enough to achieve this but I am not sure. I suspect you'd be best to deploy an additional proxy and expose it externally and configure it directly (avoiding the Ingress abstraction).
We have an existing website, lets say example.com, which is a CNAME for where.my.server.really.is.com.
We're now developing new services using Kubernetes. Our first service /login is ready to be deployed. Using a mock HTML server I've been able to deploy two pods with seperate services that map to example.com and example.com/login.
What I would like to do is get rid of my mock HTML server, and provide a service inside of the cluster, that points to our full website outside of the server. Then I can change the DNS for example.com to point to our kubernetes cluster and people will still get the main site from where.my.server.really.is.com.
We are using Traefik for ingress, and these are the changes I've made to the config for the website:
---
kind: Service
apiVersion: v1
metadata:
name: wordpress
spec:
type: ExternalName
externalName: where.my.server.really.is.com
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: wordpress
annotations:
kubernetes.io/ingress.class: traefik
spec:
backend:
serviceName: wordpress
servicePort: 80
rules:
- host: example.com
http:
paths:
- backend:
serviceName: wordpress
servicePort: 80
Unfortunately, when I visit example.com, rather than getting where.my.server.really.is.com, I get a 503 with the body "Service Unavailable". example.com/login works as expected
What have I missed?
Following traefik documentation on using ExternalName
When specifying an ExternalName, Træfik will forward requests to the given host accordingly and use HTTPS when the Service port matches 443.
This still requires setting up a proper port mapping on the Service from the Ingress port to the (external) Service port.
I believe you are missing the ports configuration of the Service. Something like
apiVersion: v1
kind: Service
metadata:
name: wordpress
spec:
ports:
- name: http
port: 80
type: ExternalName
externalName: where.my.server.really.is.com
You can see a full example in the docs.