Ingress - simple fanout configuration not working - kubernetes

I'm using Ubuntu 20.04.2 LTS. I installed microk8s 1.20.6 rev 2143 and experimenting with ingress. I must be missing something - but it doesn't work as I expect it to. I tracked the strange behavior down to the following configuration:
ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ubuntu
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: my-ubuntu
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
- path: /nginx
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
nginx-service.yaml:
apiVersion: v1
kind: Service
metadata:
name: nginx
spec:
ports:
- port: 80
name: http
- port: 443
name: https
type: ClusterIP
selector:
app: nginx
Now,
curl my-ubuntu/ # this returns Welcome page, as expected
curl my-ubuntu/nginx # this returns Welcome page, as expected
curl my-ubuntu/bad-page.html # this returns 404 Not Found, as expected
curl my-ubuntu/nginx/bad-page.html # this returns Welcome page. WHY?
Any request under my-ubuntu/nginx/* returns Welcome page, even when the url is correct and should have returned different content. Did I configure something wrong?
I was able to reproduce the same strange behavior using Docker for Windows + WSL2 + Ubuntu + ingress installed using:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.41.2/deploy/static/provider/cloud/deploy.yaml
EDIT
nginx-deployment.yaml I used:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx
labels:
app: nginx
spec:
replicas: 1
revisionHistoryLimit: 0
selector:
matchLabels:
app: nginx
template:
metadata:
labels:
app: nginx
spec:
containers:
- image: nginx
imagePullPolicy: Always
name: nginx
When I try /nginx/ instead of /nginx like #HarshManvar suggested, I get this behavior:
curl my-ubuntu/ # this returns Welcome page, as expected
curl my-ubuntu/bad-page.html # this returns 404 Not Found, as expected
curl my-ubuntu/nginx # this returns 404 Not Found
curl my-ubuntu/nginx/ # this returns Welcome page
curl my-ubuntu/nginx/bad-page.html # this returns Welcome page
Kubernetes Ingress documentation about Simple fanout also does use /nginx pattern but not working as described above.

https://kubernetes.github.io/ingress-nginx/examples/rewrite/ explains how to use rewrite-target annotation. I was able to make it work with the following ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ubuntu
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: localhost
http:
paths:
- path: /(.*)
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
- path: /nginx($|/.*)
pathType: Prefix
backend:
service:
name: nginx
port:
number: 80
Each path defines regular expression with ( ), which yields $1, $2, etc., aka. regex capture group variables. Now you put rewrite-target using those variables, and that will be the actual URL that is passed to the service's container that handles the request.
Maybe there is another way, but this is the only way I was able to make it work.

Related

kubernetes ingress-nginx ignore special characters in path

I'm trying to have a rule listening to a specific path containing a dollar sign like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: metadata-ingress
annotations:
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/use-regex: "false"
spec:
ingressClassName: public
tls:
- hosts:
- mydomain.com
rules:
- host: mydomain.com
http:
paths:
- path: /api/v2/$metadata
pathType: Prefix
backend:
service:
name: busybox
port:
number: 8280
I don't want any url rewrite or anything fancy, just want this specific path to be caught and forwarded to this service.
Without the "$" it works.
I thought disabling regex with use-regex: "false" would fix it, but no.
I also tried using the url encoded value for $ : %24metadata but it doesn't help either.
I also tried to use "exact" instead of "prefix" as the pathType but no.
I can't reproduce your problem, but I thought I walk through my test setup and you can tell me if anything is different. For the purpose of testing different paths, I have two deployments using the traefik/whoami image (this just provides a useful endpoint that shows us -- among other things -- the hostname and path involved in the request).
That looks like:
apiVersion: v1
kind: Service
metadata:
labels:
app: example
component: app1
name: example-app1
spec:
ports:
- name: http
port: 80
targetPort: http
selector:
app: example
component: app1
---
apiVersion: v1
kind: Service
metadata:
labels:
app: example
component: app2
name: example-app2
spec:
ports:
- name: http
port: 80
targetPort: http
selector:
app: example
component: app2
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: example
component: app1
name: example-app1
spec:
selector:
matchLabels:
app: example
component: app1
template:
metadata:
labels:
app: example
component: app1
spec:
containers:
- image: docker.io/traefik/whoami:latest
name: whoami
ports:
- containerPort: 80
name: http
---
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: example
component: app2
name: example-app2
spec:
selector:
matchLabels:
app: example
component: app2
template:
metadata:
labels:
app: example
component: app2
spec:
containers:
- image: docker.io/traefik/whoami:latest
name: whoami
ports:
- containerPort: 80
name: http
I've also deployed the following Ingress resource, which looks mostly like yours, except I've added a second paths config so that we can compare requests that match /api/v2/$metadata vs those that do not:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
annotations:
cert-manager.io/cluster-issuer: house
nginx.ingress.kubernetes.io/enable-cors: "true"
nginx.ingress.kubernetes.io/ssl-redirect: "true"
name: example
spec:
ingressClassName: nginx
rules:
- host: example.apps.infra.house
http:
paths:
- backend:
service:
name: example-app1
port:
name: http
path: /
pathType: Prefix
- backend:
service:
name: example-app2
port:
name: http
path: /api/v2/$metadata
pathType: Prefix
tls:
- hosts:
- example.apps.infra.house
secretName: example-cert
With these resources in place, a request to https://example.apps.infra.house/ goes to app1:
$ curl -s https://example.apps.infra.house/ | grep Hostname
Hostname: example-app1-596fcf48bd-dqhvc
Whereas a request to https://example.apps.infra.house/api/v2/$metadata goes to app2:
$ curl -s https://example.apps.infra.house/api/v2/\$metadata | grep Hostname
Hostname: example-app2-8675dc9b45-6hg7l
So that all seems to work.
We can, if we are so inclined, examine the nginx configuration that results from that Ingress. On my system, the nginx ingress controller runs in the nginx-ingress namespace:
$ kubectl -n nginx-ingress get deploy
NAME READY UP-TO-DATE AVAILABLE AGE
ingress-nginx-controller 1/1 1 1 8d
The configuration lives in /etc/nginx/nginx.conf in the container. We can cat the file to stdout and look for the relevant directives:
$ kubectl -n nginx-ingress exec deploy/ingress-nginx-controller cat /etc/nginx/nginx.conf
...
location /api/v2/$metadata/ {
...
}
...
Based on your comment, the following seems to work:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: example
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "true"
nginx.ingress.kubernetes.io/rewrite-target: "/$2"
cert-manager.io/cluster-issuer: house
spec:
ingressClassName: nginx
tls:
- hosts:
- example.apps.infra.house
secretName: example-cert
rules:
- host: example.apps.infra.house
http:
paths:
- path: /app1(/|$)(.*)
pathType: Prefix
backend:
service:
name: example-app1
port:
name: http
# Note the use of single quotes (') here; this is
# important; using double quotes we would need to
# write `\\$` instead of `\$`.
- path: '/api/v2/\$metadata'
pathType: Prefix
backend:
service:
name: example-app2
port:
name: http
The resulting location directives look like:
location ~* "^/api/v2/\$metadata" {
...
}
location ~* "^/app1(/|$)(.*)" {
...
}
And a request for the $metadata path succeeds.

Only allow requests to certain paths using k8s ingress

I've set up an ingress to route traffic to my http server, however I would like to leave some routes inaccessible from outside of the cluster.
Example routes:
/status -> end point to determine service status
/users/names -> returns users
/users/ages -> returns ages
current ingress:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: my-namespace
name: my-app-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: localhost
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: my-service
port:
number: 8080
this works currently but leaves all routes accessible. What I want to do is only have routes that fall under the /users path open, so that would be both /users/names and /users/ages. That would leave /status inaccessible from outside of the cluster. Is this achievable from changing the ingress configuration? Any help would be appreciated.
Just specify the path that you want to expose via the ingress like this:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: my-namespace
name: my-app-ingress
annotations:
kubernetes.io/ingress.class: nginx
spec:
rules:
- host: localhost
http:
paths:
- pathType: Prefix
path: /users # <- add the path here
backend:
service:
name: my-service
port:
number: 8080

Kubernetes Ingress with AWS ALB Controller

I am using an EKS cluster and deployed AWS Load Balancer Controller for ingress. Using a dummy application (taken Nginx image) deployed one service. Just for the test, I am trying to access the same service from 3 different ingress paths: / and /foo/ and /bar/
AWS is successfully provisioning an ALB while ingress kubernetes object is getting deployed.
I am able to access the dummy application (Nginx homepage) by hitting external ALB on path '/'.
http://alb-dns/ # it's opening the Nginx homepage
But http://alb-dns/foo/ and http://alb-dns/bar/ are throwing "404 Not Found" error.
I searched through the internet and probably the issue is with URL rewrite. But the problem is, AWS ALB Controller does not support URL rewrite.
I tried lot of options mentioned in the below links but none of them works:
https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/#actions
https://github.com/kubernetes-sigs/aws-load-balancer-controller/issues/835
Is there any alternative to URL redirect? How do I get Nginx Homepage using http://alb-dns/foo/ url?
Or it's better to dump AWS LB Controller and move to Nginx Controller?
Thanks in advance.
These are Kubernetes manifests I am using:
---
# Source: game-app/templates/game-app-namespace.yaml
apiVersion: v1
kind: Namespace
metadata:
name: ns-fargate-app
labels:
app.kubernetes.io/name: sample-game-app
---
# Source: game-app/templates/game-app-service.yaml
apiVersion: v1
kind: Service
metadata:
namespace: ns-fargate-app
name: game-app-service
spec:
ports:
- port: 80
targetPort: 80
protocol: TCP
type: NodePort
selector:
app.kubernetes.io/name: sample-game-app
---
# Source: game-app/templates/game-app-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
namespace: ns-fargate-app
name: sample-game-app
spec:
selector:
matchLabels:
app.kubernetes.io/name: sample-game-app
replicas: 3
template:
metadata:
labels:
app.kubernetes.io/name: sample-game-app
spec:
containers:
- image: public.ecr.aws/nginx/nginx:1.21
imagePullPolicy: Always
name: sample-game-app
ports:
- containerPort: 80
# Source: game-app-ingress/templates/game-app-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: ns-fargate-app
name: game-app-ingress
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
kubernetes.io/ingress.class: alb
spec:
rules:
- http:
paths:
- path: /foo/
pathType: Prefix
backend:
service:
name: game-app-service
port:
number: 80
- path: /bar/
pathType: Prefix
backend:
service:
name: game-app-service
port:
number: 80
- path: /
pathType: Prefix
backend:
service:
name: game-app-service
port:
number: 80
AWS ALB (Active)
Three ALB Rules
http:/alb-dns**/** (Working fine)
http:/alb-dns**/foo** (Throwing 404 error)
See https://kubernetes.io/docs/concepts/services-networking/ingress/#the-ingress-resource.
Prefix / matches all paths.
Your paths /foo/ and /bar/ are redundant in game-app-ingress.yaml. Does your Nginx instance respond to /foo/ or /bar/? If not, that's why you're getting the 404.
See Add context path for an app on nginx for adding additional contexts to your Nginx instance for /foo/ and /bar/.
Since game-app-service has the root path, you can remove the paths for /foo/ and /bar/.

Dynamic Routing by Hostname only - Kubernetes Ingress Rules

I have a K8s cluster with multiple different services deployed and would like to use a single Ingress to route each incoming request to the appropriate service via a unique hostname DNS.
Currently, I've only been able to resolve a request when using the root path i.e. service-123.app.com.
As soon as I try to make a request with a path it doesn't resolve. The paths are valid paths to each service. For example, service-123.app.com/page/12345 would be expected by the application.
I might not fully understand how K8s Ingress rules are expected to work, but I hoped that it would match based on hostname only and simply forward on the path to the appropriate service.
Am I missing something very simple here? Any help is much appreciated. Thanks!
Here are my config files.
Ingress.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
......
name: app-name
namespace: default
spec:
rules:
- host: service-123.app.com
http:
- path: "/*"
backend:
serviceName: service-123
servicePort: 80
- host: service-456.app.com
http:
paths:
- path: "/*"
backend:
serviceName: service-456
servicePort: 80
service.yaml
---
apiVersion: v1
kind: Service
metadata:
annotations: {}
labels:
app: service-123
name: service-123
namespace: default
spec:
ports:
- name: port8080
port: 80
targetPort: 8080
selector:
app: service-123
type: NodePort
Not sure which K8s and ingress controller you are using, but in the later K8s you can specify the pathType which takes care of path wildcards more nicely.
You would have something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
......
name: app-name
namespace: default
spec:
rules:
- host: service-123.app.com
http:
- path: /
pathType: Prefix
backend:
serviceName: service-123
servicePort: 80
- host: service-456.app.com
http:
paths:
- path: /
pathType: Prefix
backend:
serviceName: service-456
servicePort: 80
If you are using an nginx ingress controller a good way to see the right nginx configuration is by looking at the actual nginx.conf generated by the ingress controller.
$ kubectl cp <nginx-ingress-controller-pod>:nginx.conf nginx.conf
$ cat nginx.conf

Generating a redirect with traefik ingress on k3s?

I'm running prometheus and grafana under k3s, accessible (respectively) at http://monitoring.internal/prometheus and http://monitoring.internal/grafana. The grafana Ingress object, for example, looks like:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: grafana
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: monitoring.internal
http:
paths:
- path: /grafana
pathType: Prefix
backend:
service:
name: grafana
port:
number: 3000
This works fine, except that if you land at
http://monitoring.internal/, you get a 404 error. I would like
requests for http://monitoring.internal/ to redirect to
http://monitoring.internal/grafana. I could perhaps create another
service that runs something like darkhttpd ... --forward-all http://monitoring.internal/grafana, and create an Ingress object
that would map / to that service, but it seems like there ought to
be a way to do this with Traefik itself.
It looks like I'm running Traefik 2.4.8 locally:
$ kubectl -n kube-system exec -it deployment/traefik -- traefik version
Version: 2.4.8
Codename: livarot
Go version: go1.16.2
Built: 2021-03-23T15:48:39Z
OS/Arch: linux/amd64
I've found this documentation for 1.7 that suggests there is an annotation for exactly this purpose:
traefik.ingress.kubernetes.io/app-root: "/index.html": Redirects
all requests for / to the defined path.
But setting that on the grafana ingress object doesn't appear to have
any impact, and I haven't been able to find similar docs for 2.x
(I've looked around
here, for
example).
What's the right way to set up this sort of redirect?
Since I haven't been able to figure out traefik yet, I thought I'd post my solution here in case anyone else runs into the same situation. I am hoping someone comes along who knows The Right Way to to do this, and if I figure out I'll update this answer.
I added a new deployment that runs darkhttpd as a simple director:
apiVersion: apps/v1
kind: Deployment
metadata:
name: redirector
spec:
replicas: 1
template:
spec:
containers:
- name: redirector
image: docker.io/alpinelinux/darkhttpd
ports:
- containerPort: 8080
args:
- --forward-all
- http://monitoring.internal/grafana
A corresponding Service:
apiVersion: v1
kind: Service
metadata:
name: redirector
spec:
ports:
- port: 8080
protocol: TCP
targetPort: 8080
And the following Ingress object:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: redirector
annotations:
kubernetes.io/ingress.class: traefik
spec:
rules:
- host: monitoring.internal
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: redirector
port:
number: 8080
These are all deployed with kustomize, which takes care of
adding labels and selectors in the appropriate places. The
kustomization.yaml look like:
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- deployment.yaml
- ingress.yaml
- service.yaml
commonLabels:
component: redirector
With all this in place, requests to http://monitoring.internal/ hit the redirector pod.