Path based routing issues Traefik as Ingress Controller - kubernetes

I'm running through what looks like a configuration issue! I am using traefik as ingress controller within kubernetes and I have an ingress to route some URLs to route some frontends to various backends. Let's say I have something like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
kubernetes.io/ingress.class: traefik
traefik.frontend.rule.type: ReplacePathRegex
spec:
rules:
- host: foo.io
http:
paths:
- path: /api/authservice/(.*) /$1
backend:
serviceName: auth
servicePort: 8901
- path: /api/svcXXX/v1/files/cover/(.*) /v1/files/cover/$1
backend:
serviceName: files
servicePort: 8183
- path: /api/svcXXX/v1/files/image/(.*) /v1/files/image/$1
backend:
serviceName: files
servicePort: 8183
Using Postman (or any other client), if I POST a request on http://foo.io/api/authservice/auth/oauth/token, while looking in the access logs, it seems that it is routed to http://foo.io/api/svcXXX/v1/files/image/(.*) /v1/files/image/$1. I'm seeing this in the access logs:
[03/Jul/2018:12:57:17 +0000] "POST /api/authservice/auth/oauth/token HTTP/1.1" 401 102 "-" "PostmanRuntime/7.1.5" 15 "foo.io/api/svcXXX/v1/files/image/(.*) /v1/files/image/$1" 37ms
Am I doing something wrong ?

Note: since the documentation is changed, I've updated the links, but content on the documentation pages would be different.
ReplacePathRegex is a modifier rule. According to documentation:
Modifier rules only modify the request. They do not have any impact on routing decisions being made.
Following is the list of existing modifier rules:
AddPrefix: /products: Add path prefix to the existing request path prior to forwarding the request to the backend.
ReplacePath: /serverless-path: Replaces the path and adds the old path to the X-Replaced-Path header. Useful for mapping to AWS Lambda or Google Cloud Functions.
ReplacePathRegex: ^/api/v2/(.*) /api/$1: Replaces the path with a regular expression and adds the old path to the X-Replaced-Path header. Separate the regular expression and the replacement by a space.
To route requests, you should use matchers:
Matcher rules determine if a particular request should be forwarded to a backend.
Separate multiple rule values by , (comma) in order to enable ANY semantics (i.e., forward a request if any rule matches). Does not work for Headers and HeadersRegexp.
Separate multiple rule values by ; (semicolon) in order to enable ALL semantics (i.e., forward a request if all rules match).
###Path Matcher Usage Guidelines
This section explains when to use the various path matchers.
Use Path if your backend listens on the exact path only. For instance,
Path: /products would match /products but not /products/shoes.
Use a *Prefix* matcher if your backend listens on a particular base
path but also serves requests on sub-paths. For instance, PathPrefix: /products would match /products but also /products/shoes and
/products/shirts. Since the path is forwarded as-is, your backend is
expected to listen on /products.
Use a *Strip matcher if your backend listens on the root path (/) but
should be routable on a specific prefix. For instance,
PathPrefixStrip: /products would match /products but also
/products/shoes and /products/shirts. Since the path is stripped prior
to forwarding, your backend is expected to listen on /. If your
backend is serving assets (e.g., images or Javascript files), chances
are it must return properly constructed relative URLs. Continuing on
the example, the backend should return /products/shoes/image.png (and
not /images.png which Traefik would likely not be able to associate
with the same backend). The X-Forwarded-Prefix header (available since
Traefik 1.3) can be queried to build such URLs dynamically.
Instead of distinguishing your backends by path only, you can add a
Host matcher to the mix. That way, namespacing of your backends
happens on the basis of hosts in addition to paths.
Full list of matchers and their descriptions can be found here

Related

How to pass extra http headers to Okteto pod?

I've deployed the Duende IdentityServer to Okteto Cloud: https://id6-jeff-tian.cloud.okteto.net/.
Although the endpoint is https from the outside, the inside pods still think they are behind HTTP protocol. You can check the discovery endpoint to find out: https://id6-jeff-tian.cloud.okteto.net/.well-known/openid-configuration
That causes issues during some redirecting. So how to let the inner pods know that they are hosted in https scheme?
Can we pass some headers to the IdP to tell it the original https schema?
These headers should be forwarded to the inner pods:
X-Forwarded-For: Holds information about the client that initiated the request and subsequent proxies in a chain of proxies. This parameter may contain IP addresses and, optionally, port numbers.
X-Forwarded-Proto: The value of the original scheme, should be https in this case.
X-Forwarded-Host: The original value of the Host header field.
I searched from some aspnet documentations and found this: https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/proxy-load-balancer?source=recommendations&view=aspnetcore-6.0, however, I don't know how to configure the headers in Okteto, or in any k8s cluster.
Is there anyone who can shed some light here?
My ingress configurations is as follows (https://github.com/Jeff-Tian/IdentityServer/blob/main/k8s/app/ingress.yaml):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: id6
annotations:
dev.okteto.com/generate-host: id6
spec:
rules:
- http:
paths:
- backend:
service:
name: id6
port:
number: 80
path: /
pathType: ImplementationSpecific
The headers that you mention are being added to the request when it’s forwarded to your pods.
Could you dump the headers on the receiving end?
Not familiar with Duende, but does it have a setting to specify the “public URL”? That’s typically what I’ve done in the past for similar setups.

How to use ingress so that services can talk to each other?

On AWS EKS, I have three pods in a cluster each of which have been exposed by services. The problem is the services can not communicate with each other as discussed here Error while doing inter pod communication on EKS. It has not been answered yet but further search said that it can be done through Ingress. I am having confusion as to how to do it? Can anybody help ?
Code:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: ingress-test
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: server-service
port:
number: 8000
My server-service has APIs like /api/v1/getAll, /api/v1/updateAll, etc.
So, what should I write in path and for a database service what should I do??
And say in future I make another microservice and open another service which has APIs like /api/v1/showImage, /api/v1/deleteImage will I have to write all paths in ingress or is their another way for it to work?
A Ingress is a really good solution to expose both a frontend and a backend at the same domain with different paths (but reading your other question, it will be of no help in exposing the database)
With this said, you don't have to write all the paths in the Ingress (unless you want to) as you can instead use pathType: Prefix as it is already in your example.
Let me link you to the documentation Examples which explain how it works really well. Basically, you can add a rule with:
path: /api
pathType: Prefix
In order to expose your backend under /api and all the child paths.
The case where you put a second backend, which wants to be exposed under /api as the first one, is way more complex instead. If two Pods wants to be exposed at the same paths, you will probably need to list all the subpaths in a way that differentiate them.
For example:
Backed A
/api/v1/foo/listAll
/api/v1/foo/save
/api/v1/foo/delete
Backend B
/api/v1/bar/listAll
/api/v1/bar/save
/api/v1/bar/delete
Then you could expose one under subPath /api/v1/foo (Prefix) and the other under /api/v1/bar (Prefix).
As another alternative, you may want to expose the backends at different paths from what they actually expect using a rewrite target rule.

Can Ingress route requests based on ip?

I have been with K8s-ingress well so far but I have a question.
Can ingress route requests based on IP?
I've already know that ingress do routing based on hosts like a.com, b.com... to each services and URI like path /a-service/, /b-service/ to each services.
However, I'm curious with the idea that Ingress can route by IP? I'd like requests from my office(certain ip) to route a specific service for tests.
Does it make sense? and any idea for that?
If this is just for testing I would just whitelist the IP. You can read the docs about nginx ingress annotations
You can specify allowed client IP source ranges through the nginx.ingress.kubernetes.io/whitelist-source-range annotation. The value is a comma separated list of CIDRs, e.g. 10.0.0.0/24,172.10.0.1.
Example yaml might look like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: whitelist
annotations:
nginx.ingress.kubernetes.io/whitelist-source-range: "1.1.1.1/24"
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
serviceName: echoheaders
servicePort: 80
Also it looks like you can do that in Istio (I did not tried it) in kind ServiceRole and ServiceRoleBinding for specifying detailed access control requirements. For this you would use source.ip property. It's explained on Constraints and Properties
This is not part of the main Ingress abstraction as you noted, however many Ingress Controllers offer extra features through annotations or secondary CRDs. So in theory it could be added like that. I don't think any do routing like this though, so in practical terms, probably not available off the shelf.
As coderanger stated in his answer, ingress does not have it by default.
I'm not sure if IP based routing is the way to proceed, because how will you test/hit actual deployments/services from Office IP's when needed?
I think you can add a check to perform routing based on IP and header. For ex: you can pass a header 'redirect-to-test: true'. So if you set this to false, you can still access the production services.

Is there an order to how an Ingress rule is chosen?

Is there an order to Ingress rule specification i.e. will the first qualifying rule be honored?
The intention of following spec is to route all requests that do not have headers Host: foo.com and Host: bar.com and route them to service3. I am not sure if the spec is syntactically correct and more so, if it will serve the desired purpose?
spec:
rules:
- host: foo.com
http:
paths:
- backend:
serviceName: service1
servicePort: http
path: /
- host: bar.com
http:
paths:
- backend:
serviceName: service2
servicePort: http
path: /a/b/c
- http:
paths:
- path: /a/b/c
backend:
serviceName: service3
servicePort: http
Don't think it matters, but I am using the Contour Ingress controller.
I am not familar with Contour, I just quickly browse the doc.
How Contour works with Ingress rule is not clear from its document. I think Contour perfer using its CRD IngressRoute to specify how request rule works.
So I infer your Ingress behavior from offical kubernetes ingress rule:
Request with host: foo.com will route to service1 or service3
Request with host: bar.com will route to service2 or service3
Other request will route to service3
What you have created in syntactically correct and should route http://*/a/b/c and http://*/a/b/c/* to service3 in most ingress controllers.
An Ingress definition is just data that is supplied to an ingress controller though. The implementation of converting that data into config is ingress controller specific.
Code
Contours route config looks to be rooted by a "virtualhost" name. In the route.go code I can't see any handling of the "no virtualhost" case.
From the route.go tests it looks like a virtualhost of * is how the default host is handled.
This sorting of virtualhosts would hopefully always put * in the right place for contour to default as you describe but then I think there is also an interface for this config to be applied to envoy, the actual proxy process.
So it would appear (without testing) that no matter what order you put the Ingress definition in, contour will sort out the default host routes for you as '*'
This kind of makes sense when you consider contour also supports a custom resource definition of IngressRoute which only allows one virtualhost per definition. These CRDs as a group have no specific ordering so the sort is required.

cert-manager is creating new ingress with acme responder instead of modifying the existing

I'm trying to use cert-manager to issue a certificate via LetsEncrypt.
I've followed through with the steps here http://docs.cert-manager.io/en/latest/getting-started/index.html
However, my existing ingress is not being modified (I assume it needs to modify it due to adding a path for .well-known/....
Instead I see an ingress created for this with a name like: cm-acme-http-solver-kgpz6? Which is rather confusing?
If I get the yaml for that ingress I see the following for rules:
spec:
rules:
- host: example.com
http:
paths:
- backend:
serviceName: cm-acme-http-solver-2dd97
servicePort: 8089
path: /.well-known/acme-challenge/2T2D_XK1-zIJJ9_f2ANlwR-AcNTm3-WenOExNpmUytY
How exactly is this meant to work? As the documentation seems rather sparse.
The record you are seeing is for the challenge. It needs to succeed to configure the cert. If you are using "example.com" as the domain then it will not succeed. To get this to work you'll need to configure a DNS record for a valid hostname so that LetsEncrypt can resolve the domain and complete the check.
Usually you will not even see the challenge ingress resource. It usually runs the challenge and then removes itself as long as DNS and the hostname have been configured correctly. After it is removed the resource you created will get loaded into your ingress controller.
There are a few ingress controllers that do not support multiple ingress resources per hostname. They will load one ingress resource and ignore the other, so this is sort of a workaround/fix to the issue.