On AWS EKS, I have three pods in a cluster each of which have been exposed by services. The problem is the services can not communicate with each other as discussed here Error while doing inter pod communication on EKS. It has not been answered yet but further search said that it can be done through Ingress. I am having confusion as to how to do it? Can anybody help ?
Code:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
namespace: test
name: ingress-test
annotations:
alb.ingress.kubernetes.io/scheme: internet-facing
alb.ingress.kubernetes.io/target-type: ip
spec:
ingressClassName: alb
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: server-service
port:
number: 8000
My server-service has APIs like /api/v1/getAll, /api/v1/updateAll, etc.
So, what should I write in path and for a database service what should I do??
And say in future I make another microservice and open another service which has APIs like /api/v1/showImage, /api/v1/deleteImage will I have to write all paths in ingress or is their another way for it to work?
A Ingress is a really good solution to expose both a frontend and a backend at the same domain with different paths (but reading your other question, it will be of no help in exposing the database)
With this said, you don't have to write all the paths in the Ingress (unless you want to) as you can instead use pathType: Prefix as it is already in your example.
Let me link you to the documentation Examples which explain how it works really well. Basically, you can add a rule with:
path: /api
pathType: Prefix
In order to expose your backend under /api and all the child paths.
The case where you put a second backend, which wants to be exposed under /api as the first one, is way more complex instead. If two Pods wants to be exposed at the same paths, you will probably need to list all the subpaths in a way that differentiate them.
For example:
Backed A
/api/v1/foo/listAll
/api/v1/foo/save
/api/v1/foo/delete
Backend B
/api/v1/bar/listAll
/api/v1/bar/save
/api/v1/bar/delete
Then you could expose one under subPath /api/v1/foo (Prefix) and the other under /api/v1/bar (Prefix).
As another alternative, you may want to expose the backends at different paths from what they actually expect using a rewrite target rule.
Related
I am fairly new to Kubernetes and have just deployed my first cluster to IBM Cloud. When I created the cluster, I get a dedicated ingress subdomain, which I will be referring to as <long-k8subdomain>.cloud for the scope of this post. Now, this subdomain works for my app. For example: <long-k8subdomain>.cloud/ping works from my browser/curl just fine- I get the expected JSON response back. But, if I add this subdomain to a CNAME record on my domain provider's DNS settings (I have used Bluehost and IBM Cloud's Internet Services), I get a 404 response back from all routes. However this response is the default nginx 404 response (it says "nginx" under "404 Not Found"). I believe this means that this means the ingress load balancer is being reached, but the request does not get routed right. I am using Kubernetes version 1.20.12_1561 on VPC gen 2 and this is my ingress-config.yaml file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
nginx.ingress.kubernetes.io/configuration-snippet: |
more_set_headers "Host: <long-k8subdomain>.cloud";
spec:
rules:
- host: <long-k8subdomain>.cloud
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
I am pretty sure this problem is due to the annotations. Maybe I am using the wrong ones or I do not have enough. Ideally, I would like something like this: api..com/ to route correctly. I have also read a little bit about default backends, but I have not dove too much into that just yet. Any help would be greatly appreciated, as I have spent multiple hours trying to fix this.
Some sources I have used:
https://cloud.ibm.com/docs/containers?topic=containers-cs_network_planning
https://cloud.ibm.com/docs/containers?topic=containers-ingress-types
https://cloud.ibm.com/docs/containers?topic=containers-comm-ingress-annotations#annotations
Note: The reason why I have the second annotation is because for some reason, requests without that header were not being routed directly. So that was part of my debugging process and I just ended up leaving it as I am not sure if that annotation solves that, so I left it for now.
For the NGINX ingress controller to route requests for your own domain's CNAME record to the service instead of the IBM Cloud one, you need a rule in the ingress where the host identifies your domain.
For instance, if your domain's DNS entry is api.example.com, then change the resource YAML to:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress-resource
annotations:
kubernetes.io/ingress.class: "public-iks-k8s-nginx"
spec:
rules:
- host: api.example.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-service-name
port:
number: 80
You should not need the second annotation for this to work.
If you want both of the hosts to work, then you could add a second rule instead of replacing host in the existing one.
First a little background:
We currently have several websites and services hosted on a Plesk server and I am setting up a bare-metal development server to provide an area where we can test updates, etc. before going to production. I am using a 3 node kubernetes cluster running microk8s on Ubunutu 20.04.01. The services we host are pretty diverse: we have a couple Moodle sites, a few Wordpress sites, a site running limesurvery, an instance of Mantis bugtracker, and a few more. I have successfully gotten most of the sites containerized and running on k8s. I can also access each individual site either through a NodePort or a MetalLB load balancer.
However, I'd really like to use the NGINX Ingress Controller on top of the load balancer so that I can have a consistent way to access the sites without using a bunch of IP addresses (or in the NodePort case, ports that change). No matter what I've done, I cannot seem to get the Ingress to do what I want. I simply want to do the following:
http://<LB IP Address>/bugtracker to access the Mantis Bug Tracker site
http://<LB IP Address>/moodle1 to access one of the Moodle sites
http://<LB IP Address>/limesurvey to access the limesurvey,
etc. I seem to be able to get to the main page of the site (e.g. index.html, index.php, etc.), but any references from there on do not work - i.e. they give path not found errors or 404 errors.
Here's a sample of my Ingress file:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*)
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
- path: /moodle(/|$)(.*)
pathType: Prefix
backend:
service:
name: moodle-svc
port:
number: 8080
This ingress does not work (I get the 404s). However, if I only have one path in the file and just use '/' it works (but I can only use it for one service):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
I think what I need is for the path (limesurvey) to continue to be pre-pended onto each URL request, but I cannot seem to make this happen. The reason I think this is because when I navigate to http://<LB IP Address>/limesurvey and click on a survey, I get a 404 error at http://<LB IP Address>/index.php/<survey number>. However, if I manually change the URL in my browser to http://<LB IP Address>/limesurvey/index.php/<survey number> it will access the survey (but still have 404s with supporting assets).
Am I trying to do something outside of what the Ingress controller was designed for? I feel like I should be able to use the rewrite-target to accomplish this but I'm missing something critical.
I appreciate any help.
Some applications relay on static content served from different URL webserver locations and moreover do internal path routing (e.g. you hit "/" path but get served "/admin" section immediately).
In such cases creation of right Ingress rules gets more tricky, and requires you to better understand behavior and constructs of your web application, to predict all possible URL path locations that user may visit (these forced by app internal redirects too), and these sourced by html code as well.
Seems like your case with limesurvey app falls into that category:
Why do I think that?
Just try to open limesurvey public demo (https://demo.limesurvey.org/) and inspect site content.
You will learn that main page is using a lot of static files (e.g. css, javascrpt files), referenced from absolute path starting with: /tmp/assets/...
<link rel="stylesheet" type="text/css" href="/tmp/assets/2d523ae6/survey.css" />
of course variants of different path locations can be matched with single smart reg-ex pattern, to avoid creation of dozen of individual ingress rules (what you tried).
What's the issue?
rules:
- http:
paths:
- path: /limesurvey(/|$)(.*) <---- it won't match "/tmp/assets/..." location
pathType: Prefix
backend:
service:
name: limesurvey-svc
port:
number: 80
Please try to create additional Ingress rule to support static file location (watch out, I'm using old syntax of Ingress resource, adjust it to your needs):
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/rewrite-target: /$1
name: ingress-limesurvey-static
spec:
rules:
- http:
paths:
- backend:
serviceName: limesurvey-svc
servicePort: 80
path: /?(.*)
Best Solution (in my opinion)
You should define custom public URL within your application directly. Detailed information can be found in Advanced path settings, see publicurl option.
This way you wouldn't need to define internal reference for static files, however it should be done during installation.
I have two different deployments creating two different pods that spins up two different containers serves different purposes. but as a coincidence the port being exposed by both of those containers is 8080.
I created a single service with two ports 8080 and 8081(type=LoadBalancer) to expose both of those deployments. when I hit the LoadBalancer url I get back response from container 1 and after hitting refresh few times I get back the response from container 2. This behavior is same on both ports.
I know that changing the port exposed on the dockerfile of one of those containers would solve this problem. but just out of curiosity as a newbie to the kubernetes, is there any different approach to handle this scenario?
You could use Ingress. Here is an example.
Instead of creating one Service for both pods. Create one Service per pod. Make sure the selector labels are different for both. Set type to NodePort. Then create an Ingress with rules like.
spec:
rules:
- host: cafe.example.com
http:
paths:
- path: /tea
backend:
serviceName: tea-svc
servicePort: 80
- path: /coffee
backend:
serviceName: coffee-svc
servicePort: 80
Now there are many ingress solutions out there. Ingress in k8s is just a networking spec. All it is, is a data model that represents your networking logic. The various ingress controllers take that spec and implement the logic with their given solution. Here is a link to the docs for nginx ingress controller. https://www.nginx.com/products/nginx/kubernetes-ingress-controller/
I have been with K8s-ingress well so far but I have a question.
Can ingress route requests based on IP?
I've already know that ingress do routing based on hosts like a.com, b.com... to each services and URI like path /a-service/, /b-service/ to each services.
However, I'm curious with the idea that Ingress can route by IP? I'd like requests from my office(certain ip) to route a specific service for tests.
Does it make sense? and any idea for that?
If this is just for testing I would just whitelist the IP. You can read the docs about nginx ingress annotations
You can specify allowed client IP source ranges through the nginx.ingress.kubernetes.io/whitelist-source-range annotation. The value is a comma separated list of CIDRs, e.g. 10.0.0.0/24,172.10.0.1.
Example yaml might look like this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: whitelist
annotations:
nginx.ingress.kubernetes.io/whitelist-source-range: "1.1.1.1/24"
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /
backend:
serviceName: echoheaders
servicePort: 80
Also it looks like you can do that in Istio (I did not tried it) in kind ServiceRole and ServiceRoleBinding for specifying detailed access control requirements. For this you would use source.ip property. It's explained on Constraints and Properties
This is not part of the main Ingress abstraction as you noted, however many Ingress Controllers offer extra features through annotations or secondary CRDs. So in theory it could be added like that. I don't think any do routing like this though, so in practical terms, probably not available off the shelf.
As coderanger stated in his answer, ingress does not have it by default.
I'm not sure if IP based routing is the way to proceed, because how will you test/hit actual deployments/services from Office IP's when needed?
I think you can add a check to perform routing based on IP and header. For ex: you can pass a header 'redirect-to-test: true'. So if you set this to false, you can still access the production services.
I've recently setup a Kubernetes cluster and I am brand new to all of this so it's quite a bit to take in. Currently I am trying to setup and Ingress for wordpress deployments. I am able to access through nodeport but I know nodeport is not recommended so I am trying to setup the ingress. I am not exactly sure how to do it and I can't find many guides. I followed this to setup the NGINX LB https://github.com/nginxinc/kubernetes-ingress/tree/master/examples/complete-example and I used this to setup the WP Deployment https://docs.docker.com/ee/ucp/admin/configure/use-nfs-volumes/#inspect-the-deployment
I would like to be able to have multiple WP deployments and have an Ingress that resolves to the correct one, but I really can't find much information on it. Any help is greatly appreciated!
You can configure your ingress to forward traffic to a different service depending on path.
An example of such a confugration is this:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /foo
backend:
serviceName: s1
servicePort: 80
- path: /bar
backend:
serviceName: s2
servicePort: 80
Read the kubernetes documentation on ingress for more info.
PS: In order for this to work you need an ingress controller like the one in the links in your question.
If you are on AWS, I highly recommend ALB ingress controller in conjunction with external-dns. These in combination with Wordpress Multisite give you some powerful options when it comes to providing dynamic ingress to new sites.
If you start running into any wonky issues (e.g. unable to login to the admin, redirect loops, disappearing media) after getting that all set up, I wrote a guide on some of the more common issues people run into when running Wordpress on Kubernetes, might be worth having a look!