We have one service which has two exposed ports- one is for server and second is for management. Kubernetes cluster have NGINX controller running. Is there any way to expose both the ports of service in Ingress YAML file?
Details:
Service server Port : 8081
Service Management Port: 8082
We tried with two "backends" in Ingress YAML file but it works only for default port which is 80 and fail for any other port.
ingress.yml file is as below:
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: bw-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
spec:
rules:
- host: my.host.net
http:
paths:
- path: "/controller/"
backend:
serviceName: bw-svc1
servicePort: 80
- path: "/actuator/"
backend:
serviceName: bw-svc2
servicePort: 5612
The indentation level of both paths is not equal in your config sample.
Other than that your config looks correct and should result in the following behavior:
All traffic coming to the cluster via my.host.net/controller will be sent to service bw-svc1:80
All traffic coming to the cluster via my.host.net/actuator will be sent to service bw-svc2:5612
If this is not what you'd like to achieve, could you please re-phrase your question?
Related
I am trying to setup Prometheus in k8 cluster, able to run using helm. Accessing dashboard when i expose prometheus-server as LoadBalancer service using external ip.
Same does not work when I try to configure this service as ClusterIP and making it as backend using ingress controller. Receiving 404 error, any thoughts on how to troubleshoot this?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
with above ingress definition in place, url “http://<>/prometheus/ getting redirected to http://<>/graph/ and then 404 error page getting rendered. When url adjusted to http://<>/prometheus/graph some of webcontrols gets rendered with lots of errors on browser console.
Prometheus might be expecting to have control over the root path (/).
Please change the Ingress to prometheus.example.com and it should work fine. (Changing it to a subdomain)
Please change your Ingress configuration file, add host field:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: prometheus.example.com
http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
then apply changes executing command:
$ kubectl aply -f your_ingress_congifguration_file.yaml
The host header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.
Please take a look here: hosts-header.
Ingress definition: ingress.
Useful information: helm-prometheus.
Useful documentation: ingress-path-matching.
I created a service and each service is creating a new load balancer, I don't want to create a new load balancer for each service. For that, I found solution ingress controller but it's not happening.
I will try to describe the objects you need in just words.
You don't need to create a load balancer for each service. When you're using an ingress controller (like nginx), the ingress controller itself will be the type load balancer. All your other services need to be something like ClusterIP type.
Afterwards you can decide how to link your ClusterIP services with the Nginx LoadBalancer: create an ingress for each service or one ingress that exposes each service based on some rule (like paths as #harsh-manvar shows in the post above).
When you say "it's not happening", it would be good if you could provide details on your setup.
In order for Nginx ingress controller to work, it needs to be defined either as a NodePort or LoadBalancer service type. The examples provided in the nginx documentation are using LoadBalancer. However, LoadBalancer only works when your cluster supports this object (that means running in most cloud providers like AWS/GCP/Azure/DigitalOcean or newer versions of minikube). On the other hand, NodePort will expose the ingress controller on the Kubernetes node where it runs (when using minikube, that usually means a VM of sorts which then needs to be port forwarded to be accessible).
To use ingress in a local environment, you can look into minikube. All you need is to run minikube addons enable ingress and it will deploy an nginx controller for you. Afterwards, all you need to do is define an ingress and depending on your setup you may need to use kubectl port-forward to port forward port 80 on an nginx controller pod to a local port on your machine.
There are different types of services: ClusterIP, NodePort, LoadBalancer and ExternalName. You can specify it in spec.type. Actually the default one, when not specified is not LoadBalancer, but ClusterIP, so in your case, simply leave away the type: LoadBalancer definition and use your serviceName as backend in your ingress resource. Example:
spec:
rules:
- host: your.fully.qualified.host.name
http:
paths:
- backend:
serviceName: your-internal-service-name
servicePort: 80
path: /
Keep in mind that for some cloud providers there's also the possibility to use an internal LoadBalancer without a public IP. This is done by adding an annotation to the service configuration. For Azure AKS it looks like this:
metadata:
annotations:
service.beta.kubernetes.io/azure-load-balancer-internal: "true"
For Google's GKE the annotation is cloud.google.com/load-balancer-type: "Internal"
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress
annotations:
kubernetes.io/ingress.class: "nginx"
certmanager.k8s.io/cluster-issuer: wordpress-prod
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
tls:
- hosts:
- test.test.com
secretName: prod
rules:
- host: test.test.com
http:
paths:
- path: /service-1
backend:
serviceName: service-1
servicePort: 80
- path: /service-2
backend:
serviceName: service-2
servicePort: 5000
Sharing here documentation for ingress to target multiple services you can redirect to multi-service.
Using this you can access services like
https://test.test.com/service-1
https://test.test.com/service-2
Following documentation you should do the following.
More information: kubernetes.github.com
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
name: rewrite
namespace: default
spec:
rules:
- host: rewrite.bar.com
http:
paths:
- backend:
serviceName: http-svc
servicePort: 80
path: /something(/|$)(.*)
For example, the ingress definition above will result in the following rewrites:
rewrite.bar.com/something rewrites to rewrite.bar.com/
rewrite.bar.com/something/ rewrites to rewrite.bar.com/
rewrite.bar.com/something/new rewrites to rewrite.bar.com/new
I have an ingress defined as:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: foo-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: zaz-address
kubernetes.io/ingress.allow-http: "false"
ingress.gcp.kubernetes.io/pre-shared-cert: foo-bar-com
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /zaz/*
backend:
serviceName: zaz-service
servicePort: 8080
Then the service zap-service is a nodeport defined as:
apiVersion: v1
kind: Service
metadata:
name: zaz-service
namespace: default
spec:
clusterIP: 10.27.255.88
externalTrafficPolicy: Cluster
ports:
- nodePort: 32455
port: 8080
protocol: TCP
targetPort: 8080
selector:
app: zap
sessionAffinity: None
type: NodePort
The nodeport is successfully selecting the two pods behind it serving my service. I can see in the GKE services list that the nodeport has an IP that looks internal.
When I check in the same interface the ingress, it also looks all fine, but serving zero pods.
When I describe the ingress on the other hand I can see:
Rules:
Host Path Backends
---- ---- --------
foo.bar.com
/zaz/* zaz-service:8080 (<none>)
Which looks like the ingress is unable to resolve the service IP. What am I doing wrong here? I cannot access the service through the external domain name, I am getting an error 404.
How can I make the ingress translate the domain name zaz-service into the proper IP so it can redirect traffic there?
Seems like the wildcards in the path are not supported yet.
Any reason why not using just the following in your case?
spec:
rules:
- host: foo.bar.com
http:
paths:
- path: /zaz
backend:
serviceName: zaz-service
servicePort: 8080
My mistake was, as expected, not reading the documentation thoroughly.
The port stated in the Ingress path is not a "forwarding" mechanism but a "filtering" one. In my head it made sense that it would be redirecting http(s) traffic to port 8080, which is the one where the Service behind was listening to, and the Pod behind the service too.
Reality was that it would not route traffic which was not port 8080 to the service. To make it cleaner I changed the port in the Ingress from 8080 to 80 and in the Service the front-facing port from 8080 to 80 too.
Now all requests coming from the internet can reach the server successfully.
I have an Ingress object set up to route traffic to the appropriate Service based on the Url path. I would like to access/expose this Ingress object within another Pod. I'm wondering if this is possible?
I tried to set up a Service on the Ingress but that didn't seem to work.
So, for whatever reason (ssr, lots of microservices, etc) you want to access k8s resources using their ingress path mapping, instead of calling each service by its internal name.
For example, you have an ingress config like that:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: example.com
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-service
servicePort: 80
- path: /api/cart/?(.*)
backend:
serviceName: cart-service
servicePort: 80
and you want to access auth-service using http://example.com/api/auth instead of http://auth-service.
All you have to do is replace domain part (example.com in our case) with ingress service url. It depends on your configuration and environment, but usually it looks like http://[SERVICE_NAME].[NAMESPACE], for example:
GCP - http://ingress-nginx-controller.ingress-nginx
Helm ingress nginx - http://my-release-ingress-nginx-controller (here we are
using only service name part, because helm installs ingress in
default namespace)
Minikube - if you are using minikube ingress
addon, then you might run into problem where you cannot access
ingress, then just use helm version. (dont disable ingress addon - just install helm version alongside of it)
Get namespaces: kubectl get namespaces
Get service names inside namespace kubectl get services -n [NAMESPACE].
If you have assigned a host name, you also have to provide the domain name and IP address of the cluster to the /etc/hosts file. When you access a service via Ingress from outside the cluster, this is the file that is consulted for host name resolution.
However, a pod running inside a cluster does not have access to this /etc/hosts file. It has its own /etc/hosts file. To use ingress, the pod needs to have the same domain name and IP address entry in it's own /etc/hosts file.
To achieve this, you have to use hostAliases. Here's a sample of how that works:
apiVersion: v1
kind: Pod
metadata:
...
spec:
hostAliases:
- ip:<IP address>
hostnames:
- <host name>
For more detail on hostAliases, go to this link
I have spend so much time on this issue. I found very simple solution. I am using Mac Docker Desktop 3.3.1.
My Kubernetes Version: 1.19.7
I am trying to access UI URL from another pod running in the cluster.
My UI Service
apiVersion: v1
kind: Service
metadata:
name: my-ui-service
spec:
type: LoadBalancer
selector:
app: my-ui
ports:
- protocol: TCP
port: 8080
targetPort: 8080
Ingress for the service
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- host: my-site.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: my-ui-service
port:
number: 8080
I have used NGINX Ingress Controller.
Command to run the Ingress Controller:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.48.1/deploy/static/provider/cloud/deploy.yaml
Once the controller is ready run the command to see the status of ingress.
kubectl get ingress
Now see the description of the ingress:
kubectl describe ingress my-ingress
Here you will find
Rules:
Host Path Backends
---- ---- --------
my-site.com
/ my-ui-service:8080 (10.1.2.198:8080)
In any pod in the cluster you can access the domain my-site.com by using my-ui-service:8080.
Inside your cluster your pods use services to reach other pods.
From outside the cluster a client may use ingress to reach services.
Ingress resource allows connection to services.
So your pod need to be reachable by a service (my-svc-N in the following example), which you're going to use in your ingress definition.
Take a look at this example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ing
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
backend:
serviceName: default-http-backend
servicePort: 80
rules:
- host: my-kube.info
http:
paths:
- path: /
backend:
serviceName: my-svc-1
servicePort: 80
- host: cheeses.all
http:
paths:
- path: /aaa
backend:
serviceName: my-svc-2
servicePort: 80
- path: /bbb
backend:
serviceName: my-svc-3
servicePort: 80
I have multiple MSA on k8s on GKE. Each is on separate subdomain like:
msa1.example.com
msa2.example.com
I have it in single ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: main-ingress
annotations:
kubernetes.io/ingress.global-static-ip-name: lalala-ip-1
kubernetes.io/ingress.allow-http: "false"
spec:
tls:
- hosts:
- msa1.example.com
secretName: msa1-tls
backend:
serviceName: sink
servicePort: 80
rules:
- host: msa1.example.com
http:
paths:
- path: /.well-known/*
backend:
serviceName: letsencrypt
servicePort: 80
- path: /*
backend:
serviceName: lalala
servicePort: 80
- host: msa2.example.com
http:
paths:
- path: /*
backend:
serviceName: lalala2
servicePort: 80
... and all is nice.
The thing is, that I want to have each MSA in separate file.
Problem is this kubernetes.io/ingress.global-static-ip-name: lalala-ip-1 line. If I have it in two ingresses only first started is bounded to IP, but other ones no.
Is there a way, to share IP on GKE ingress controller between two ingresses?
A way around it could be to run your own nginx-ingress controller in your cluster and expose it via LoadBalancer service type. Then you would have 1 IP for your ingress and be able to serve all ingresses via nginx controller by adding annotation kubernetes.io/ingress.class: "nginx"
Reference: https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress/
Confirmed my comment:
Only one resource at a time can use a static external IP address.
https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address
GKE has recently added support for the new Kubernetes Gateway API. Both the GKE Gateway implementation as well as the Kubernetes Gateway API specification are still in alpha at this point.
The Kubernetes Gateway-API, is intended to support use cases, where you have a central Gateway (with a single IP), but want different Routes (with different hostnames or paths), managed in separate objects or even namespaces.
References:
https://gateway-api.sigs.k8s.io/
https://cloud.google.com/kubernetes-engine/docs/concepts/gateway-api