I am trying to setup Prometheus in k8 cluster, able to run using helm. Accessing dashboard when i expose prometheus-server as LoadBalancer service using external ip.
Same does not work when I try to configure this service as ClusterIP and making it as backend using ingress controller. Receiving 404 error, any thoughts on how to troubleshoot this?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
with above ingress definition in place, url “http://<>/prometheus/ getting redirected to http://<>/graph/ and then 404 error page getting rendered. When url adjusted to http://<>/prometheus/graph some of webcontrols gets rendered with lots of errors on browser console.
Prometheus might be expecting to have control over the root path (/).
Please change the Ingress to prometheus.example.com and it should work fine. (Changing it to a subdomain)
Please change your Ingress configuration file, add host field:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ops-ingress
annotations:
#nginx.org/server-snippet: "proxy_ssl_verify off;"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: prometheus.example.com
http:
paths:
- path: /prometheus(/|$)(.*)
backend:
serviceName: prometheus-server
servicePort: 80
then apply changes executing command:
$ kubectl aply -f your_ingress_congifguration_file.yaml
The host header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.
Please take a look here: hosts-header.
Ingress definition: ingress.
Useful information: helm-prometheus.
Useful documentation: ingress-path-matching.
Related
I am trying to deploy Jaeger all-in-one image in a kubernetes cluster.
Jaeger is not in the root of the url, meaning it's accessible through https://somedomain.com/xyz/jaeger
I have an ingress rule which seems to be pointing correctly to a Service which is also referencing fine the pod in a deployment (I can see all this in Rancher UI).
But somehow when I try to access, nginx is throwing a 502 Bad Gateway error.
This is how the ingress rule looks like
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
namespace: my-namespace
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: somedomain.com
http:
# Jaeger
- path: /xyz/jaeger(/|$)(.*)
pathType: Prefix
backend:
service:
name: jaeger
port:
number: 16868
Then in the pod definition I tried using the QUERY_BASE_PATH env var setting it to /xyz/jaeger but that made no difference at all.
The problem was an incorrect port being specified.
16868 instead of 16686
So I am in the process of migrating my bare metal cluster onto GKE and ran into an issue with the ingress. On my bare metal cluster, I used the ingress controller from nginxinc which worked fine. Below is a sample of an Ingress file of a particular deployment:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
namespace: mynamespace
name: app-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/ssl-redirect: "false"
nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
rules:
- host: myhost
http:
paths:
- path: /dev/appname(/|$)(.*)
backend:
serviceName: app
servicePort: 80
Basically, when a user types in myhost/dev/appname, myhost is resolved to a HAProxy server. The proxy then routes the request to the NodePort that the ingress service is running on.
I tried to do the same thing on my GKE cluster with the only exception being that the Ingress controller on the GKE cluster is exposed using a LoadBalancer as per the documentation
However I get a 502 error if I try to access the URL.
TL;DR: Looking for the best way to access various applications (deployments) deployed on a GKE cluster using URL's like: myhost/dev/firstapp and myhost/dev/secondapp etc.
You can use Kong Ingress as your ingress controller on GKE for your path based ingresses. You can install Kong Ingress from GCP Marketplace. It is easy to integrate and also supports various plugins for authenticating, monitoring etc.
You'll get detailed information and installation instructions from https://github.com/Kong/google-marketplace-kong-app#basic-usage
I would follow this guide on setting up Nginx on GKE. The ingress looks like below:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-resource
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /hello
backend:
serviceName: hello-app
servicePort: 8080
You should be able to access your app externally http://external-ip-of-ingress-controller/hello
Now to debug 502 issue verify that the health-checks from the Loadbalancer to your app is passing or not.
I have an Ingress object set up to route traffic to the appropriate Service based on the Url path. I would like to access/expose this Ingress object within another Pod. I'm wondering if this is possible?
I tried to set up a Service on the Ingress but that didn't seem to work.
So, for whatever reason (ssr, lots of microservices, etc) you want to access k8s resources using their ingress path mapping, instead of calling each service by its internal name.
For example, you have an ingress config like that:
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: ingress-service
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/use-regex: 'true'
spec:
rules:
- host: example.com
http:
paths:
- path: /api/users/?(.*)
backend:
serviceName: auth-service
servicePort: 80
- path: /api/cart/?(.*)
backend:
serviceName: cart-service
servicePort: 80
and you want to access auth-service using http://example.com/api/auth instead of http://auth-service.
All you have to do is replace domain part (example.com in our case) with ingress service url. It depends on your configuration and environment, but usually it looks like http://[SERVICE_NAME].[NAMESPACE], for example:
GCP - http://ingress-nginx-controller.ingress-nginx
Helm ingress nginx - http://my-release-ingress-nginx-controller (here we are
using only service name part, because helm installs ingress in
default namespace)
Minikube - if you are using minikube ingress
addon, then you might run into problem where you cannot access
ingress, then just use helm version. (dont disable ingress addon - just install helm version alongside of it)
Get namespaces: kubectl get namespaces
Get service names inside namespace kubectl get services -n [NAMESPACE].
If you have assigned a host name, you also have to provide the domain name and IP address of the cluster to the /etc/hosts file. When you access a service via Ingress from outside the cluster, this is the file that is consulted for host name resolution.
However, a pod running inside a cluster does not have access to this /etc/hosts file. It has its own /etc/hosts file. To use ingress, the pod needs to have the same domain name and IP address entry in it's own /etc/hosts file.
To achieve this, you have to use hostAliases. Here's a sample of how that works:
apiVersion: v1
kind: Pod
metadata:
...
spec:
hostAliases:
- ip:<IP address>
hostnames:
- <host name>
For more detail on hostAliases, go to this link
I have spend so much time on this issue. I found very simple solution. I am using Mac Docker Desktop 3.3.1.
My Kubernetes Version: 1.19.7
I am trying to access UI URL from another pod running in the cluster.
My UI Service
apiVersion: v1
kind: Service
metadata:
name: my-ui-service
spec:
type: LoadBalancer
selector:
app: my-ui
ports:
- protocol: TCP
port: 8080
targetPort: 8080
Ingress for the service
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
spec:
rules:
- host: my-site.com
http:
paths:
- pathType: Prefix
path: /
backend:
service:
name: my-ui-service
port:
number: 8080
I have used NGINX Ingress Controller.
Command to run the Ingress Controller:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.48.1/deploy/static/provider/cloud/deploy.yaml
Once the controller is ready run the command to see the status of ingress.
kubectl get ingress
Now see the description of the ingress:
kubectl describe ingress my-ingress
Here you will find
Rules:
Host Path Backends
---- ---- --------
my-site.com
/ my-ui-service:8080 (10.1.2.198:8080)
In any pod in the cluster you can access the domain my-site.com by using my-ui-service:8080.
Inside your cluster your pods use services to reach other pods.
From outside the cluster a client may use ingress to reach services.
Ingress resource allows connection to services.
So your pod need to be reachable by a service (my-svc-N in the following example), which you're going to use in your ingress definition.
Take a look at this example:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ing
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
backend:
serviceName: default-http-backend
servicePort: 80
rules:
- host: my-kube.info
http:
paths:
- path: /
backend:
serviceName: my-svc-1
servicePort: 80
- host: cheeses.all
http:
paths:
- path: /aaa
backend:
serviceName: my-svc-2
servicePort: 80
- path: /bbb
backend:
serviceName: my-svc-3
servicePort: 80
I can't get the nginx controller to route based on the hostname. The YAML below doesn't work - traffic goes to the default back-end / I get a 404. However, if I remove the value for host, the ingress controller successfully routes traffic to my-service. The service works successfully if I place it behind a load balancer but I want to have multiple services working for different host names so I want to use an ingress controller and use a single IP. Thoughts?
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nginx
annotations:
ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: test1.mydomain.com
http:
paths:
- path: /
backend:
serviceName: my-service
servicePort: 80
The yaml looks slightly different than the rewrite example located here. The yaml is valid and kubectl apply or create should work but not produce the results you are expecting. Do you need the rewrite annotation or could you remove it and the back end service will respond without issue? If you don't need to rewrite anything try removing the yaml to just look like:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-nginx
spec:
rules:
- host: test1.mydomain.com
http:
paths:
- backend:
serviceName: my-service
servicePort: 80
Is it possible to have a fallback service for Kubernetes ingresses in the event that none of the normal pods are live/ready? In other words, how would you go about presenting a friendly "website down" page to visitors if all pods crashed or went down somehow?
Right now, a page appears that says "default backend - 404" if that happens.
Here's what we tried, to no avail:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
spec:
backend:
serviceName: website-down-service
servicePort: 80
rules:
- host: example.com
http:
paths:
- path: /
backend:
serviceName: example-service
servicePort: 80
For reference, we're testing locally with Minikube and deploying to the cloud on Google's Container Engine.
If using Nginx then default backend annotation should do the trick, sample:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: ingress-name
namespace: your-namespace
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/default-backend: fallback-backend
spec:
<your spec here>
For the Nginx Ingress Controller there is a flag --default-backend-service, which currently points to the service showing the "default backend - 404" message. Just replace it with the service you want. See https://github.com/kubernetes/ingress/tree/master/controllers/nginx#command-line-arguments
If you're using another Ingress Controller, I expect it to have a similar option.