I am trying to deploy the kube-prometheus-stack.
I have added it as a dependency in the Chart.yaml as below.
...
dependencies:
- name: kube-prometheus-stack
version: 13.4.1
repository: https://prometheus-community.github.io/helm-charts
...
I have also configured an ingress rule to route the /grafana/?(.*) path to the service solutions-helm-grafana at port 80.
- path: /grafana/?(.*)
pathType: Prefix
backend:
service:
name: helm-grafana
port:
number: 80
However, when I try open /grafana/ in the browser, it returns a 404 after redirecting to /login. What templates do I need to add to successfully deploy ? Are there any examples that I can refer to ?
Hi #Moses can you try removing the ?(.*) from the path?
404 comes up when the ingress is not registered with the ingress controller.
Probably because the release has not been deployed successfully.
Try the following steps to debug the issue:-
Check if the pod have been deployed using kubectl get pods.
Try debugging the ingress object. kubectl describe ing <ing_object_name>.
Check if the Endpoints have been created using kubectl get ep.
Next,get the service endpoint using kubectl get service.
Use a busybox pod to curl and check whether the Grafana is being served via the service above.
Update:-
Add the following configuration to serve Grafana on a subpath
env:
GF_SERVER_DOMAIN: <domain>
GF_SERVER_ROOT_URL: https://<domain>/grafana/
GF_SERVER_SERVE_FROM_SUB_PATH: true
and use this path in ingress:
path: /grafana/
Sources:
Run Grafana behind reverse-proxy
Grafana confiugration root_url
Related
I am trying to deploy a SpringbBoot Java application hosted on Apache Tomcat on a Kubernetes cluster using Ngninx Ingress for URL routing. More specifically, I am deploying on Minikube on my local machine, exposing the SpringBoot application as a Cluster IP Service, and executing the
minikube tunnel
command to expose services to my local machine. Visually, the process is the following...
Browser -> Ingress -> Service -> Pod hosting docker container of Apache Tomcat Server housing SpringBoot Java API
The backend service requires a header "SM_USER", which for the moment can be any value. When running the backend application as a Docker container with port forwarding, accessing the backend API works great. However, when deploying to a Kubernetes cluster behind an Nginx ingress, I get 403 errors from the API stating that the SM_USER header is missing. I suspect the following. My guess is that the header is included with the request to the ingress, but removed when being routed to the backend service.
My setup is the following.
Deploying on Minikube
minikube start
minikube addons enable ingress
eval $(minikube docker-env)
docker build . -t api -f Extras/DockerfileAPI
docker build . -t ui -f Extras/DockerfileUI
kubectl apply -f Extras/deployments-api.yml
kubectl apply -f Extras/deployments-ui.yml
kubectl expose deployment ui --type=ClusterIP --port=8080
kubectl expose deployment api --type=ClusterIP --port=8080
kubectl apply -f Extras/ingress.yml
kubectl apply -f Extras/ingress-config.yml
edit /etc/hosts file to resolve mydomain.com to localhost
minikube tunnel
Ingress YAML
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: my-ingress
annotations:
kubernetes.io/ingress.class: "nginx"
# Tried forcing the header by manual addition, no luck (tried with and without)
nginx.ingress.kubernetes.io/configuration-snippet: |
proxy_set_header SM_USER $http_Admin;
spec:
rules:
- host: mydomain.com
http:
paths:
- path: /ui
pathType: Prefix
backend:
service:
name: ui
port:
number: 8080
- path: /api
pathType: Prefix
backend:
service:
name: api
port:
number: 8080
Ingress Config YAML
kind: ConfigMap
apiVersion: v1
metadata:
name: nginx-config
namespace: default
labels:
app.kubernetes.io/name: my-ingress
app.kubernetes.io/part-of: my-ingress
data:
# Tried allowing underscores and also using proxy protocol (tried with and without)
enable-underscores-in-headers: "true"
use-proxy-protocol: "true"
When navigating to mydomain.com/api, I expect to receive the ROOT API interface but instead receive the 403 error page indicating that the SM_USER is missing. Note, this is not a 403 forbidden error regarding the ingress or access from outside the cluster, the specific SpringBoot error page I receive is from within my application and custom to indicate that the header is missing. In other words, my routing is definitely correct, and I am able to access the API, its just that the header is missing.
Are there configs or parameters I am missing? Possibly an annotation?
This is resolved. The issue was that the config was being applied in the Application namespace. Note, even if the Ingress object is in the application namespace, if you are using minikubes built in ingress functionality, the ConfigMap must be applied in the Nginx ingress namespace.
How do I map a service containing a static webpage under a subpath using ambassador in kubernetes ?
this is my yaml file
apiVersion: getambassador.io/v2
kind: Mapping
metadata:
name: grafana
spec:
prefix: /grafana/
method: GET
service: monitoring-grafana.default:80
timeout_ms: 40000
**
and this is the response i get when trying to navigate
If you're seeing this Grafana has failed to load its application files
This could be caused by your reverse proxy settings.
If you host grafana under subpath make sure your grafana.ini root_url setting includes subpath
If you have a local dev build make sure you build frontend using: yarn start, yarn start:hot, or yarn build
Sometimes restarting grafana-server can help
**
Have you read through the Monitoring page on the Ambassador Docs? There's a bit about implementing Prometheus + Grafana if that helps: https://www.getambassador.io/docs/latest/howtos/prometheus/
To start playing with Kubernetes I created a small project where I create a three nodes cluster on my local machine using Vagrant and Vbox:
https://github.com/sasadangelo/k8s-cluster
Just typing vagrant up my cluster is up and running and I can start to play with it. I got familiar with the main concepts and commands and created a simple "Hello World" application composed by a docker image with an NGINX web server where you can connect via browser and a "Hello World" message appears with the hostname. The hostname helps me to understand which Pod responded to my request.
Here the project:
https://github.com/sasadangelo/k8s-tutorials/tree/master/hello-k8s
Currently, I can create a deployment with kubectl create exposing it as a service with kubectl expose and then increase replica with kubect scale. At the moment, I am able to use use NodePort service, so with kubectl describe service I can see on which IP the 5 Pods listen, I connect to them and everything works fine.
My problem is that now I want to load balance the traffic. I want to connect to a single IP and as soon as I click on the browser Reload button I want to see that different Pod names appear.
I know that Kubernetes by default offers only NodePort service and if I want a load balancer I need something like an Ingress. I also know that a simple reverse proxy like Ingress Nginx is a good implementation for my needs.
However, I read lot of tutorials but I have hard time understanding how I should configure it to achieve what I need.
Here code I have so far:
https://github.com/sasadangelo/k8s-cluster/tree/master/ingress
Can anyone help me how to fix Ingress code to achieve what I need?
Kube Proxy which is a kubernetes component provides load balancing at L4 layer via iptables load balancing.So with ClusterIP type service itself you get load balancing between pods. But with clusterIP type service you can not access it from outside the cluster i.e from a browser. Nodeport type service provides access to the pods from outside the cluster with load balancing at L4 layer by kube proxy.
LoadBalancer and Ingress provides load balancing and rule based routing at L7 layer if that's what you are looking for.
Looking at your code you have yamls for nginx ingress controller and exposed it as Nodeport. What you need next is to create an ingress resource
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-world.info
http:
paths:
- path: /
backend:
serviceName: web
servicePort: 8080
Once you create above ingress and you have a ClusterIP type service for your pod with a name web which exposed port 8080, You should be able to access it.
Here is the doc on using nginx ingress for an hello world application on Minikube which you can follow to make ingress work in your setup as well.
Here is what I have done in detail to solve the issue. First of all, I installed Nginx Ingress resources:
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/provider/baremetal/service-nodeport.yaml
then I create an Nginx Ingress resource in a ingress-nginx.yaml file:
---
apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
name: hello-k8s-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: hello-k8s.info
http:
paths:
- path: /
backend:
serviceName: hello-k8s
servicePort: 80
and installed it with the command:
kubectl apply -f ingress-nginx.yaml
You can see code here.
Inside the cluster (doing vagrant ssh on whatever node) I tested:
curl -H "Host: hello-k8s.info" IP
where IP is got from the command:
kubectl get ingress
Suppose it is 10.97.139.101. Outside the cluster, on my Mac (Running on 3 VirtualBox on my Mac) to access the application I needed to add a route:
sudo route add -net 10.97.0.0/16 192.168.205.10
where 192.168.205.10 is the IP of the master node. In /etc/hosts I added the line:
10.97.139.101 hello-k8s.info
Now typing hello-k8s.info in the browser I see the "Hello World" web page appear.
Thanks to Arghya Sadhu for help.
In my kubernetes cluster I would like to do monitoring so I installed grafana.
I would like to access the grafana dashboard as http://example.com/monitoring, so I tried to include this in my ingress configuration
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: example-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
rules:
- host: example.com
http:
paths:
- path: /monitoring/(.*)
backend:
serviceName: grafana
servicePort: 80
The idea is to add other paths there as well, for example / for the website.
I noticed that Grafana redirects http://example.com/monitoring to http://example.com/login. Of course this should have been http://example.com/monitoring/login. What would be the preferred way to fix this. Can it be done with ingress or should I somehow tell Grafana that it is behind a /monitoring path (if possible)?
I've installed grafana using this using Helm.
UPDATE: I've modified as suggested below the grafana chart's file values.yaml as follows
grafana.ini:
server:
domain: example.com
root_url: http://example.com/monitoring/
Now I get:
And the heml command I use to install grafana:
$> helm install stable/grafana -f values.yaml --set persistence.enabled=true --set persistence.accessModes={ReadWriteOnce} --set persistence.size=8Gi -n grafana
This is a common problem with services that are behind an HTTP reverse-proxy. Luckily, Grafana offers a way to let it know the context path it is running behind.
In grafana.ini (which is most possibly supplied via a ConfigMap to its Kubernetes deployment), you need to specify the variables like the following:
[server]
domain = example.com
root_url = http://example.com/monitoring/
See the full documentation here: https://grafana.com/docs/installation/behind_proxy/
I have the bare metall kubernetes pod running tomcat application on port 8085. If it would be common server, the app would be accessible via http://<server-ip>:8085/app. My goal is to expose the tomcat on Kubernetes node's address and the same port as used in tomcat.
I am able to expose and access app using Node Port service - but it is inconvenient that port is always different.
I tried to setup traefik ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: test-tag2
spec:
rules:
- host: kubernetes.example.com #in my conf I use node's domain name
http:
paths:
- path: /test
backend:
serviceName: test-tag2
servicePort: 8085
And I can see result in Traefik's dashboard, but still if I navigate to http://kubernetes.example.com/test/app I get nothing.
I've tried a bunch of ways to configure that and still no luck.
Is it actually possible to expose my pod in this way?
Did you try specifying a nodePort value in the service yaml? If specified, kubernetes will create service on the specified NodePort. If the nodePort is not available , kubernetes doesn't create the service.
Refer to this answer for more details:
https://stackoverflow.com/a/43944385/1237402