We are using grafana to visualize the influx data. There are multiple dashboard created in. Because of some technical issue there may not be new data in Influx to display in the dashboard because of some downtime.
Is there a possibilities that I can add a panel in all the dashboard with an alert message of the downtime. So that dash board users don't have to go anywhere and notified about the downtime there itself.
Thanks
I think that it's not possible to configure a pop up like you want with grafana.
Find another notification channel (e-mail,discord,slack,...).
If you really want a pop up, this wont be configured in Grafana but in Javascript. To do that, you'll have to custom your Grafana page.For that, i can't help ou.
Related
I am currently setting up grafana alerts. How do I customize my message template so my alert email shows The ip address of the server, the state of the server and the node/instance?
Thank you.
I figured it out once, then recently I updated my grafana instance that wiped my work and I had to figure it out again. It was tough the first time.
You can use the labels that are made available through prometheus in your summary and description sections in your alerts by using the syntax:
{{$labels.instance}}
{{$labels.value}}
https://prometheus.io/docs/prometheus/latest/configuration/template_examples/
The only catch is that you have to use Math expression in the last condition in your alert rule for the labels to be available in the Summary section of the alert.
For example, in our personal alerts we will use something like:
Machine {{$labels.instance}} is not reporting status via win-exporter.
The machine could be offline or the service could be stopped.
I've been looking through the docs and grafana community but can't seem to find a definitive answer to this.
I have Grafana configured witha PostgreSQL datasource and created a dashboard to monitor the number of new sessions being created in my database. This works, and I can see a graph of sessions being generated over time.
My question is regarding where the numbers are recorded, if anywhere? If I have this graph on my dashboard, does it go away and make the query every single time the page is loaded? My main concern is that a user can change the time period, going back potentially years, which would hammer the database grouping all those sessions into time intervals.
If they are not stored anywhere with Grafana, how are people managing this? Would we need to use another 'middle man' to receive all of the stats, and use this as the datasource instead of the PostgreSQL database?
When you configured the datasource, you are telling grafana to pick data from there. In your case it is your postgres db.
Grafana does not copy data anywhere else.
If you want that then you should store that data somewhere else. You can setup monitoring for postgresql and store metrics in some engines like prometheus.
We are evaluating Stackdriver as an alternative to our ELK-stack, I'm missing a few features that I have in kibana (1).
Most important I don't find a way to show the actual logs in a Stackdriver Dashboard, I can only show graphs based on the logs. Changing between two tabs all the time (2 and 3) and adapting the filters on both of them seems very inconvenient for log/error analysis.
Is there a way that I can have a dashboard that also shows logs (based on the filters in the dashboard search)?
There is currently no way to show raw log files in the Metrics Dashboard unfortunately.
You can file a feature request to add this functionality to Stackdriver.
Problem:
I have a dashboard in Grafana which monitors the healthiness of my monitoring services: Prometheis, Alertmanagers, Pushgateways and Grafana itself. It shows simple Up/Down status of these services in Singlestat panels.
When one of my Premetheus (I have one in each datacenter) is down, Singlestat panel which is backed with this Prometheus as a datasource is loading 30s, until it shows "Request error".
Even worse, when I want to have only one panel for each Prometheus instance and combine results from all Prometheis that monitor them (Prometheis in my setup monitor each other). For this I use --mixed-- data source, and in this case, when one of used datasources is down Singlestat panel loads forever, and as down datasource is added in all my Singlestat panels for Prometheis, all these panels load forever.
Also when one of Prometheis stops working, I have a very long loading time of some Grafana pages:
Configuration -> Datasources
and
Dashboards -> Home.
But this is not always, sometimes it loads normally.
Investigations:
I investigated Query timeout in Grafana datasource (set it for 1s), but without any effect on this problem.
I have also tried to add datasource variable. It solves the problem only partially and I am not satisfied with it:
I have a combo box with datasources in Dashboard and Singlestat panel for each Prometheus backed with this variable dastasource. Problem is that I have to change through all the Prometheis in a combo box to see the whole picture for Prometheus services.
Similar it is possible to create Singlestat panels for all combinations of datasources and Prometheus instances (in my case 3 x 3 panels) but it is not intuitive and gets worse and worse with each Prometheus servers I will add in the future.
Question:
Is there any way how to handle unreachable datasources, that dashboards will continue to work?
Maybe I have to add some component to my setup, but I think it should be done in Grafana (although it seems it is not possible).
I use tidb database, but when I want to build monitor. the grafana can't display data. It display "no data points". I check the network in the chrome. and docker logs. they look like right. I don't know why.
Sorry, I want to ask question,but I don't have reputation to post images. so I write the issue in the github.
https://github.com/pingcap/tidb/issues/7509#issuecomment-416438806
Thanks for your feedback. You need to use pushgateway v0.4.0 instead of pushgateway (0.5.1).
You could try https://github.com/pingcap/tidb-docker-compose or https://github.com/pingcap/tidb-operator directly.
We're working on remove pushgateway, and use prometheus to pull metrics directly.