how to append two variable inside vector,
[30d] should be replaced with [${days}${consd}]
my actual query (part of the query)
method=~"[[Method]]",le="+t"}[30d])) * 100
here i want replace the 30d with two grafana variable say one
grafana variable "days" as text box defaulted with 40, which will be edited by user.
grafana variable "consd" as constant defaulted with value "d".
now i need to update the above query like below but its not working its just taking only the first value.
method=~"[[Method]]",le="+t"}[${days}${consd}])) * 100
method=~"[[Method]]",le="+t"}[${days}d])) * 100
Related
I have a table in Grafana that has several columns and uses the gauge display mode. Setting the min/max values for these columns is troublesome. If the column is a percentage value the max is always known so can be hard coded as 100 or 1. But for example a column displaying database sizes the max is not known and will change.
I am running Grafana 8.1.2 so tried the new 'config from query results' transform for the first time. This works fine to alter the values of a single column but not for more than one.
Grafana Table
As you can see in the attached picture I have set the max for the database size using the new transform but I also need to be able to set the max for the log size column too.
The dashboard has 2 queries in it, both for MSSQLServer. Query A returns the results in a table format and Query B returns the config settings: query result
I've then got the transform set up as follows: transform set up
Is there a way to set the min\max settings for multiple columns using this new transform that I'm missing or some other technique to do it. Unfortunately (for me) Grafana seems to favour time series data so isn't as configurable for table data.
I'm using prometheus with grafana. I have a usecase where I have to take variables dynamically and need to perform divide operation which to be performed for each variable which is coming dynamically so can plot graph at each variable level.
eg. first metrics is -
rate(container_cpu_usage_seconds_total{id="/",instance=~'${INSTANCE:pipe}'}[5m])
where ${INSTANCE:pipe} getting dynamically
which needs to be divided by -
machine_cpu_cores{kubernetes_io_hostname=~'${INSTANCE:pipe}'}
and i want result in format -
1 entry per variable
eg.
vars result
var1 - 102
var2 - 23
var3 - 453
note (var1,var2,var3 are nothing but dynamically passed variables and result is nothing value return by divide operation)
Thanks in advance
After trying some queries found the solution -
My use-case has 2 metrics as below -
container_cpu_usage_seconds_total
machine_cpu_cores
In both metrics I found common label as kubernetes_io_hostname
I grouped both the metrics with the above label with the following queries -
(sort_desc ( max (rate (container_cpu_usage_seconds_total{id="/",kubernetes_io_role="node"}[5m])) BY (kubernetes_io_hostname)
sort_desc(max (machine_cpu_cores{kubernetes_io_role="node"}) BY (kubernetes_io_hostname ))
So my data has only 1 label named kubernetes_io_hostname
Then I did the division of the above 2 metrics and then got the result for the kubernetes_io_hostname label
If you need more info on this let me know in the comment section.
I'm using Grafana table panel to show status of my application.
Currently in Prometheus metrics, I'm returning 1 and 0. Where, 1 mean pass and 0 means fail.
On table view it is also showing as 0 & 1.
Can I change the Prometheus query to return 'PASS' when value is 1, 'FAIL' otherwise?
I faced similar for a graph. I had to convert int value to text for metric. I managed the situation by converting the value to text by using CAST. Simply use the CAST in your metric column. Please note that, I'm using PostgreSQL as a DataSource.
Please check below image for the casting example:
I believe you should look the way to change the display in Grafana.
Under Edit option of the panel
Check the overrides option where you can select
Override fields with name and select value mappings
Add Value mappings as per your requirement
I have also took the screenshot as per the numbers for your reference.
I have multiple Prometheus instances providing the same metric, such as:
my_metric{app="foo", state="active", instance="server-1"} 20
my_metric{app="foo", state="inactive", instance="server-1"} 30
my_metric{app="foo", state="active", instance="server-2"} 20
my_metric{app="foo", state="inactive", instance="server-2"} 30
Now I want to display this metric in a Grafana singlestat widget. When I use the following query...
sum(my_metric{app="foo", state="active"})
...it, of course, sums up all values and returns 40. So I tell Prometheus to sum it by instance...
sum(my_metric{app="foo", state="active"}) by (instance)
...which results in a "Multiple Series Error" in Grafana. Is there a way to tell Prometheus/Grafana to only use the first of the results?
I don't know of a distinct, but I think this would work too:
topk(1, my_metric{app="foo", state="active"} by (instance))
Check out the second to last example in here:
https://prometheus.io/docs/prometheus/latest/querying/examples/
One way I just found is to additionally do an average over all values:
avg(sum(my_metric{app="foo", state="active"}) by(instance))
If you need to return an arbitrary time series out of multiple matching time series, then this can be done with topk() or bottomk() functions. For example, the following query returns a single time series with the maximum value out of multiple time series which match my_metric{app="foo", state="active"}:
topk(1, my_metric{app="foo", state="active"})
You need to set instant query option in Grafana when using topk(). Otherwise topk(1, ...) may return multiple time series when it is used for building a graph with range query. This is because topk(1, ...) selects a single time series with the max value individually per each point on the graph. Different points on the graph may have different time series with the max value. There is a workaround, which allows returning a single series out of many series on a graph in alternative Prometheus-like systems such as VictoriaMetrics. It provides topk_* and bottomk_* functions for this purpose. See, for example, topk_last or topk_avg.
Note that topk() has no common grounds with DISTINCT from SQL. If you need to select distinct label values with PromQL, then you need to use count(...) by (label). It will return unique label values for the given label alongside the number of unique time series per each label value. For example, count(my_metric) by (app) will return unique app label names for time series with my_metric name. This is roughly equivalent to the following SQL with DISTINCT clause:
SELECT DISTINCT app FROM my_metric
See count() docs for details.
I have two exporters for feeding data into prometheus - the node exporter and the elasticsearch exporter. I'm trying to combine sources from both exporters into one query, but unfortunately get "No data points" in the graph.
Each of the series successfully shows data:
elasticsearch_jvm_memory_max_bytes{cluster="$cluster", name=~"$node"}
node_memory_MemTotal{name=~"$node"}
This is the result when I try to subtract the two series from one another:
node_memory_MemTotal{name=~"$node"} - elasticsearch_jvm_memory_max_bytes{cluster="$cluster", name=~"$node"}
What am I missing here?
Thanks.
The subtraction you are trying here is more complex than it reads in the beginning. On both sides of the - operator are queries that can result in one or more time series. So the operation requested works as follows: Execute the query on the left hand side and get a result of one or more time series. A time series means a unique combination of a metric and all its labels and their values. Then a second query for your right hand side is executed which also results in one or more time series. Now to calculate the results, only those combinations with matching label combinations are used.
For your example this means that the metrics from node_exporter and from the elasticsearch_exporter have different label names (or even only different values for the labels). When there are no combinations that exist on both sides, you will see the empty result. For details on how operators are applied, please see the prometheus docs.
To solve your problem, you could do the following:
Check the metrics of both left and right side on their own
Evaluate if there are additional labels that could be ignored
See if there is a good label to match on (e.g. instance / node / hostname)
Use the ignoring(a,b,c) on the required side(s) to drop superfluous dimensions, e.g. the job
Try the following query:
node_memory_MemTotal{name=~"$node"}
- on(name)
sum(elasticsearch_jvm_memory_max_bytes{cluster="$cluster", name=~"$node"}) by (name)
It works in the following way:
It selects all the time series matching the node_memory_MemTotal{name=~"$node"} time series selector.
It selects all the time series matching the elasticsearch_jvm_memory_max_bytes{cluster="$cluster", name=~"$node"} selector.
It groups time series found at step 2 by name label value and sums time series in each group with sum() aggregate function. The end result of the sum(...) by (name) is per-name sums.
It finds pairs of time series with identical name label value from the step 1 and step 3 and calculates the difference between the first and the second time series in each pair. The on(name) modifier is used for limiting the set of labels, which are used for finding time series pairs with matching labels. See more details about this process here.