I used grafana to display data from cloudwatch. I found grafana shows value incorrectly, for example, from this graph, the test_value is 1.000 at time 2021-02-28 07:29:00,
however, from this graph, u can see the test_value is still 1.000 at time 2021-02-28 10:29:00, while the bar graph shows there should be no test_value at this time slot;
it is very confused to see this? maybe the grafana setting is wrong? any suggestion?
You have sparse metric, so Grafana is showing the closest previous value. I would switch Hower tooltip Mode to Single instead of All series to avoid confusion.
You can also use CloudWatch Metric Math with FILL() function to fill the missing values of a metric with the specified filler value when the metric values are sparse.
Related
I want to make a Grafana dashboard with multiple dynamic interval graphs of the same time series. However I want the current intervals to be displayed in the graph legend.
Obviously the multiplication doesn't work in the alias field. Is there any way to have it calculated in the alias?
I've tried naively multiplying it in the alias field and did some googling on dynamic alias calculation, but didn't find anything
I have a Singlestat panel and Graph panel that use an identical query, (Singlestat & Graph query). And, the Singlestat is set to max (Singlestat setting).
Unfortunately, the graph clearly shows a maximum greater than the max singlestat (714 vs ~800): Singlestat vs Graph. Judging from the sparklines on the Singlestat, it seems like the Singlestat's calculations are less granular than the graph's. Can anyone explain why this would be if they're using the same base query? The other singlestat functions (like Min, Avg, etc.) seem to work fine. It's just max that I'm seeing this issue with.
Note: I reviewed the other Grafana Singlestat vs Graph posts, but this appears to be a different issue.
If you take a look at the first image you linked to, you'll notice there is a Min step input, with a default value of 5m. That's where your lower resolution comes from. You may set that explicitly to your scrape interval (or less, to make sure you don't lose any samples due to jitter in the scrape interval, although that may end up being costly), but if you increase your dashboard range enough you'll:
(a) likely have a singlestat max value that's higher than anything on the graph (because your graph is now lower resolution than the singlestat source data); and
(b) will hit Prometheus' 11K samples limit if you zoom out to a range longer than 11K times the scrape interval.
Your best bet is to use PromQL to calculate the max value to display in your singlestat panel. You'll still have to deal with (a) above (low resolution graph when the range is long) but it's going to be the actual max (as much as the fact that you're actually sampling values at some fixed interval allows) and it's going to be more efficient.
Problem is that given your query -- sum(jvm_thread_count) -- there is no way of putting that into a single PromQL query with max_over_time. You'd have to define a recorded rule (something like instance:jvm_thread_count:sum = sum(jvm_thread_count) and then have your singlestat panel display the results of the max_over_time(instance:jvm_thread_count:sum[$__range_s]) instant query (check the Instant checkbox in your singlestat settings).
Background: Soon after my performance test, the response time details of the business transactions will be fed into splunk, from where I need to generate trend graph for couple of transactions for certain period of time.
Now I am able to group the response time data for these transactions but in visualization, not able to generate the chart.
Refer:SS01.jpg for more details.
query used: index=xyz source=abc (Period!=Period) Transaction_Name=Search OR Address_Book OR Policy | chart values(Average) as Average by Transaction_Name
I want the chart to appear in either format A or B as it appears on SS02.jpg.
Please help me on this.
Thanks.
You have very low values (i.e. 0.046, 0.099, etc) in the query output and very high value in the chart Y axis (100), in order to get the desired picture just reduce the maximum value for the vertical axis like:
To make the changes permanent you can amend charting.secondaryAxis.maximumNumber parameter value like it described in Controlling chart (y axis) range
Personally I prefer BM.Sense output for performance test results analysis.
I have an % CPU usage grafana graph.
The problem is that the source data is collected by collectd as Jiffies.
I am using the following formula:
collectd|<ServerName>|cpu-*|cpu-idle|value|nonNegativeDerivative()|asPercent(-6000)|offset(100)
The problem is that when I increase the time range (to 30 days for example), the grafana is aggregating the data and since it is accumulative numbers (And not percentage or something it can make a simple average), the data in the graph is becoming invalid.
Any idea how to create a better formula?
Have you looked at the aggregation plugin (read type) to compute averages?
https://collectd.org/wiki/index.php/Plugin:Aggregation/Config
it is very strange that you have to use the nonNegativeDerivative function for a CPU metric. nonNegativeDerivative should only be used for ever increasing counters, not a gauge like metric like CPU
I am new to Graphite and can't understand how to do this:
I have a large number of time-metrics (celery metrics) in format stats.timers.*.median
I want to show:
Top N metrics with average value above X
Display them on one graph with the names of metrics
Now I have averageAbove(stats.timers.*.median,50) but it displays graphs without names and renders strangely and in bad scale. Help, please! :)
You will need to chain a few functions together in order to get the desired result.
limit(sortByMaxima(averageAbove(stats.timers.*.median, X)), N)
Starting the the averageAbove as the base.
The next thing you want to do is get all the metrics in order, "top-to-bottom" by using sortByMAxima.
Then you can limit the results that are rendered with the limit function.
You might not be rending the legend if you have too many metrics for the size of the graph. You can do 3 things.
Make the graph larger
Reduce the number of metrics using limit
Force the legend to be displayed via hideLegend