Simple cumulative increase in Prometheus - grafana

I have an application that increments a Prometheus counter when it receives a particular HTTP request. The application runs in Kubernetes, has multiple instances and redeploys multiple times a day. Using the query http_requests_total{method="POST",path="/resource/aaa",statusClass="2XX"} produces a graph displaying cumulative request counts per instance as is expected.
I would like to create a Grafana graph that shows the cumulative frequency of requests received over the last 7 days.
My first thought was use increase(...[7d]) in order to account for any metrics starting outside of the 7 day window (like in the image shown) and then sum those values.
I've come to the realisation that sum(increase(http_requests_total{method="POST",path="/resource/aaa",statusClass="2XX"}[7d])) does in fact give the correct answer for points in time. However, resulting graph isn't quite what was asked for because the component increase(...) values increase/decrease along the week.
How would I go about creating a graph that shows the cumulative sum of the increase in these metrics over the passed 7 days? For example, given the simplified following data
| Day | # Requests |
|-----|------------|
| 1 | 10 |
| 2 | 5 |
| 3 | 15 |
| 4 | 10 |
| 5 | 20 |
| 6 | 5 |
| 7 | 5 |
| 8 | 10 |
If I was to view a graph of day 2 to day 8 I would like the graph to render a line as follows,
| Day | Cumulative Requests |
|-----|---------------------|
| d0 | 0 |
| d1 | 5 |
| d2 | 20 |
| d3 | 30 |
| d4 | 50 |
| d5 | 55 |
| d6 | 60 |
| d7 | 70 |
Where d0 represents the initial value in the graph
Thanks

Prometheus doesn't provide functionality, which can be used for returning cumulative increase over multiple time series on the selected time range.
If you still need this functionality, then try VictoriaMetrics - Prometheus-like monitoring solution I work on. It allows calculating cumulative increase over multiple counters. For example, the following MetricsQL query returns cumulative increase over all the time series with http_requests_total name on the selected time range in Grafana:
running_sum(sum(increase(http_requests_total)))
How does it work?
It calculates increase per each time series with the http_requests_total name. Note that the increase() in the query above doesn't contain lookbehind window in square brackets. VictoriaMetrics automatically sets the lookbehind window to the step value, which is passed by Grafana to /api/v1/query_range endpoint. The step value is the interval between points on the graph.
It sums increases returned at step 1 with the sum() function individually per each point on the graph.
It calculates cumulative increase over per-step increases returned at step 2 with the running_sum function.

If I understood your question's idea correctly, I think I managed to create such graph with a query like this
sum(max_over_time(counterName{someLabel="desiredlabelValue"}[7d]))
A graph produced by it looks like the blue one:
The reasons why the future part of the graph decreases are both because the future processing hasn't obviously yet happened and because the more-than-7-days-old processing slides out of the moving 7-day inspection window.

Related

Normalize Count Measure in Tableau

I am trying to create a plot similar to those created by Google's ngram viewer. I have the ngrams that correspond to year, but some years have much more data than others; as a result, plotting from absolute counts doesn't get me the information I want. I'd like to normalize it so that I get the counts as a percentage of the total samples for that year.
I've found ways to normalize data to ranges in Tableau, but nothing about normalizing by count. I also see that there is a count distinct function, but that doesn't appear to do what I want.
How can I do this in Tableau?
Thanks in advance for your help!
Edit:
Here is some toy data and the desired output.
Toy Data:
+---------+------+
| Pattern | Year |
+---------+------+
| a | 1 |
| a | 1 |
| a | 1 |
| b | 1 |
| b | 1 |
| b | 1 |
| a | 2 |
| b | 2 |
| a | 3 |
| b | 4 |
+---------+------+
Desired Output:
Put [Year] on the Columns shelf, and if it is really a Date field instead of a number - choose any truncation level you'd like or choose exact date. Make sure to treat it as a discrete dimension field (the pill should be blue)
Put [Number of Records] on the Rows shelf. Should be a continuous measure, i.e. SUM([Number of Records])
Put Pattern on the Color shelf.
At this point, you should be looking at a graph raw counts. To convert them to percentages, right click on the [Number of Records] field on the Rows shelf, and choose Quick Table Calc->Percent of Total. Finally, right click on [Number of Records] a second time, and choose Compute Using->Pattern.
You might want to sort the patterns. One easy way is to just drag them in the color legend.

Calculate median absolute difference using Crystal Reports 2013

I have a list of employees and turn-around times, like so:
order | employee | turn-around
------------------------------
1 | Mark | 1
2 | Mark | 2
3 | Mark | 10
4 | John | 1
5 | John | 5
6 | John | 20
7 | Chad | 15
8 | Chad | 20
9 | Chad | 60
So, as you can see, the data ends to be skewed somewhat, and so I'd like to summarize each employee by their median turn-around:
employee | median turn-around
-----------------------------
Mark | 2
John | 5
Chad | 20
I'd also like to present each employee with a comparison of how they're doing compared to the other employees. For this summary, I'd like to use the difference from the median of the medians:
employee | median turn-around | median absolute difference
----------------------------------------------------------
Mark | 2 | -3
John | 5 | 0
Chad | 20 | +15
I'd like to have this automatically done in Crystal Reports 2013 so each employee gets their own page with a histogram of their turn-around times, their median turn-around time, and how it compares to the median of all the other employees' median turn-around times.
Alas, my crystal-fu is failing me in the last part. I have grouped the records by employee, created a formula field to calculate the turn-around time in the details, and created a formula to retrieve the median turn-around for the employee in the group footer. I've managed to create my histogram. But I cannot for the life of me figure out how to aggregate the group medians and report the median of that median without querying the same data again using a subreport. Is it possible to accomplish this without a subreport?

How to create a chart of cumulative sums per date from a chart of sums per date?

I have that kind of data in a google spreadsheet:
| Day 1 | Day 2 | Day 3
Marc | 10 | 5 | 8
Amy | - | 15 | 3
What I would like is a chart that shows that marc had a total of 10 on day 1, 15 on day 2 and 23 on day 3 and Amy didn't exist on day 1 then had 15 on day 2 then 18 on day 3.
The ideal would be an automated solution that generates directly the chart but I guess i could extract some data to an other sheet and use for my chart. For instance:
| Day 1 | Day 2 | Day 3
Marc | 10 | 15 | 23
Amy | - | 15 | 18
If that's the only solution, how would I generate such a table automatically based on the input of my first table knowing that new names may be added and each day a column will be added ?
Thanks
Simple way
Just make plot with bars and see cumulative sums in chart. You won't see exact number though:
Hard Way
Or prepare another table with new calculated sums. Suppose, your data is placed on sheet \1/. Add another sheet and paste formulas:
="1!R2C2:"&"R"&COUNTA('1'!A:A)+1&"C"&counta('1'!1:1)+1 in cell A1 to count work range
={'1'!B1:1} in cell B1 to copy 'Days' labels.
={'1'!A2:A} in cell A2 to copy names
And finally paste this hard formula in cell B2:
=mmult(ArrayFormula(MMULT(ArrayFormula(row(INDIRECT(A1,0))^0),SPLIT(ArrayFormula(CONCATENATE("-"&INDIRECT(A1,0))),"-"))*ArrayFormula(--(SPLIT(ArrayFormula(CONCATENATE("-"&if(COLUMN(INDIRECT(A1,0)),row(INDIRECT(A1,0))))),"-")=ArrayFormula(row(OFFSET('1'!A2,,,COUNTA('1'!A:A))))))),ArrayFormula(--(TRANSPOSE(SPLIT(ArrayFormula(CONCATENATE("-"&if(row(INDIRECT(A1,0)),COLUMN(INDIRECT(A1,0))))),"-"))<=ArrayFormula(COLUMN(OFFSET('1'!B1,,,1,COUNTA('1'!1:1)))))))
Then plot your new calculated data to make something like this:
Note that I changed "-" by 0 to make data look like numbers.
Also zero's are listed as empty string ''. This is made by custom number format: 0;0;
Please, look at Working example

Generate a graph based on percentage in excel

I have a table in excel with the below structure
Names | Pass | Fail |
= ==== == ==== ==== == =====
NameA | 2 | 3 |
NameB | 6 | 7 |
NameC | 3 | 4 |
The Pass/Fail details im getting from a series of rows using CountIF formula.
If i generate a graph now for this table in excel. I get the details based on the count.
Eg: For Name A, out of 5 rows - 2 are pass and 3 are Fail
I wanted to acheive this graph interms of percentage as Y-axis in the graph which says out of 100% - 40% are pass adn 60% percent are fail.
Can someone please help me out with this?
If you want to plot a percentage, calculate it in the sheet, and plot the calculations.
Alternatively, if you use a stacked 100% chart (column or bar), the bars will be scaled so they show the percentage. However, the data will still be the input values, and data labels will show these values and not percentages, and you will have both bars in the chart.

Bootstrap weighted data - Matlab

I have a simple dataset with values and absolute frequencies, like the table below:
value|freq
-----------
1 | 10
3 | 20
4 | 10
3 | 10
And now I'd like to calculate the frequency table, like:
value| %
-----------
1 | 1/5
3 | 3/5
4 | 1/5
And last step, I'd like to compute the bootstrap CI with matlab. I have a lot of rows in the dataset.
I've calculated the frequency table via grpstatscommand in Matlab, but I don't know how I can use it in the boostrp function in matlab.
Any help or suggestions would be really appreciated.