I am getting these metrics by default from prometheus:
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.005"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.01"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.025"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.05"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.1"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.25"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="0.5"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="1"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="2.5"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="5"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="10"} 33
gin_gin_request_duration_bucket{host="localhost:8080",method="GET",path="/dummy1",le="+Inf"} 33
Related
I have a matrix A of size 2500 x 500. I want to sum each 10 columns and get the result as a matrix B of size 2500 x 50. That is, the first column of B is the sum of the first 10 columns of A, the second column of B is the sum of second 10 columns of A, and so on.
How can I do that without a for loop? Since I have to do that hundreds of times and it is highly time consuming to do that using for loop.
First, we "block reshape" A, such that we have the desired number of columns. Therefore, we shamelessly steal the code from the great Divakar, and put in some minimal effort to generalize it. Then, we just need to sum along the second axis, and reshape to the original form.
Here's an example with five columns to be summed:
% Sample input data
A = reshape(1:100, 10, 10).'
[r, c] = size(A);
% Number of columns to be summed
n_cols = 5;
% Block reshape to n_cols, see https://stackoverflow.com/a/40508999/11089932
B = reshape(permute(reshape(A, r, n_cols, []), [1, 3, 2]), [], n_cols);
% Sum along second axis
B = sum(B, 2);
% Reshape to original form
B = reshape(B, r, c / n_cols)
That's the output:
A =
1 2 3 4 5 6 7 8 9 10
11 12 13 14 15 16 17 18 19 20
21 22 23 24 25 26 27 28 29 30
31 32 33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70
71 72 73 74 75 76 77 78 79 80
81 82 83 84 85 86 87 88 89 90
91 92 93 94 95 96 97 98 99 100
B =
15 40
65 90
115 140
165 190
215 240
265 290
315 340
365 390
415 440
465 490
Hope that helps!
This can be done with splitapply. An advantage of this approach is that it works even if the group size does not divide the number of columns (the last group is smaller):
A = reshape(1:120, 12, 10).'; % example 10×12 data (borrowed from HansHirse)
n_cols = 5; % number of columns to sum over
result = splitapply(#(x)sum(x,2), A, ceil((1:size(A,2))/n_cols));
In this example,
A =
1 2 3 4 5 6 7 8 9 10 11 12
13 14 15 16 17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32 33 34 35 36
37 38 39 40 41 42 43 44 45 46 47 48
49 50 51 52 53 54 55 56 57 58 59 60
61 62 63 64 65 66 67 68 69 70 71 72
73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96
97 98 99 100 101 102 103 104 105 106 107 108
109 110 111 112 113 114 115 116 117 118 119 120
result =
15 40 23
75 100 47
135 160 71
195 220 95
255 280 119
315 340 143
375 400 167
435 460 191
495 520 215
555 580 239
I have written this code in q for solving Euler 18 problem,as described in the link below, using recursion.
https://stackoverflow.com/questions/8002252/euler-project-18-approach
Though the code works, it is not efficient and gets stack overflow at pyramids of sizes greater than 3000. How could I make this code much more efficient.I believe the optimum code can be less than 30 characters.
pyr:{[x]
lsize:count x;
y:x;
$[lsize <=1;y[0];
[.ds.lastone:x[lsize - 1];
.ds.lasttwo:x[lsize - 2];
y:{{max (.ds.lasttwo)[x] +/: .ds.lastone[x],.ds.lastone[x+1]}each til count .ds.lasttwo};
$[(count .ds.lasttwo)=1;y:{max (.ds.lasttwo) +/: .ds.lastone[x],.ds.lastone[x+1]}0;y:y[]];
x[lsize - 2]:y;
pyr[-1_x]]]
}
To properly implement this logic in q you need to use adverbs.
First, to quickly find the rolling maximums you can use the prior adverb. For example:
q)input:(75;95 64;17 47 82;18 35 87 10;20 04 82 47 65;19 01 23 75 03 34;88 02 77 73 07 63 67;99 65 04 28 06 16 70 92;41 41 26 56 83 40 80 70 33;41 48 72 33 47 32 37 16 94 29;53 71 44 65 25 43 91 52 97 51 14;70 11 33 28 77 73 17 78 39 68 17 57;91 71 52 38 17 14 91 43 58 50 27 29 48;63 66 04 68 89 53 67 30 73 16 69 87 40 31;04 62 98 27 23 09 70 98 73 93 38 53 60 04 23)
q)last input
4 62 98 27 23 9 70 98 73 93 38 53 60 4 23
q)1_(|) prior last input
62 98 98 27 23 70 98 98 93 93 53 60 60 23
That last line outputs the a vector with the maximum value between each successive pair in the input vector. Once you have this you can add it to the next row and repeat.
q)foo:{y+1_(|) prior x}
q)foo[input 14;input 13]
125 164 102 95 112 123 165 128 166 109 122 147 100 54
Then, to apply this function over the whole use the over adverb:
q)foo over reverse input
,1074
EDIT: The approach above can be generalized further.
q provides a moving max function mmax. With this you can find "the x-item moving maximum of numeric y", which generalizes the use of prior above. For example, you can use this to find the moving maximum of pairs or triplets in the last row of the input:
q)last input
4 62 98 27 23 9 70 98 73 93 38 53 60 4 23
q)2 mmax last input
4 62 98 98 27 23 70 98 98 93 93 53 60 60 23
q)3 mmax last input
4 62 98 98 98 27 70 98 98 98 93 93 60 60 60
mmax can be used to simplify foo above:
q)foo:{y+1_ 2 mmax x}
What's especially nice about this is that it can be used to generalize to variants of this problem with wider triangles. For example, the triangle below has two more values on each row and from any point on a row you can move to the left, middle, or right of the row below it.
5
5 6 7
6 7 3 9 1
I just received a large data array of random numbers. 20 numbers per line, 600,000 lines in a CSV file. The numbers are separated by a space instead of a comma thus postgresql reads it as one long string per line and I cannot insert the proper data attribute for the data.
Each set of numbers will have a unique id. Each number is 2 digits long. I want the ability to count the amount of times a certain number was entered. Get the frequency of each number between certain ID's.
My question:
What data type should I use to insert the data so it is recognized as integers instead of text?
How do I replace the space with a comma?
Do I need to replace the space with a comma?
Currently running Postgres 9.6, PgAdmin 4.
Bonus if answer is provided in PgAdmin as well.
Also here is a sample
Excel
Numbers
06 18 20 21 24 32 36 40 44 47 50 52 55 57 60 61 62 68 72 79
03 05 12 13 14 16 17 18 24 28 33 34 35 39 44 55 62 63 64 67
09 10 12 13 15 25 30 31 36 42 43 44 46 48 51 57 65 69 75 79
08 12 15 20 27 33 34 37 41 43 44 45 54 55 60 61 66 70 72 76
CSV FILE
Numbers06 18 20 21 24 32 36 40 44 47 50 52 55 57 60 61 62 68 72 79
03 05 12 13 14 16 17 18 24 28 33 34 35 39 44 55 62 63 64 67
09 10 12 13 15 25 30 31 36 42 43 44 46 48 51 57 65 69 75 79
08 12 15 20 27 33 34 37 41 43 44 45 54 55 60 61 66 70 72 76
or the file with the id numbers
CSV
ID, Numbers
1253842,06 18 20 21 24 32 36 40 44 47 50 52 55 57 60 61 62 68 72 79
1253843,03 05 12 13 14 16 17 18 24 28 33 34 35 39 44 55 62 63 64 67
1253844,09 10 12 13 15 25 30 31 36 42 43 44 46 48 51 57 65 69 75 79
1253845,08 12 15 20 27 33 34 37 41 43 44 45 54 55 60 61 66 70 72 76
1253846,04 06 07 09 11 12 13 14 18 20 21 26 30 36 37 41 43 48 74 79
1253847,01 11 14 15 35 37 38 43 46 48 49 51 53 57 64 65 66 70 76 77
1253848,01 03 14 17 20 22 24 25 38 42 46 54 56 57 60 61 66 72 78 80
Here's the error message
>
ERROR: malformed array literal: "06 18 20 21 24 32 36 40 44 47 50 52 55 57 60 61 62 68 72 79"
DETAIL: Array value must start with "{" or dimension information.
CONTEXT: COPY Quick numbers, line 2, column numbers : "06 18 20 21 24 32 36 40 >44 47 50 52 55 57 60 61 62 68 72 79"
What's the fastest way to create a 8x8 matrix filled with 1-64 by row. The help docs say i should even be able to fill a matrix with an array, but i can't seem to make it work. I've been told it can be done more easily than i do it, but I've not seen it done. Here's an idea of what i'm looking for...
v26 =
1 2 3 4 5 6 7 8
9 10 11 12 13 14 15 16
17 18 19 20 21 22 23 24
25 26 27 28 29 30 31 32
33 34 35 36 37 38 39 40
41 42 43 44 45 46 47 48
49 50 51 52 53 54 55 56
57 58 59 60 61 62 63 64
but to get it to do this, I had to do a row-by-row fill with ...
v26 = [1:8; 9:16; 17:24; 25:32; 33:40; 41:48; 49:56; 57:64]
make a sequence, then you reshape it:
m = reshape(1:64, [8 8])';
You have to transpose it in the end b/c matlab is column major.
Continue my research,
I need another similar logic with ismember that has execution time faster.
this part of my code and the matrix.
StartPost =
14 50 30 1 72 44 76 68 63 80 25 41;
14 50 30 1 61 72 42 46 67 77 81 27;
35 23 8 54 19 70 48 75 66 79 2 84;
35 23 8 54 82 72 78 68 19 2 48 66;
69 24 36 52 63 47 5 18 11 82 1 15;
69 24 36 52 48 18 1 12 80 63 6 84;
73 38 50 7 1 33 24 68 29 20 62 84;
73 38 50 7 26 61 65 32 22 18 2 69]
for h=2:2:8,
...
done=all(ismember(StartPost(h,1:4),StartPost(h-1,1:4)));
...
end
I checked that code by using Profile viewer. I got that in this part that made my code took time execution slowly.
Anyone has experience about this logic, please share. thanks
MATLAB has several undocumented built-in functions which can help you achieve the same results as other functions, only faster.
In your case, you can use ismembc:
done = all(ismembc(StartPost(h, 1:4), sort(StartPost(h-1, 1:4)))));
Note that ismembc(A, B) requires matrix B to be sorted and not to contain any NaNs values.
Here's the execution time difference for your example:
tic
for h = 2:2:8
done = all(ismember(StartPost(h, 1:4), StartPost(h-1, 1:4)));
end
toc
Elapsed time is 0.029888 seconds.
tic
for h = 2:2:8
done = all(ismembc(StartPost(h, 1:4), sort(StartPost(h-1, 1:4))));
end
toc
Elapsed time is 0.006820 seconds.
This is about ~50 times faster.