HALog - Connect and response times percentiles - haproxy

When I run the following command to parse haproxy logs, the output doesn't contain any headers, and I'm not able to understand the meanings of the numbers in each of the columns.
Command halog -pct < haproxy.log > percentiles.txt
the output that I see is:
0.1 3493 18 0 0 0
0.2 6986 25 0 0 0
0.3 10479 30 0 0 0
0.4 13972 33 0 0 0
0.5 17465 37 0 0 0
0.6 20958 40 0 0 0
0.7 24451 43 0 0 0
0.8 27944 46 0 0 0
0.9 31438 48 0 0 0
1.0 34931 49 0 0 0
1.1 38424 50 0 0 0
1.2 41917 51 0 0 0
1.3 45410 52 0 0 0
1.4 48903 53 0 0 0
1.5 52396 55 0 0 0
1.6 55889 56 0 0 0
1.7 59383 57 0 0 0
1.8 62876 58 0 0 0
1.9 66369 60 0 0 0
2.0 69862 61 0 0 0
3.0 104793 74 0 0 0
4.0 139724 80 0 1 0
5.0 174656 89 0 1 0
6.0 209587 94 0 1 0
7.0 244518 100 0 1 0
8.0 279449 106 0 1 0
9.0 314380 112 0 1 0
10.0 349312 118 0 1 0
15.0 523968 144 0 1 0
20.0 698624 168 0 1 0
25.0 873280 180 0 2 0
30.0 1047936 190 0 2 0
35.0 1222592 200 0 3 0
40.0 1397248 210 0 3 0
45.0 1571904 220 0 4 0
50.0 1746560 230 0 6 0
55.0 1921216 241 0 7 0
60.0 2095872 258 0 9 0
65.0 2270528 279 0 10 0
70.0 2445184 309 0 16 0
75.0 2619840 354 1 18 0
80.0 2794496 425 1 20 0
85.0 2969152 545 1 22 0
90.0 3143808 761 1 39 1
91.0 3178740 821 1 80 1
92.0 3213671 921 1 217 1
93.0 3248602 1026 1 457 1
94.0 3283533 1190 1 683 1
95.0 3318464 1408 1 889 1
96.0 3353396 1721 1 1107 1
97.0 3388327 2181 1 1328 1
98.0 3423258 2902 1 1555 1
98.1 3426751 3000 1 1580 1
98.2 3430244 3094 1 1607 1
98.3 3433737 3196 1 1635 1
98.4 3437231 3301 1 1666 1
98.5 3440724 3420 1 1697 1
98.6 3444217 3550 1 1731 1
98.7 3447710 3690 1 1770 1
98.8 3451203 3848 1 1815 1
98.9 3454696 4030 1 1864 1
99.0 3458189 4249 1 1923 2
99.1 3461682 4490 1 1993 2
99.2 3465176 4766 2 2089 2
99.3 3468669 5085 2 2195 2
99.4 3472162 5441 3 2317 97
99.5 3475655 5899 5 2440 365
99.6 3479148 6517 11 2567 817
99.7 3482641 7403 14 2719 1555
99.8 3486134 8785 16 2992 2779
99.9 3489627 11650 997 3421 4931
100.0 3493121 85004 4008 20914 71716
The first column looks to be the percentile, (like P50, P90, P99, etc) but the what are the values in the 2nd, 3rd, 4th, 5th and 6th columns? Also, are they total values (halog reports total times when provided with other options), or average values or maximum values?


<percentile> <request count> <Request Time*> <Connect Time**> <Response Time***> <Data Time****>
* Referred to as TR in the documentation.
** Referred to as Tc in the documentation.
*** Referred to as Tr in the documentation.
**** Referred to as Td in the documentation.
The source provides some good pointers.

Related

Netlogo BehaviourSpace weird Output and Multiple Runs

For some weird reason the behaviorSpace in netlogo runs the same value pairs model multiple times even though repetitions are 1. I can't understand why. The output file in table format looks like this. I don't know what's up with double quotes.
BehaviorSpace results (NetLogo 5.2.0)
/home/abhishekb/new_models/basic/try4.nlogo
experiment1
10/26/2015 02:34:28:770 +0530
min-pxcor max-pxcor min-pycor max-pycor
-7 7 -7 7
[run number] knt k threshold scale [step]
16 0 75 0.1 1 2535
7 0 54 0.1 1 0 8715
5 0 47 0.3 1 0 9374
10 0 61 0.1 1 0 8841
"22 0 89 0.1 1 0 3664"8" 0 54 0.3 1 0 12001" 0 40 0.1 1 0 10727
2" 013" 0 68 0.1 1 0"22 0 89 0.1 1 0 4449
128" 0 103 0.1 1 0 2805
4 0 47 0.1 1 0 1200119" 0 82 0.1 1 0 12001"1 0 40 0.1 1 0 12001
3"
26" 0 96 0.3 1 0 4800
9" 0 103 0.3 1 0 43321" 0 82 0.6 1 0 7385
31 0 110 0.1 1 0 2976
1" 0 40 0.1 1 0 12001
4" 0 89 0.6 1 0 6389
25 0 96 0.1 1 0 7517
26 0 96 0.3 1 0 5479
9""27" 0 96 0.6 1 0 6117
28 0 103 0.1 1 0 2219
29 0 103 0.3 1 0 6411
30 0 103 0.6 1 0 5693
31 0 110 0.1 1 0 3985
78 500 61 0.6 1 0 9720
79 500 68 0.1 1 0 6067
80 500 68 0.3 1 0 6795
81 500 68 0.6 1 0 8305
82 500 75 0.1 1 0 4416
83 500 75 0.3 1 0 5742
84 500 75 0.6 1 0 7399
85 500 82 0.1 1 0 5306
86 500 82 0.3 1 0 5388
01"
87 500 82 0.6 1 0 6869
88 500 89 0.1 1 0 12001
89 500 89 0.3 1 0 5097
90 500 89 0.6 1 0 6478
91 500 96 0.1 1 0 2275
92 500 96 0.3 1 0 4693" 500 96 0.6 1 0 6395
94"94 500 103 0.1 1 0 12001
95 500 103 0.3 1 0 3984
96 500 103 0.6 1 0 5440
97 500 110 0.1 1 0 1893
98 500 110 0.3 1 0 37299" 500 110 0.6 1 0 5275
100 750 40 0.1 1 0 12001
101" 750 40 0.3 1 0 12001
102 750 40 0.6 1 0 12001
"
103""" 750 47 0.1 1 0 11911
"
104""" 750 47 0.3 1 0 12001
105 750 47 0.6 1 0 11821
750" 54 0.1 1 0 12001
107 750 54 0.3 1 0 8703
811108" 750 54 0.6 1 0 10099
5"
,61" 0.1 1 0 12001
110 750 61 0.3 1 0 7453
0111" 750111" 750 61 0.6 1 0 9051
112 750 68 0.1 1 0 12001
68 0.3 1 0 12001
BehaviourSpace Code:
##$###$##
NetLogo 5.2.1
##$###$##
##$###$##
##$###$##
<experiments>
<experiment name="experiment1" repetitions="1" runMetricsEveryStep="false">
<setup>check-setup-percent
file-write-values</setup>
<go>go</go>
<final>write-to-file</final>
<timeLimit steps="12000"/>
<exitCondition>count inboxturtles with[exit = 1 and exited = false] = 0</exitCondition>
<steppedValueSet variable="knt" first="0" step="250" last="2500"/>
<steppedValueSet variable="k" first="40" step="7" last="110"/>
<enumeratedValueSet variable="threshold">
<value value="0.1"/>
<value value="0.3"/>
<value value="0.6"/>
</enumeratedValueSet>
<enumeratedValueSet variable="scale">
<value value="1"/>
</enumeratedValueSet>
</experiment>
</experiments>
##$###$##
##$###$##
default
0.0
-0.2 0 0.0 1.0
0.0 1 1.0 0.0
0.2 0 0.0 1.0
link direction
true
0
Line -7500403 true 150 150 90 180
Line -7500403 true 150 150 210 180
##$###$##
1
##$###$##

Changing index of matrix

I'm trying to change the following code so that the first matrix will become the second matrix:
function BellTri = matrix(n)
BellTri = zeros(n);
BellTri(1,1) = 1;
for i = 2:n
BellTri(i,1) = BellTri(i-1,i-1);
for j = 2:i
BellTri(i,j) = BellTri(i - 1,j-1) + BellTri(i,j-1);
end
end
BellTri
First matrix (when n = 7)
1 0 0 0 0 0 0
1 2 0 0 0 0 0
2 3 5 0 0 0 0
5 7 10 15 0 0 0
15 20 27 37 52 0 0
52 67 87 114 151 203 0
203 255 322 409 523 674 877
Second matrix
1 1 2 5 15 52 877
1 3 10 37 151 674 0
2 7 27 114 523 0 0
5 20 87 409 0 0 0
15 67 322 0 0 0 0
52 255 0 0 0 0 0
203 0 0 0 0 0 0

An option is to cyclically permute the columns using circshift.
function [BellTri, Second] = matrix(n)
BellTri = zeros(n);
BellTri(1,1) = 1;
for i = 2:n
BellTri(i,1) = BellTri(i-1,i-1);
for j = 2:i
BellTri(i,j) = BellTri(i - 1,j-1) + BellTri(i,j-1);
end
end
Second = BellTri;
for i = 1:n
Second(:, i) = circshift(Second(:,i), 1-i);
end
for i = n-1:-1:2
Second(1, i) = Second(1, i-1);
end
end
Input: [BellTri, Second] = matrix(7)
Output:
BellTri =
1 0 0 0 0 0 0
1 2 0 0 0 0 0
2 3 5 0 0 0 0
5 7 10 15 0 0 0
15 20 27 37 52 0 0
52 67 87 114 151 203 0
203 255 322 409 523 674 877
Second =
1 1 2 5 15 52 877
1 3 10 37 151 674 0
2 7 27 114 523 0 0
5 20 87 409 0 0 0
15 67 322 0 0 0 0
52 255 0 0 0 0 0
203 0 0 0 0 0 0

One approach:
out = zeros(size(A));
out(logical(fliplr(triu(ones(size(A,1)))))) = A(logical(tril(ones(size(A,1)))));
Note: As Divakar pointed out, there should be a typo in the first row. This method gives the corrected one.
Results:
A = [1 0 0 0 0 0 0;
1 2 0 0 0 0 0;
2 3 5 0 0 0 0;
5 7 10 15 0 0 0;
15 20 27 37 52 0 0;
52 67 87 114 151 203 0;
203 255 322 409 523 674 877];
>> out
out =
1 2 5 15 52 203 877
1 3 10 37 151 674 0
2 7 27 114 523 0 0
5 20 87 409 0 0 0
15 67 322 0 0 0 0
52 255 0 0 0 0 0
203 0 0 0 0 0 0

2D vs 3D FFT in Matlab/Octave

Say I have this matrix in memory and I want to calculate the 3D FFT
T =
0 1 2 3
4 5 6 7
8 9 10 11
12 13 14 15
16 17 18 19
20 21 22 23
24 25 26 27
28 29 30 31
32 33 34 35
36 37 38 39
40 41 42 43
44 45 46 47
44 45 46 47
52 53 54 55
56 57 58 59
60 61 62 63
real(fft2(T))
ans =
2000 -32 -32 -32
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
real(fftn(T))
ans =
2000 -32 -32 -32
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
-144 0 0 0
-128 0 0 0
-112 0 0 0
-128 0 0 0
Why am I getting the same result? How 3D FFTs can be done in Matlab/Octave?

A 3D-FFT should be applied to a 3D-array. If you apply the 3D-FFT to a 2D-array you get the same result as a 2D-FFT, because there is no third dimension in the array.
Think about it this way: an N-dimensional FFT is just N 1-dimensional FFT's, one along each dimension. If there is no third dimension in the array, the FFT along that dimension does nothing.

need to insert datetime in every row of vmstat output

I need to insert datetime in every vmstat line that has value.
I can create a function like this:
function insert_datetime {
while read line
do
printf "$line"
date '+ %m-%d-%Y %H:%M:%S'
done
}
then call vmstat as below:
'vmstat 3 5 | insert_datetime'
but this line puts date time to every line, including dashes (--) and any rows that has text. How can I exclude rows that has dahses and text?
kthr memory page faults cpu 04-23-2013 10:19:49
----- ----------- ------------------------ ------------ ----------------------- 04-23-2013 10:19:49
r b avm fre re pi po fr sr cy in sy cs us sy id wa pc ec 04-23-2013 10:19:49
0 0 45688088 4094129 0 0 0 0 0 0 45 12172 2840 1 1 99 0 0.35 2.2 04-23-2013 10:19:49
2 0 45694135 4088082 0 0 0 0 0 0 451 56350 21818 3 1 97 0 0.73 4.5 04-23-2013 10:19:52
1 0 45694137 4088061 0 0 0 0 0 0 303 24568 951 3 1 96 0 0.82 5.1 04-23-2013 10:19:55
1 0 45694138 4087739 0 0 0 0 0 0 445 9170 1504 2 0 98 0 0.64 4.0 04-23-2013 10:19:58
4 0 45703145 4078732 0 0 0 0 0 0 335 47175 1306 4 1 95 0 1.01 6.3 04-23-2013 10:20:01
I needed to look like this:
kthr memory page faults cpu
----- ----------- ------------------------ ------------ -----------------------
r b avm fre re pi po fr sr cy in sy cs us sy id wa pc ec
0 0 45688088 4094129 0 0 0 0 0 0 45 12172 2840 1 1 99 0 0.35 2.2 04-23-2013 10:19:49
2 0 45694135 4088082 0 0 0 0 0 0 451 56350 21818 3 1 97 0 0.73 4.5 04-23-2013 10:19:52
1 0 45694137 4088061 0 0 0 0 0 0 303 24568 951 3 1 96 0 0.82 5.1 04-23-2013 10:19:55
1 0 45694138 4087739 0 0 0 0 0 0 445 9170 1504 2 0 98 0 0.64 4.0 04-23-2013 10:19:58
4 0 45703145 4078732 0 0 0 0 0 0 335 47175 1306 4 1 95 0 1.01 6.3 04-23-2013 10:20:01

Why not just use vmstat -t? It seems to be exactly what you are looking for. Here is some sample output
[root#web5 vmstat]# vmstat -t 1
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ ---timestamp---
r b swpd free buff cache si so bi bo in cs us sy id wa st
1 0 15704 193236 189628 595868 9 3 25 16 15 20 11 1 88 1 0 2013-05-22 13:32:36 JST
0 0 15704 193212 189628 595868 0 0 0 0 22 20 0 0 100 0 0 2013-05-22 13:32:37 JST
0 0 15704 193212 189628 595868 0 0 0 0 19 12 0 0 100 0 0 2013-05-22 13:32:38 JST
0 0 15704 193212 189628 595868 0 0 0 0 10 11 0 0 100 0 0 2013-05-22 13:32:39 JST
0 0 15704 193212 189628 595868 0 0 0 96 34 25 0 1 99 0 0 2013-05-22 13:32:40 JST
0 0 15704 193212 189628 595868 0 0 0 0 10 9 0 0 100 0 0 2013-05-22 13:32:41 JST
0 0 15704 193212 189628 595868 0 0 0 0 14 23 0 0 100 0 0 2013-05-22 13:32:42 JST
executed on CentOS6.3 with procps 3.2.8
[root#web5 uptime]# vmstat -V
procps version 3.2.8

Use awk:
vmstat 3 5 | awk '/^ *[0-9]/{$0=$0 " " strftime("%m-%d-%Y %T")};1'

Try:
function insert_datetime {
while read line
do
printf "$line"
if [[ "$line" =~ [0-9].* ]]; then
date '+ %m-%d-%Y %H:%M:%S'
else
echo
fi
done
}

sed can give you answer too... in much cleaner & portable (across shells) way:
vmstat 3 5 | sed '/^ *[0-9].*/s/.*/printf "&";date "+ %m-%d-%Y %H:%M:%S"/e'
All lines starting with a number are appended date in required format.

Set colorbar ranges in 3d graph in matlab

I would like to create a three dimensional surface using this:
>> a=X
a =
Columns 1 through 8
0 50 100 150 200 250 300 350
Columns 9 through 16
400 450 500 550 600 650 700 750
Columns 17 through 21
800 850 900 950 1000
>> b=Y
b =
0
50
100
150
200
250
300
350
400
>> c=Z
c =
Columns 1 through 8
0 0 0 0 0 0 0 0
16 32 67 98 127 164 194 234
120 171 388 773 1086 1216 1770 2206
189 270 494 1978 2755 3134 5060 10469
133 166 183 348 647 937 1446 2304
192 162 154 113 161 189 266 482
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Columns 9 through 16
0 0 0 0 0 0 0 0
366 604 529 504 346 226 228 179
4027 11186 10276 5349 2560 1322 996 799
27413 76387 37949 15591 5804 2654 1803 1069
9844 24152 14772 4613 1777 849 459 290
1288 2623 1538 582 280 148 90 56
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0
Columns 17 through 21
0 0 0 0 0
108 94 79 0 0
646 476 612 0 0
884 858 722 0 0
266 215 139 0 0
48 48 31 0 0
0 0 0 0 0
0 0 0 0 0
0 0 0 0 0
>> surf(X,Y,Z)
At the same time I would like to define that Z values < = 1803 will be shown with red color on the surface graph, 1803 < Z < 2755 yellow and Z > = 2755 green. The limits of the colorbar can be the min and max values of Z (from 0 to 76387). How can I set the ranges of the colorbar in order to get this result?

This will do as you ask:
%# add red (row 1), yellow (row 2), green (row 2)
map = [1 0 0; 0 1 1; 0 1 0];
%# set the new map as the current map
colormap(map);
colors = zeros(size(c)); %# create colors array
colors(c <= 1803) = 1; %# red (1)
colors(c > 1803 & c < 2755) = 2; %# yellow (2)
colors(c >= 2755) = 3; %# green (3)
%# and pass it into surf
surf(a,b,c, colors)