How to count requests per hour? - command-line

Question

I have access log file with data only per 1 day like:
10.2.21.120 "-" - [26/Jan/2013:19:15:11 +0000] "GET /server/ad?uid=abc&type=PST HTTP/1.1" 202 14 10
10.2.21.120 "-" - [26/Jan/2013:19:17:22 +0000] "GET /server/ad?uid=abc&type=PST HTTP/1.1" 204 14 9
10.2.22.130 "-" - [26/Jan/2013:19:27:53 +0000] "GET /server/ad?uid=abc&type=PST HTTP/1.1" 200 14 8
I am using the following command:
awk '$9 == 200 { s++ } END { print s / NR * 100; }' access.log

Answer 1

This awk may help you
$ awk -F[:\ ] '{count[$5]++}; $12 == 200 { hour[$5]++} END { for (i in hour) print i, hour[i]/count[i]*100 }' input
Test
$ cat input
10.1.20.123 "1.1.1.1" - [15/Oct/2014:12:14:17 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 10
10.1.20.123 "1.1.1.1" - [15/Oct/2014:12:14:17 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 100 3014 10
10.1.20.123 "1.1.1.1" - [15/Oct/2014:13:14:26 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 9
10.1.20.123 "1.1.1.1" - [15/Oct/2014:13:24:55 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 8
$ awk -F[:\ ] '{count[$5]++}; $12 == 200 { hour[$5]++} END { for (i in hour) print i, hour[i]/count[i]*100 }' input
12 50
13 100
What it does
{count[$5]++} array count stores the number of occurence of each hour from the log file.
$12 == 200 { hour[$5]++} Now it the log is success, that is $12 == 200 then the corresponding value in hour array is incremented.
So count[13] will contain total enteries from hour 13 where as hour[13] would contain count of succuessfull entries
END { for (i in hour) print i, hour[i]/count[i]*100 } prints the hour, percentage

Answer 2

I would use something like this:
awk -F"[ :]" '
{tot[$5]++; if ($(NF-2)==200) succ[$5]++}
END {for (i in tot) printf "%d %d/%d %.2f%\n", i, succ[i], tot[i], succ[i]/tot[i]*100}' file
This defines : or space as field separators. Hence, the hour is stored in the 5th field and the return code in the one before the penultimate (NF-2) - we could also use the 12th as you do, but this allows support for longest input in the log.
Then, it keeps track of the amount of lines in the tot[] array and the successful ones in succ[] array.
Finally it prints the result
Here we have some more data with different hours and return codes:
$ cat a
10.1.20.123 "1.1.1.1" - [15/Oct/2014:13:14:17 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 10
10.1.20.123 "1.1.1.1" - [15/Oct/2014:13:14:26 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 9
10.1.20.123 "1.1.1.1" - [15/Oct/2014:13:24:55 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 100 3014 8
10.1.20.123 "1.1.1.1" - [15/Oct/2014:17:24:55 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 200 3014 8
10.1.20.123 "1.1.1.1" - [15/Oct/2014:17:24:55 +0000] "POST /server/ad?uid=abc&type=PST HTTP/1.1" 404 3014 8
Let's run the script:
$ awk -F"[ :]" '{tot[$5]++; if ($(NF-2)==200) succ[$5]++} END {for (i in tot) printf "%d %d/%d %.2f%\n", i, succ[i], tot[i], succ[i]/tot[i]*100}' a
13 2/3 66.67%
17 1/2 50.00%

Answer 3

% cat success.awk
NF==11{
split($4,a,":") ; hour = a[2]
total[hour] += 1
if($9==200) success[hour] += 1
}
END{
for(h in total) print h, 100*(0+success[h])/total[h]"% ("0+success[h]"/"total[h]")"
}
% awk -f success.awk mylog
13 66.6667% (2/3)
17 50% (1/2)
%