awk on a date variable - date

What is working :
echo "Oct 12 2021" | awk '{print "date -d\""$1FS$2FS$3"\" +%Y%m%d"}' | bash
how could I use a variable on a bash script ?
mydate="Oct 12 2021"
awk -v dateformat= ''$mydate" '{print "date -d\""$1FS$2FS$3"\" +%Y%m%d"}'

You obviously wouldn't really use awk to call date on a single value but I assume you have something more than that in mind in your real code so this is what you asked for (call GNU date from awk using an awk variable as the date):
mydate="Oct 12 2021"
awk -v date="$mydate" 'BEGIN {
system("date -d\047" date "\047 +\047%Y%m%d\047")
}'
20211012
or if you prefer:
awk -v date="$mydate" 'BEGIN {
cmd = "date -d\047" date "\047 +\047%Y%m%d\047"
print ( (cmd | getline line) > 0 ? line : date"=N/A" )
close(cmd)
}'
20211012
Don't do either of those though. It's very slow to call an external command from awk and you don't need to when all you want to do is change text from one format to another:
$ awk -v date="$mydate" 'BEGIN {
split(date,d);
mth = (index("JanFebMarAprMayJunJulAugSepOctNovDec",d[1])+2)/3
printf "%04d%02d%02d\n", d[3], mth, d[2]
}'
20211012
Also, if you have GNU awk it has built-in time functions so even if what you wanted to do wasn't just shuffling bits of text around you could call the internal mktime() and strftime() instead of the external date:
awk -v date="$mydate" 'BEGIN {
split(date,d)
mth = (index("JanFebMarAprMayJunJulAugSepOctNovDec",d[1])+2)/3
secs = mktime(d[3]" "mth" "d[2]" 12 0 0")
print strftime("%Y%m%d",secs)
}'
20211012

Your original example uses awk to prepare arguments that are then passed to the Linux date command running in bash; the date formatting is performed by the date command.
You don't need awk for this, the date command -d option is very flexible in the format it receives.
mydate="Oct 3 2021"
date -d "$mydate" +%Y%m%d
Will give you a formatted date. You can assign the result to a variable using the $() syntax - run a command and assign result
formattedDate=$(date -d "$mydate" +%Y%m%d)
echo $formattedDate

Related

Replace a date string in sed with the ouput of a custom function with arguments

I have a bash script which echos out an html file like
this ./foo.sh > out.html In the html there are timestamps in the
following format 2019-02-24T17:02:19Z.
I wrote a function to
convert the time stamp to the time delta between the timestamp
and now.
my_uptime(){
START=$(date -d "$1" "+%s")
NOW=$(date "+%s")
STD=$(echo "($NOW-$START)/3600" | bc)
MIN=$(echo "(($NOW-$START)/60)%60" | bc)
echo "Uptime $STD h $MIN min"
}
Now I want to replace the timestamp with the output of my_uptime
directly in the stream. I tried this:
echo "<p>some html</p>
2019-02-24T17:02:19Z
<p>some more html</p>" | sed -r "s/[0-9\-]+T[0-9:]+Z/$(my_uptime \0)/"
This fails because the command substitution doesn't recognize the
back reference and puts in a literal 0. Is there another way to
achieve this? Preferably directly in the stream.
... | sed -r "s/[0-9\-]+T[0-9:]+Z/$(my_uptime \0)/"
This code is attempting to pass the matched value from sed's s/// into the shell function. However, $(...) is expanded before sed even sees it.
Using sed is probably not appropriate here.
Here's a perl replacement that effectively combines your shell function and sed:
... | perl -ple '
if (/([0-9-]+T[0-9:]+Z)/) {
$s = `date -d "$1" +%s`;
$n = time;
$h = int(($n-$s)/3600);
$m = int(($n-$s)/60)%60;
$_ = "Uptime $h h $m min";
}'
You could probably do something similar in awk.

Replace a character with #(hash symble) only in 5th & 6th field

I am trying to Replace a character with #(hash symble) only in 5th & 6th field.
eg. I have to replace 'Z' with '#' only in 5th & 6th field (using perl or AWK script). And remaining fields containng 'Z' symbol should not be affected.
(just I'm updating the post to replace double quote(") instead of Z by #. Can I achive this? thanks for precious help)
eg: i/p file:
aa",bb,ccc,ddd,eee",ddd",fff
aa1",ba1,ccc1,"ddd1,eee"1,ddd1,fff1
z,aa2,bb2",ccc2,ddd2","eee2",ddd2,fff2"
Expected O/p file:
aa",bb,ccc,ddd,eee#,ddd#,fff
aa1",ba1,ccc1,#ddd1,eee#1,ddd1,fff1
aa2,bb2",ccc2,ddd2#,#eee2#,ddd2,fff2"
Thanks.
$ awk 'BEGIN{FS=OFS=","} {for (i=5;i<=6;i++) gsub(/Z/,"#",$i)} 1' file
x,aaZ,bb,ccc,ddd,eee#,dddZ,fff
y,aa1Z,ba1,ccc1,#ddd1,eee#1,ddd1,fff1
z,aa2,bb2Z,ccc2,ddd2#,#eee2,ddd2,fff2Z
Since its only two filed, loop can be omitted.
awk -F, -v OFS=, '{gsub(/Z/,"#",$5);gsub(/Z/,"#",$6)} 1' file
x,aaZ,bb,ccc,ddd,eee#,dddZ,fff
y,aa1Z,ba1,ccc1,#ddd1,eee#1,ddd1,fff1
z,aa2,bb2Z,ccc2,ddd2#,#eee2,ddd2,fff2Z
To replace " in fifth and sixth field:
awk -F, -v OFS=, '{gsub(/\"/,"#",$5);gsub(/\"/,"#",$6)} 1' file
aa",bb,ccc,ddd,eee#,ddd#,fff
aa1",ba1,ccc1,"ddd1,eee#1,ddd1,fff1
z,aa2,bb2",ccc2,ddd2#,#eee2#,ddd2,fff2"
Here is a Perl way to do the job:
perl -anF, -e '$"=","; s/Z/#/ for (#F)[4,5];print"#F";' < in1.txt
If you have mutiple Z in a field, you could use:
perl -anF, -e '$"=","; s/Z/#/g for (#F)[4,5];print"#F";' < in1.txt
Output:
aaZ,bb,ccc,ddd,eee#,ddd#,fff
aa1Z,ba1,ccc1,Zddd1,eee#1,ddd1,fff1
aa2,bb2Z,ccc2,ddd2Z,#eee2,ddd2,fff2Z
Edit according to comment:
in1.txt
aa",bb,ccc,ddd,eee",ddd",fff
aa1",ba1,ccc1,"ddd1,eee"1,ddd1,fff1
aa2,bb2",ccc2,ddd2","eee2,ddd2,fff2"
Command:
perl -anF'','' -e '$"=",";s/"/#/ for (#F)[4,5];print"#F";' < in1.txt
result:
aa",bb,ccc,ddd,eee#,ddd#,fff
aa1",ba1,ccc1,"ddd1,eee#1,ddd1,fff1
aa2,bb2",ccc2,ddd2",#eee2,ddd2,fff2"

jenkins rest get duration in human format

I need to get job duration via curl command.
I can get access to it
curl -s $JENKINS_JOB_URL/146/api/json?tree=duration
{"duration":14182142}
How convert the 14182142 to "3 hr 56 min" ? ( via ksh, or in the curl command )
Thanks
You could always use some python:
python -c"import datetime;print datetime.timedelta(milliseconds=14182142);"
3:56:22.142000
Is awk acceptible?
curl -s $JENKINS_JOB_URL/146/api/json?tree=duration | awk -F : '{ print $2 }' | awk -F } '{ printf "%i hr %i min", ($1 / 3600000), ($1 % 3600000)/60000 }'
If anyone knows of a nice way to get the number out of the string and into awk, I can remove the nasty double-invocation of awk.

How to add timestamp to pipe output?

I need to add a timestamp in front of the output of a long-executing command (a "tcpdump", in my use-case...).
It - very simplified - looks like this one:
(echo A1; sleep 3; echo B2) | perl -MPOSIX -pe 'print strftime "%T ", localtime $^T; s/\d//'
which gives this kind of output:
16:10:24 A
16:10:24 B
i.e.: perl's localtime is (obviously) called when perl is invoked.
Instead I need this kind of result:
16:10:24 A
16:10:27 B
i.e.: time stamp should be relative to the input's generation time...
Any smart (or no so smart :-) solution?
Just remove the $^T from your Perl command. That way, you will use the current time instead of the process start time. See the docs for $^T.
However, a more elegant formulation with Perl would be:
... | perl -MPOSIX -ne's/\d//; print strftime("%T ", localtime), $_'
You could pipe the output to:
awk '{ print strftime("%T"), $0; }'
Example:
while : ; do echo hey; sleep 1; done | awk '{ print strftime("%T"), $0; }'
20:49:58 hey
20:49:59 hey
20:50:00 hey
20:50:01 hey
20:50:02 hey
20:50:03 hey
20:50:04 hey
20:50:05 hey
Alternatively, you could use ts:
ts '%T'
(echo A1; sleep 3; echo B2) | perl -MPOSIX -pe 'print strftime "%T ", localtime; s/\d//'
Works excellent for me. Why you added $^T there?
on Linux, you can use tcpdump -tttt to print the timestamp before each output line.
# tcpdump -tttt -c 1 2>/dev/null
2013-12-13 23:42:12.044426 IP 10.0.2.15.ssh > 10.0.2.2.53466: Flags [P.], seq 464388005:464388121, ack 16648998, win 65535, length 116
If your tcpdump doesn't have the -tttt option, you should use awk from devnull

Extract data between two strings using either AWK or SED

I'm trying to extract data/urls (in this case - someurl) from a file that contains them within some tag ie.
xyz>someurl>xyz
I don't mind using either awk or sed.
I think the best, easiest, way is with cut:
$ echo "xyz>someurl>xyz" | cut -d'>' -f2
someurl
With awk can be done like:
$ echo "xyz>someurl>xyz" | awk 'BEGIN { FS = ">" } ; { print $2 }'
someurl
And with sed is a little bit more tricky:
$ echo "xyz>someurl>xyz" | sed 's/\(.*\)>\(.*\)>\(.*\)/\2/g'
someurl
we get blocks of something1<something2<something3 and print the 2nd one.
grep was born to extract things:
kent$ echo "xyz>someurl>xyz"|grep -Po '>\K[^>]*(?=>)'
someurl
you could kill a fly with a bomb of course:
kent$ echo "xyz>someurl>xyz"|awk -F\> '$0=$2'
someurl
If your grep supports P option then you can use lookahead and lookbehind regular expression to identify the url.
$ echo "xyz>someurl>xyz" | grep -oP '(?<=xyz>).*(?=>xyz)'
someurl
This is just a sample to get you started not the final answer.