I have to find a particular string first, then i need to pick up a value in that string, which is changing aftrr every run. How can i do this.
Stage pro begin_time { size is 54mb and 3 sec }
Stage pro1 begin_time {.....}
I have very big file. I have to specifically look for begin_time string first then look for its value and store it somewhere. How can i do this.
I have tried it using grep but i don't know how to store value of 3 here. Similarly i have many cases here, how can i handle this situation
Thanks for the reply. I want to store it as value 3 with respect to pro, then some other value corresponding to pro1. Second, time mentioned here is a start time. I have same information for each of the stage pro,pro1 but with an end time. I have to extract end time as well. Then i have to caluculate net time elapsed, i.e. time between start time and end time
cat aaa
Stage pro begin_time { size is 54mb and 3 sec }
Stage pro1 begin_time {.....}
cat aaa |awk '$3=="begin_time" {print $9}' |grep .
3
if you wish to store it :
var=$(cat aaa |awk '$3=="begin_time" {print $9}' |grep .)
echo $var
3
If you wish to map pro with 3 then,
cat aaa | awk '$3=="begin_time" {print $2,$9}' |grep .
pro 3
Related
I'm writting some zsh functions using the powerful completion feature. The computation of my completions take some times and I want to make use of the completion caching policy. From the zsh manual (https://zsh.sourceforge.io/Doc/Release/Completion-System.html) I found this code snippet
example_caching_policy () {
# rebuild if cache is more than a week old
local -a oldp
oldp=( "$1"(Nm+7) )
(( $#oldp ))
}
I couldn't find any explanation on the (Nm+7) syntax, what does Nm means ? With try and error I could find out that for example Nms+1 would change the cache policy to 1 second, while Nmh+1 to 1 hour. But where can I find the general (NmX+N) construct explanation ?
Same what does exactly means the line (( $#oldp )) ?
I can explain the (Nm+7)
man zshexpn, search for Glob Qualifiers
a[Mwhms][-|+]n
files accessed exactly n days ago. Files accessed within the last n days are selected using a negative
value for n (-n). Files accessed more than n days ago are selected by a positive n value (+n). Op‐
tional unit specifiers `M', `w', `h', `m' or `s' (e.g. `ah5') cause the check to be performed with
months (of 30 days), weeks, hours, minutes or seconds instead of days, respectively. An explicit `d'
for days is also allowed.
Any fractional part of the difference between the access time and the current part in the appropriate
units is ignored in the comparison. For instance, `echo *(ah-5)' would echo files accessed within the
last five hours, while `echo *(ah+5)' would echo files accessed at least six hours ago, as times
strictly between five and six hours are treated as five hours.
m[Mwhms][-|+]n
like the file access qualifier, except that it uses the file modification time.
N stands for NULL_GLOB, if zsh matches nothing, it will remove the pattern.
Without this N option, if it matches nothing it will print an error.
Example with 4 files
$ touch lal # = updates file modification date to now
$ ls
lal lil lol tut
$ ls l*(m+7)
lil lol
# files older than 7 days starting with l
$ ls l*(m-7)
lal
# files younger than 7 days starting with l
$ ls l*(m+200)
zsh: no match
# no files older than 200 days
$ ls l*(Nm+200)
lal lil lol tut
# N = NULL_GLOB made disappear the non-matching pattern so it's just ls
I am trying to filter a pcap file in tshark wit a lua script and ultimately output it to a .csv. I am most of the way there but I am still running into a few issues.
This is what I have so far
tshark -nr -V -X lua_script:wireshark_dissector.lua -r myfile.pcap -T fields -e frame.time_epoch -e Something_UDP.field1 -e Something_UDP.field2 -e Something_UDP.field3 -e Something_UDP.field4 -e Something_UDP.field5 -e Something_UDP.field6 -e Something_UDP.field15 -e Something_UDP.field16 -e Something_UDP.field18 -e Something_UDP.field22 -E separator=,
Here is an example of what the frames look like, sort of.
frame 1
time: 1626806198.437893000
Something_UDP.field1: 0
Something_UDP.field2: 1
Something_UDP.field3:1
Something_UDP.field5:1
Something_UDP.field6:1
frame 2
time: 1626806198.439970000
Something_UDP.field8: 1
Something_UDP.field9: 0
Something_UDP.field13: 0
Something_UDP.field14: 0
frame 3
time: 1626806198.440052000
Something_UDP.field15: 1
Something_UDP.field16: 0
Something_UDP.field18: 1
Something_UDP.field19:1
Something_UDP.field20:1
Something_UDP.field22: 0
Something_UDP.field24: 0
The output I am looking for would be
1626806198.437893000,0,1,1,,1,1,1,,,,,
1626806198.440052000,,,,,,,,,1,0,,1,1,1,,0,0,,,,
That is if the frame contains one of the fields I am looking for it will output its value followed by a comma but if that field isn't there it will output a comma. One issue is that not every frame contains info that I am interested in and I don't want them to be outputted. Part of the issue with that is that one of the fields I need is epoch time and that will be in every frame but that is only important if the other fields are there. I could use awk or grep to do this but wondering if it can all be done inside tshark. The other issue is that the fields being requested will com from a text file and there may be fields in the text file that don't actually exist in the pcap file and if that happens I get a "tshark: Some fields aren't valid:" error.
In short I have 2 issues.
1: I need to print data only it the fields names match but not if the only match is epoch.
2: I need it to work even if one of the fields being requested doesn't exist.
I need to print data only it the fields names match but not if the only match is epoch.
Try using a display filter that mentions all the field names in which you're interested, with an "or" separating them, such s
-Y "Something_UDP.field1 or Something_UDP.field2 or Something_UDP.field3 or Something_UDP.field4 or Something_UDP.field5 or Something_UDP.field6 or Something_UDP.field15 or Something_UDP.field16 or Something_UDP.field18 or Something_UDP.field22"
so that only packets containing at least one of those fields will be processed.
I need it to work even if one of the fields being requested doesn't exist.
Then you will need to construct the command line on the fly, avoiding field names that aren't valid.
One way, in a script, to test whether a field is valid is to use the dftest command:
dftest Something_UDP.field1 >/dev/null 2>&1
will exit with a status of 0 if there's a field named "Something_UDP.field1" and will exit with a status of 2 if there isn't; if the scripting language you're using can check the exit status of a command to see if it succeeds, you can use that.
I have a log file that has text (a list of filenames), which are of this format:
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401164500025
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401170000022
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401171500018
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401173000018
Now, in a ksh script, I'm trying to retrieve two lists with the lines - one that have date OLDER date and other list that have NEWER date for a give date: 20170401 17:12
The first two lines into one older_list and last two lines into newer_list:
like,
older_list file has these:
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401164500025
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401170000022
newer_list file has these:
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401171500018
/dir1/dir2/dir3/dir4/dir5/dir6/dir7/us.ca.sf.release123.20170401173000018
can you guys please throw me the script that can handle this.
our is an old version
oslevel
5.3.0.0
Thanks
You could use for example awk tool in order to achieve your task:
Example:
> TIME=201704011700 # for same matter you could set it with current date TIME=$(date %Y%m%d%H%M)
> awk -F'\.' -v time=$TIME '{if (substr($5,1,12) >= time) {print $0 }}' your_input_log_file > fitered_after_$TIME
> awk -F'\.' -v time=$TIME '{if (substr($5,1,12) < time) {print $0 }}' your_input_log_file > fitered_before_$TIME
Explanation:
TIME is an variable defined in your shell
awk parameter -F
will define the . as a field delimiter thus your field we are
interested in as per your example would be $5.
-v option will
define for awk a variable time before starting with the value of
your TIME shell variable.
I have a very simple mp3 player, and the order it plays audio files are based on the file names, and the rule is there must be a 3-size number in the beginning of file name, such as:
001file.mp3
002file.mp3
003file.mp3
I want to write a fish shell sortmp3 to add numbers to the files of a directory. Say directory myfiles contains files:
aaa.mp3
bbb.mp3
ccc.mp3`
When I run sortmp3 myfiles, the file names will be changed to:
001aaa.mp3
002bbb.mp3
003ccc.mp3
But my question is:
how to generate some sequential numbers?
how to make sure the size of each number is exactly 3?
I would write this, which makes no assumptions about how many files there are in a directory:
function sortmp3
set -l files *
set -l i
for i in (seq (count $files))
echo mv $files[$i] (printf "%03d%s" $i $files[$i])
end
end
Remove the "echo" if you like how it works.
You can generate sequential numbers with the seq tool - an external program.
This will only take care of the first part, it won't pad to three characters.
To do that, there's a variety of choices:
printf '%s\n' 00(seq 0 99) | rev | cut -c 1-3 | rev
printf '%s\n' 00(seq 0 99) | sed 's/^.*\(...\)$/\1/'
The 00(seq 0 99) part will generate numbers from "1" to "99" with two zeroes prepended - ie. from "001" to "0099". The later parts of the pipeline remove the superfluous zeroes again.
Or with the next fish version, you can use the new string tool:
string sub -s -3 -- 00(seq 0 99)
Depending on your specific situation you should use the "seq" command to generate sequential numbers or the "math" command to increment a counter. To format the number with a predictable number of leading zeros use the "printf" command:
set idx 12
printf '%03d' $idx
Let's say I have the first column of the following dataset in a file and I want to emulate the flag in the second column so I export only that row tied to a flag = 1 (dataset is pre-sorted by the target column):
1 1
1 0
1 0
2 1
2 0
2 0
I could run awk 'NR==1 {print; next} seen[$1]++ {print}' dataset but would run into a problem for very large files (seen keeps growing). Is there an alternative to handle this without tracking every single unique value of the target column (here column #1)? Thanks.
So you only have the first column? And would like to generate the second? I think a slightly different awk command could work
awk '{if (last==$1) {flag=0} else {last=$1; flag=1}; print $0,flag}' file.txt
Basically you just check if the first field matches the last one you've seen. Since it's sorted, you don't have to keep track of everything you've seen, only the last one to know if the value is different.
Seems like grep would be fine for this:
$ grep " 1" dataset