So I have a txt file where I need to extract every third number and print it to separate file using Terminal. The txt file is just a long list of numbers, tab delimited:
18 25 0 18 24 5 18 23 5 18 22 8.2 ...
I know there is a way to do this using sed or awk, but so far I've only been able to extract every third line by using:
awk 'NR%3==1' testRain.txt > rainOnly.txt
So here's the answer (or rather, the answer I utilized!):
xargs -n1 < input.txt | awk '!(NR%3)' > output.txt
This gives you an output.txt that has every third number of the original file as a separate line.
A quick pipe line to extract every 3rd number:
$ xargs -n1 < file | sed '3~3!d'
0
5
5
8.2
If you don't want each number on a newline throw the result back through xargs:
$ xargs -n1 < file | sed '3~3!d' | xargs
0 5 5 8.2
Use redirection to store the output in a new file:
$ xargs -n1 < file | sed '3~3!d' | xargs > new_file
With awk using a simple for loop you could do:
$ awk '{for(i=3;i<=NF;i+=3)print $i}' file
0
5
5
8.2
or (adds a trailing tab):
$ awk '{for(i=3;i<=NF;i+=3)printf "%s\t",$i;print ""}' file
0 5 5 8.2
Or by setting the value of RS (adds trailing newline):
$ awk '!(NR%3)' RS='\t' file
0
5
5
8.2
$ awk '!(NR%3)' RS='\t' ORS='\t' file
0 5 5 8.2
You can print every third character by substituting the next two with nothing, globally. When the count straddles a newline, using Perl might be the simplest solution:
perl -p000 -e 's/(.)../$1/gs'
If you want the first, fourth etc character from every line, a line-oriented tool like sed suffices:
sed 's/\(.\)../\1/g'
Using grep -P
grep -oP '([^\t]+\t){2}\K[^\t\n]+' file
0
5
5
8.2
This might work for you (GNU sed):
sed -r 's/(\S+\s){3}/\1/g;s/\s$//' file
#user2718946
Your solution was close, but here you are without xarg.
awk 'NR%3==1' RS=" " file
18
18
18
18
Different start:
awk 'NR%3==0' RS=" " file
0
5
5
8.2
Related
I want sed command to delete a line from which which contain more than 10 number and which are not start with 91
My file content is like this
919876543210
789876543210
9012345678
12345678901
9865746321
And need output like this.
919876543210
9012345678
9865746321
I have tried awk 'length>=4' 1.txt | grep -v ^91
this will show me
789876543210
12345678901
What you can do:
ignore all lines starting with "91", /^91/!
otherwise delete anything with at least 11 characters
(assuming you do not care about the difference between digits, letters, whitespace...),
{/.........../d}
Altogether:
sed '/^91/!{/.........../d}'
With the -E it would be more elegantly possible.
(My sed is GNU sed version 4.2.1)
Simple, clear, portable, efficient, etc. with awk:
$ awk '/^91/ || length()<11' file
919876543210
9012345678
9865746321
I want to remove lines from file by regex pattern using sed just like in this question Delete lines in a text file that containing a specific string, but only inside a range of lines (not in the whole file). I want to do it starting from some line number till the end of file.
This is how I've done it in combination with tail:
tail -n +731 file|sed '/some_pattern/d' >> file
manually remove edited range in file from previous step
Is there a shorter way to do it with sed only?
Something like sed -i '731,1000/some_pattern/d' file?
You can use this sed,
sed -i.bak '731,1000{/some_pattern/d}' yourfile
Test:
$ cat a
1
2
3
13
23
4
5
$ sed '2,4{/3/d}' a
1
2
23
4
5
You need $ address to match end of file. With GNU sed:
sed -i '731,${/some_pattern/d;}' file
Note that this can be slower than tail -n +number, because sed will start processing at start of file instead of doing lseek() like tail.
(With BSD sed you need sed -i '' ...)
sed is for simple substitutions on individual lines, that is all. For anything even marginally more interesting an awk solution will be clearer, more robust, portable, maintainable, extensible and better in just about ever other desirable attribute of software.
Given this sample input file:
$ cat file
1
2
3
4
1
2
3
4
1
2
3
4
The following script will print every line except a line containing the number 3 that occurs after the 6th line of the input file:
$ awk '!(NR>6 && /3/)' file
1
2
3
4
1
2
4
1
2
4
Want to only do the deletion between lines 6 and 10? No problem:
$ awk '!(NR>6 && NR<10 && /3/)' file
1
2
3
4
1
2
4
1
2
3
4
Want the skipped lines written to a log file? No problem:
awk 'NR>6 && /3/{print > "log";next} {print}' file
Written to stderr?
awk 'NR>6 && /3/{print | "cat>&2";next} {print}' file
Want a count of how many lines you deleted also written to stderr?
awk 'NR>6 && /3/{print | "cat>&2"; cnt++; next} {print} END{print cnt | "cat>&2"}' file
ANYTHING you want to do additionally or differently will be easy and build on what you start with. Try doing any of the above, or just about anything else, with a sed script that satisfies your original requirement.
awk to the rescue!
awk '!(NR>=731 && /pattern/)' input > output
A text file (file name: 1.txt):
Incoming_Queries_A: 13201096
Incoming_Queries_A6: 946
Incoming_Queries_AAAA: 1288191
Incoming_Queries_ANY: 31280
Incoming_Queries_AXFR: 5
Incoming_Queries_CNAME: 410
Incoming_Queries_DS: 20
Incoming_Queries_MX: 854
Incoming_Queries_NS: 97217
Incoming_Queries_PTR: 1011409
Incoming_Queries_SOA: 5006
Incoming_Queries_SPF: 1
Incoming_Queries_SRV: 3555
Incoming_Queries_TXT: 511
Incoming_Requests_IQUERY: 11
Incoming_Requests_NOTIFY: 1
Incoming_Requests_QUERY: 15640501
Incoming_Requests_STATUS: 1
Incoming_Requests_UPDATE: 5
I want to remove all strings before tab (include tab) in a line of text and print the output(example: 13201096 ) to standard out.
Example:
# egrep -i "Incoming_Queries_A:" ./1.txt | sed 's/.Incoming_Queries_A:\t//'
Output:
Incoming_Queries_A: 13201096
But I only want to output 13201096
How to fix it? thanks
sed doesn't automatically handle escaped chars (such as \t and \n) on bash. You can handle it in two different ways:
You can replace the \t by an actual tab in your expression. To hit a tab in the terminal, you do Control-V, then hit the TAB key: Ctrl-V and then Tab.
(this one seems far more elegant, IMO) you can force sed to interpret your \t, by placing a $ before your substitution string. This way, your command would be like:
egrep -i "Incoming_Queries_A:" ./1.txt | sed $'s/Incoming_Queries_A:\t//'
(I removed that . before Incoming_Queries_A: - probably a typo/desperate tentative)
Hope that helps.
Since you only need the second column, you can use cut:
cut -f2 file.txt
With awkyou can do this like that:
awk '{print $2}' 1.txt
13201096
946
1288191
31280
5
410
20
854
97217
1011409
5006
1
3555
511
11
1
15640501
1
5
Or awk '/Incoming_Queries_A:/ {print $2}' /tmp/t.txt for get only the line you wanted
You could try the below GNU sed command,
sed -r 's/^.*\t//' file
I want to delete some specific rows in a file. For example I have a file containing these rows:
+1 1:-51.000000
+1 1:-47.000000
+1 1:-53.000000
+1 1:-48.000000
+1 1:-49.000000
+1 1:-42.000000
I want to delete the last 3 rows. How to do that in bash file?
nl=`wc -l fileName | awk 's=$1-3{print s}'`; head -n $nl fileName > file_withoutlast3rows.txt
wc -l calculates the total number of lines in the file. Using awk, you can print the total number of lines -3. Then use head with -n option to read only that many lines. You can also put that many lines into a new file using >.
You can also use awk to do this :
awk -v nl=$(wc -l <fileName) 'NR<(nl-2)' fileName
Simply count the number of lines, calculate the first index, and use sed -i to delete it.
COUNT=$(wc -l < file)
INDEX=$((COUNT > 3 ? COUNT - 2 : 1))
sed -i "$INDEX,\$d" file
Or just use bash's readarray:
readarray LINES < file
LAST=$((${#LINES[#]} > 3 ? ${#LINES[#]} - 3 : 0))
printf "%s" "${LINES[#]:0:LAST}" > file
This might work for you (GNU sed):
sed '1N;$!N;$d;P;D' file
for say 5 lines:
sed ':a;$!N;s/.*/&/m5;Ta;$d;P;D' file
Using only awk with no external commands like wc.
Read the file twice
awk 'NR==FNR {i++;next} (i-FNR)>=3' file file
Is fairly easy to strip the first and last character from a string using awk/sed?
Say I have this string
( 1 2 3 4 5 6 7 )
I would like to strip parentheses from it.
How should I do this?
sed way
$ echo '( 1 2 3 4 5 6 7 )' | sed 's/^.\(.*\).$/\1/'
1 2 3 4 5 6 7
awk way
$ echo '( 1 2 3 4 5 6 7 )' | awk '{print substr($0, 2, length($0) - 2)}'
1 2 3 4 5 6 7
POSIX sh way
$ var='( 1 2 3 4 5 6 7 )'; var="${var#?}"; var="${var%?}"; echo "$var"
1 2 3 4 5 6 7
bash way
$ var='( 1 2 3 4 5 6 7 )'; echo "${var:1: -1}"
1 2 3 4 5 6 7
If you use bash then use the bash way.
If not, prefer the posix-sh way. It is faster than loading sed or awk.
Other than that, you may also be doing other text processing, that you can combine with this, so depending on the rest of the script you may benefit using sed or awk in the end.
why doesn't this work? sed '..' s_res.temp > s_res.temp ?
This does not work, as the redirection > will truncate the file before it is read.
To solve this you have some choices:
what you really want to do is edit the file. sed is a stream editor not a file editor.
ed though, is a file editor (the standard one too!). So, use ed:
$ printf '%s\n' "%s/^.\(.*\).$/\1/" "." "wq" | ed s_res.temp
use a temporary file, and then mv it to replace the old one.
$ sed 's/^.\(.*\).$/\1/' s_res.temp > s_res.temp.temp
$ mv s_res.temp.temp s_res.temp
use -i option of sed. This only works with GNU-sed, as -i is not POSIX and GNU-only:
$ sed -i 's/^.\(.*\).$/\1/' s_res.temp
abuse the shell (not recommended really):
$ (rm test; sed 's/XXX/printf/' > test) < test
On Mac OS X (latest version 10.12 - Sierra) bash is stuck to version 3.2.57 which is quite old. One can always install bash using brew and get version 4.x which includes the substitutions needed for the above to work.
There is a collection of bash versions and respective changes, compiled on the bash-hackers wiki
To remove the first and last characters from a given string, I like this sed:
sed -e 's/^.//' -e 's/.$//'
# ^^ ^^
# first char last char
See an example:
sed -e 's/^.//' -e 's/.$//' <<< "(1 2 3 4 5 6 7)"
1 2 3 4 5 6 7
And also a perl way:
perl -pe 's/^.|.$//g'
If I want to remove the First (1) character and the last two (2) characters using sed.
Input "t2.large",
Output t2.large
sed -e 's/^.//' -e 's/..$//'
`