I need to print a line with any matches occurrences between '
This is my source line:
(2,'one.com',0,NULL,1,1,1,0),(3,'two.com',0,NULL,1,1,1,0),(4,'three.com',0,NULL,1,1,1,0),(5,'four.com',0,NULL,0,1,1,0),(6,'five.com',0,NULL,0,1,1,0), etc....
I need to print only:
one.com
two.com
three.com
four.com
five.com
You can for example use this grep:
$ grep -Po "(?<=')[^']*(?=',)" file
one.com
two.com
three.com
four.com
five.com
It prints from a given ' up to next ',.
Here is a simple awk to get the info:
awk -v RS=\' 'NR%2==0' file
one.com
two.com
three.com
four.com
five.com
It will print all fro start ' to stop '
Can also be written like this:
awk '!(NR%2)' RS=\' file
Related
Given a file, for example:
potato: 1234
apple: 5678
potato: 5432
grape: 4567
banana: 5432
sushi: 56789
I'd like to grep for all lines that start with potato: but only pipe the numbers that follow potato:. So in the above example, the output would be:
1234
5432
How can I do that?
grep 'potato:' file.txt | sed 's/^.*: //'
grep looks for any line that contains the string potato:, then, for each of these lines, sed replaces (s/// - substitute) any character (.*) from the beginning of the line (^) until the last occurrence of the sequence : (colon followed by space) with the empty string (s/...// - substitute the first part with the second part, which is empty).
or
grep 'potato:' file.txt | cut -d\ -f2
For each line that contains potato:, cut will split the line into multiple fields delimited by space (-d\ - d = delimiter, \ = escaped space character, something like -d" " would have also worked) and print the second field of each such line (-f2).
or
grep 'potato:' file.txt | awk '{print $2}'
For each line that contains potato:, awk will print the second field (print $2) which is delimited by default by spaces.
or
grep 'potato:' file.txt | perl -e 'for(<>){s/^.*: //;print}'
All lines that contain potato: are sent to an inline (-e) Perl script that takes all lines from stdin, then, for each of these lines, does the same substitution as in the first example above, then prints it.
or
awk '{if(/potato:/) print $2}' < file.txt
The file is sent via stdin (< file.txt sends the contents of the file via stdin to the command on the left) to an awk script that, for each line that contains potato: (if(/potato:/) returns true if the regular expression /potato:/ matches the current line), prints the second field, as described above.
or
perl -e 'for(<>){/potato:/ && s/^.*: // && print}' < file.txt
The file is sent via stdin (< file.txt, see above) to a Perl script that works similarly to the one above, but this time it also makes sure each line contains the string potato: (/potato:/ is a regular expression that matches if the current line contains potato:, and, if it does (&&), then proceeds to apply the regular expression described above and prints the result).
Or use regex assertions: grep -oP '(?<=potato: ).*' file.txt
grep -Po 'potato:\s\K.*' file
-P to use Perl regular expression
-o to output only the match
\s to match the space after potato:
\K to omit the match
.* to match rest of the string(s)
sed -n 's/^potato:[[:space:]]*//p' file.txt
One can think of Grep as a restricted Sed, or of Sed as a generalized Grep. In this case, Sed is one good, lightweight tool that does what you want -- though, of course, there exist several other reasonable ways to do it, too.
This will print everything after each match, on that same line only:
perl -lne 'print $1 if /^potato:\s*(.*)/' file.txt
This will do the same, except it will also print all subsequent lines:
perl -lne 'if ($found){print} elsif (/^potato:\s*(.*)/){print $1; $found++}' file.txt
These command-line options are used:
-n loop around each line of the input file
-l removes newlines before processing, and adds them back in afterwards
-e execute the perl code
You can use grep, as the other answers state. But you don't need grep, awk, sed, perl, cut, or any external tool. You can do it with pure bash.
Try this (semicolons are there to allow you to put it all on one line):
$ while read line;
do
if [[ "${line%%:\ *}" == "potato" ]];
then
echo ${line##*:\ };
fi;
done< file.txt
## tells bash to delete the longest match of ": " in $line from the front.
$ while read line; do echo ${line##*:\ }; done< file.txt
1234
5678
5432
4567
5432
56789
or if you wanted the key rather than the value, %% tells bash to delete the longest match of ": " in $line from the end.
$ while read line; do echo ${line%%:\ *}; done< file.txt
potato
apple
potato
grape
banana
sushi
The substring to split on is ":\ " because the space character must be escaped with the backslash.
You can find more like these at the linux documentation project.
Modern BASH has support for regular expressions:
while read -r line; do
if [[ $line =~ ^potato:\ ([0-9]+) ]]; then
echo "${BASH_REMATCH[1]}"
fi
done
grep potato file | grep -o "[0-9].*"
I have a file as below
NAME(BOLIVIA) TYPE(SA)
APPLIC(Java) IP(192.70.xxx.xx)
NAME(BOLIVIA) TYPE(SA)
APPLIC(Java) IP(192.71.xxx.xx)
I am trying to extract the values NAME and IP using sed:
cat file1 |
sed ':a
N
$!ba
s/\n/ /g' | sed -n 's/.*\(NAME(BOLI...)\).*\(IP(.*)\).*/\1 \2/p'
However, I'm only getting the output:
NAME(BOLIVIA) IP(192.71.xxx.xx)
What I would like is:
NAME(BOLIVIA) IP(192.70.xxx.xx)
NAME(BOLIVIA) IP(192.71.xxx.xx)
Would appreciate it if someone could give me a pointer on what I'm missing.
TIA
Your first sed commands reformats the file into one long line. You could have used tr -d "\n" for this, but that is not the problem.
The problem is in the second part, where the .* greedy eats as much as possible until finding the last match.
Your solution could be "fixed" with the ugly
# Do not use this:
sed -zn 's/[^\n]*\(NAME(BOLI...)\)[^\n]*\n[^\n]*\(IP([^)]*)\)[^\n]*/\1 \2/gp' file1
Possible solutions:
cat file1 | paste -d " " - - | sed -n 's/.*\(NAME(BOLI...)\).*\(IP(.*)\).*/\1 \2/p'
# or
grep -Eo "(NAME\(BOLI...\)|IP\(.*\))" file1 | paste -d " " - -
# or
printf "%s %s\n" $(grep -Eo "(NAME\(BOLI...\)|IP\(.*\))" file1)
In case you are ok with awk could you please try following. Written and tested in link
https://ideone.com/bJDzgf with shown samples only.
awk '
match($0,/^NAME\([^)]*/){
name=substr($0,RSTART+5,RLENGTH-5)
next
}
match($0,/IP\([^)]*/){
print name,substr($0,RSTART+3,RLENGTH-3)
name=""
}
' Input_file
This might work for you (GNU sed):
sed -n '/NAME/{N;/IP/s/\s.*\s/ /p}' file
If a line contains NAME and the following line contains IP remove everything between and print the result.
An alternative shorter awk:
awk '$1 ~ /^NAME/ {nm = $1} $2 ~ /^IP/ {print nm, $2}' file
NAME(BOLIVIA) IP(192.70.xxx.xx)
NAME(BOLIVIA) IP(192.71.xxx.xx)
The issue in your script is the use .* which matches in a greedy way
so that you have only the first NAME(BOLI...) and last IP(.*)
If you can use python :
#!/bin/bash
python -c '
import re, sys
for ar in re.findall(r"(NAME\(BOLI.*?\)).*?(IP\(.*?\))", sys.stdin.read(), re.DOTALL):
print(*ar)
' < input-file
I have a fairly complex text file file1.txt that hasn't been munged properly. The file is tab-delimited however, i.e. each string is separated by \t.
I would like to write a script/use a Unix command that parses this entire file for a certain string string1: which will print the line after the colon until stopping at \t.
The text file looks like this:
...kjdafhldkhlfak\tSTRING1:Iwanttokeepthis\tfadfasdafldafh\tSTRING1:andthis\tafsdkfasldh....
So the grep like function outputs
Iwanttokeepthis
andthis
In Perl, I know how to print a string if it occurs with
perl -wln -e 'print if /\bSTRING1\b/' file1.txt
How would one revise this to print the line between STRING1: and \t?
With Perl:
$ echo $'kjdafhldkhlfak\tSTRING1:Iwanttokeepthis\tfadfasdafldafh\tSTRING1:andthis\tafsdkfasldh' > /tmp/file
perl -lne 'while (/STRING1:([^\t]+)\t/g) {print $1}' /tmp/file
Iwanttokeepthis
andthis
Or, as stated in comments:
$ perl -nle'print for /STRING1:([^\t]*)\t/g' /tmp/file
Iwanttokeepthis
andthis
With GNU grep:
grep -Po 'STRING1:\K.*?(?=\t)' file
Output:
Iwanttokeepthis
andthis
See: The Stack Overflow Regular Expressions FAQ
Which is the simple and fast UNIX command to print all lines from the last occurrence of a pattern to the end of the file ?
sed -n '/pattern/,$p' file
This sed command prints from the first occurrence onwards.
This might work for you (GNU sed):
sed 'H;/pattern/h;$!d;x;//!d' file
Stashes the last pattern and following lines in the hold space and at end-of-file prints them out.
Or using the same method in awk:
awk '{x=x ORS $0};/pattern/{x=$0};END{if(x ~ //)print x}' file
However on my machine jaypals way with sed seems to be the quickest:
tac file | sed '/pattern/q' | tac
Reverse the file, print until the first pattern, exit and reverse the file.
tac file | awk '/pattern/{print;exit}1' | tac
Here's a Perlish way to do it:
perl -ne '$seen = 1, #a = () if /pattern/; push #a, $_; END { print #a if $seen }' file
Simplest solution is just to use a regex matching on the entire file:
perl -0777 -ne 'print $1 if /pattern(.*?)$/' file
A standalone awk:
awk '/pattern/{delete a;c=0}{a[c++]=$0}END{for (i=0;i<c;i++){print a[i]}}' file
Here is an pure awk
awk 'FNR==NR {if ($0~/pattern/) f=FNR;next} FNR==f {a=1}a' file{,}
It reads the file twice, and first time set a flag for last found of pattern, then print form pattern and out.
Or you can store data in an array like this:
awk '/pattern/ {f=NR} {a[NR]=$0} END {for (i=f;i<=NR;i++) print a[i]}' file
Using GNU awk for multi-char RS and gensub():
$ awk -v RS='^$' -v ORS= '{print gensub(/.*(pattern)/,"\\1","")}' file
e.g.:
$ cat file
a
b
c
b
d
$ awk -v RS='^$' -v ORS= '{print gensub(/.*(b)/,"\\1","")}' file
b
d
The above simply deletes from the start of the file up to just before the last occurrence of "b".
I want to delete all the rows/lines in a file that has a specific character, '?' in my case. I hope there is a single line command in Bash or AWK or Perl. Thanks
You can use sed to modify the file "in-place":
sed -i "/?/d" file
Alternatively, use grep:
grep -v "?" file > newfile.txt
Even better, just a single line using sed
sed '/?/d' input
use -i to edit file in place.
perl -i -ne'/\?/ or print' file
or
perl -i -pe's/^.*?\?.*//s' file
Here are already grep, sed and perl solutions - only for fun, pure bash one:
pattern='?'
while read line
do
[[ "$line" =~ "$pattern" ]] || echo "$line"
done
translated
for every line on the STDIN
match it for the pattern =~
and if the match is not successful || - print out the line
awk '!($0~/?/){print $0}' file_name