What is this sed command doing? and is there any online utility that kind of explains sed a little bit, like regex?
sed -i '1s/$/|,a Type,b Type,c Type/;/./!b;1!s/$/|,,,/' textflile.txt
I think in the beginning it is adding csv a type, b type, c type at the end of the line but what does the rest of the command too
I don't know of any such utility, but let me explain using a text editor:
sed -i '1s/$/|,a Type,b Type,c Type/;/./!b;1!s/$/|,,,/' textflile.txt
^ ^ ^ ^ ^^ ^^ ^
| | | | || || |
modify | End Non-empty || || input
the | of lines || |Negation, file
file | line only || |i.e. lines 2,3,...
in | || |
place | || First
First line Negation, i.e.| line
empty lines only|
Branch to
script end,
i.e. skip the rest
In other words, it adds |,a type, b Type,c Type to the first line, doesn't change empty lines, and adds |,,, to all the remaining lines.
sed -i '1s/$/|,a Type,b Type,c Type/;/./!b;1!s/$/|,,,/' textflile.txt
can be written as
sed -i '
1 s/$/|,a Type,b Type,c Type/
/./! b
1! s/$/|,,,/
' textflile.txt
on line 1 only, add some text to the end of the line
if the line is empty ("matches 1 character, not"), goto next "cycle" (i.e., print current line and go to next line)
on every line except line 1, add "|,,," to the end of the line
So, it looks like you're adding some blank fields to a CSV file.
info sed contains the complete sed manual.
This doesn't answer your question but it's important for people to know and requires more space and formatting than a comment so: FYI to do what #choroba says that sed script does, i.e.
it adds |,a type, b Type,c Type to the first line,
doesn't change empty lines,
and adds |,,, to all the remaining lines.
is just this in awk:
awk '
NR==1 { print $0 "|,a type, b Type,c Type"; next }
!NF { print }
NF { print $0 "|,,," }
'
or if you're familiar with ternary expressions and want to remove the redundant code:
awk '{
sfx = "|," (NR==1 ? "a type, b Type,c Type" : ",,")
print $0 (NF ? sfx : "")
}'
I want to replace every occurrence of a string in a file but in a way that the new text will be numbered. Example:
from:
foo bar a
foo bar b
foo bar c
to:
bar baz1 a
bar baz2 b
bar baz3 c
How to do this in command line?
One way with awk:
$ awk '{gsub(/foo bar/,"bar baz"++i)}1' file
bar baz1 a
bar baz2 b
bar baz3 c
perl -lape '#F[0,1] = ($F[1], "baz". ++$i); $_= "#F"' file
Using shell.
#! /usr/bin/bash
i=0
while read a b c
do
((i++))
echo "bar baz$i $c"
done < file
Is it possible to encapsulate the following pseudocode using sed?
for line in lines:
if line == "foo":
print "FOO"
else:
print "- " + line
Here's the first thing I tried:
> echo 'foo
> bar
> baz' | sed -e 's/^foo$/FOO/' -e 's/^/- /'
- FOO
- bar
- baz
This is incorrect since both substitutions are applied to the first line.
Is it possible to tell sed to perform a maximum of one substitution per line?
You can limit what lines a substitution affects, by prefixing it with a pattern:
sed -e '/^foo$/! s/^/- /' -e '/^foo$/ s//FOO/' infile
A better alternative is to use the t branch command which will go to the next line if the previous substitution succeeded:
sed 's/^foo$/FOO/; t; s/^/- /' infile
Or the more portable:
sed -e 's/^foo$/FOO/' -e t -e 's/^/- /' infile
Output in both cases:
FOO
- bar
- baz
my textfile looks like this
foo.en 14 :: xyz 1;foo bar 2;foofoo 5;bar 9
bar.es 18 :: foo bar 4;kjp bar 2;bar 6;barbar 8
Ignoring text before the :: delimiter, is there a one liner unix command (many pipes allowed) or one liner perl script that extract the text such that yields the output of unique words delimited by ; ?:
xyz
foo bar
foofoo
bar
kjp bar
barbar
i've tried looping through the textfile with a python script but i'm looking for a one-liner for the task.
ans = set()
for line in open(textfile):
ans.add(line.partition(" :: ")[1].split(";").split(" ")[:-1])
for a in ans:
print a
With Perl:
perl -nle 's/.*?::\s*//;!$s{$_}++ and print for split /\s*\d+;?/' input
Description:
s/.*?::\s*//; # delete up to the first '::'
This part:
!$s{$_}++ and print for split /\s*\d+;?/
can be rewritten like this:
foreach my $word (split /\s*\d+;?/) { # for split /\s*\d+;?/
if (not defined $seen{$word}}) { # !$s{$_}
print $word; # and print
}
$seen{$word}++; # $s{$_}++
}
Since the increment in !$s{$_}++ is a post increment, Perl first test for the false condition and then does the increment. An undefined hash value has the value 0. If the test fails, i.e., $s{$_} was previously incremented, then the and part is skipped due to short circuiting.
cat textfile | sed 's/.*:://g' | tr '[0-9]*;' '\n' | sort -u
Explanation:
sed 's/.*:://g' Take everything up to and including `::` and replace it with nothing
tr '[0-9];' '\n' Replace numbers and semicolon with newlines
sort -u Sort, and return unique instances
it does result in a sorted output, I believe...
You can try this:
$ awk -F ' :: ' '{print $2}' input.txt | grep -oP '[^0-9;]+' | sort -u
bar
barbar
foo bar
foofoo
kjp bar
xyz
If your phrases contains numbers, try this perl regex: '[^;]+?(?=\s+\d+(;|$))'
With only awk :
$ awk -F' :: ' '{
gsub(/[0-9]+/, "")
split($2, arr, /;/ )
for (a in arr) arr2[arr[a]]=""
}
END{
for (i in arr2) print i
}' textfile.txt
And a one-liner version :
awk -F' :: ' '{gsub(/[0-9]+/, "");split($2, arr, /;/ );for (a in arr) arr2[arr[a]]="";}END{for (i in arr2) print i}' textfile.txt
How would like to join two lines usung awk or sed?
For example, I have data like below:
abcd
12:12:12:12:12:12:12:12
efgh001_01
45:45:45:45:45:45:45:45
ijkl7464746
78:78:78:78:78:78:78:78
and I need output like below:
abcd 12:12:12:12:12:12:12:12
efgh001_01 45:45:45:45:45:45:45:45
ijkl7464746 78:78:78:78:78:78:78:78
Running this almost works, but I need the space or tab:
awk '!(NR%2){print$0p}{p=$0}'
You're almost there:
awk '(NR % 2 == 0) {print p, $0} {p = $0}'
With sed you can do that as follows:
sed -n 'N;s/\n/ /p' file
where:
N reads next line
s replaces the new line character with a space to join both lines properly
p prints the result
This might work for you:
sed '$!N;s/\n/ /' file
or this:
paste -sd' \n' file