file1 contains:
first=stan
last=smith
I want to create file2 which contains
first=homer
last=simpson
script.sh contains
#!/bin/bash
sed s/stan/$1/ file1 >tempfile
sed s/smith/$2/ tempfile >file2
rm tempfile
script.sh homer simpson does what I want.
Is there a better way to do this in a bash script without creating and deleting tempfile?
Yes you can:
$ s1="homer"
$ s2="simpson"
$ sed -e "s/stan/$s1/g" -e "s/smith/$s2/g" file1 > file2
$ cat file2
first=homer
last=simpson
As you can see the -e option is used to perform two different sed commands in the same line.
In a bash script:
#!/bin/bash
sed -e "s/stan/$1/g" -e "s/smith/$2/g" file1 > file2
Sed supports multiple expressions. So you can do something like:
sed -e "s/stan/$1/" -e "s/smith/$2/" file1 > file2
I also recommend to use double quotes to prevent wordsplitting.
Related
I am trying to insert the contents of a file1.txt into file2.txt using sed. The content of file1.txt is just a single line, which is a path.
I want it to be added as a prefix to each line in file2.txt as well as add another / character.
$ cat file1.txt
/psot/rot8888/orce/db/tier/data/tine
$ cat file2.txt
o1_mf_users_abchwfg_.dbf
o1_mf_toptbs2_abchrq0_.dbf
o1_mf_toptbs1_abchrl2_.dbf
o1_mf_toptbs1_abchtlf_.dbf
Desired output should be like:
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
Tried command:
$ sed '/o1/ r file1.txt' file2.txt >> test.txt
$ cat test.txt
o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine
o1_mf_toptbs1_abchtlf_.dbf
/psot/rot8888/orce/db/tier/data/tine
You can use pr for this without having to worry about sed metacharacters, delimiters, etc.
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d/
$ pr -mts"$(< f1)" /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
Where -m allows pasting files parallely and -s is the separator between the files to be merged. Here, /dev/null is used as a dummy for one of the files as only the separator has to be prefixed.
If you need to add some more characters after the contents of file containing the prefix:
$ cat ip.txt
abcd.xyz
123.txt
foo_baz.txt
$ cat f1
/a/b/c/d
$ pr -mts"$(< f1)"'/' /dev/null ip.txt
/a/b/c/d/abcd.xyz
/a/b/c/d/123.txt
/a/b/c/d/foo_baz.txt
This will work using any awk in any shell on every UNIX box:
$ awk 'NR==FNR{p=$0; next} {print p "/" $0}' file1 file2
/psot/rot8888/orce/db/tier/data/tine/o1_mf_users_abchwfg_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs2_abchrq0_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchrl2_.dbf
/psot/rot8888/orce/db/tier/data/tine/o1_mf_toptbs1_abchtlf_.dbf
This might work for you (GNU sed):
sed '1h;1d;G;s/\(.*\)\n\(.*\)/\2\1/' file1 file2
Copy file1 into the hold space and append it to each line in file2. Using regexp and back references, manipulate the two lines into one in the correct order.
Alternative:
sed 'x;s/.*/cat file1/e;G;s/\n//' file2
Insert file1 into the hold space, append the current line of file2, and remove the newline connecting them.
A third way:
sed 'r file1' file2 | sed -E 'N;s/(.*)\n(.*)/\2\/\1/'
I have this list of files:
$ more files
one_this_2017_1_abc.txt
two_that_2018_1_abc.txt
three_another_2017_10.abc.txt
four_again_2018_10.abc.txt
five_back_2018_1a.abc.txt
I would like to get this output:
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY.abc.txt
four_again_XXXX_YY.abc.txt
five_back_XXXX_YY.abc.txt
I am trying to remove the year and the bit after the year and replace them with another string--this is to generate test cases.
I can get the year just fine, but it's that one or two character piece after it I can't seem to match.
This should work, right?
~/test_cases
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\{1,2\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Except it doesn't for the 2 character cases.
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\
{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Doesn't work for the two character cases either, and this works not at all (but according to the docs it should):
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[[:alnum:]]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
Other random experiments that don't work:
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\+_/_YY_/'
one_YY_XXXX_1_abc.txt
two_YY_XXXX_1_abc.txt
three_YY_XXXX_10.abc.txt
four_YY_XXXX_10.abc.txt
five_YY_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{1\}_/_YY_/'
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
$ cat files | sed -e 's/_[[:digit:]]\{4\}_/_XXXX_/' -e 's/_[a-zA-Z0-9]\{2\}_/_YY_/'
one_this_XXXX_1_abc.txt
two_that_XXXX_1_abc.txt
three_another_XXXX_10.abc.txt
four_again_XXXX_10.abc.txt
five_back_XXXX_1a.abc.txt
Tried with both GNU sed version 4.2.1 under Linux and sed (GNU sed) 4.4 under Cygwin.
And yes, I realize I can pipe this through multiple sed calls to get it to work, but that regex SHOULD work, right?
if your Input_file is same as shown sample then following may help you in same.
sed 's/\([^_]*\)_\([^_]*\)_\(.*_\)\(.*\)/\1_\2_XXXX_YY_\4/g' Input_file
Output will be as follows.
one_this_XXXX_YY_abc.txt
two_that_XXXX_YY_abc.txt
three_another_XXXX_YY_10.abc.txt
four_again_XXXX_YY_10.abc.txt
five_back_XXXX_YY_1a.abc.txt
I have the following sed command:
sed -i -E "s/\{\{(.*)\}\}/$(echo "$\1")/g" test.conf
In test.conf, I have this:
this is a {{TEST}}
and this is an {{ANSWER}} here.
And I have the follow environment variables set:
export TEST=1234
export ANSWER=5678
When I run the sed command, I end up with this result:
this is a $TEST
and this is an $ANSWER here.
I want 1234 and 5678 there respectively. Is there a reason the echo command is interpreting things literally?
Backreferences are used internally by a single sed command. The echo has no idea about sed backreferences and would have been invoked by the shell before the sed command has even run so the $(echo "$\1") is outputing $\1 so
sed -i -E "s/\{\{(.*)\}\}/$(echo "$\1")/g" test.conf
is really:
sed -i -E "s/\{\{(.*)\}\}/$\1/g" test.conf
hence the output you are seeing.
Anyway, sed is for simple subsitutions on individual lines, for anything else you should be using awk:
$ export TEST=1234 ANSWER=5678
$ awk 'match($0,/(.*)\{\{(.*)\}\}(.*)/,a){$0=a[1] ENVIRON[a[2]] a[3]} 1' file
this is a 1234
and this is an 5678 here.
The above uses GNU awk for the 3rd arg to match(), with other awks it'd be:
$ awk 'match($0,/\{\{(.*)\}\}/){$0=substr($0,1,RSTART-1) ENVIRON[substr($0,RSTART+2,RLENGTH-4)] substr($0,RSTART+RLENGTH)} 1' file
this is a 1234
and this is an 5678 here.
If anyone suggests running eval or similar on the sed output - don't do it (google eval is evil and friends), just use the awk command above for simple string operations.
You can use perl which conveniently has a %ENV hash variable, see perldoc for more info
perl -pe 's/\{\{(.*)\}\}/$ENV{$1}/' test.conf
This is probably a trivial one:
I have a file (my.file) with these lines:
>h1_c1
>h1_c2
>h1_c3
>h2_c1
>h2_c2
>h2_c3
and I want to change it in place to be:
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h3
I thought this ought to do it:
sed -i 's/\(\>\)\(h1\)\(\_\)\(.*\)/\1 \4 \3 \2/g' my.file
sed -i 's/\(\>\)\(h2\)\(\_\)\(.*\)/\1 \4 \3 \2/g' my.file
but it doesn't seem to work. How do I do it?
The obvious sed for your example is:
$ sed -i~ -e 's/^>\(h[0-9]\)_\(c[0-9]\)/>\2_\1/' *.foo
I tested this and it works for your example file.
Try this awk
awk -F">|_" '{print ">"$3"_"$2}' my.file > tmp && mv tmp my.file
awk -F">|_" '{print ">"$3"_"$2}' my.file
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h2
You can try this sed,
sed 's/>\(h[1-2]\)_\(.*\)/>\2_\1/' yourfile
(OR)
sed -r 's/>(h[1-2])_(.*)/>\2_\1/' yourfile
kent$ sed -r 's/>([^_]*)_(.*)/>\2_\1/' f
>c1_h1
>c2_h1
>c3_h1
>c1_h2
>c2_h2
>c3_h2
you add -i if you want it to happen "in-place"
I need to do the following for hundreds of files:
Append the name of the file (which may contain spaces) to the end of each line in the file.
It seems to me there should be some way to do this:
sed -e 's/$/FILENAME/' *
where FILENAME represents the name of the current file. Is there a sed variable representing the current filename? Or does anyone have a different solution using bash, awk, etc.?
I'm sure there are other ways to do it, I'd use perl:
perl -p -i -e 's/$/$ARGV/;' *
Some versions of sed support the "--in-place" argument so you can condense Tyler's solution to
for i in * ; do
sed -e "s/\$/$i/" --in-place "$i"
done
You could do it with a bash script
for i in *
do
sed -e "s/\$/$i/" "$i"
done
One-liner version:
for i in * ; do sed -e "s/\$/$i/" "$i" ; done
Edit: If you want to replace the contents of the file with the new, name-appended lines, do this:
TFILE=`mktemp`
for i in *
do
sed -e "s/\$/$i/" "$i" > $TFILE
cp -f $TFILE "$i"
done
rm -f $TFILE
awk '{print $0,FILENAME}' > tmpfile
In BASH, I'd do something to the effect of:
for f in *; do echo $f >> $f; done
More or less how Tyler suggested, just with some modifications to allow for spaces in the name. I was hoping for a one-liner though...
(
OLDIFS=$IFS
IFS=$'\n'
for f in *
do
IFS=OLDIFS
sed -e "s/\$/$f/" $f > tmpfile
mv tmpfile $f
IFS=$'\n'
done
)
This might work for you:
printf "%s\n" * | sed 's/.*/sed -i "s|$| &|" &/' | bash