Can't find this anywhere in my teacher's lectures and no one seems to have answered it online.
How do I use sed to take lines from one file and place them in between the lines of another file?
I know it's probably a simple question but I swear I can't find how to do it.
I attempted
sed '2r myfile.txt' mydata.txt > mydata.txt
but it deletes the mydata.txt lines entirely
If your mydata.tx is a typo and you are using mydata.txt, you cannot redirect to the input file, it will truncate it to empty before you even get started. Use sed -i '2r myfile.txt' mydata.txt or else write it to a file with another filename and rename it if necessary. (The -i will cause sed to update the input file with the output, basically automatically doing what I suggest.)
Related
I'd like to add a line on top of my output if my input file has a specific word.
However, if I'm just looking for specific string, then as I understand it, it's too late. The first line is already in the output and I can't prepend to it anymore.
Here's an exemple of input.
one
two
two
three
If I can find a line with, say, the word two, I'd like to add a new line before the first one, with for example FOUND. I want that line prepended only once, even if there are several matches.
So an input file without any two would remain unchanged, and the example file above would become:
FOUND
one
two
two
three
I know how to prepend with i\, but can't get the context right. From what I understood that would be around:
1{
/two/{ # This will will search "two" in the first line, how to look for it in the whole file ?
1i\
FOUND
}
}
EDIT:
I know how to do it using other languages/methods, that's not my question.
Sed has advanced features to work on several lines at once, append/prepend lines and is not limited to substitution. I have a sed file already filled with expressions to modify a python source file, which is why I'd prefer to avoid using something else. I want to be able to add an import at the beginning of a file if a certain class is used.
A Perl solution:
perl -i.bak -0077 -pE 'say "FOUND" if /two/;' in_file
The Perl one-liner uses these command line flags:
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
-i.bak : Edit input files in-place (overwrite the input file). Before overwriting, save a backup copy of the original file by appending to its name the extension .bak.
-E : Tells Perl to look for code in-line, instead of in a file. Also enables all optional features. Here, enables say.
-0777 : Slurp files whole.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
sed is for doing s/old/new on individual strings, that's not what you're trying to do so you shouldn't bother trying to use sed. There's lots of ways to do this, this one will be very efficient, robust and portable to all Unix systems:
$ grep -Fq 'two' file && echo "FOUND"; cat file
FOUND
one
two
two
three
To operate on a stream instead of (or in addition to) a file and without needing to read the whole input into memory:
awk 'f{print; next} {buf[NR]=$0} /two/{print "FOUND"; for (i=1;i<=NR;i++) print buf[i]; f=1}'
e.g.:
$ cat file | awk 'f{print; next} {buf[NR]=$0} /two/{print "FOUND"; for (i=1;i<=NR;i++) print buf[i]; f=1}'
FOUND
one
two
two
three
That awk script will also work using any awk in any shell on every Unix box.
I am struggling with another issue, this time related to redirecting output to a file (>). I have a couple of files named like foo-ab.in foo-abc.in foo-abcd.in bar-ab.in bar-abc.in bar-abcd.in and I want to go through each file, change some small part of it with sed and then rewrite the results to something like foo-ab2.in, foo-abc2.in and foo-abcd2.in for each file. What I have so far is:
for file in foo bar; do sed -n 's/text to change/replacement/ p' $file*.in> &1; done
however I get the error:
-bash: syntax error near unexpected token `&'
I understand that I am probably doing this all wrong, but I want to accomplish this:
for file in foo bar; do sed -n 's/text to change/replacement/ p' $file*.in > $file*2.in; done
where $file*2.in has exactly the same name as $file.in with a 2 appended in the middle, but since there are multiple foo with either ab, abc, or abcd endings, I don't know how to accomplish this. I already have my sed command worked out, it is just the correct file name to output to that is tripping me up.
Apparently an answer similar to this already exists here: wildcard input files
But for clarity and the sake of closing the question, the answer is to change the command as follows:
for file in foo*.in bar*.in; do sed -n 's/text to change/replacement p' $file > "${file%.*}"2.in; done
now it inputs file foo-ab.in through file foo-abcd.in, does the sed change, and outputs file foo-ab2.in through foo-abcd2.in (and the same for bar*.in).
I want to insert a range of lines from a file, say something like 210,221r before the first occurrence of a pattern in a bunch of other files.
As I am clearly not a GNU sed expert, I cannot figure how to do this.
I tried
sed '0,/pattern/{210,221r file
}' bunch_of_files
But apparently file is read from line 210 to EOF.
Try this:
sed -r 's/(FIND_ME)/PUT_BEFORE\1/' test.text
-r enables extendend regular expressions
the string you are looking for ("FIND_ME") is inside parentheses, which creates a capture group
\1 puts the captured text into the replacement.
About your second question: You can read the replacement from a file like this*:
sed -r 's/(FIND_ME)/`cat REPLACEMENT.TXT`\1/' test.text
If replace special characters inside REPLACEMENT.TXT beforehand with sed you are golden.
*= this depends on your terminal emulator. It works in bash.
In https://stackoverflow.com/a/11246712/4328188 CodeGnome gave some "sed black magic" :
In order to insert text before a pattern, you need to swap the pattern space into the hold space before reading in the file. For example:
sed '/pattern/ {
h
r file
g
N
}' in
However, to read specific lines from file, one may have to use a two-calls solution similar to dummy's answer. I'd enjoy knowing of a one-call solution if it is possible though.
I use sed to do a simple replacement to headers in a file.
Sometimes they need to be replaced, sometimes not.
It works fine, but is long because it reads the the files every time (hundreds of MB).
However there is a pattern that separates the header from the content.
How do I tell sed to stop processing the file after encountering a certain pattern ?
Example :
blabla headers that I want to edit here but maybe not FRAME some more content here
Let's say that want to remove "want" from the headers, but the word may or may not be in said headers. I know that I want to stop processing the file at FRAME.
sed -i '0,/\(pattern1\|pattern2\)/s//pattern1/' * ; # TODO stop at FRAME
You can use the q command to quit the sed processing the rest of the input
sed -i '0,/\(pattern1\|pattern2\)/s//pattern1/' * ; /FRAME/q'
/FRAME/ pattern matches the line containing FRAME upon which the command q is excecuted
OR
You can specify an address range from start of the file till it encounters FRAME as
sed '0, /FRAME/ s/old/new'
You can use awk
awk '/pattern stop/ {f=1} !f {sub(/old data/,"new data")} 1' file
This will replace old data with "new data" as long as pattern stop is not found.
To write data back to original file:
awk 'code' file > tmp && mv tmp file
I know a line number in a file, wherein I want to keep the first word and delete the rest till the end of the line. How do I do this using sed ?
So lets say, I want to go to line no 10 in a file, which looks like this -
goodword "blah blah"\
and what i want is
goodword
I have tried this - sed 's/([a-z])./\1/'
But this does it on all the lines in a file. I want it only on one specified line.
If by "first word" you mean "everything up to the first space", and if by "retain this change in the file itself" you mean that you don't mind creating a new file with the same name as the previous file, and if you have a sed that supports -i, you can probably just do:
sed -i '10s/ .*//' input-file
If you want to be more restrictive in the definition of a word, you can use '10s/\([a-z]*\).*/\1/'
Can you use grep or awk to grab just one line, and then pipe it into sed (if grep or awk couldn't do the entire job for you) to work on just one line? I think the key here is isolating that one line first, and then worrying about extracting something from it.
Using awk
awk 'NR==10 {print $1}' file
goodword