It's easy to find instructions on how to print lines with 65 characters or more (Google "sed one-liners), but I can't figure out the syntax to actually edit a line (i.e., do a substitution) on such a line.
To make changes on the lines which has 65 or above characters.
sed '/^.\{65,\}/s/.*/llll/' file
This would replace all the characters on the lines which has 65 or above characters to llll. Substitution works on those lines only.
^.\{65,\} search pattern which matches only those lines which satisfies the given condition. \{65,\} called repetition quantifier which repeats the previous token (that is .) 65 or more times. To save the changes made, you need to add inline edit -i parameter to your sed command.
Actually, I was able to piece it together. All I wanted to do was to replace "foo" with "bar," but only if the line has 65 characters or more. Turns out it looks like this:
sed '/\(^.\{65\}\)/s/foo/bar/' num.tmp
This might work for you (GNU sed):
sed 's/./&/65;T;s/pattern/replacement/' file
Replace character 65 by itself and if the substitution fails bail out otherwise ...
I think you need the -i option.
$ man sed
Output:
...
-i[SUFFIX], --in-place[=SUFFIX]
edit files in place (makes backup if extension supplied)
Related
France 211 55 Europe
Japan 144 120 Asia
Germany 96 61 Europe
England 94 56 Europe
Taiwan 55 144 Asia
North Korea 44 2134 Asia
The above is my data file.
There are empty lines in it.
There are no spaces or tabs in those empty lines.
I want to remove all empty lines in the data.
I did a search Delete empty lines using SED has given the perfect answer.
Before that, I wrote two sed code myself:
sed -r 's/\n\n+/\n/g' cou.data
sed 's/\n\n\n*/\n/g' cou.data
And I tried awk gsub, not successful either.
awk '{ gsub(/\n\n*/, "\n"); print }' cou.data
But they don't work and nothing changes.
Where did I do wrong about my sed code?
Use the following sed to delete all blank lines.
sed '/./!d' cou.data
Explanation:
/./ matches any character, including a newline.
! negates the selector, i.e. it makes the command apply to lines which do not match the selector, which in this case is the empty line(s).
d deletes the selected line(s).
cou.data is the path to the input file.
Where did you go wrong?
The following excerpt from How sed Works states:
sed operates by performing the following cycle on each line of input: first, sed reads one line from the input stream, removes any trailing newline, and places it in the pattern space. Then commands are executed; each command can have an address associated to it: addresses are a kind of condition code, and a command is only executed if the condition is verified before the command is to be executed.
When the end of the script is reached, unless the -n option is in use, the contents of pattern space are printed out to the output stream, adding back the trailing newline if it was removed.8 Then the next cycle starts for the next input line.
I've intentionally emboldened the parts which are pertinent to why your sed examples are not working. Given your examples:
They seem to disregard that sed reads one line at a time.
The trailing newlines, (\n\n and \n\n\n in your first and second example respectively), which you're trying to match don't actually exist. They've been removed by the time your regexp pattern is executed and then reinstated when the end of the script is reached.
RobC's answer is great if your lines are terminated by newline (linefeed or \n) only, because SED separates lines that way. If your lines are terminated by \r\n (or CRLF) - which you may have your reasons for doing even on a unix system - you will not get a match, because from sed's perspective the line isn't empty - the \r (CR) counts as a character. Instead you can try:
sed '/^\r$/d' filename
Explanation:
^ matches the start of the line
\r matches the carriage return
$ matches the end of the line
d deletes the selected line(s).
filename is the path to the input file.
I want to insert a range of lines from a file, say something like 210,221r before the first occurrence of a pattern in a bunch of other files.
As I am clearly not a GNU sed expert, I cannot figure how to do this.
I tried
sed '0,/pattern/{210,221r file
}' bunch_of_files
But apparently file is read from line 210 to EOF.
Try this:
sed -r 's/(FIND_ME)/PUT_BEFORE\1/' test.text
-r enables extendend regular expressions
the string you are looking for ("FIND_ME") is inside parentheses, which creates a capture group
\1 puts the captured text into the replacement.
About your second question: You can read the replacement from a file like this*:
sed -r 's/(FIND_ME)/`cat REPLACEMENT.TXT`\1/' test.text
If replace special characters inside REPLACEMENT.TXT beforehand with sed you are golden.
*= this depends on your terminal emulator. It works in bash.
In https://stackoverflow.com/a/11246712/4328188 CodeGnome gave some "sed black magic" :
In order to insert text before a pattern, you need to swap the pattern space into the hold space before reading in the file. For example:
sed '/pattern/ {
h
r file
g
N
}' in
However, to read specific lines from file, one may have to use a two-calls solution similar to dummy's answer. I'd enjoy knowing of a one-call solution if it is possible though.
I have a whole bunch of files, and I wish to change something like this:
My line of text
My other line of text
Into
My line of text\\
My other line of text
Seems simple, but somehow it isn't. I have tried sed s,"\n\n","\\\\\n", as well as tr '\n' '\\' and about 20 other incarnations of these commands.
There must be something going on which I don't understand... but I'm completely lost as to why nothing is working. I've had some comical things happening too, like when cat'ing out the file, it doesn't print newlines, only writes over where the rest was written.
Does anyone know how to accomplish this?
sed works on lines. It fetches a line, applies your code to it, fetches the next line, and so forth. Since lines are treated individually, multiline regexes don't work quite so easily.
In order to use multiline regexes with sed, you have to first assemble the file in the pattern space and then work on it:
sed ':a $!{ N; ba }; s/\n\n/\\\\\n/g' filename
The trick here is the
:a $!{ N; ba }
This works as follows:
:a # jump label for looping
$!{ # if the end of the input has not been reached
N # fetch the next line and append it to what we already have
ba # go to :a
}
Once this is over, the whole file is in the pattern space, and multiline regexes can be applied to it. Of course, this requires that the file is small enough to fit into memory.
sed is line-oriented and so is inappropriate to try to use on problems that span lines. You just need to use a record-oriented tool like awk:
$ awk -v RS='^$' -v ORS= '{gsub(/\n\n/,"\\\\\n")}1' file
My line of text\\
My other line of text
The above uses GNU awk for multi-char RS.
Here is an awk that solve this:
If the the blank lines could contains tabs or spaces, user this:
awk '!NF{a=a"//"} b{print a} {a=$0;b=NF} END {print a}' file
My line of text//
My other line of text
If blank line is just blank with nothing, this should do:
awk '!NF{a=a"//"} a!=""{print a} {a=$0} END {print a}' file
This might work for you (GNU sed):
sed 'N;s|\n$|//|;P;D' file
This keeps 2 lines in the pattern space at any point in time and replaces an empty line by a double slash.
I have a simple text file containing several lines of data where each line has got exactly 26 characters.
E.g.
10001340100491938001945591
10001340100491951002049591
10001340100462055002108507
10001340100492124002135591
10001340100492145002156507
10001340100472204002205591
Now, I want to replace 12th and 13th character if only these two characters are 49 with characters 58.
I tried it like this:
sed 's/^(.{12})49/\58/' data.txt
but am getting this error:
sed: -e expression #1, char 18: invalid reference \5 on `s' command's RHS
Thanks in advance
The captured group is \1, so you want to put the 11 (not 12) characters, then 58:
sed -E 's/^(.{11})49/\158/' data.txt
You also need -E or -r if you don't want to escape square and curly brackets.
With your input, the command changes 49 to 58 in lines 1, 2, 4 and 5.
If you like to try an awk solution:
awk 'substr($0,12,2)==49 {$0=substr($0,1,11)"58"substr($0,14)}1' file
10001340100581938001945591
10001340100581951002049591
10001340100462055002108507
10001340100582124002135591
10001340100582145002156507
10001340100472204002205591
In your expression you are replacing the first 14 characters (if you got it right). But you need to replace 11 plus the two (12th, 13th). More importantly, sed is fussy about escaping brackets, so you need backslashes in front of \( and { etc. Finally - the number of the capture group is 1. You omitted that number.
Putting it all together, you get
sed 's/^\(.\{11\}\)49/\158/'
There are flags you can use in sed to make the expressions more "regular" (changing the meaning of ( vs \(). What flag that is depends on your platform (version of sed), I believe.
For all lines that have .to I want to add expect( at the start.
For example for
blob.to 20
leave me
Thing.to(30)
Other.to {all}
leave me tooo
far_in.to stuff
I would like to see:
expect(blob.to 20
leave me
expect(Thing.to(30)
expect(Other.to {all}
leave me tooo
expect(far_in.to stuff
Currently I have:
sed -i '/.to/s/^[[:space]]*/expect(/' ../_spec_seded/"$file"
but am getting sed: -e expression #1, char 28: unterminateds' command`
from it
How do I resolve the error which I don't see yet and also 'preserve' spaces when doing this kind of substitution.
Bonus appreciation if you can exclude lines that have }.to instead of [any character except '}'].to
n.b. I deal with closing the parens separately.
p.s. unix site down right now.
Your leading spaces vanish because you choose not to preserve those. You could say:
sed '/.to/ s/^\([[:space:]]*\)/\1expect(/' filename
Alternatively, you could say:
sed '/\.to/ s/\(\S\)/expect(\1/' filename
For your input, it'd result in:
expect(blob.to 20
leave me
expect(Thing.to(30)
expect(Other.to {all}
leave me tooo
expect(far_in.to stuff
Bonus appreciation if you can exclude lines that have }.to instead of
[any character except '}'].to
If I understand it correct, you want to say:
sed '/[^}]\.to/ s/\(\S\)/expect(\1/' filename
This might work for you (GNU sed):
sed 's/\S\+\.to/expect(&/' file