I have over 1000 text files where I need to remove the first 6 lines and also remove some boilerplate text from the bottom of each file.
I am trying this in one command in sed, if possible.
Here are the individual commands
sed -i.bak '/Some text as starting delimiter/,/Some text as an ending delimiter/d' My-File.txt
sed -e '1,6d' < MyFile.txt
Can these be combined ?
How so?
Thanks
Try this:
sed -i.bak -e '/Some text as starting delimiter/,/Some text as an ending delimiter/d' -e '1,6d' My-File.txt
or
sed -i.bak '/Some text as starting delimiter/,/Some text as an ending delimiter/d;1,6d' My-File.txt
Related
I have this content in a file where I want to replace spaces at certain positions with pipe symbol (|). I used sed for this, but it is replacing all the spaces in the string. But I don't want to replace the space for the 3rd and 4th string.
How to achieve this?
Input:
test test test test
My attempt:
sed -e 's/ /|/g file.txt
Expected Output:
test|test|test test
Actual Output:
test|test|test|test
sed 's/ /\
/3;y/\n / |/'
As newline cannot appear in a sed pattern space, you can change the third space to a newline, then change all newlines and spaces to spaces and pipes.
GNU sed can use \n in the replacement text:
sed 's/ /\n/3;y/\n / |/'
If the original input doesn't contain any pipe characters, you can do
sed -e 's/ /|/g' -e 's/|/ /3' file
to retain the third white space. Otherwise see other answers.
You could replace the 'first space' twice, e.g.
sed -e 's/ /|/' -e 's/ /|/' file.txt
Or, if you want to specify the positions (e.g. the 2nd and 1st spaces):
sed -e 's/ /|/2' -e 's/ /|/1' file.txt
Using GNU sed to replace the first and second one or more whitespace chunks:
sed -i -E 's/\s+/|/;s/\s+/|/' file
See the online demo.
Details
-i - inline replacements on
-E - POSIX ERE syntax enabled
s/\s+/|/ - replaces the first one or more whitespace chars
; - and then
s/\s+/|/ the second one or more whitespace chars on each line (if present).
Keep it simple and use awk, e.g. using any awk in any shell on every Unix box no matter what other characters your input contains:
$ awk '{for (i=1;i<NF;i++) sub(/ /,"|")} 1' file
test|test|test test
The above replaces all but the last " " on each line. If you want to replace a specific number, e.g. 2, then just change NF to 2.
I'm trying to replace the text using the sed, but it's showing some error. Not getting where I'm getting wrong.
sed -i 's/process.env.REDIRECT_URI/http:\/\/test-domain.apps.io/\callback/g' input.txt
Have this :
process.env.REDIRECT_URI
Replace this with :
http://test-domain.apps.io
Try:
sed -i 's/process.env.REDIRECT_URI/http:\/\/test-domain.apps.io/g' input.txt
Notes:
The original command has a spurious string /\callback. All that was needed to make the code work was to remove it.
. is a wildcard. If you want to be sure that you are matching periods, they should be escaped:
sed -i 's/process\.env\.REDIRECT_URI/http:\/\/test-domain.apps.io/g' input.txt
Sometimes, its clearer if one doesn't have to escape /. One can use a separator of one's choice. For example, use #:
sed -i 's#process\.env\.REDIRECT_URI#http://test-domain.apps.io#g' input.txt
If you did want /callback in the output, use:
sed -i 's/process\.env\.REDIRECT_URI/http:\/\/test-domain.apps.io\/callback/g' input.txt
or:
sed -i 's#process\.env\.REDIRECT_URI#http://test-domain.apps.io/callback#g' input.txt
27211;18:05:03479;20161025;0;0;0;0;10991;0;10991;000;0;0;000;1000000;0;0;000;0;0;0;82
Second string after ; is time. gg:mm:sssss:. I just want to be gg:mm:ss:
Like so:
27211;18:05:03;20161025;0;0;0;0;10991;0;10991;000;0;0;000;1000000;0;0;000;0;0;0;82
I tried with cut but it deletes everything after n'th occurance of character, and for now I am stuck, please help.
give this one liner a try:
awk -F';' -v OFS=";" 'sub(/...$/,"",$2)+1' file
It removes the last 3 chars from column 2.
update with sed one liner
If you are a fan of sed:
sed -r 's/(;[^;]*)...;/\1;/' file
With sed:
sed -r 's/^([^;]+;[^;]+)...;/\1;/' file
(Or)
sed -r 's/^([^;]+;[0-9]{2}:[0-9]{2}:[0-9]{2})...;/\1;/' file
It also can be something like sed 's/(.*)([0-9]{2}\:){2}([0-9]{3})[0-9]*\;(.*)/\1\2\3\4/g'
It is not very clean, but at least is more clear for me.
Regards
I'd use perl for this:
perl -pe 's/(?<=:\d\d)\d+(?=;)//' file
That removes any digits between "colon-digit-digit" and the semicolon (first match only, not globally in the line).
If you want to edit the file in-place: perl -i -pe ...
With sed:
sed -E 's/(:[0-9]{2})[0-9]{3}/\1/' file
or perl:
perl -pe's/:\d\d\B\K...//' file
Have a file that has been created incorrectly. There are several space delimited fields in the file but one text field has some unwanted newlines. This is causing a big problem.
How can I remove these characters but not the wanted line ends?
file is:
'Number field' 'Text field' 'Number field'
1 Some text 999999
2 more
text 111111111
3 Even more text 8888888888
EOF
So there is a NL after the word "more".
I've tried sed:
sed 's/.$//g' test.txt > test.out
and
sed 's/\n//g' test.txt > test.out
But none of these work. The newlines do not get removed.
tr -d '\n' does too much - I need to remove ONLY the newlines that are preceded by a space.
How can I delete newlines that follow a space?
SunOS 5.10 Generic_144488-09 sun4u sparc SUNW,Sun-Fire-V440
A sed solution is
sed '/ $/{N;s/\n//}'
Explanation:
/ $/: whenever the line ends in space, then
N: append a newline and the next line of input, and
s/\n//: delete the newline.
It might be simplest with Perl:
perl -p0 -e 's/ \n/ /g'
The -0 flag makes Perl read the entire file as one line. Then we can substitute using s in the usual way. You can, of course, also add the -i option to edit the file in-place.
How can I delete newlines that follow a space?
If you want every occurrence of $' \n' in the original file to be replaced by a space ($' '), and if you know of a character (e.g. a control character) that does not appear in the file, then the task can be accomplished quite simply using sed and tr (as you requested). Let's suppose, for example, that control-A is a character that is not in the file. For the sake of simplicity, let's also assume we can use bash. Then the following script should do the job:
#!/bin/bash
A=$'\01'
tr '\n' "$A" | sed "s/ $A/ /g" | tr "$A" '\n'
I have values in a file like this ' value-to-remove '(without the ' characters). I want to use sed to run through the file and replace the values including the space before and after. I am running this via a bash script.
How can I do this?
The sed command I'm using at the moment replaces the values but leaves behind the two spaces.
sed -i 's/ '$value' / /g' test.conf
In script I have
sed -i -e 's/\s'$DOMAIN'-'$SITE'\s/\s/g' gitosis.conf
echoed as
sed -i -e s/\sffff.com-eeee\s/\s/g test.conf
Not working though.
IMHO your sed does not know '\s', so use [ \t], and use double quotes, otherwise your variables will not expand. e.g.:
sed -i -e "s/[ \t]'$DOMAIN'-'$SITE'[ \t]/ /g" gitosis.conf
Let me know if this is what you need
echo 'Some values to remove value-to-remove and more' | sed -e 's/\svalue-to-remove\s/CHANGED/g'
output: Some values to removeCHANGEDand more