Okay after googling for some minutes it seems this is the regular way to prefix each line of output with sed
But I get an error I don't understand.
What does this mean and how can I fix this?
$ sed 's/^/#/' test.txt
sed: -e expression #1, char 0: no previous regular expression
What my test.txt looks like - it's really just a test
### test.txt
a
b
c
d
e
f
g
h
Oh yeah.. the version
$ sed --version
GNU sed version 4.2.1
Copyright (C) 2009 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE,
to the extent permitted by law.
You can use awk to solve this too:
cat file
a
b
To add # to all line:
awk '{$0="#"$0}1' file
#a
#
#b
To add # to all non blank line
awk 'NF{$0="#"$0}1' file
#a
#b
Below code will do the same using capturing group but it won't touch the blank lines.
sed 's/^\(.\)/#\1/' test.txt
try this if you want to add # at the start of a blank line also.
sed 's/^\(.\|\)/#\1/' file
Example:
$ cat f
a
b
$ sed 's/^\(.\)/#\1/' f
#a
#b
$ sed 's/^\(.\|\)/#\1/' f
#a
#
#b
Related
I want to replace only the first occurence of version: * in a file.
So I have a working sed command that work with GNU sed (source):
sed -i '0,/\(.*"version"\): "\(.*\)",/s//\1: '"\"${NEW_VERSION}\",/" package-lock.json
My problem is that i am executing this in scripts that also can run without GNU sed.
When i replace by sed -i '1,/\(.*"version"\): "\(.*\)",/s//\1: '"\"${NEW_VERSION}\",/" package-lock.json then it work without GNU sed but i have the following error when GNU sed is available:
sed: -e expression #1, char 0: no previous regular expression
EDIT: my main goal
As requested, here is my initial goal:
In a package.json and/or a package-lock.json , i want to replace the first occurence of version: X.X.X by version: Y.Y.Y where $NEW_VERSION containers Y.Y.Y
Using sed:
sed -i.bak -E '/(version: ).*/!{p;d;}
s//\1'"$NEW_VERSION"'/
:a
n
ba
' file
Alternatively this awk would also work:
awk -v ver="$NEW_VERSION" '!done && /^version:/{$2=ver; done=1} 1' file
You could check first occurrence by for example storing something in hold space.
sed '
# If hold space is empty
x;/^$/{x;
# If there is a pattern, replace it and..
/\("version": "\).*",/{
s//\1'"$NEW_VERSION"'"/1
# and hold the line.
h;
};x
};x
'
I'm going to simplify the expressions, since I'm not exactly sure what you're trying to match with the double quotes and the comma, and I think they obscure the main point. To replace just the first occurrence of foo with repl, you can do:
sed -e s/foo/repl/ -e ta -e p -e d -e :a -e n -e ba
The t command branches to the :a after a replacement is made, and the commands after :a just read and print each line without trying the substitution.
eg:
$ printf '%s\n' qux foo bar baz foo | sed -e s/foo/repl/ -e ta -e p -e d -e :a -e n -e ba
qux
repl
bar
baz
foo
But, this is really a lot easier with awk:
awk '/foo/ && !a{gsub("foo", "repl"); a = 1}1'
I have function that prints a header that needs to be applied across several files, but if I utilize a sed process substitution the lines prior to the last have a backslash \ on them.
E.g.
function print_header() {
cat << EOF
-------------------------------------------------------------------
$(date '+%B %d, %Y # ~ %r') ID:$(echo $RANDOM)
EOF
}
If I then take a file such as test.txt:
line 1
line 2
line 3
line 4
line 5
sed "1 i $(print_header | sed 's/$/\\/g')" test.txt
I get:
-------------------------------------------------------------------\
November 24, 2015 # ~ 11:18:28 AM ID:13187
line 1
line 2
line 3
line 4
line 5
Notice the troublesome backslash at the end of the first line, I'd like to not have that backslash appear. Any ideas?
I would use cat for that:
cat <(print_header) file > file_with_header
This behavior depends on the sed dialect. Unfortunately, it's one of the things which depends on which version you have.
To simplify debugging, try specifying verbatim text. Here's one from a Debian system.
vnix$ sed '1i\
> foo\
> bar' <<':'
> hello
> goodbye
> :
foo
bar
hello
goodbye
Your diagnostics appear to indicate that your sed dialect does not in fact require the backslash after the first i.
Since you are generating the contents of the header programmatically anyway, my recommended solution would be to refactor the code so that you can avoid this conundrum. If you don't want cat <<EOF test.txt then maybe experiment with sed 1r/dev/stdin' <<EOF test.txt (I could not get 1r- to work, but /dev/stdin should be portable to any Linux.)
Here is my kludgy fix, if you can find something more elegant I'll gladly credit you:
sed "1 i $(print_header | sed 's/$/\\/g;$s/$/\x01/')" test.txt | tr -d '\001'
This puts an unprintable SOH (\x01) ascii Start Of Header character after the inserted text, that precludes the backslashes and then I run it over tr to delete the SOH chars.
I have variable data with digits and minus "[-]", "[0-9]". It's may be:
source NUMBER ->result NUMBER AFTER MODIFY
XXX ->0.XXX,
XXXX ->X.XXX,
XXXXX ->XX.XXX,
-XXX -> -0.XXX,
-XXXX ->-X.XXX,
-XXXXX ->-XX.XXX,
Can this be done with sed?
I'd say:
sed -r 's/[0-9]{3}$/.&/; s/^(-?)\./\10./' filename
That is:
s/[0-9]{3}$/.&/ # put a dot before the last three digits in a line
s/^(-?)\./\10./ # if the result of this begins with . or -., insert a 0
# before the .
-r requires GNU sed. If you're on BSD or Mac OS X, which comes with BSD sed, you could use
sed 's/[0-9]\{3\}$/.&/;s/^\(-\?\)\./\10./' filename
That's the same thing with basic instead of extended regex syntax.
Addendum: Come to think of it, this appears to be equivalent to
awk '{ printf("%.3f\n", $0 / 1000) }' filename
sed 's/\(-*\)\(.*\)\(...\)/\10\2.\3/;s/0\([1-9]\)/\1/' YourFile
another way, remove 0 if needed and without back reference
perl -pe 's/(\d{3})\b/.$1/;
s/\B\./0./' file
line1 : 222<word-boundary> --> .222
line2 : <non-word-boundary>.222 --> 0.222
I just copied couple of files from windows to unix and they all have ^M at the end. I know how to remove them using vi, but I can only do one file at a time, is there a way I can do it for all the files in the folder?. There are like 60 files and manually doing it for all of them is time consuming!.
I'm open to using other tools as well!
PS: The OS is Solaris
Thanks
For posterity, let's post the solution from within VI. You can remove the Ctrl-M at the end of every line like this:
:%s/^V^M$//
Note that this is what you type, wnere ^V means Ctrl-V and ^M means Ctrl-M. The idea here is that ^V will "escape" the following ^M, so that you can match it in the substitution regex.
And the % expression means "do this on every line".
Note that this may or may not work in vim, depending on your settings.
But your question asks how to do this in vi, in which you can't easily make a change to multiple files. If you're open to using other tools, please indicate so in your question.
You can use sed on a single file or stream:
$ printf 'one\r\ntwo\r\n' > /tmp/test.txt
$ od -c < /tmp/test.txt
0000000 o n e \r \n t w o \r \n
0000012
$ sed -i'' -e 's/^M$//' /tmp/test.txt
$ od -c < /tmp/test.txt
0000000 o n e \n t w o \n
0000010
$
In this case, in /bin/sh in FreeBSD, I escaped the ^M by ... you guessed it ... using ^V.
When using sed's -i option, you can specify multiple files and they will all be modified in place, perhaps eliminating the need to wrap this in a script. If you want to put this into a script anyway, I recommend you try to do so, and then ask for help if it doesn't work. That's the StackOverflow Way. :-)
Or just use Jonathan's for loop example. You don't need temp files.
UPDATE
If your sed does not have a -i option, then you can still do this pretty easily in a for loop:
[ghoti#pc ~]$ od -c /tmp/test1.txt
0000000 o n e \r \n t w o \r \n
0000012
[ghoti#pc ~]$ for f in /tmp/test*.txt; do sed -e 's/^M$//' "$f" > /tmp/temp.$$ && mv -v /tmp/temp.$$ "$f"; done
/tmp/temp.26687 -> /tmp/test1.txt
/tmp/temp.26687 -> /tmp/test2.txt
[ghoti#pc ~]$ od -c /tmp/test1.txt
0000000 o n e \n t w o \n
0000010
If you don't have a dos2unix or dtou command on your machine, you can use tr instead:
for file in "$#" # LIst of files passed as argument to script
do
tr -d '\015' < "$file" > tmp.$$
cp tmp.$$ "$file"
done
rm tmp.$$
You can add trap commands around that to clean up if you interrupt. Using cp instead of mv preserves owner, permissions, symlinks, hard links.
use the command dos2ux.
dos2ux file >file2
I need to replace the pattern ### with the current line number.
I managed to Print in the next line with both AWK and SED.
sed -n "/###/{p;=;}" file prints to the next line, without the p;, it replaces the whole line.
sed -e "s/###/{=;}/g" file used to make sense in my head, since the =; returns the line number of the matched pattern, but it will return me the the text {=;}
What am i Missing? I know this is a silly question. I couldn't find the answer to this question in the sed manual, it's not quite clear.
If possible, point me what was i missing, and what to make it work. Thank you
Simple awk oneliner:
awk '{gsub("###",NR,$0);print}'
Given the limitations of the = command, I think it's easier to divide the job in two (actually, three) parts. With GNU sed you can do:
$ sed -n '/###/=' test > lineno
and then something like
$ sed -e '/###/R lineno' test | sed '/###/{:r;N;s/###\([^\n]*\n\)\([^\n]*\)/\2\1/;tr;:c;s/\n\n/\n/;tc}'
I'm afraid there's no simple way with sed because, as well as the = command, the r and GNU extension R commands don't read files into the pattern space, but rather directly append the lines to the output, so the contents of the file cannot be modified in any way. Hence piping to another sed command.
If the contents of test are
fooo
bar ### aa
test
zz ### bar
the above will produce
fooo
bar 2 aa
test
zz 4 bar
This might work for you (GNU sed):
sed = file | sed 'N;:a;s/\(\(.*\)\n.*\)###/\1\2/;ta;s/.*\n//'
An alternative using cat:
cat -n file | sed -E ':a;s/^(\s*(\S*)\t.*)###/\1\2/;ta;s/.*\t//'
As noted by Lev Levitsky this isn't possible with one invocation of sed, because the line number is sent directly to standard out.
You could have sed write a sed-script for you, and do the replacement in two passes:
infile
a
b
c
d
e
###
###
###
a
b
###
c
d
e
###
Find the lines that contain the pattern:
sed -n '/###/=' infile
Output:
6
7
8
11
15
Pipe that into a sed-script writing a new sed-script:
sed 's:.*:&s/###/&/:'
Output:
6s/###/6/
7s/###/7/
8s/###/8/
11s/###/11/
15s/###/15/
Execute:
sed -n '/###/=' infile | sed 's:.*:&s/^/& \&/:' | sed -f - infile
Output:
a
b
c
d
e
6
7
8
a
b
11
c
d
e
15
is this ok ?
kent$ echo "a
b
c
d
e"|awk '/d/{$0=$0" "NR}1'
a
b
c
d 4
e
if match pattern "d", append line number at the end of the line.
edit
oh, you want to replace the pattern not append the line number... take a look the new cmd:
kent$ echo "a
b
c
d
e"|awk '/d/{gsub(/d/,NR)}1'
a
b
c
4
e
and the line could be written like this as well: awk '1+gsub(/d/,NR)' file
one-liner to modify the FILE in place, replacing LINE with the corresponding line number:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs -IX sed -i 'X s/LINE/X/' FILE
Following on from https://stackoverflow.com/a/53519367/29924
If you try this on osx the version of sed is different and you need to do:
seq 1 `wc -l FILE | awk '{print $1}'` | xargs --verbose -IX sed -i bak "X s/__line__/X/" FILE
see https://markhneedham.com/blog/2011/01/14/sed-sed-1-invalid-command-code-r-on-mac-os-x/