I have multiple lines in a text file and some are empty.
hasjdh lashd
aksl asldh l lasjdh
I want to add * to the start of all the non empty lines.
*hasjdh lashd
*aksl asldh l lasjdh
how to do it
$ sed 's/^./*&/' file
*hasjdh lashd
*aksl asldh l lasjdh
This might work for you (GNU sed):
sed '/\S/s/^/*/' file
If the line contains a non-whitespace character, insert an * before the first character of that line.
I am using the following command to strip leading and trailing spaces from a file A.csv
sed "s/^ \+//g;s/[ \t]*$//;s/ \{1,\}/ /g" <A.csv> B.csv
Here is an example to A.csv
"a"," v b","z"
"a"," vd","z"
"a"," v, b, c ","z "
"a"," vb ","z "
The problem is that not all leading and trailing spaces are removed as shown below:
"a"," v b","z"
"a"," vd","z"
"a"," v, b, c ","z "
"a"," vb ","z "
Below is an example of what I was expecting:
"a","v b","z"
"a","vd","z"
"a","v, b, c","z"
"a","vb","z"
How can I get this right?
sed 's/" \+/"/g;s/[ \t]*"/"/g;s/ \{1,\}/ /g' A.csv
The ouput:
"a","v b","z"
"a","vd","z"
"a","v, b, c","z"
"a","vb","z"
Your own command, only s/ \{1,\}/ /g is working.
Thing is, sed will treat csv file as a simple text file, without knowing the commas and quotes are for columns.
So the ^ and $ will only match the beginning and the end of each line.
Also you forgot to put g to the second s.
You can't/shouldn't do this properly with just sed. I recommend switching to some better language that can work with CSV files.
There is also a tool called csvtool:
$ cat /path/to/trim
#!/usr/bin/env bash
shopt -s extglob
for c; do
c=${c##*([[:space:]])} c=${c%%*([[:space:]])}
printf '"%s"\n' "${c//'"'/'""'}"
done | paste -sd,
$ csvtool call /path/to/trim A.csv
"a","v b","z"
"a","vd","z"
"a","v, b, c","z"
"a","vb","z"
As much as I like csvtool for simple stuff, this will unfortunately be painfully slow! It took my VBox nearly 15 seconds to process a short 4000-line CSV.
This might work for you (GNU sed):
sed -r 's/"\s*([^[:space:]"]+(\s*[^[:space:]"]+)*)\s*"/"\1"/g' file
Remove immediate white space either side of a pair of double quotes, globally throughout the file.
If I have a string and I want to use sed such that a colon is inserted after the first 8 characters and then after every 2 characters after it, what would the sed pattern look like and what would the replacement pattern look like?
awk has substr() function, it can solve your problem eaiser:
awk -v m="8" -v n="2"
'{a=substr($0,1,m);
b=substr($0,m+1);
gsub(".{"n"}","&:",b)}$0=a":"b'
You can change the m and n to desired value (error handling was not done in my one-liner). If we do a test:
$ awk -v m="8" -v n="2" '{a=substr($0,1,m);b=substr($0,m+1);gsub(".{"n"}","&:",b)}$0=a":"b'<<<"aaaaaaaaaaaaaaa"
aaaaaaaa:aa:aa:aa:a
This might work for you (GNU sed):
sed 's/../&:/4g' file
Insert a : after the 4th occurrence of 2 characters and then globally throughout the line.
So let's say I have a file
a b c
a b
a
and I want to append a line below of the line containing a b and c. So I write the command :
sed -i '/s*a b c/a new line'
But it doesn't work. What is wrong with the sed command that I wrote?
You pattern is incorrect. Try this instead
sed -e '/a *b *c/a new line' file.txt
If you want to modify the original file in-place, add the -i flag. It would probably better to test it without it first though.
You need to match the line (or unique part) of the first line, do:
sed -i '/^a[[:blank:]]*b[[:blank:]]*c$/a new line' file.txt
^a[[:blank:]]* matches a at the start of the line followed by any number of whitespaces, similarly b[[:blank:]]* matches b followed by any number of whitespaces
c$ matches c at the end of the line
Example:
$ cat file.txt
a b c
a
a
b
$ sed '/^a[[:blank:]]*b[[:blank:]]*c$/a new line' file.txt
a b c
new line
a b
a
I am using sed to find and replace text, e.g.:
set -i 's/a/b/g' ./file.txt
This replaces every instance of a with b in the file. I need to add an exception, such that sed replaces every instance of a with b, except for the first appearance in the file, e.g.:
There lived a bird who liked to eat fish.
One day he fly to a tree.
This becomes:
There lived a bird who liked to ebt fish.
One dby he fly to b tree.
How can I modify my sed script to only replace every instance of a with b, except for the first occurrence?
I have GNU sed version 4.2.1.
This might work for you (GNU sed):
sed 's/a/b/2g' file
or
sed ':a;s/\(a[^a]*\)a/\1b/;ta' file
This can be taylored e.g.
sed ':a;s/\(\(a[^a]*\)\{5\}\)a/\1b/;ta' file
will start replacing a with b after 5 a's
You can do a more complete implementation with a script that's more complex:
#!/bin/sed -nf
/a/ {
/a.*a/ {
h
s/a.*/a/
x
s/a/\n/
s/^[^\n]*\n//
s/a/b/g
H
g
s/\n//
}
: loop
p
n
s/a/b/g
$! b loop
}
The functionality of this is easily explained in pseudo-code
if line contains "a"
if line contains two "a"s
tmp = line
remove everything after the first a in line
swap tmp and line
replace the first a with "\n"
remove everything up to "\n"
replace all "a"s with "b"s
tmp = tmp + "\n" + line
line = tmp
remove first "\n" from line
end-if
loop
print line
read next line
replace all "a"s with "b"s
repeat loop if we haven't read the last line yet
end-loop
end-if
One way is to replace all and then reverse the first replacement (thanks potong):
sed -e 'y/a/\n/' -e 's/\n/a/g' -e 'y/\n/b/'
Newline serves as an intermediate so strings beginning with b work correctly.
The above works line-wise, if you want to apply it to the whole file, first make the whole file into one line:
<infile tr '\n' '^A' | sed 'y/a/\n/; s/\n/a/; y/\n/b/' | tr '^A' '\n'
Or more briefly using the sed command from potong's answer:
<infile tr '\n' '^A' | sed 's/a/b/2g' | tr '^A' '\n'
Note ^A (ASCII 0x01) can be produced with Ctrl-vCtrl-a. ^A in tr can be replaced by \001.
This assumes that the file contains no ^A.