I have a sed oneliner which removes excess whitespace:
sed -e 's/^\s*//' -e 's/\s*$//' -e 's/\s{2,}/ /g'
When I test it on " \tone1 two\t3three\t ", the sed removes the whitespace at the beginning and end of the line but doesn't match the excess whitespace between words, and sed returns \tone1 two\t3three. What I want is \tone1 two 3three, so sed -e 's/[ \t]{2,}/ /g' is not functioning.
regexr.com shows the expression as functional.
My version is GNU sed version 4.2.1.
{ and } need to be escaped in basic regex mode that sed uses.
However, you can use this sed with a single substitution with alternation:
sed -E 's/^[[:blank:]]+|[[:blank:]]+$|[[:blank:]]{2,}//g' file
POSIX character class [[:blank:]] matches a space or tab characters.
Related
I have this content in a file where I want to replace spaces at certain positions with pipe symbol (|). I used sed for this, but it is replacing all the spaces in the string. But I don't want to replace the space for the 3rd and 4th string.
How to achieve this?
Input:
test test test test
My attempt:
sed -e 's/ /|/g file.txt
Expected Output:
test|test|test test
Actual Output:
test|test|test|test
sed 's/ /\
/3;y/\n / |/'
As newline cannot appear in a sed pattern space, you can change the third space to a newline, then change all newlines and spaces to spaces and pipes.
GNU sed can use \n in the replacement text:
sed 's/ /\n/3;y/\n / |/'
If the original input doesn't contain any pipe characters, you can do
sed -e 's/ /|/g' -e 's/|/ /3' file
to retain the third white space. Otherwise see other answers.
You could replace the 'first space' twice, e.g.
sed -e 's/ /|/' -e 's/ /|/' file.txt
Or, if you want to specify the positions (e.g. the 2nd and 1st spaces):
sed -e 's/ /|/2' -e 's/ /|/1' file.txt
Using GNU sed to replace the first and second one or more whitespace chunks:
sed -i -E 's/\s+/|/;s/\s+/|/' file
See the online demo.
Details
-i - inline replacements on
-E - POSIX ERE syntax enabled
s/\s+/|/ - replaces the first one or more whitespace chars
; - and then
s/\s+/|/ the second one or more whitespace chars on each line (if present).
Keep it simple and use awk, e.g. using any awk in any shell on every Unix box no matter what other characters your input contains:
$ awk '{for (i=1;i<NF;i++) sub(/ /,"|")} 1' file
test|test|test test
The above replaces all but the last " " on each line. If you want to replace a specific number, e.g. 2, then just change NF to 2.
I have to collect a select query data to a CSV file. I want to use a sed command to replace \n from the data by a space.
I'm using this:
query | sed "s/\n/ /g" > file.csv .......
But it is not working. Only \ is getting removed, while it should also remove n and add a space. Please suggest something.
You want to replace newline with space, not necessarily using sed.
Use tr:
tr '\n' ' '
\n is special to sed: it stands for the newline character. To replace a literal \n, you have to escape the backslash:
sed 's/\\n/ /g'
Notice that I've used single quotes. If you use double quotes, the backslash has a special meaning if followed by any of $, `, ", \, or newline, i.e., "\n" is still \n, but "\\n" would become \n.
Since we want sed to see \\n, we'd have to use one of these:
sed "s/\\\n/ /g" – the first \\ becomes \, and \n doesn't change, resulting in \\n
sed "s/\\\\n/ /g" – both pairs of \\ are reduced to \ and sed gets \\n as well
but single quotes are much simpler:
$ sed 's/\\n/ /g' <<< 'my\nname\nis\nrohinee'
my name is rohinee
From comments on the question, it became apparent that sed had nothing to do with removing the backslashes; the OP tried
echo my\nname\nis | sed 's/\n/ /g'
but the backslashes are removed by the shell:
$ echo my\nname\nis
mynnamenis
so even if the correct \\n were used, sed wouldn't find any matches. The correct way is
$ echo 'my\nname\nis' | sed 's/\\n/ /g'
my name is
I stumble upon the command sed -e 's/ /\'$'\n/g'that supposedly takes an input and split all spaces into new lines. Still, I don't quite get how the '$' works in the command. I know that s stands for substitute, / / stands for the blank spac, \n stands for new line and /g is for global replacement, but not sure how \'$' fits in the picture. Anybody who can shed some light here will be much appreciated.
Basically it's meant for platform portability. With GNU sed it would be just
sed -e 's/ /\n/g'
because GNU sed is able to interpret \n as new line.
However, other versions of sed, like the BSD version (that comes with MacOS) do not interprete \n as newline.
That's why the command is build out of two parts
sed -e 's/ /\' part2: $'\n/g'
The $'\n/g' is an ANSI C string parsed by the shell before executing sed. Escape sequences like \n will get expanded in such strings. Doing so, the author of the command passed a literal new line (0xa) to the sed command rather than passing the escape sequence \n. (0x5c 0x6e).
One more thing, since the newline (0xa) is a command separator in sed, it needs to get escaped. That's why the \ at the end of the first part.
Alternatively you could just use a multiline version:
sed -e 's/ /\
/g'
Btw, I would have written the command like
sed -e 's/ /\'$'\n''/g'
meaning just putting the $'\n' into the ANSI C string. Imo that's better to understand.
how can I replace a tab or empty spaces with a single semicolon?
This is what I'm trying:
sed -i.bak 's/(\s)+|(\t)+/,/g' aws_lab.txt
+/ was unexpected at this time.
Thanks
Just use the appropriate POSIX character class:
sed 's/[[:space:]]\{1,\}/,/g' aws_lab.txt
Very few seds will recognize the \s shorthand for [[:space:]] and since a tab is one of the characters included in [:space:] you dont need to specify it separately. Also, + is an ERE metacharacter, not BRE as supported by sed by default so you'd need to add the -E arg to sed to use it (only supported by GNU and OSX sed variants).
This might work for you (GNU sed):
sed -i.bak 's/[ \t][ \t]*/:/' file
Or
sed -ri.bak 's/[ \t]+/:/' file
Where there is an explicit space infront of the tab:
sed -i.bak 's/[ \t][ \t]*/:/g' file
Will replace repeated patterns throughout a line.
Am running BASH and UNIX utilities on Windows 7.
Have a file that contains a vertical tab. The binary symbol is 0x0B. The octal symbol is 013. I need to replace the symbol with a blank space.
Have tried this sed approach but it fails:
sed -e 's/'$(echo "octal-value")'/replace-word/g'
Specifically:
sed -e 's/'$(echo "\013")'/ /g'
Update:
Following this advice I use GNU sed and this approach:
sed -i 's:\0x0B: :g' file
but the stubborn vertical tab is still in the file.
What is the correct way to replace a non-printable character with a printable character?
Sed should recognise special characters:
sed -e 's/\x0b/ /g'
In answer to why the -e? If you use more than one sed expression, then each one must be preceded by the -e. So, for example:
echo foo bar bas zer | sed -e 's/zer/oh my/g' -e 's/bas/baz/'
would result in:
foo bar baz oh my
thus performing 2 different sed changes ('scripts) with only a single invocation. See sed man pages for more details.
(the above example is, obviously, contrived. I, however, have seen a sed command in a script with 78 individual -e 'scripts'!)
If you only have one 'script', then the -e is optional, obviously.