I'm running GnuWin32 under Windows 10
I'm trying to run the following sed one-liner using the Gnu Bash shell:
sed -f <(sed -E 's_(.+)\t(.+)_s/\1/\2/g_' C:/dictionary.txt) C:/content.txt
The file substitute sed statement converts dictionary entries into sed expressions. The main sed uses them for the content replacements.
It is described in How to awk to read a dictionary and replace words in a file?
dictionary.txt looks like this:
aluminium<tab>aluminum
analyse<tab>analyze
white spirit<tab>mineral spirits
stag night<tab>bachelor party
savoury<tab>savory
potato crisp<tab>potato chip
mashed potato<tab>mashed potatoes
content.txt looks like this:
The container of white spirit was made of aluminium.
We will use an aromatic method to analyse properties of white spirit.
No one drank white spirit at stag night.
Many people think that a potato crisp is savoury, but some would rather eat mashed potato.
...
more sentences
When running GnuWin32/sed in GnuBash-shell under windows 10, I receive the following error message:
syntax error near unexpected token <(s
How to re-formulate the script to run under GnuWin32/sed under windows 10?
with thanks to https://stackoverflow.com/users/2836621/mark-setchell and https://stackoverflow.com/users/5403468/tiw the solution works when using cygwin64
One way is to write the inner sed output to a temporary file first, use it, and then delete it:
sed -r "s_(.+)\t(.+)_s/\1/\2/g_" C:/dictionary.txt>tmp_script.sed
sed -f tmp_script.sed C:/content.txt
del tmp_script.sed
Another way, based on Mr. Mark Setchell's comment, plus little tweak, with cygwin installed,
this work on both bash and batch:
sed -r "s_(.+)\t(.+)_s/\1/\2/g_" C:/dictionary.txt | sed -f /dev/stdin C:/content.txt
I was trying to install docker in Ubuntu, and following the instructions I have come across a very strange sed command which I do not understand (this seems to be used to set-up bash autocompletion for docker):
sudo sed -i '$acomplete -F _docker docker' /etc/bash_completion.d/docker.io
What is that command doing? The -i command means in-place editing, but what does the $acomplete -F _docker docker mean? The $ is matching the last line, but what is it doing? I do not even recognize any sed command there! For example, a substitution command would look like:
$s/in/out/
Could somebody explain that expression for me?
It appends complete -F _docker docker to the file.
From sed's manual:
a \
text Append text, which has each embedded newline preceded by a backslash.
I'm using Sed on a Mac. I am trying to do a simple string replace on a file that is not in the directory. I do:
sed -i 's/old/new/' /Users/A/file
and it says invalid command code A.
What do I need to do?
The -i option in OSX/BSD sed is a little different than the GNU/Linux version in that it requires a backup extension to be given, even if it's an empty string (which means that no backup will be made). The "invalid command code" error message occurs because s/old/new is taken as the backup extension and /Users/A/file is taken as the script (where A is seen as an invalid command name). So it needs to be something like:
sed -i '' 's/old/new/' /Users/A/file
if you have perl:
perl -p -i -e 's/old/new/' /Users/A/file
The -i in sed is not a standard across OS's (not a POSIX standard). This should work every time:
cp /Users/A/file /Users/A/file.sed
cat /Users/A/file.sed | sed 's/old/new' > /Users/A/file
I have this line inside a file:
ULNET-PA,client_sgcib,broker_keplersecurities
,KEPLER
I try to get rid of that ^M (carriage return) character so I used:
sed 's/^M//g'
However this does remove everything after ^M:
[root#localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities^M,KEPLER
[root#localhost tmp]# sed 's/^M//g' test
ULNET-PA,client_sgcib,broker_keplersecurities
What I want to obtain is:
[root#localhost tmp]# vi test
ULNET-PA,client_sgcib,broker_keplersecurities,KEPLER
Use tr:
tr -d '^M' < inputfile
(Note that the ^M character can be input using Ctrl+VCtrl+M)
EDIT: As suggested by Glenn Jackman, if you're using bash, you could also say:
tr -d $'\r' < inputfile
still the same line:
sed -i 's/^M//g' file
when you type the command, for ^M you type Ctrl+VCtrl+M
actually if you have already opened the file in vim, you can just in vim do:
:%s/^M//g
same, ^M you type Ctrl-V Ctrl-M
You can simply use dos2unix which is available in most Unix/Linux systems. However I found the following sed command to be better as it removed ^M where dos2unix couldn't:
sed 's/\r//g' < input.txt > output.txt
Hope that helps.
Note: ^M is actually carriage return character which is represented in code as \r
What dos2unix does is most likely equivalent to:
sed 's/\r\n/\n/g' < input.txt > output.txt
It doesn't remove \r when it is not immediately followed by \n and replaces both with just \n. This fails with certain types of files like one I just tested with.
alias dos2unix="sed -i -e 's/'\"\$(printf '\015')\"'//g' "
Usage:
dos2unix file
If Perl is an option:
perl -i -pe 's/\r\n$/\n/g' file
-i makes a .bak version of the input file
\r = carriage return
\n = linefeed
$ = end of line
s/foo/bar/g = globally substitute "foo" with "bar"
In awk:
sub(/\r/,"")
If it is in the end of record, sub(/\r/,"",$NF) should suffice. No need to scan the whole record.
This is the better way to achieve
tr -d '\015' < inputfile_name > outputfile_name
Later rename the file to original file name.
I agree with #twalberg (see accepted answer comments, above), dos2unix on Mac OSX covers this, quoting man dos2unix:
To run in Mac mode use the command-line option "-c mac" or use the
commands "mac2unix" or "unix2mac"
I settled on 'mac2unix', which got rid of my less-cmd-visible '^M' entries, introduced by an Apple 'Messages' transfer of a bash script between 2 Yosemite (OSX 10.10) Macs!
I installed 'dos2unix', trivially, on Mac OSX using the popular Homebrew package installer, I highly recommend it and it's companion command, Cask.
This is clean and simple and it works:
sed -i 's/\r//g' file
where \r of course is the equivalent for ^M.
Simply run the following command:
sed -i -e 's/\r$//' input.file
I verified this as valid in Mac OSX Monterey.
remove any \r :
nawk 'NF+=OFS=_' FS='\r'
gawk 3 ORS= RS='\r'
remove end of line \r :
mawk2 8 RS='\r?\n'
mawk -F'\r$' NF=1
Does anybody know which version of sed is required to get option -i to work? I am on CentOS5 and I am getting this error.
If you're going to be using -i with sed you're doing it wrong. sed is a stream editor and it should be used to edit streams, not files, as -i wants to do.
If you want to edit a file, you should be using ed. ed is a line editor and it should be used to edit files. IMO, that's the tool you want to be using.
btw, -i is a GNUism. from the wikipedia:
GNU sed added several new features. The best-known is in-place editing of files (i.e., replace the original file with the result of applying the sed program), which was later included in BSD sed too. This feature is nowadays often used instead of ed scripts: for example,
I don't think you can get -i to work then.
I think this other SO question may help you out:
sed -i + what the same option in SOLARIS
Perhaps the solution isn't as nice as sed -i, however.