I'm trying to migrate this working command
docker-compose $(find docker-compose* | sed -e "s/^/-f /") up -d --remove-orphans
from bash to fish. The intention of this command is to get this
docker-compose -f docker-compose.backups.yml ... -f docker-compose.wiki.yml up -d --remove-orphans
My naive try
docker-compose (find docker-compose* | sed -e "s/^/-f /") up -d --remove-orphans
is not working, though. The error is:
ERROR: .FileNotFoundError: [Errno 2] No such file or directory: './ docker-compose.backups.yml'
What is the correct translation?
The difference in behavior is due to the fact fish, sanely, only splits the output of a command capture on line boundaries. Whereas POSIX shells like bash split it on whitespace by default. That is, POSIX shells split the output of $(...) on the value of $IFS which is space, tab, and newline by default.
There are several ways to rewrite that command so it works in fish. The one that requires the smallest change is to change the sed to insert a newline between the -f and the filename:
docker-compose (find docker-compose* | sed -e "s/^/-f\n/") up -d --remove-orphans
I can't for the life of me get sed working with vagrant provisioning. I want to make a inline change to /etc/hosts.
I've verified that the sed command works when run in the shell.
Here is my Vagrantfile:
# vi: set ft=ruby :
########### Global Config ###########
machines = ["admin2"]
num_hdd_per_osd = 3
vagrant_box = %q{bento/ubuntu-18.04}
#####################################
machines.each do |machine|
Vagrant.configure("2") do |config|
config.vm.define machine do |node| #name vagrant uses to reference this VM
node.vm.box = vagrant_box
node.vm.hostname = machine
node.vm.network "private_network", ip: "192.168.0.#{ machines.index(machine) + 10}"
node.vm.provider "virtualbox" do |vb|
# Display the VirtualBox GUI when booting the machine
vb.gui = false
vb.name = machine # name virtualbox uses to refer to this vm
# Customize the amount of memory on the VM:
vb.memory = "1048"
# Core Count
vb.cpus = "2"
end
if node.vm.hostname.include? "admin"
node.vm.provision "shell", inline: <<-SHELL
sed -i.bak -e 's,\\(127\\.0\\.0\\.1[[:space:]]*localhost\\),\\1aa,' /etc/hosts
SHELL
end
end
end
end
I should see /etc/hosts changed to 127.0.0.1 localhostaa but it is unchanged.
What is wrong?
EDIT: I updated the code with the suggestion from Alex below. It now uses inline: <<-SHELL and escaped ALL escapes (so double escape). It Works!
The problem there is your Vagrantfile is Ruby code, and your sed script is inside a Ruby here string.
If you try this simplified Ruby script:
# test.rb
puts <<-SHELL
sudo sed -i.bak -e 's,\(127\.0\.0\.1[[:space:]]*localhost\),\1aa,' /etc/host
SHELL
You may see the problem:
▶ ruby test.rb
sudo sed -i.bak -e 's,(127.0.0.1[[:space:]]*localhost),aa,' /etc/host
That is, the \1 and other \ have been interpreted by Ruby prior to interpolation in the here string.
The best option for you is to use the <<'SHELL' notation, similar to what you would do in Bash:
node.vm.provision "shell", inline: <<-'SHELL'
sed -i.bak -e 's,\(127\.0\.0\.1[[:space:]]*localhost\),\1aa,' /etc/hosts
SHELL
The other option would be to escape the backslash in \1. Also, note that, as far as I can tell, the call to sudo is not required there either.
If, however, you need to interpolate a string in this script, you could do something like this:
# test.rb
mystring = 'aa'
$script = "sed -i.bak -e '" +
's,\(127\.0\.0\.1[[:space:]]*localhost\),\1' + "#{mystring},' /etc/hosts"
And then in your provisioner:
node.vm.provision "shell", inline: $script
See also this related answer.
I've successfully used the following sed command to search/replace text in Linux:
sed -i 's/old_link/new_link/g' *
However, when I try it on my Mac OS X, I get:
"command c expects \ followed by text"
I thought my Mac runs a normal BASH shell. What's up?
EDIT:
According to #High Performance, this is due to Mac sed being of a different (BSD) flavor, so my question would therefore be how do I replicate this command in BSD sed?
EDIT:
Here is an actual example that causes this:
sed -i 's/hello/gbye/g' *
If you use the -i option you need to provide an extension for your backups.
If you have:
File1.txt
File2.cfg
The command (note the lack of space between -i and '' and the -e to make it work on new versions of Mac and on GNU):
sed -i'.original' -e 's/old_link/new_link/g' *
Create 2 backup files like:
File1.txt.original
File2.cfg.original
There is no portable way to avoid making backup files because it is impossible to find a mix of sed commands that works on all cases:
sed -i -e ... - does not work on OS X as it creates -e backups
sed -i'' -e ... - does not work on OS X 10.6 but works on 10.9+
sed -i '' -e ... - not working on GNU
Note Given that there isn't a sed command working on all platforms, you can try to use another command to achieve the same result.
E.g., perl -i -pe's/old_link/new_link/g' *
I believe on OS X when you use -i an extension for the backup files is required. Try:
sed -i .bak 's/hello/gbye/g' *
Using GNU sed the extension is optional.
This works with both GNU and BSD versions of sed:
sed -i'' -e 's/old_link/new_link/g' *
or with backup:
sed -i'.bak' -e 's/old_link/new_link/g' *
Note missing space after -i option! (Necessary for GNU sed)
Had the same problem in Mac and solved it with brew:
brew install gnu-sed
and use as
gsed SED_COMMAND
you can set as well set sed as alias to gsed (if you want):
alias sed=gsed
Or, you can install the GNU version of sed in your Mac, called gsed, and use it using the standard Linux syntax.
For that, install gsed using ports (if you don't have it, get it at http://www.macports.org/) by running sudo port install gsed. Then, you can run sed -i 's/old_link/new_link/g' *
Your Mac does indeed run a BASH shell, but this is more a question of which implementation of sed you are dealing with. On a Mac sed comes from BSD and is subtly different from the sed you might find on a typical Linux box. I suggest you man sed.
Insead of calling sed with sed, I do ./bin/sed
And this is the wrapper script in my ~/project/bin/sed
#!/bin/bash
if [[ "$OSTYPE" == "darwin"* ]]; then
exec "gsed" "$#"
else
exec "sed" "$#"
fi
Don't forget to chmod 755 the wrapper script.
Sinetris' answer is right, but I use this with find command to be more specific about what files I want to change. In general this should work (tested on osx /bin/bash):
find . -name "*.smth" -exec sed -i '' 's/text1/text2/g' {} \;
In general when using sed without find in complex projects is less efficient.
I've created a function to handle sed difference between MacOS (tested on MacOS 10.12) and other OS:
OS=`uname`
# $(replace_in_file pattern file)
function replace_in_file() {
if [ "$OS" = 'Darwin' ]; then
# for MacOS
sed -i '' -e "$1" "$2"
else
# for Linux and Windows
sed -i'' -e "$1" "$2"
fi
}
Usage:
$(replace_in_file 's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' "./mysql/.env")
Where:
, is a delimeter
's,MASTER_HOST.*,MASTER_HOST='"$MASTER_IP"',' is pattern
"./mysql/.env" is path to file
As the other answers indicate, there is not a way to use sed portably across OS X and Linux without making backup files. So, I instead used this Ruby one-liner to do so:
ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml
In my case, I needed to call it from a rake task (i.e., inside a Ruby script), so I used this additional level of quoting:
sh %q{ruby -pi -e "sub(/ $/, '')" ./config/locales/*.yml}
Here's how to apply environment variables to template file (no backup need).
1. Create template with {{FOO}} for later replace.
echo "Hello {{FOO}}" > foo.conf.tmpl
2. Replace {{FOO}} with FOO variable and output to new foo.conf file
FOO="world" && sed -e "s/{{FOO}}/$FOO/g" foo.conf.tmpl > foo.conf
Working both macOS 10.12.4 and Ubuntu 14.04.5
Here is an option in bash scripts:
#!/bin/bash
GO_OS=${GO_OS:-"linux"}
function detect_os {
# Detect the OS name
case "$(uname -s)" in
Darwin)
host_os=darwin
;;
Linux)
host_os=linux
;;
*)
echo "Unsupported host OS. Must be Linux or Mac OS X." >&2
exit 1
;;
esac
GO_OS="${host_os}"
}
detect_os
if [ "${GO_OS}" == "darwin" ]; then
sed -i '' -e ...
else
sed -i -e ...
fi
sed -ie 's/old_link/new_link/g' *
Works on both BSD & Linux with gnu sed
With Sed, I want to use "!" to exclude lines that matches "he". Here is an example.
echo "hello" |sed "/he/!s/hello/hi/"
To my surprise, my ubuntu 14 returns
"bash !s/hello/hi: event not found"
error. Any ideas? How could I exclude line ranges corresponding to a pattern with Sed?
You have history expansion enabled. You need to disable it with set +H.
Example
Let's enable history expansion and run your command:
$ set -H
$ echo "hello" |sed "/he/!s/hello/hi/"
bash: !s/hello/hi/: event not found
Now, let's disable it and observe that the command now runs correctly and without error:
$ set +H
$ echo "hello" |sed "/he/!s/hello/hi/"
hello
Alternative
If you want to keep history expansion enabled, then single-quote your string:
$ set -H
$ echo "hello" |sed '/he/!s/hello/hi/'
hello
I am trying to us the perl -pi -e to edit the a line in a xml file. If I run the perl -pi -e command from the command line it works fine, but once I put in my script and use the system command I get the error listed below:
su: invalid option -- i
Try `su --help' for more information.
print "Please enter virtualhost 1 - Example - ucisha.com:";
my $virtualhost1 = <>;
system("ssh -t <HOST> \"sudo su - root -c perl -pi -e 's/xmlNamespaceAware=\"false\">/xmlNamespaceAware=\"false\"> <Alias>$virtualhost1<\/Alias>/g' /tcserver/springsource-tc-server-node/UCISjvm/conf/krh.xml\"");
You need to add quote marks around the argument you pass to -c.
At the moment you have -c perl and then -pi is taken as another argument to su not perl.