setup new database in ubuntu using a script [duplicate] - postgresql

I have a script where I need to start a command, then pass some additional commands as commands to that command. I tried
su
echo I should be root now:
who am I
exit
echo done.
... but it doesn't work: The su succeeds, but then the command prompt is just staring at me. If I type exit at the prompt, the echo and who am i etc start executing! And the echo done. doesn't get executed at all.
Similarly, I need for this to work over ssh:
ssh remotehost
# this should run under my account on remotehost
su
## this should run as root on remotehost
whoami
exit
## back
exit
# back
How do I solve this?
I am looking for answers which solve this in a general fashion, and which are not specific to su or ssh in particular. The intent is for this question to become a canonical for this particular pattern.

Adding to tripleee's answer:
It is important to remember that the section of the script formatted as a here-document for another shell is executed in a different shell with its own environment (and maybe even on a different machine).
If that block of your script contains parameter expansion, command substitution, and/or arithmetic expansion, then you must use the here-document facility of the shell slightly differently, depending on where you want those expansions to be performed.
1. All expansions must be performed within the scope of the parent shell.
Then the delimiter of the here document must be unquoted.
command <<DELIMITER
...
DELIMITER
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<END
a=1
mylogin=$(whoami)
echo a=$a
echo mylogin=$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=0
mylogin=leon
a=0
mylogin=leon
2. All expansions must be performed within the scope of the child shell.
Then the delimiter of the here document must be quoted.
command <<'DELIMITER'
...
DELIMITER
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<'END'
a=1
mylogin=$(whoami)
echo a=$a
echo mylogin=$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=1
mylogin=root
a=0
mylogin=leon
3. Some expansions must be performed in the child shell, some - in the parent.
Then the delimiter of the here document must be unquoted and you must escape those expansion expressions that must be performed in the child shell.
Example:
#!/bin/bash
a=0
mylogin=$(whoami)
sudo sh <<END
a=1
mylogin=\$(whoami)
echo a=$a
echo mylogin=\$mylogin
END
echo a=$a
echo mylogin=$mylogin
Output:
a=0
mylogin=root
a=0
mylogin=leon

A shell script is a sequence of commands. The shell will read the script file, and execute those commands one after the other.
In the usual case, there are no surprises here; but a frequent beginner error is assuming that some commands will take over from the shell, and start executing the following commands in the script file instead of the shell which is currently running this script. But that's not how it works.
Basically, scripts work exactly like interactive commands, but how exactly they work needs to be properly understood. Interactively, the shell reads a command (from standard input), runs that command (with input from standard input), and when it's done, it reads another command (from standard input).
Now, when executing a script, standard input is still the terminal (unless you used a redirection) but the commands are read from the script file, not from standard input. (The opposite would be very cumbersome indeed - any read would consume the next line of the script, cat would slurp all the rest of the script, and there would be no way to interact with it!) The script file only contains commands for the shell instance which executes it (though you can of course still use a here document etc to embed inputs as command arguments).
In other words, these "misunderstood" commands (su, ssh, sh, sudo, bash etc) when run alone (without arguments) will start an interactive shell, and in an interactive session, that's obviously fine; but when run from a script, that's very often not what you want.
All of these commands have ways to accept commands by ways other than in an interactive terminal session. Typically, each command supports a way to pass it commands as options or arguments:
su root -c 'who am i'
ssh user#remote uname -a
sh -c 'who am i; echo success'
Many of these commands will also accept commands on standard input:
printf 'uname -a; who am i; uptime' | su
printf 'uname -a; who am i; uptime' | ssh user#remote
printf 'uname -a; who am i; uptime' | sh
which also conveniently allows you to use here documents:
ssh user#remote <<'____HERE'
uname -a
who am i
uptime
____HERE
sh <<'____HERE'
uname -a
who am i
uptime
____HERE
For commands which accept a single command argument, that command can be sh or bash with multiple commands:
sudo sh -c 'uname -a; who am i; uptime'
As an aside, you generally don't need an explicit exit because the command will terminate anyway when it has executed the script (sequence of commands) you passed in for execution.

If you want a generic solution which will work for any kind of program, you can use the expect command.
Extract from the manual page:
Expect is a program that "talks" to other interactive programs according to a script. Following the script, Expect knows what can be expected from a program and what the correct response should be. An interpreted language provides branching and high-level control structures to direct the dialogue. In addition, the user can take control and interact directly when desired, afterward returning control to the script.
Here is a working example using expect:
set timeout 60
spawn sudo su -
expect "*?assword" { send "*secretpassword*\r" }
send_user "I should be root now:"
expect "#" { send "whoami\r" }
expect "#" { send "exit\r" }
send_user "Done.\n"
exit
The script can then be launched with a simple command:
$ expect -f custom.script
You can view a full example in the following page: http://www.journaldev.com/1405/expect-script-example-for-ssh-and-su-login-and-running-commands
Note: The answer proposed by #tripleee would only work if standard input could be read once at the start of the command, or if a tty had been allocated, and won't work for any interactive program.
Example of errors if you use a pipe
echo "su whoami" |ssh remotehost
--> su: must be run from a terminal
echo "sudo whoami" |ssh remotehost
--> sudo: no tty present and no askpass program specified
In SSH, you might force a TTY allocation with multiple -t parameters, but when sudo will ask for the password, it will fail.
Without the use of a program like expect any call to a function/program which might get information from stdin will make the next command fail:
ssh use#host <<'____HERE'
echo "Enter your name:"
read name
echo "ok."
____HERE
--> The `echo "ok."` string will be passed to the "read" command

Related

Can you get full command line from process ID (including command line arguments, etc)?

This question is in addition to the question asked here: https://unix.stackexchange.com/questions/163145/how-to-get-whole-command-line-from-a-process. On my system, the following command results in a PID (as expected):
CUDA_VISIBLE_DEVICES=4,5 python3 main.py 1> out.txt 2> err.txt &
Now, the methods in the stack exchange link above provide many solutions. However, when trying these solutions, I only receive the following information:
python3 main.py
Is there a way to return the entire command line "CUDA_VISIBLE_DEVICES=4,5 python3 main.py 1> out.txt 2> err.txt &", not just the portion "python3 main.py"?
No.
Assuming you're on a Linux system, you can find the individual bits, but you can't put it together.
Assume also that the process's PID is in $pid
The CUDA_VISIBLE_DEVICES=4,5 variable gets added to the environment of the python command. You can find it in /proc/$pid/environ but you can't tell which of those variables were specified on the command line: the user could have written
export CUDA_VISIBLE_DEVICES=4,5
python3 main.py 1> out.txt 2> err.txt &
The file redirections are available in /proc/$pid/fd:
/proc/$pid/fd/1 is a symbolic link to out.txt
/proc/$pid/fd/2 is a symbolic link to err.txt
I don't know how to tell if a process is running in the background.
Since you're just interested in the environment: with bash
declare -A environ
while IFS='=' read -r -d '' var value; do
environ["$var"]="$value"
done < /proc/$pid/environ
echo "process has CUDA_VISIBLE_DEVICE value ${environ[CUDA_VISIBLE_DEVICE]}"

Multiple mongo commands in kubernetes not working

I am attempting to automate the following series of commands which work correctly into a BASH script:
kubectl exec -it mongo-pod -- bash
mongo DBNAME
db.auth("theUser", "thePw")
db.theCollection.find()
The script I am using is as follows:
#!/bin/bash
kubectl exec -it mongo-pod -- bash -c "mongo DBNAME && /
db.auth("theUser", "thePw") && /
db.theCollection.find()"
I have tried the following:
Executing multiple commands( or from a shell script) in a kubernetes pod
but any commands that are added after the first using & or && are not executed. For example just using "mongo DBNAME" correctly opens the prompt and sets it to the correct db, but adding any other command with && causes all commands to fail with the following:
bash: -c line 0: syntax error near unexpected token 'theUser'
All of the comments are spot on, but at least two things I have the highest confidence I have the answer to:
First, you have the line continuation character wrong; it should be \ and not /. It actually wouldn't even be required if you switched bash into "exit on error" mode, with
kubectl exec -it mongo-pod -- bash -ec "mongo DBNAME
echo 'this command only runs if mongo exits a-ok'
exit 1
and this never will run
"
However, the other mistake is around the quoting characters used: if you have bash -c " then you must either use the single-quote for the interior string literals, or escape them with \". You can actually see what I'm talking about by looking at the syntax highlighting of the shell snippet in your question. Observe that the string literal is red, but then the text theUser as well as thePw are both black -- that's because they are outside the string literal since the string stopped at the first " it encountered -- the one present in db.auth("
It is almost always the case that you'll want to use single quotes when invoking bash remotely like that, for several reasons but the most relevant is that you can then use db.auth("something") without having to unnecessarily escape the double quotes.
Since mongo (like many interpreters such a node and python) wants you to either type in it interactively, provide the input on its "standard input", or give it a local file containing commands, you will want to change the invocation to one of those strategies depending on your needs.
A very convenient way of redirecting standard input without having to use echo or printf and its associated quoting hell is to use what are called "here documents" (abbreviated "heredocs") in bash:
kubectl exec -it mongo-pod -- bash -ec 'mongo DBNAME<<"FOO"
db.auth("theUser", "thePw")
printjson(db.theCollection.find())
FOO
'
That causes bash to transmit almost all characters between the two "heredoc delimiters" to the standard input of the command. If you quote the delimiter, as I have with the [arbitrary] word FOO, then the contents are not subject to variable expansion, command interpolation, etc, which can be one more mechanism to avoid backtick and dollarsign weirdness.

same aliases and functions for user and root in fish shell

How can I make aliases defined in ~/.config/fish/config.fish and functions defined in ~/.config/fish/functions available for the root user as well?
symlinking to /root/.config/fish didn't do the job. Furthermore, I'd also like to be able to use my functions and aliases via sudo, which is currently not working as well.
How do you guys do that?
I don't have a solution for the first problem (I don't have any functions that are useful for the root user), but the second one (functions not working as an argument for sudo) can be solved with a wrapper function:
function sudo
if functions -q $argv[1]
set argv fish -c "$argv"
end
command sudo $argv
end
The issue is of course that sudo is an external command (usually at /usr/bin/sudo) that cannot know about fish internals like function definitions. So this wrapper will call fish if you give it a function name. Of course, since this launches a new fish session, some things, like cd will not be useful because it won't alter the state of the main fish session.
I use a symlink to my config.fish and works perfectly. Have you checked the permissions?
777 lrwxrwxrwx 1 root root 35 Dez 21 2015 /root/.config/fish/config.fish -> /home/lara/.config/fish/config.fish
In newer versions you can also use the directory /etc/fish/ for system-wide configuration
The problem with sudo I solved with an alias:
# -E just preserv env. It's not really
# necessary, but may be useful.
function sudo; command sudo -sE $argv; end
from sudo man page:
-s, --shell
Run the shell specified by the SHELL environment variable if it is set or the shell specified by the invoking user's
password database entry. If a command is specified, it is passed to
the shell for execution via
the shell's -c option. If no command is specified, an interactive shell is executed.
Old question that I came across while attempting to do the same thing myself.
Neither existing solution from #faho or #LaraMaia solves the first part of the question, which is really what I needed for my use-case. Both solutions only handled functions that are defined in ~/.config/fish/functions. In-memory functions and aliases that have not been "saved" do not work.
So here's a version that does work for aliases/in-memory functions:
function sudo
if functions -q -- "$argv[1]"
set cmdline (
for arg in $argv
printf "\"%s\" " $arg
end
)
set -x function_src (string join "\n" (string escape --style=var (functions "$argv[1]")))
set argv fish -c 'string unescape --style=var (string split "\n" $function_src) | source; '$cmdline
command sudo -E $argv
else
command sudo $argv
end
end
Commented and slightly expanded version at this Gist.

sh: variable substitution with heredoc

cat "${pos}" | /usr/bin/iconv -f CP1251 -t UTF-8 | uniq | sed -En "/^CLIENT_ID.*/!p" | while read line
do
.....
......
cat >> "$TMPFILE" << EOF
INSERT INTO ......;
EOF
done
As you can see each iteration writes a SQL statement to a tmp-file.
I launched this script from a regular interactive shell and got the expected output. Launched from a cron job - nothing.
After investigating I found a problem. When I use "$TMPFILE" without "" the script works ok. Why does this happen?
OS: FreeBSD, bourne shell.
IIRC, cron doesn't source all the files that a login shell does, so you will end up with different settings for environment variables. Could be the path $TMPFILE is pointing to contains spaces when run from cron for example.
Also, on some systems (depending on setup), cron uses a different shell. So if you start your script from command line, for example /usr/bin/sh might be used, whereas when started by cron, /bin/sh is used. (I have no experience with *BSD, but I have observed this on linux.)

How to execute many (~15) bash commands in Perl?

I know of system() and qx(), but I need to execute ~15 bash commands. E.g.
mkdir, chown, edquota -p user1 -u user2, cp -r, su - username, git, rm, ln -s
Question
Is there an efficient way to execute many Bash commands in Perl?
I don't care in this case about the output.
First, I'd use the equivalent Perl function for as many of those bash command as I could, which is most of the ones you included in your post. Then, for the rest of them I'd either use system() or qx() or backticks or one of the IPC:: modules (such as IPC::Run or IPC::Open3).
Use bash syntax for many commands. Separate them with ; or && or whatever takes your fancy (man bash).
$ perl -E 'system qq{date; date}'
In Linux, I like POE framework's POE::Wheel::Run module for running system commands (and code blocks) asynchronously. You say you do not care about the output, but if you need it in the future POE::Wheel::Run has an elegant interface allowing us to interact with the process.
my $s = <<END;
echo "1"
echo "2"
echo "3"
END
system("$s");