How can I store output string to a variable AND display in console - perl

I have a perl script that prints a message. This script is being called by GNU make. In my GNU make, I want to display the message printed out by the script AND store it in a variable also.
I'm doing it this way.
result=`$(PERL) parse.pl report.log` #parse the report
echo $(result) #echo the message here
ifneq ($(strip $$(result)),) #check if message is empty
#if not empty, search for filepath string pattern and exit
echo filepath
exit 1
endif
But it is not displaying the string message from parse.pl.

You are capturing into a shell variable, but then trying to echo a makefile variable (and even if you tried to echo the shell variable, that wouldn't work because make runs each line in a separate shell process).
Changing it to echo the shell varible and all to run in one shell should work:
foo:
result=`$(PERL) parse.pl report.log`; \
echo $$result
but whatever you later need to do to use the captured result would also need to be in the same shell execution.
Apparently you can capture into a makefile variable too, which may be more convenient:
foo:
$(eval result := $(shell $(PERL) parse.pl report.log))
echo $(result)

The critical thing to keep in mind with make is first, that the entire makefile is parsed before any rules are run, and second a makefile has two completely distinct syntaxes in it: makefile syntax for most of it, and shell syntax for the recipes. The shell syntax is run by the shell, not by make: make just starts a shell, hands over the recipe, and waits for the shell to exit to see if it worked or not.
As a result of this you CANNOT combine make constructs like ifeq with shell commands and their results: it cannot work because all the make constructs are parsed first, while the makefile is being read in, and the shell commands are not run until much later, when the target is to be built.
In your case you need to write the entire thing in shell syntax, because you want things to depend on the shell invocation.
So, like this:
foo:
result=`$(PERL) parse.pl report.log`; \
echo $$result; \
if [ "$$result" = "" ]; then \
echo filepath; \
exit 1; \
fi
Note how each line ends with a backslash, so it's appended to the previous line instead of being a separate line: make runs each separate line in a different shell.
Alternatively if you have a new-enough GNU make you can use the one shell feature:
.ONESHELL:
foo:
result=`$(PERL) parse.pl report.log`
echo $$result
if [ "$$result" = "" ]; then
echo filepath
exit 1
fi

Related

why doesn't make -C change $PWD as seen through a scripting language such as Perl?

Here is temp/Makefile:
all:
echo $$PWD
echo $(CURDIR)
perl -e 'print $$ENV{"PWD"}'
and now
$make -C temp
make: Entering directory `/home/mgaleck/temp'
/home/mgaleck/temp
/home/mgaleck/temp
/home/mgaleck
make: Leaving directory `/home/mgaleck/temp'
Why is the third value without temp?
According to Make manual, -C option causes to "change the directory" (working directory?) first.
Same thing happens with Python.
Because the PWD environment variable doesn't hold the current working directory; it holds whatever the current working directory was the last time sh set it. Anything other than a shell starting up, or a shell executing the cd builtin (or a similar builtin like pushd, in shells that have it), has no effect on PWD, and relying on PWD anywhere except in the shell is probably a silly idea. Use getcwd (C), Cwd::getcwd (Perl), os.getcwd (Python), etc. instead.

How can I make a function run every time cd successfully changes to another directory within sh on FreeBSD?

I'm using sh as my shell on FreeBSD but I want to be able to have a pretty prompt like the one bash gives me on Ubuntu. There are two things that the FreeBSD implementation of sh seems to lack as far as PS1 escape characters go:
The \w works but does not expand $HOME to ~, so this is something I have already hacked up myself
I can use PS1 to update the prompt on the terminal, but as far as I can tell it is not possible to use the PS1 variable to update the title bar as well. ESC and BEL fail to set the title as one would expect if they were using bash or ksh
Here is my .shrc file
update_prompt() {
case "$PWD" in
"$HOME"*)
pretty_pwd="~${PWD#*"${HOME}"}"
;;
"/usr$HOME"*)
pretty_pwd="~${PWD#*"/usr${HOME}"}"
;;
*)
pretty_pwd="$PWD"
;;
esac
case "$TERM" in
xterm*|rxvt*)
PS1="[$USER#\\h $pretty_pwd]\\$ "
;;
*)
;;
esac
printf "\\033]0;[%s#$(hostname -s): %s]\\007" "$USER" "$pretty_pwd"
}
update_prompt
So when I fire up a terminal or log in via ssh, it gives the pretty prompt that I like. But now I need this function to run every time that cd is executed and returns an exit status of 0.
I was going to use an alias that was something like:
alias cd="cd $1 && update_prompt"
but that was before I realized that aliases do not except arguments. How might I go about doing something like this?
You can use a function instead of an alias:
cd() {
command cd "$#" && update_prompt
}
Just put it into ~/.shrc. You have to use command here to let sh know that you are referring to the actual cd builtin command instead of the function you've just defined.
Refer to the sh(1) manual page for the details on how to make sh(1) source the ~/.shrc file when it starts:
Therefore, a user should place commands that are to be executed only at login
time in the .profile file, and commands that are executed for every shell
inside the ENV file. The user can set the ENV variable to some file by placing
the following line in the file .profile in the home directory, substituting for
.shrc the filename desired:
ENV=$HOME/.shrc; export ENV
I use this trick in my cd alias manager. Here's a link to the source code of the function: https://github.com/0mp/goat/blob/v2.5.0/libgoat.sh#L31-L57
You can do it with alias+arguments if you swap the commands:
$ alias cd="echo change; cd"
$ pwd
/nas
$ cd /
change
$ pwd
/
$ cd /etc
change
$ pwd
/etc
$

Syntax error in script

My script try execute mvn clean install in all projects but before it tried switch to dev branch and pull it.
successString="[INFO] BUILD SUCCESS";
file="mvnoutput";
red=$'\e[1;31m';
grn=$'\e[1;32m';
end=$'\e[0m';
function checkResult
if grep -Fxq "$successString" $file
then
echo -en "${grn}[${1}]Build ok${end}";
else
echo "${red}[${1}]Error in mvn clean install${end}";
exit 1;
fi;
end
function pullAndSwitchDevBranch
git checkout dev;
git pull origin dev;
end
cd api-pay-commons/;
pullAndSwitchDevBranch;
touch mvnoutput;
mvn clean install -U > mvnoutput;
checkResult PAY;
Why I received this error?
line 17: end: command not found ./script.sh: line 20: syntax
error near unexpected token git' ./script.sh: line 20: git checkout dev;'
You have used the "fish" tag, so I'm assuming you are running this with the fish shell.
If so: This is not a valid fish script. Fish is explicitly not POSIX-compatible, so you might want to read up on the syntax. (If not, then please correct your tag)
red=$'\e[1;31m';
Fish does not use var=value to set a variable. It uses set var value.
Fish also does not have the $'' style of quoting. Instead, backslash-escapes are interpreted outside of quotes.
So this would be set red \e"[1;31m". Alternatively, fish offers the set_color builtin, which prints the escape sequence for a given color. So this could be set red (set_color red) (or you could call set_color later).
then
Fish does not use if condition; then dosomething; fi. It's if condition; dosomething; end.
echo -en "${grn}[${1}]Build ok${end}";
Fish does not use "${var}". It also does not call the function arguments $1 et al.
This would be something like echo -ens "$grn" "[$argv[1]]" "Build ok" "$end".
exit 1
Fish currently does not allow exiting from functions. You'll have to return 1 and check the result from the outside.
Additionally, you're using semicolons all over the place, which is not typical style in any shell I'm aware of.
In case you are not using fish, but some POSIX-compatible shell (like bash), this is also not valid. The main (and possibly only) issue is that you are using function something; dosomething; end. In bash, that would be
function something () {
#dostuff
}
And in pure POSIX, that would be
something () {
#dostuff
}

How to set environment variables for a shell command

I often see this command in node.js programs: NODE_ENV=test node app.js which sets the NODE_ENV variable to test and works. I also read here https://en.wikipedia.org/wiki/Environment_variable that this should work for any shell command, but running some tests on my own, here is what I see
$ HELLO="WORLD"
$ HELLO="MARS" echo "$HELLO"
WORLD
$
I would expect this to print MARS. Is there something I am missing here?
The syntax VAR=value command means that the command will be invoked with the environment variable VAR set to VALUE, and this will apply only for the scope of that command.
However, when you are using the command line:
HELLO="MARS" echo "$HELLO"
The shell first interprets the "$HELLO" parameter, determines that it is WORLD, and then what it actually does is run:
HELLO="MARS" echo "WORLD"
So the echo may have the HELLO variable set, but it doesn't affect what it prints - it has already been interpreted before.
Doing
HELLO="MARS"; echo "$HELLO"
does something else entirely. First it sets HELLO to MARS in the current shell, and then it goes on to interpret the echo command. By this time HELLO contains MARS, not WORLD. But this is an entirely different effect - the variable HELLO stays with the value MARS, which is not the case in the command without the ;.
Your problem is that echo is just a poor choice for a demonstartion of this. You can do other demonstrations to prove that HELLO is changed properly:
HELLO="MARS" eval 'echo $HELLO'
In this case, the shell will not interpret the $HELLO because it is within a string in single quotes. It will first put MARS in HELLO, and then call the eval 'echo $HELLO' with that variable set. The eval command with then run echo $HELLO, and you'll get the output you were expecting.
This syntax is best used for things that don't use the given variable as part of the command line, but rather use it internally.
Other answers are correct, but here a refinement :
There are 2 cases in fact when defining a list of variable separated by spaces in bash whether it ends or not with a command.
VAR1=value1 VAR2=value2 ... VARn=valuen command arg1 arg2 ... argn
and
VAR1=value1 VAR2=value2 ... VARn=valuen
don't export VAR1 ... VARn the same way.
In first case VAR1 ... VARn will be set only for command and will then not be exported to current shell.
In second case VAR1 ... VARn will alter current shell.
then ( remark that ';' is very same of using a new line )
HELLO=WORLD
HELLO=MARS echo "i don't export HELLO."
echo "HELLO=$HELLO"
will display
i don't export HELLO.
HELLO=WORLD
and
HELLO=WORLD
HELLO=MARS ; echo "i did export HELLO."
echo "HELLO=$HELLO"
will display
i did export HELLO.
HELLO=MARS

bash script to build complex command syntax, print it first then execute - problems with variable expansion

I want to create scipt to faciliate producing local text file extracts from Hive.
This is to basically execute commands like below:
hive -e "SET hive.cli.print.header=true;SELECT * FROM dropme"|perl -pe 's/(?:\t|^)\KNULL(?=\t|$)//g'>extract/outbound/dropme.txt
While the above works like a charm I find it quite problematic to implement through the parametrized following script (much simplified):
#!/bin/sh
TNAME=dropme
SQL="SELECT * FROM $TNAME"
echo $SQL
echo "SQL: $SQL"
EXTRACMD="hive -e \"SET hive.cli.print.header=true;$SQL\"|perl -pe 'BEGIN{if(defined(\$_=<ARGV>)){s/\b\w+\.//g;print}}s/(?:\t|^)\KNULL(?=\t|$)//g'>extract/outbound/$TNAME.txt"
echo "CMD: $EXTRACMD";
${EXTRACMD}
When run I get: Exception in thread "main" java.lang.NumberFormatException: For input string: "e"
I know there may be many flavours you can print the text or execute command. For instance the line echo $SQL prints me list of files in the directory instead:
SELECT file1.txt file2.txt file3.txt file4.txt FROM dropme
while the next one: echo "SQL: $SQL" gives just what I want: SQL: SELECT * FROM dropme
echo "CMD: $EXTRACMD" prints the (almost) the command to be executed. Almost, as I see \t in perl code being expanded:
CMD: hive -e "SET hive.cli.print.header=true;SELECT * FROM dropme"|perl -pe 'BEGIN{if(defined($_=<ARGV>)){s\w+\.//g;print}}s/(?: |^)\KNULL(?= |$)//g'>extract/outbound/dropme.txt
Maybe that's still ok, but what I want is to be able to copy&paste this command into (other) terminal and execute as the command I put at the top. Ideally I would like that command to be exactly the same (so with \t there)
Biggest problem I have comes when I try to execute it (${EXTRACMD} line). I'm getting the error:
Exception in thread "main" java.lang.NumberFormatException: For input string: "e" …and so on, irrelevant as bash treats every 'word' as single command here. I assume as I don't even know what is really tries to run (prior print attempt obviously doesn't help)
I'm aware that I have multiple options, like:
escaping special characters in the command definition string (like I did with doublequotes)
experimenting with echo and $VAR, '$VAR' or "$VAR"
experimenting with "${EXTRACMD}" or evaluating through eval "${EXTRACMD}"
experimenting with shopt -s extglob or set -f
but as number of combinations is quite large and with my little bash experience I feel it's better to ask for good practice here so my question is:
Is there a way to print a (complex/compound shell) command first and subsequently be able to execute it (exactly as per printed output)? In this case it would be printing the exact command from the top, then executing it the same way as by manually copying that output into terminal prompt and pressing Enter.
Do not construct commands as strings. See http://mywiki.wooledge.org/BashFAQ/050 for details.
That page also talks about a built-in way of getting the shell to tell you what it is running (section 6).
If that doesn't do what you want you can also, with bash, try using printf %q\\n "${arr[*]}".