I'm running two Perl scripts in parallel in Jenkins
some shell commands
perl script 1 &
perl script 2 &
wait
some more shell commands
If one of the perl scripts fail in the middle of the execution , the job waits until the other script runs (as it is executed in parallel in background).
I want the job to stop as soon as one of the script fails and not waste time by completing the execution of other script.
Please help.
You set up a signal handler for SIGCHLD, which is a signal that is always delivered to the parent process when a child exits. I'm not aware of a mechanism to see which child process exited, but you can save the subprocess process identifiers and just kill both of them when you receive SIGCHLD:
some shell commands
perl script 1 &
pid1=$!
perl script 2 &
pid2=$!
trap "kill $pid1 $pid2" CHLD
wait
some more shell commands
The script above has the downside that it will kill the other script regardless of the exit status of the subprocess. You could in the trap, if you want to, add a check for the exit status. The subprocess could e.g. create some temp file if it succeeds and the trap could check if the file exists.
Typically with Jenkins you would have the parallel steps running as separate jobs (or projects as they are sometimes known) rather than steps in a job. This would then allow the steps to run in parallel across different slave machines and it would keep the output for the jobs in a separate place.
You would then have a controlling job running the other parts.
I like the Multijob plugin for this sort of thing.
There are alternatives which may suit better, such as Build Flow Plugin which uses a DSL to describe the jobs you want to run
I have an executable which can run perl scripts using the following command at the prompt:
blah.exe Launch.pl
The way we have our tests setup is that we call the Launch.pl from Parent.pl like this "blah.exe Launch.pl" - script within script. However, when executing the command with backticks/system command the parent .pl script execution waits till I get the handle back by closing and exiting out of the application (blah.exe). At this point the code in parent.pl continues to execute.
How do I return the handle back to the parent .pl script after I get done running the code that is contained in the Launch.pl
So, parent.pl calls "blah.exe Launch.pl"; but after running the code inside Launch.pl inside the application (blah.exe) it just sits there waiting to be exited out of so that the code in parent.pl can continue running. I need to keep the application (blah.exe) open till I am done running a bunch of scripts one after another.
Run blah.exe in the background. When you are done with the Parent.pl, terminate the application with kill.
Let's say I have 5 batch files that run sequentially one after another (executed via the Windows task scheduler on a normal Windows XP PC):
Script1.bat
Script2.bat
Script3.bat
Script4.bat
Script5.bat
Suppose one of the scripts fail (an error condition is detected -- details on how this happens is not important for my question here). How do I stop the other scripts from running if they all run within the task scheduler? For example, if Script1.bat fails, I don't want to run Script2-5.bat. If Script3.bat fails, I don't want to run Script4-5.bat, etc.
I thought about writing a flag value to a temporary file that each script would read from. At the beginning of each script (except for the first one), it will check to see if the flag is valid. The first script would clear out this flag at the beginning each time these set of batch files run.
Surely there is a better way to do this or maybe there is a standard for how to handle this type of situation? Thanks!
Write a master.bat file that conditionally calls each of the scripts in sequence. Then schedule the master instead of directly scheduling the 5 scripts.
#echo off
call Script1.bat
if %errorlevel%==0 call Script2.bat
if %errorlevel%==0 call Script3.bat
if %errorlevel%==0 call Script4.bat
if %errorlevel%==0 call Script5.bat
I am writing a Perl script and I need to execute Unix Ctrl+Z on the script.
How can I do it in Perl ?
thanks.
From perl you can send signals to processes with the function kill, which has the same name as the Unix command line tool that does the same thing. The equivalent to Ctrl+Z is running
kill -SIGTSTP pid
you need to find out what numeric value your TSTP signal has on your system. You would do this by running
kill -l TSTP
on the command line. Let's say this returns 20
Then in your Perl script you would add
kill 20 => $$;
which will send the TSTP signal to the currently running process id ($$)
Update:
as described by daxim, you can skip the 'kill -l' part and provide the name of the signal directly:
kill 'TSTP' => $$;
In bash ctrl+z stops the current job and puts it in background with %JobId you can return to this job. I'm not sure what you mean since I thought ctrl+z is caught by bash..
I've searched around but haven't quite found what I'm looking for. In a nutshell I have created a bash script to run in a infinite while loop, sleeping and checking if a process is running. The only problem is even if the process is running, it says it is not and opens another instance.
I know I should check by process name and not process id, since another process could jump in and take the id. However all perl programs are named Perl5.10.0 on my system, and I intend on having multiple instances of the same perl program open.
The following "if" always returns false, what am I doing wrong here???
while true; do
if [ ps -p $pid ]; then
echo "Program running fine"
sleep 10
else
echo "Program being restarted\n"
perl program_name.pl &
sleep 5
read -r pid < "${filename}_pid.txt"
fi
done
Get rid of the square brackets. It should be:
if ps -p $pid; then
The square brackets are syntactic sugar for the test command. This is an entirely different beast and does not invoke ps at all:
if test ps -p $pid; then
In fact that yields "-bash: [: -p: binary operator expected" when I run it.
Aside from the syntax error already pointed out, this is a lousy way to ensure that a process stays alive.
First, you should find out why your program is dying in the first place; this script doesn't fix a bug, it tries to hide one.
Secondly, if it is so important that a program remain running, why do you expect your (at least once already) buggy shell script will do the job? Use a system facility that is specifically designed to restart server processes. If you say what platform you are using and the nature of your server process. I can offer more concrete advice.
added in response to comment:
Sure, there are engineering exigencies, but as the OP noted in the OP, there is still a bug in this attempt at a solution:
I know I should check by process name
and not process id, since another
process could jump in and take the id.
So now you are left with a PID tracking script, not a process "nanny". Although the chances are small, the script as it now stands has a ten second window in which
the "monitored" process fails
I start up my week long emacs process which grabs the same PID
the nanny script continues on blissfully unaware that its dependent has failed
The script isn't merely buggy, it is invalid because it presumes that PIDs are stable identifiers of a process. There are ways that this could be better handled even at the shell script level. The simplest is to never detach the execution of perl from the script since the script is doing nothing other than watching the subprocess. For example:
while true ; do
if perl program_name.pl ; then
echo "program_name terminated normally, restarting"
else
echo "oops program_name died again, restarting"
fi
done
Which is not only shorter and simpler, but it actually blocks for the condition that you are really interested in: the run-state of the perl program. The original script repeatedly checks a bad proxy indication of the run state condition (the PID) and so can get it wrong. And, since the whole purpose of this nanny script is to handle faults, it would be bad if it were faulty itself by design.
I totally agree that fiddling with the PID is nearly always a bad idea. The while true ; do ... done script is quite good, however for production systems there a couple of process supervisors which do exactly this and much more, e.g.
enable you to send signals to the supervised process (without knowing it's PID)
check how long a service has been up or down
capturing its output and write it to a log file
Examples of such process supervisors are daemontools or runit. For a more elaborate discussion and examples see Init scripts considered harmful. Don't be disturbed by the title: Traditional init scripts suffer from exactly the same problem like you do (they start a daemon, keep it's PID in a file and then leave the daemon alone).
I agree that you should find out why your program is dying in the first place. However, an ever running shell script is probably not a good idea. What if this supervising shell script dies? (And yes, get rid of the square braces around ps -p $pid. You want the exit status of ps -p $pid command. The square brackets are a replacement for the test command.)
There are two possible solutions:
Use cron to run your "supervising" shell script to see if the process you're supervising is still running, and if it isn't, restart it. The supervised process can output it's PID into a file. Your supervising program can then cat this file and get the PID to check.
If the program you're supervising is providing a service upon a particular port, make it an inetd service. This way, it isn't running at all until there is a request upon that port. If you set it up correctly, it will terminate when not needed and restart when needed. Takes less resources and the OS will handle everything for you.
That's what kill -0 $pid is for. It returns success if a process with pid $pid exists.