I have written a shebang R script and would like to execute it from a Perl script. I currently use system ($my_r_script_path, $r_script_arg1, $r_script_arg2, ...) and my question is how can I verify the R script terminates normally (no errors or warnings).
guess I should make my R script return some true value at the end, only if everything is OK, then catch this value in Perl, but I'm not sure how to do that.
Thanks!
You can set the return value in the command quit(), eg q(status=1). Default is 0, see also ?quit. How to catch that one in Perl, is like catching any other returning value in Perl. It is saved in a special variable $? if I remember right. See also the examples in the perldoc for system, it should be illustrated there.
On a sidenote, I'd just use the R-Perl interface. You can find info and examples here :
http://www.omegahat.org/RSPerl/
Just for completeness :
At the beginning of your script, you can put something like :
options(
warn=2, # This will change all warnings into errors,
# so warnings will also be handled like errors
error= quote({
sink(file="error.txt"); # save the error message in a file
dump.frames();
print(attr(last.dump,"error.message"));
sink();
q("no",status=1,FALSE) # standard way for R to end after errors
})
)
This will save the error message, and break out of the R session without saving, with exit code 1 and without running the .Last.
Still, the R-Perl interface offers a lot more possibilities that are worth checking out if you're going to do this more often.
Related
I have a perl script for which I would like to write a few tests. If the perl script gets supplied a specific option, the user is allowed to paste or type out multi-line input and end their input using control-d.
These are the tests I want to write:
The script (when an interactive flag is supplied) waits for (multiple lines of) input on STDIN until the user hits control-d
[this test is already implemented - but provided for completeness] The script (when a flag is supplied that indicates a redirect/pipe) consumes all input on STDIN and does not wait for control-d
The script (when no input flag is provided [interactive or redirect]) does not wait for interactive user input
Test 1 toy example
I wrote a test for test 1 that confirms input was received (it works by the parent test script printing to the child's input handle and the child modifies and prints that input back out), but that test doesn't wait for an end of input signal (e.g. control-d) (which I don't know how to send anyway). So in other words, I can confirm it receives input, but I don't know how to confirm that it waits for the user to intentionally end the input entry. I.e. How do I know that the input consumption won't stop until the user types control-d?
Here's what I have so far for test 1. I wrote a 3rd little IO::Pipe::Consumer module to be able to send input to the child process that I'm testing and I wrote a toy example of the script that allows input on STDIN from a tty.
Here is a toy version of the script I'm testing:
>perl -e 'while(<STDIN>){print("*$_")}'
test
*test
^d
>
And here is the toy test code (that I want to improve) for the above script:
>perl -e '
use IO::Pipe::Consumer;
$obj = new IO::Pipe::Consumer;
$si = $obj->getSubroutineConsumer(
sub { while(<STDIN>) print("*$_") } }
);
print $si "test\n"
'
*test
>
I thought the parent would have to print an EOF (e.g. like what you get from "control-d") to end the input in the test, but the test ends immediately even though I'm not sending any such end-of-input character. I can see that it's getting, modifying, and printing the input. Is that sufficient to confirm that the script will wait for user input (and that the user will be able to intentionally end the input) or is there something else I should do to confirm it waits for all user input until the user intends to end it?
Test 2 - done
Test 3 toy - don't know how to write it...
Even if modified input spit back out is sufficient proof of "waiting for input" for test 1, I also wish to test that a script doesn't consume input on STDIN when no input option (interactive or redirect) is provided - but since it doesn't seem to wait even when I do send it input without an end-of-input signal, how would I test that the script wouldn't hang waiting for input? Note, the script has other options for consuming redirected or piped input, so my intent is specifically to know if it's waiting on input from the tty. All of the STDIN consumption options (whether from the tty or via redirect/pipe) are optional, which is why I want to write these tests.
My manual testing shows everything works as intended. I would just like some tests to assure that behavior for the future.
IO::Pipe::Consumer
I feel like the thing I'm missing is not relevant to IO::Pipe::Consumer, so WRT that, I'll just describe it instead of paste in 30 or so lines of code... All it does is it sets a pipe to the child's STDIN and gives that handle back to the parent for it to print to. I haven't put it on cpan. I'm just experimenting to see if I can use it to write these tests.
IO::Pipe::Consumer is basically the opposite of IO::Pipe::Producer (a module I published on cpan looong ago, say 2001-ish, when I was new to perl, or programming for that matter). The main difference, aside from swapping STDIN for STDOUT and Reader with Writer (and vice versa), is that the open is open(STDIN,"<",\${$stdin_pipe}).
I thought the parent would have to print an "end-of-input" (e.g. "control-d") character to end the input in the test,
Ctrl-D doesn't produce an "end of input character"; it causes the terminal to return EOF.
I don't know what IO::Pipe::Consumer is —it's not on CPAN— but I presume it creates a pipe. Exiting the program causes the writer end of the pipe to be closed and thus cause the reader end to return EOF.
is there something else I should do to confirm it waits for all user input until the user intends to end it?
<> reads until one of the following things happen:
A line feed is encountered (returning what was read including the line feed)
EOF is encountered (returning what was read up to an including the line feed)
An error is encountered (returning false).
You can confirm it waits by putting sleep statements between what you send. Be aware that buffering may interfere.
I am trying to identify misspelled words with Aspell via Perl. I am working on a Linux server without administrator privileges which means I have access to Perl and Aspell but not, for example, Text::Aspell which is a Perl interface for Aspell.
I want to do the very simple task of passing a list of words to Aspell and having it return the words that are misspelled. If the words I want to check are "dad word lkjlkjlkj" I can do this through the command line with the following commands:
aspell list
dad word lkjlkjlkj
Aspell requires CTRL + D at the end to submit the word list. It would then return "lkjlkjlkj", as this isn't in the dictionary.
In order to do the exact same thing, but submitted via Perl (because I need to do this for thousands of documents) I have tried:
my $list = q(dad word lkjlkjlkj):
my #arguments = ("aspell list", $list, "^D");
my $aspell_out=`#arguments`;
print "Aspell output = $aspell_out\n";
The expected output is "Aspell output = lkjlkjlkj" because this is the output that Aspell gives when you submit these commands via the command line. However, the actual output is just "Aspell output = ". That is, Perl does not capture any output from Aspell. No errors are thrown.
I am not an expert programmer, but I thought this would be a fairly simple task. I've tried various iterations of this code and nothing works. I did some digging and I'm concerned that perhaps because Aspell is interactive, I need to use something like Expect, but I cannot figure out how to use it. Nor am I sure that it is actually the solution to my problem. I also think ^D should be an appropriate replacement for CTRL+D at the end of the commands, but all I know is it doesn't throw an error. I also tried \cd instead. Whatever it is, there is obviously an issue in either submitting the command or capturing the output.
The complication with using aspell out of a program is that it is an interactive and command-line driver tool, as you suspect. However, there is a simple way to do what you need.
In order to use aspell's command list one needs to pass it words via STDIN, as its man page says. While I find the GNU Aspell manual a little difficult to get going with, passing input to a program via its STDIN is easy enough and we can rewrite the invocation as
echo dad word lkj | aspell list
We get lkj printed back, as due. Now this can run out of a program just as it stands
my $word_list = q(word lkj good asdf);
my $cmd = qq(echo $word_list | aspell list);
my #aspell_out = qx($cmd);
print for #aspell_out;
This prints lines lkj and asdf.
I assemble the command in a string (as opposed to an array) for specific reasons, explained below. The qx is the operator form of backticks, which I prefer for its far superior readability.
Note that qx can return all output in a string, if in scalar context (assigned to a scalar for example), or in a list when in list context. Here I assign to an array so you get each word as an element (alas, each also comes with a newline, so may want to do chomp #aspell_out;).
Comment on a list vs string form of a command
I think that it's safe to recommend to use a list-form for a command, in general. So we'd say
my #cmd = ('ls', '-l', $dir); # to be run as an external command
instead of
my $cmd = "ls -l $dir"; # to be run as an external command
The list form generally makes it easier to manage the command, and it avoids the shell altogether.
However, this case is a little different
The qx operator doesn't really behave differently -- the array gets concatenated into a string, and that runs. The very fact that we can pass it an array is incidental, and not even documented
We need to pipe input to aspell's STDIN, and shell does that for us simply. We can use a shell with command's LIST form as well, but then we'd need to invoke it explicitly. We can also go for aspell's STDIN by means other than the shell but that's more complex
With a command in a list the command name must be the first word, so that "aspell list" from the question is wrong and it should fail (there is no command named that) ... except that in this case it wouldn't (if the rest were correct), since for qx the array gets collapsed into a string
Finally, apsell nicely exposes its API in a C library and that's been utilized for the module you mention. I'd suggest to install it as a user (no privileges needed) and use that.
You should take a step back and investigate if you can install Text::Aspell without administrator privilige. In most cases that's perfectly possible.
You can install modules into your home directory. If there is no C-compiler available on the server you can install the module on a compatible machine, compile and copy the files.
I'm trying to suppress output from a code section in a script (namely the network initialization from a Caffe network). I've tried wrapping the corresponding bit of code in an evalc command
[suppressed_output, var_output] = evalc('someFunction(input)');
But this doesn't work. I've still got loads of lines of (non-error) output from the network initialization that are clogging my logs (amidst all the wanted output printed via fprintf('') in the script). I think this happens because the corresponding function is writing to STDERR (instead of STDOUT?) - the first line it prints is this warning:
WARNING: Logging before InitGoogleLogging() is written to STDERR
... and then hundreds of lines of what it is doing follow, e.g.:
I0215 15:01:51.840272 28620 upgrade_proto.cpp:66] Attempting to upgrade input file specified using deprecated input fields: tmp-def.prototxt
I0215 15:01:51.840360 28620 upgrade_proto.cpp:69] Successfully upgraded file specified using deprecated input fields.
...
Can I somehow suppress the output to STDERR (without messing with the function content)? Ideally only locally for this specific function, since I'd still like to get potential error messages.
In case it is relevant:
I call myScript via matlab command line and its output written to a log (mlexec.log) with tee:
matlab -nodesktop -nosplash -display :1 -r "try, myScript; catch e, disp(getReport(e)), end, quit force" 2>&1| tee mlexec.log
The problem here is, that in the matlab command line call, the output from STDERR is streamed to STDOUT by this "command": 2>&1. Since the .cpp file seems to stream its output to STDERR (according to the Warning), it will be forwarded to STDOUT and eventually the log.
Streaming STDERR (2) to Nirvana with 2>NUL or a different log file (e.g. 2>mlexec.stderr.log) solves the problem.
I wanted to post this in a comment but it said I had to have 50 reputation (I have 49 now...)
I think this is what you're looking for
EDIT/UPDATE:
One thing you can do is enclose a section of your code with warning on/off statements as follows:
warning('off','all')
%your code here
warning('on','all')
This should stop any warnings being output to stderr from that section. I personally do not recommend this, it's good to know what you're doing that the MATLAB runtime does not like.
Using fish shell, I'm writing very simple script that checks the command execution
#!/usr/bin/fish
command
if $status
echo "Oops error"
else
echo "Worked OK"
#...
end
And get the error message:
fish: Variables may not be used as commands. Instead, define a function like “function status; 0 $argv; end”. See the help section for the function command by typing “help function”.
The message looks pretty straight forward but no "defining function like..." nor "help function" helps solving the problem.
There is also a 'test' command, that sounds promising. But docs say it is to be used to check files...
How this simple thing should be done with fish shell?
Heh... And why all documentation is SO misleading?..
P.S. Please, don't write about 'and' command.
Fish's test command currently works exactly like POSIX test (i.e. the one you'll find in bash or similar shells). It has a couple of operations, including "-gt", "-eq", "-lt" to check if a number is bigger, equal or less than another number, respectively.
So if you want to use test, you'll do if test $status -eq 0 (a 0 traditionally denotes success). Otherwise, you can check the return value of a command by putting it in the if clause directly like if command (which will be true if the command returns 0) - that's what fish is trying to do here, which is why it complains about a variable being used in place of a command.
I'm fairly new to Perl and have been searching the interwebs for documentation for what I'm trying to do. I'm not having any luck.
I have a program that outputs information to stdout with prompts throughout. I need to make a Perl script to pipe that information to a file.
I thought I could use Expect but there seems to be a problem with the pipe after the first prompt.
Here is the part of my code:
# Run program and compare the output to the BASE file
$cmd = "./program arg1 arg2 arg3 arg4 > $outfile";
my $exp = new Expect;
$exp->spawn($cmd);
BAIL_OUT("couldn't create expect object") if (! defined $exp);
$exp->expect(2);
$exp->send("\n");
For this case there is only a single prompt for the user to press "enter". This program is small and very fast - 2 seconds is plenty of time to reach the first prompt.
The output file only contains the first half of the information.
Does anyone have any suggestions on how I can grab the second half as well?
UPDATE:
I've verified that this works with Expect by using a simple script:
spawn ./program arg1 arg2 arg3 arg4
expect "<Return>"
send "\r"
interact
Where "< Return >" is a verbose expression that the Perl script could look for.
Note: I've tried writing my Perl script to expect "< Return >"...it makes no difference.
i.e.
$exp->expect(2, '-re', "<Return>")
Any thoughts?
UPDATE2:
Hazaah! I've found a solution to my problem...completely by accident.
So, I had a mistype in some test code I made...
$exp->expect(2);
$exp->send("\r");
$exp->expect(2);
Note the trailing expect(2)...I accidentally left that in and it worked!
So, I'm trying to understand what is happening. Unix expect does not seem work this way! It appears Expect implemented in Perl "expects" anything...not just prompts?
So, I provided expect another 2 seconds to collect stdout and I am able to get everything.
If anyone can offer some more detailed information as to what is going on here I'd love to understand what is going on.
Try sending \r instead of \n - you're trying to emulate a carriage return, not a newline, and the tty settings might not be translating them.
ALSO:
A suggestion from the Expect docs FAQ section, which seems likely given your accidental solution:
My script fails from time to time without any obvious reason. It
seems that I am sometimes loosing output from the spawned program.
You could be exiting too fast without giving the spawned program
enough time to finish. Try adding $exp->soft_close() to terminate the
program gracefully or do an expect() for 'eof'.
Alternatively, try adding a 'sleep 1' after you spawn() the program.
It could be that pty creation on your system is just slow (but this is
rather improbable if you are using the latest IO-Tty).
Standard unix/tcl expect doesn't exit in interactive mode, which could give your program enough time to finish running.
It's been a while since I've used Expect, but I'm pretty sure you need to provide something for Expect to match the prompt against:
$exp->expect( 2, 'Press enter' );
for example.