I'm trying to suppress output from a code section in a script (namely the network initialization from a Caffe network). I've tried wrapping the corresponding bit of code in an evalc command
[suppressed_output, var_output] = evalc('someFunction(input)');
But this doesn't work. I've still got loads of lines of (non-error) output from the network initialization that are clogging my logs (amidst all the wanted output printed via fprintf('') in the script). I think this happens because the corresponding function is writing to STDERR (instead of STDOUT?) - the first line it prints is this warning:
WARNING: Logging before InitGoogleLogging() is written to STDERR
... and then hundreds of lines of what it is doing follow, e.g.:
I0215 15:01:51.840272 28620 upgrade_proto.cpp:66] Attempting to upgrade input file specified using deprecated input fields: tmp-def.prototxt
I0215 15:01:51.840360 28620 upgrade_proto.cpp:69] Successfully upgraded file specified using deprecated input fields.
...
Can I somehow suppress the output to STDERR (without messing with the function content)? Ideally only locally for this specific function, since I'd still like to get potential error messages.
In case it is relevant:
I call myScript via matlab command line and its output written to a log (mlexec.log) with tee:
matlab -nodesktop -nosplash -display :1 -r "try, myScript; catch e, disp(getReport(e)), end, quit force" 2>&1| tee mlexec.log
The problem here is, that in the matlab command line call, the output from STDERR is streamed to STDOUT by this "command": 2>&1. Since the .cpp file seems to stream its output to STDERR (according to the Warning), it will be forwarded to STDOUT and eventually the log.
Streaming STDERR (2) to Nirvana with 2>NUL or a different log file (e.g. 2>mlexec.stderr.log) solves the problem.
I wanted to post this in a comment but it said I had to have 50 reputation (I have 49 now...)
I think this is what you're looking for
EDIT/UPDATE:
One thing you can do is enclose a section of your code with warning on/off statements as follows:
warning('off','all')
%your code here
warning('on','all')
This should stop any warnings being output to stderr from that section. I personally do not recommend this, it's good to know what you're doing that the MATLAB runtime does not like.
Related
I have a perl script for which I would like to write a few tests. If the perl script gets supplied a specific option, the user is allowed to paste or type out multi-line input and end their input using control-d.
These are the tests I want to write:
The script (when an interactive flag is supplied) waits for (multiple lines of) input on STDIN until the user hits control-d
[this test is already implemented - but provided for completeness] The script (when a flag is supplied that indicates a redirect/pipe) consumes all input on STDIN and does not wait for control-d
The script (when no input flag is provided [interactive or redirect]) does not wait for interactive user input
Test 1 toy example
I wrote a test for test 1 that confirms input was received (it works by the parent test script printing to the child's input handle and the child modifies and prints that input back out), but that test doesn't wait for an end of input signal (e.g. control-d) (which I don't know how to send anyway). So in other words, I can confirm it receives input, but I don't know how to confirm that it waits for the user to intentionally end the input entry. I.e. How do I know that the input consumption won't stop until the user types control-d?
Here's what I have so far for test 1. I wrote a 3rd little IO::Pipe::Consumer module to be able to send input to the child process that I'm testing and I wrote a toy example of the script that allows input on STDIN from a tty.
Here is a toy version of the script I'm testing:
>perl -e 'while(<STDIN>){print("*$_")}'
test
*test
^d
>
And here is the toy test code (that I want to improve) for the above script:
>perl -e '
use IO::Pipe::Consumer;
$obj = new IO::Pipe::Consumer;
$si = $obj->getSubroutineConsumer(
sub { while(<STDIN>) print("*$_") } }
);
print $si "test\n"
'
*test
>
I thought the parent would have to print an EOF (e.g. like what you get from "control-d") to end the input in the test, but the test ends immediately even though I'm not sending any such end-of-input character. I can see that it's getting, modifying, and printing the input. Is that sufficient to confirm that the script will wait for user input (and that the user will be able to intentionally end the input) or is there something else I should do to confirm it waits for all user input until the user intends to end it?
Test 2 - done
Test 3 toy - don't know how to write it...
Even if modified input spit back out is sufficient proof of "waiting for input" for test 1, I also wish to test that a script doesn't consume input on STDIN when no input option (interactive or redirect) is provided - but since it doesn't seem to wait even when I do send it input without an end-of-input signal, how would I test that the script wouldn't hang waiting for input? Note, the script has other options for consuming redirected or piped input, so my intent is specifically to know if it's waiting on input from the tty. All of the STDIN consumption options (whether from the tty or via redirect/pipe) are optional, which is why I want to write these tests.
My manual testing shows everything works as intended. I would just like some tests to assure that behavior for the future.
IO::Pipe::Consumer
I feel like the thing I'm missing is not relevant to IO::Pipe::Consumer, so WRT that, I'll just describe it instead of paste in 30 or so lines of code... All it does is it sets a pipe to the child's STDIN and gives that handle back to the parent for it to print to. I haven't put it on cpan. I'm just experimenting to see if I can use it to write these tests.
IO::Pipe::Consumer is basically the opposite of IO::Pipe::Producer (a module I published on cpan looong ago, say 2001-ish, when I was new to perl, or programming for that matter). The main difference, aside from swapping STDIN for STDOUT and Reader with Writer (and vice versa), is that the open is open(STDIN,"<",\${$stdin_pipe}).
I thought the parent would have to print an "end-of-input" (e.g. "control-d") character to end the input in the test,
Ctrl-D doesn't produce an "end of input character"; it causes the terminal to return EOF.
I don't know what IO::Pipe::Consumer is —it's not on CPAN— but I presume it creates a pipe. Exiting the program causes the writer end of the pipe to be closed and thus cause the reader end to return EOF.
is there something else I should do to confirm it waits for all user input until the user intends to end it?
<> reads until one of the following things happen:
A line feed is encountered (returning what was read including the line feed)
EOF is encountered (returning what was read up to an including the line feed)
An error is encountered (returning false).
You can confirm it waits by putting sleep statements between what you send. Be aware that buffering may interfere.
There is a line in a library that I can't take out:
put oResults format "x(80)" skip.
I have a program that is calling the library that doesn't have a default output so this line errors out.
I know I can just send output in my program somewhere but I want to fix it so you don't have to have a output. Seek function maybe?
EDIT: 10.2b
I only get an error in unix.
In a unix environment this line:
put oResults format "x(80)" skip.
errors out.
but if you put a:
if seek(output) <> ? then
put oResults format "x(80)" skip.
it doesn't error.
in a unix environment line:
put oResults format "x(80)" skip.
errors out.
but if you put a:
if seek(output) <> ? then
put oResults format "x(80)" skip.
it doesn't error.
You are running in batch mode. You should always be redirecting your output at the OS level when you run in batch mode. Something like this:
bpro -p test.p > errors.out 2>&1
Not redirecting output will pretty much always lead to the error that you are seeing.
If you are embedding the bpro, mbpro or _progres -b or whatever command in a script that needs to show that output or otherwise work with it you would typically use "cat" or "tail -f" on the output file.
I have a matlab file that takes in a file. I would like to run that program in the matlab shell, such as prog. I need to implement it so that it takes a number of arguments, such as "prog filename.txt 1 2 which would mean that i can use filename.txt and 1 2 as variables in my program.
Thank you!
In order to make a script accept arguments from the command line, you must first turn it into a function that will get the arguments you want, i.e if your script is named prog.m, put as the first line
function []=prog(arg1, arg2)
and add an end at the end (assuming that the file has only one function). It's very important that you call the function the same name as the file.
The next thing is that you need to make sure that the script file is located at the same place from where you call the script, or it's located at the Matlab working path, otherwise it'll not be able to recognize your script.
Finally, to execute the script you use
matlab -r "prog arg1 arg2"
which is equivalent to calling
prog(arg1,arg2)
from inside Matlab.
*- tested in Windows and Linux environments
Once your function is written in a separate file, as discussed by the other answer you can call it with a slightly more complicated setup to make it easier to catch errors etc.
There is useful advice in this thread about ensuring that Matlab doesn't launch the graphical interface and quits after finishing the script, and reports the error nicely if there is one.
For example:
matlab -nodisplay -nosplash -r "try, prog(1, 'file.txt'), catch me, fprintf('%s / %s\n',me.identifier,me.message), exit(1), end, exit(0)"
The script given to Matlab would read as follows if line spaces were added:
% Try running the script
try
prog(1, 'file.txt')
catch me
% On error, print error message and exit with failure
fprintf('%s / %s\n',me.identifier,me.message)
exit(1)
end
% Else, exit with success
exit(0)
I'm fairly new to Perl and have been searching the interwebs for documentation for what I'm trying to do. I'm not having any luck.
I have a program that outputs information to stdout with prompts throughout. I need to make a Perl script to pipe that information to a file.
I thought I could use Expect but there seems to be a problem with the pipe after the first prompt.
Here is the part of my code:
# Run program and compare the output to the BASE file
$cmd = "./program arg1 arg2 arg3 arg4 > $outfile";
my $exp = new Expect;
$exp->spawn($cmd);
BAIL_OUT("couldn't create expect object") if (! defined $exp);
$exp->expect(2);
$exp->send("\n");
For this case there is only a single prompt for the user to press "enter". This program is small and very fast - 2 seconds is plenty of time to reach the first prompt.
The output file only contains the first half of the information.
Does anyone have any suggestions on how I can grab the second half as well?
UPDATE:
I've verified that this works with Expect by using a simple script:
spawn ./program arg1 arg2 arg3 arg4
expect "<Return>"
send "\r"
interact
Where "< Return >" is a verbose expression that the Perl script could look for.
Note: I've tried writing my Perl script to expect "< Return >"...it makes no difference.
i.e.
$exp->expect(2, '-re', "<Return>")
Any thoughts?
UPDATE2:
Hazaah! I've found a solution to my problem...completely by accident.
So, I had a mistype in some test code I made...
$exp->expect(2);
$exp->send("\r");
$exp->expect(2);
Note the trailing expect(2)...I accidentally left that in and it worked!
So, I'm trying to understand what is happening. Unix expect does not seem work this way! It appears Expect implemented in Perl "expects" anything...not just prompts?
So, I provided expect another 2 seconds to collect stdout and I am able to get everything.
If anyone can offer some more detailed information as to what is going on here I'd love to understand what is going on.
Try sending \r instead of \n - you're trying to emulate a carriage return, not a newline, and the tty settings might not be translating them.
ALSO:
A suggestion from the Expect docs FAQ section, which seems likely given your accidental solution:
My script fails from time to time without any obvious reason. It
seems that I am sometimes loosing output from the spawned program.
You could be exiting too fast without giving the spawned program
enough time to finish. Try adding $exp->soft_close() to terminate the
program gracefully or do an expect() for 'eof'.
Alternatively, try adding a 'sleep 1' after you spawn() the program.
It could be that pty creation on your system is just slow (but this is
rather improbable if you are using the latest IO-Tty).
Standard unix/tcl expect doesn't exit in interactive mode, which could give your program enough time to finish running.
It's been a while since I've used Expect, but I'm pretty sure you need to provide something for Expect to match the prompt against:
$exp->expect( 2, 'Press enter' );
for example.
I want to call matlab in bash non-interactively and use its result outside Matlab.
For example, I have a script test.m
rand(3,4)
quit
When I execute in bash
$ matlab -nosplash -nodesktop -nodisplay -r test
Warning: No window system found. Java option 'MWT' ignored
< M A T L A B (R) >
Copyright 1984-2008 The MathWorks, Inc.
Version 7.7.0.471 (R2008b)
September 17, 2008
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
ans =
0.8147 0.9134 0.2785 0.9649
0.9058 0.6324 0.5469 0.1576
0.1270 0.0975 0.9575 0.9706
Is it possible to suppress the start message of Matlab and only show the results also without "ans=".
Note I am asking a general question not just for this example.
Thanks and regards!
Try using the -logfile command line option:
-logfile log - Make a copy of any output to the command window
in file log. This includes all crash reports.
Then you can easily remove the first few lines using any way you want (sed for example). Example:
matlab.exe -nosplash -nodesktop -nojvm -logfile out.log -r 'rand(3,3), exit'
sed '1,5d' out.log
Also if you are running from a script where you need it to finish running before continuing, use the -wait option:
-wait - MATLAB is started by a separate starter program
which normally launches MATLAB and then immediately
quits. Using the -wait option tells the starter
program not to quit until MATLAB has terminated.
This option is useful when you need to process the
the results from MATLAB in a script. The call to
MATLAB with this option will block the script from
continuing until the results are generated.
More info on MATLAB startup options can be found here, or in the matlab executable reference pages: Windows/Unix
You could use the Unix command "tail +n" to remove the first n lines of output. That header looks like 10 lines, so this will strip it.
$ matlab -nosplash -nodesktop -nodisplay -r test | tail +10
This is a little fragile, though, since warnings (like that "no window system") will get stripped, and the header size will vary depending on what warnings happen (and those warnings are useful diagnostics). Also, that warning might be on STDERR instead of STDOUT, so "tail +9" might be what you need.
A more robust approach could be to modify the Matlab script to write to a separate file using fopen/fprintf/fclose. That way the header, warnings, errors, etc from Matlab will be separated from the formatted output you want. To get the "disp" output to go to that separate file handle, you can capture it using evalc. The outfile could be specified using an argument to test() in the -r message, and the $$ env variable (the bash process's PID) incorporated in the file name to prevent collisions in a multiprocess environment.
function test(ppid)
outfile = sprintf('outfile-%d.tmp', ppid);
fh = fopen(outfile, 'w');
myvar = rand(3,4);
str = evalc('disp(myvar)');
fprintf(fh, '%s', str);
fclose(fh);
To invoke it from bash, use this calling form. (May be minor syntax problems here; I don't have a Unix box to test on right now.)
% matlab -nosplash -nodisplay -r "test($$)" -logfile matlab-log-$$.tmp
Let's say your bash PID is 1234. Now you've got your output in outfile-1234.tmp and a Matlab log in matlab-log-1234.tmp. Stick them in /tmp if you don't want to be dependent on pwd. You could extend this to create multiple output files from a single matlab invocation, saving the startup costs if you need to compute multiple things.
I'd recommend saving the output to a file, and then reading in that file. This approach is slightly more complicated, but less fragile as formats change etc. It gives you much more control. You'll find plenty of scripts on the web to transform Matlab files to a different host language.
Example:
A = randn(3, 2);
save temp_output.mat A
# Later, read temp_output.mat in whichever language you desire.
To suppress the display of ans =, you can use the DISP function:
disp(rand(3,4));
To suppress that first warning message, you can try adding in the option -nojvm to see if that helps.
To suppress everything else, you can try this solution from a MathWorks Newsgroup thread that addresses the same problem.
Calling MATLAB like this
matlab -nodisplay <test.m &>matlab.output
will dump all of the startup messages and other displayed output into the matlab.output file (which can be named anything you want). If you then (following Peter's suggestion) have test.m save the result that you need to a file using
csvwrite('temp_output.txt',A)
or another appropriate output function you can then read in this file and proceed.