I have a software which gets its inputs and outputs from files in text. I just need to run the software with a given input file SIM.DATA using the following command in matlab
[status, result]=system('eclrun -v 2014.2 SIM.DATA',-echo')
if I have multiple .DATA files like SIM1.DATA to SIM4.DATA, and I use a parfor lop to run the software with different input DATa files, it gives and error but when I run it with for loop it works just fine.
I have very basic knowledge of parralel computing and cluster computing, could you please let me know what is wrong with my approach? here is a piece of code i run.
parfor labindex=1:4
cmd{labindex} = ['eclrun -v 2014.2 e300 SIM',num2str(labindex),'.DATA'];
end
parfor labindex=1:4
[status, result] = system([cmd{labindex},'.DATA'],'-echo');
end
Related
I need help to use mcc -mv in a for loop.
Specifically, I have matlab files names as Myfiles_k.m where k runs from 1:n.
I want to do something like the following
for i=1:n
fname = ['Myfiles_',num2str(i),'.m'];
mcc -mv fname
end
This leads to the following error:
Could not determine type of the MATLAB file 'fname'. Please make sure that you are compiling MATLAB Program files.
Could anyone help with letting me know what am I doing wrong here?
The command
mcc -mv fname
is interpreted as
mcc('-mv','fname')
That is, the arguments are seen as strings. You have a variable containing a string you want to pass to mcc, this requires:
mcc('-mv',fname)
I am executing windows '.exe' file in 'cmd' prompt for various inputs through Matlab. The commands as follows.
for i = 1:n
filename = sprintf('input_%d.dat',i);
string = sprintf('!sfbox.exe %s', filename);
eval(string)
end
All input files are present and independent of each other. But if I attempt to parallelize the execution using 'parfor' as follows,
parfor i = 1:n
filename = sprintf('input_%d.dat',i);
string = sprintf('!sfbox.exe %s', filename);
eval(string)
end
I get an error, but the code runs serially without stopping
Explanation
MATLAB runs parfor loops on multiple MATLAB workers that have
multiple workspaces. The indicated function might not access the
correct workspace; therefore, its usage is invalid.
Is there a correct way to execute the eval using parfor?
(PS: I tried manually executing several .exe files in cmd prompt and it is feasible to run several .exe files at same time in command prompt. Problem is the way I attempt to do it in Matlab. Please suggest better methods.)
You are hitting issues with Matlab not knowing what eval is actually doing. While you know that it's doing the right thing, the eval command could be executing anything. There is a little documentation on transparency issues using eval statements in parfor and spmd statements.
Switching to use an feval statement should solve your problem, as Matlab will know that the only thing going into that statement is a string. More directly, you can use the system command to directly execute an arbitrary string in the cmd prompt from matlab.
parfor i = 1:n
filename = sprintf('input_%d.dat',i);
string = sprintf('sfbox.exe %s', filename);
system(string);
end
I am iterating through a large test matrix in MATLAB and calling second-party proprietary software (running in MATLAB) each time. I cannot edit the software source code. Sometimes, the software hangs, so I want to exit it after a certain amount of time and move on to the next iteration.
In pseudocode, I'm doing this:
for i = 1:n
output(i) = proprietary_software(input(i));
end
How can I skip to the next iteration (and possibly save output(i)='too_long') if the proprietary software is taking too long?
You will need to call Matlab from another instance of Matlab. The other instance of Matlab will run the command and release control to the first instance of Matlab to wait while it either saves the results or reaches a certain time. In this case, it will wait 30 seconds.
You will need 1 additional function. Make sure this function is on the Matlab path.
function proprietary_software_caller(input)
hTic=tic;
output=proprietary_software(input);
hToc=toc(hTic);
if hToc<30
save('outfile.mat','output');
end
exit;
end
You will need to modify your original script this way
[status,firstPID] = str2double(system('for /f "tokens=2 delims=," %F in (''tasklist /nh /fi "imagename eq Matlab.exe" /fo csv) do #echo %~F'')'));
for i = 1:n
inputStr=num2str(input(i));
system(['matlab.exe -nodesktop -r proprietary_software_caller\(',inputStr,'\)&']);
hTic=tic;
hToc=toc(hTic);
while hToc<30 || ~(exist('outfile.mat','file')==2)
hToc=toc(hTic);
end
if hToc>=30
output(i)= 'too_long';
[status,allPIDs]=str2double(system('for /f "tokens=2 delims=," %F in (''tasklist /nh /fi "imagename eq Matlab.exe" /fo csv) do #echo %~F'')'));
allPIDs(allPIDs==firstPID)=[];
for a=1:numel(allPIDs)
[status,cmdout]=system(['taskkill /F /pid ' sprintf('%i',allPIDs(a))]);
end
elseif exist('outfile.mat','file')==2
loadedData=load('outfile.mat');
output(i)=loadedData.output;
delete('outfile.mat');
end
end
I hope this helps.
You are essentially asking for a way to implement a timeout on MATLAB code. This can be surprisingly tricky to implement. The first thing to state is that if the MATLAB code in question cannot terminate itself, either by exiting cleanly or throwing an error, then it is not possible to terminate the code without quitting or killing the MATLAB process in question. For example, throwing an error in an externally created timer does not work; the error is caught.
The first question to ask is therefore:
Can the over-running code be made to terminate itself?
This depends on the cause to the over-run, and also your access to the source code:
If the program gets stuck in an infinite (or very long-running) loop, either in MATLAB code or a mex file for which you have source code, or which calls a user-defined callback each iteration, then you can get this code to terminate itself.
If the program gets stuck inside a MATLAB builtin, or a p-code file or mex file for which you don't have the source code, and doesn't have support for calling a callback regularly, then it won't be possible for you to get the code to terminate itself.
Let's address the first case. The easiest way to get the code to terminate itself is to get it to throw an error, which is caught by the caller, if it exceeds the timeout time. E.g. in the OP's case:
for i = 1:n
tic();
try
output(i) = proprietary_software(input(i));
catch
end
end
with the following code somewhere in the over-running loop, or called in a loop callback or mex file:
assert(toc() < 10, 'Timed out');
Now for the second case. You need to kill this MATLAB process, so it makes sense for this to be a MATLAB process you have spawned from your current MATLAB session. You can do this using a system call similar to this:
system('matlab -nodisplay -r code_to_run()')
While it is possible for a MATLAB process to quit itself in some situations which could be of use here (e.g. a timer function calling quit('force')), the most reliable way of killing a MATLAB process is to do it with a system call, using taskkill (Windows) or kill (Linux/Mac).
A framework using the approach of spawning and killing timed-out MATLAB processes might work like this:
Using system calls, launch one or more new MATLAB processes from your MATLAB session, running the code you want.
Use the file system or a memory mapped file to communicate between the MATLAB processes the function inputs, loop progress, outputs, process ids and timeout times.
Use the original MATLAB process to check the timeout times haven't been reached, or if so to terminate the process in question and instantiate a new one.
Use the original MATLAB process to collect up the function outputs (either from the filesystem or memory mapped file) and exit. Workers should terminate when there is no more work left
I provide a sketch only because a full working implementation of this approach is fairly involved, and in fact it has already been implemented and is publicly available in the batch_job toolbox. In the OP's case, using this toolbox (with a 10 second timeout) you'd call:
output = batch_job(#proprietary_software, input(:)', '-timeout', 10);
Note that for the toolbox to work, its root directory needs to be on your MATLAB path at startup.
I am iterating through a large test matrix in MATLAB and calling second-party proprietary software (running in MATLAB) each time. I cannot edit the software source code. Sometimes, the software hangs, so I want to exit it after a certain amount of time and move on to the next iteration.
In pseudocode, I'm doing this:
for i = 1:n
output(i) = proprietary_software(input(i));
end
How can I skip to the next iteration (and possibly save output(i)='too_long') if the proprietary software is taking too long?
You will need to call Matlab from another instance of Matlab. The other instance of Matlab will run the command and release control to the first instance of Matlab to wait while it either saves the results or reaches a certain time. In this case, it will wait 30 seconds.
You will need 1 additional function. Make sure this function is on the Matlab path.
function proprietary_software_caller(input)
hTic=tic;
output=proprietary_software(input);
hToc=toc(hTic);
if hToc<30
save('outfile.mat','output');
end
exit;
end
You will need to modify your original script this way
[status,firstPID] = str2double(system('for /f "tokens=2 delims=," %F in (''tasklist /nh /fi "imagename eq Matlab.exe" /fo csv) do #echo %~F'')'));
for i = 1:n
inputStr=num2str(input(i));
system(['matlab.exe -nodesktop -r proprietary_software_caller\(',inputStr,'\)&']);
hTic=tic;
hToc=toc(hTic);
while hToc<30 || ~(exist('outfile.mat','file')==2)
hToc=toc(hTic);
end
if hToc>=30
output(i)= 'too_long';
[status,allPIDs]=str2double(system('for /f "tokens=2 delims=," %F in (''tasklist /nh /fi "imagename eq Matlab.exe" /fo csv) do #echo %~F'')'));
allPIDs(allPIDs==firstPID)=[];
for a=1:numel(allPIDs)
[status,cmdout]=system(['taskkill /F /pid ' sprintf('%i',allPIDs(a))]);
end
elseif exist('outfile.mat','file')==2
loadedData=load('outfile.mat');
output(i)=loadedData.output;
delete('outfile.mat');
end
end
I hope this helps.
You are essentially asking for a way to implement a timeout on MATLAB code. This can be surprisingly tricky to implement. The first thing to state is that if the MATLAB code in question cannot terminate itself, either by exiting cleanly or throwing an error, then it is not possible to terminate the code without quitting or killing the MATLAB process in question. For example, throwing an error in an externally created timer does not work; the error is caught.
The first question to ask is therefore:
Can the over-running code be made to terminate itself?
This depends on the cause to the over-run, and also your access to the source code:
If the program gets stuck in an infinite (or very long-running) loop, either in MATLAB code or a mex file for which you have source code, or which calls a user-defined callback each iteration, then you can get this code to terminate itself.
If the program gets stuck inside a MATLAB builtin, or a p-code file or mex file for which you don't have the source code, and doesn't have support for calling a callback regularly, then it won't be possible for you to get the code to terminate itself.
Let's address the first case. The easiest way to get the code to terminate itself is to get it to throw an error, which is caught by the caller, if it exceeds the timeout time. E.g. in the OP's case:
for i = 1:n
tic();
try
output(i) = proprietary_software(input(i));
catch
end
end
with the following code somewhere in the over-running loop, or called in a loop callback or mex file:
assert(toc() < 10, 'Timed out');
Now for the second case. You need to kill this MATLAB process, so it makes sense for this to be a MATLAB process you have spawned from your current MATLAB session. You can do this using a system call similar to this:
system('matlab -nodisplay -r code_to_run()')
While it is possible for a MATLAB process to quit itself in some situations which could be of use here (e.g. a timer function calling quit('force')), the most reliable way of killing a MATLAB process is to do it with a system call, using taskkill (Windows) or kill (Linux/Mac).
A framework using the approach of spawning and killing timed-out MATLAB processes might work like this:
Using system calls, launch one or more new MATLAB processes from your MATLAB session, running the code you want.
Use the file system or a memory mapped file to communicate between the MATLAB processes the function inputs, loop progress, outputs, process ids and timeout times.
Use the original MATLAB process to check the timeout times haven't been reached, or if so to terminate the process in question and instantiate a new one.
Use the original MATLAB process to collect up the function outputs (either from the filesystem or memory mapped file) and exit. Workers should terminate when there is no more work left
I provide a sketch only because a full working implementation of this approach is fairly involved, and in fact it has already been implemented and is publicly available in the batch_job toolbox. In the OP's case, using this toolbox (with a 10 second timeout) you'd call:
output = batch_job(#proprietary_software, input(:)', '-timeout', 10);
Note that for the toolbox to work, its root directory needs to be on your MATLAB path at startup.
I have a matlab file that takes in a file. I would like to run that program in the matlab shell, such as prog. I need to implement it so that it takes a number of arguments, such as "prog filename.txt 1 2 which would mean that i can use filename.txt and 1 2 as variables in my program.
Thank you!
In order to make a script accept arguments from the command line, you must first turn it into a function that will get the arguments you want, i.e if your script is named prog.m, put as the first line
function []=prog(arg1, arg2)
and add an end at the end (assuming that the file has only one function). It's very important that you call the function the same name as the file.
The next thing is that you need to make sure that the script file is located at the same place from where you call the script, or it's located at the Matlab working path, otherwise it'll not be able to recognize your script.
Finally, to execute the script you use
matlab -r "prog arg1 arg2"
which is equivalent to calling
prog(arg1,arg2)
from inside Matlab.
*- tested in Windows and Linux environments
Once your function is written in a separate file, as discussed by the other answer you can call it with a slightly more complicated setup to make it easier to catch errors etc.
There is useful advice in this thread about ensuring that Matlab doesn't launch the graphical interface and quits after finishing the script, and reports the error nicely if there is one.
For example:
matlab -nodisplay -nosplash -r "try, prog(1, 'file.txt'), catch me, fprintf('%s / %s\n',me.identifier,me.message), exit(1), end, exit(0)"
The script given to Matlab would read as follows if line spaces were added:
% Try running the script
try
prog(1, 'file.txt')
catch me
% On error, print error message and exit with failure
fprintf('%s / %s\n',me.identifier,me.message)
exit(1)
end
% Else, exit with success
exit(0)