I wounder what is the most efficient way to run a program, given as executable, from Matlab many times in a for loop. At the moment I use the following Code:
for i = 1:100
system('MyProgram.exe');
% Do something with the output from the .exe
end
So, from the profiler I know that 99,9% of the time is used in the execution of the Program itself. My question is basically if there is a more efficient way to run executables in general from within Matlab?
I have read that everytime I run an exe like described above, a process is created which has to initialize the Matlab runtime environment... Is there possibly a way to avoid this by only doing the initialization once and from there on run the programm multiple times?
I am guessing your can't directly modify the .exe's you are given, so perhaps there is a way to instead of calling the .exe directly, you could call a .bash shell script.
I would imagine that if you do this and within the shell script check to see if a workspace is already open to associate the execution of the .exe with a specific process ID. Although I would guess that when the executable finishes it closes the session.
Just throwing some stuff out there :P I have had lots of trouble with how Matlab handles this kind of thing (Also things like Excel).
Hope you figure this out.
EDIT: I found some possible examples here Example Descriptions
-Kyle
Related
I want to have two MATLAB windows open on the same computer. The desired scenario is as follows: MATLAB window 1 is continuously running a script that has nothing to do with MATLAB window 2. At the same time, MATLAB window 2 is running a script that continuously checks for a certain condition, and if it is met, then it will terminate the script running on MATLAB window 1, and then terminate its own script as well. I want to have two MATLAB windows instead of one since I believe it will be more time efficient for what I am trying to do. I found an interesting "KeyInject" program at http://au.mathworks.com/matlabcentral/fileexchange/40001-keyinject , but I was wondering if there is a simpler way already built into MATLAB.
Do you want simple, or a flexible, infinitely expandable version 1.0? Simple would be to trigger System A via a file created by System B.
Simple would have System B create a file, then System A would check for the file with the command
if exist ( fileName, 'file' )
then do your shutdown commands. On startup, System A would delete the file with
delete ( fileName );
The second option is to use the udp command. UDP allows any data to be sent between processes, whether on the same computer or over a network. (See https://www.mathworks.com/help/instrument/udp.html for more info).
I see several ways:
Restructure to avoid this XY problem
Use (mat) files (as Hoki suggested), possibly using the parallel computing toolbox to keep everything in one MATLAB session.
Write some MEX functions that communicate with each other via a global pipe.
Write an Auto(Hot)key script.
Option 2 is probably easiest. Take a look at events and listeners if you write in OOP, otherwise, you'd have to poll inside a loop
Option 3 is harder and way more time consuming to implement, but allows for much faster detection of the condition, and much faster data transfer between the sessions. Use only if speed is essential...but I guess that doesn't apply :)
Option 4: the AutoHotkey solution is probably the most Horrible Thing® you could do on an already Horrible Construction®, but oh what fun!! In both MATLAB sessions, you create a (hidden) figure with the name Window1 or Window2, respectively. These window names are something that AutoHotkey can easily track. If the conditions are met, you update the corresponding window name, triggering the remainder of the AutoHotkey script: press a button in the other window! If you need to transfer data between the windows: you can create basic edit boxes in both GUIs, and copy-paste the data between them. If you're on Linux: you can use Autokey for the same purpose, but by then you're basically writing Python code doing the heavy lifting, so just use Python.
Or, you know, use KeyInject. Less fun.
I'm currently doing some combustion engine analysis which has lead me to try and pass some specific heats from EES to matlab, by using EES-macros (.emf files) to generate the properties. This works great for simple tasks where the properties are just assigned to variables in the macros which is then exported and read by Matlab.
Now, I'm interested in getting the properties of products in chemical equilibrium calculations, so I need to solve coupled equations in EES. This poses a problem since you can't have unassigned stuff on the right hand side in EES-macros.
The above problem was quickly solved simply by solving the equations for the equilibrium composition in a reguler .ees-file and then exporting the results. But this has led to another problem:
Once I call my Matlab-script the procedure starts "hanging" just before the specific heats are returned. I've found that the script completes once you manually close the now-opened EES-window, but this is not viable since i need to make several hundreds of imports.
The problem doesn't occur when using EES-macros instead of files, since in these you can simply use the Quit statement in the end, but as mentioned macros are not an option for this. Does anyone know of an equivalent statement that you can use in an EES-FILE? I've also tried to shut down EES with a system-command in my script: system('taskkill /F /IM EES.EXE');. But this doesn't seem to be able to find the EES-task, although it appears in the task manager and in the taskbar (the statement is tested, it works if you open EES manually).
Any help is very much appreciated, thanks in advance!
Regards
You can use a macro file to solve the EES file and then quit the program.
Example.emf contains:
Open C:\Example.ees
Solve
Quit
And then the MATLAB system call
system('$EESPath\ees.exe C:\Example.emf');
will do the job.
You will need to leverage the $Export directive to place the results into an external file that MATLAB can then import.
Is there a way to call Matlab functions from outside, in particular by the Windows cmd (but also the Linux terminal, LUA-scripts, etc...), WITHOUT opening a new instance of Matlab each time?
for example in cmd:
matlab -sd myCurrentDirectory -r "function(parameters)" -nodesktop -nosplash -nojvm
opens a new instance of Matlab relatively fast and executes my function. Opening and closing of this reduced matlab prompt takes about 2 seconds (without computations) - hence for 4000 executions more than 2 hours. I'd like to avoid this, as the called function is always located in the same workspace. Can it be done in the same instance always?
I already did some research and found the possibility of the MATLAB COM Automation Server, but it seems quite complicated to me and I don't see the essential steps to make it work for my case. Any advices for that?
I'm not familiar with c/c++/c# but I'm thinking about the use of python (but just in the worst case).
Based on the not-working, but well thought, idea of #Ilya Kobelevskiy here the final workaround:
function pipeConnection(numIterations,inputFile)
for i=1:numIterations
while(exist('inputfile','file'))
load inputfile;
% read inputfile -> inputdata
output = myFunction(inputdata);
delete('inputfile');
end
% Write output to file
% Call external application to process output data
% generate new inputfile
end;
Another convenient solution would be to compile an executable of the Matlab function:
mcc -m myfunction
run this .exe-file using cmd:
cd myCurrentDirectory && myfunction.exe parameter1 parameter2
Be aware that the parameters are now passed as strings and the original .m-file needs to be adjusted considering that.
further remarks:
I guess Matlab still needs to be installed on the system, though
it is not necessary to run it.
I don't know how far this method is limited respectively the complexity of the
underlying function.
The speed-up compared to the initial apporach given in the question is
relatively small
Amongst the several methods exposed here, there is one workaround that should reduce the execution time of your multiple matlab calls. The idea is to run a custom function multiple times within on matlab session.
For example, myRand.m function is defined as
function r = myRand(a,b)
r = a + (b-a).*rand;
Within the matlab command window, we generate the single line command like this
S = [1:5; 1:5; 101:105];
cmd_str = sprintf('B(%d) = myRand(%d,%d);', S)
It generates the following command string B(1) = myRand(1,101);B(2) = myRand(2,102);B(3) = myRand(3,103);B(4) = myRand(4,104);B(5) = myRand(5,105); that is executed within a single matlab session with
matlab -nojvm -nodesktop -nosplash -r "copy_the_command_string_here";
One of the limitation is that you need to run your 4000 function calls in a row.
I like approach proposed by Magla, but given the constrains stated in your comment to it, it can be improved to still run single function in one matlab session.
Idea is to pipe your inputs and outputs. For inputs, you can check if certain input file exists, if it does, read input for your function from it, do work, write output to another file to signal script/function processing results that it matlab function is done and is waiting for the next input.
It is very straightforwad to implement using disk files, with some effort it is probably possible to do through memory disk (i.e., open input/output fiels in RAM).
function pipeConnection(numIterations,inputFile,outputFile)
for i=1:numIterations
while(!isfile(inputFile))
sleep(50);
end;
% Read inputs
output = YourFunction(x,y,z);
% Write output to file, go to next iteration
end;
return;
If number of iterations is unknown when you start, you can also encode exit conditions in input file rather than specifying number of iterations right away.
If you're starting up MATLAB from the command line with the -r option in the way you describe, then it will always start a new instance as you describe. I don't believe there's a way around this.
If you are calling MATLAB from a C/C++ application, MATLAB provides the MATLAB engine interface, which would connect to any running instance of MATLAB.
Otherwise the MATLAB Automation Server interface that you mention is the right way to go. If you're finding it complicated, I would suggest posting a separate question detailing what you've tried and what difficulties you're having.
For completeness, I'll mention that MATLAB also has an undocumented interface that can be called directly from Java - however, as it's undocumented it's very difficult to get right, and is subject to change across versions so you shouldn't rely on it.
Edit: As of R2014b, MATLAB makes available the MATLAB Engine for Python, via which you can automate MATLAB from a Python script. And as of R2016b, there is also the MATLAB Engine for Java. If anyone was previously considering the undocumented Java techniques mentioned above, this would now be the way to go.
Is there a way to call Matlab functions from outside, in particular by the Windows cmd (but also the Linux terminal, LUA-scripts, etc...), WITHOUT opening a new instance of Matlab each time?
for example in cmd:
matlab -sd myCurrentDirectory -r "function(parameters)" -nodesktop -nosplash -nojvm
opens a new instance of Matlab relatively fast and executes my function. Opening and closing of this reduced matlab prompt takes about 2 seconds (without computations) - hence for 4000 executions more than 2 hours. I'd like to avoid this, as the called function is always located in the same workspace. Can it be done in the same instance always?
I already did some research and found the possibility of the MATLAB COM Automation Server, but it seems quite complicated to me and I don't see the essential steps to make it work for my case. Any advices for that?
I'm not familiar with c/c++/c# but I'm thinking about the use of python (but just in the worst case).
Based on the not-working, but well thought, idea of #Ilya Kobelevskiy here the final workaround:
function pipeConnection(numIterations,inputFile)
for i=1:numIterations
while(exist('inputfile','file'))
load inputfile;
% read inputfile -> inputdata
output = myFunction(inputdata);
delete('inputfile');
end
% Write output to file
% Call external application to process output data
% generate new inputfile
end;
Another convenient solution would be to compile an executable of the Matlab function:
mcc -m myfunction
run this .exe-file using cmd:
cd myCurrentDirectory && myfunction.exe parameter1 parameter2
Be aware that the parameters are now passed as strings and the original .m-file needs to be adjusted considering that.
further remarks:
I guess Matlab still needs to be installed on the system, though
it is not necessary to run it.
I don't know how far this method is limited respectively the complexity of the
underlying function.
The speed-up compared to the initial apporach given in the question is
relatively small
Amongst the several methods exposed here, there is one workaround that should reduce the execution time of your multiple matlab calls. The idea is to run a custom function multiple times within on matlab session.
For example, myRand.m function is defined as
function r = myRand(a,b)
r = a + (b-a).*rand;
Within the matlab command window, we generate the single line command like this
S = [1:5; 1:5; 101:105];
cmd_str = sprintf('B(%d) = myRand(%d,%d);', S)
It generates the following command string B(1) = myRand(1,101);B(2) = myRand(2,102);B(3) = myRand(3,103);B(4) = myRand(4,104);B(5) = myRand(5,105); that is executed within a single matlab session with
matlab -nojvm -nodesktop -nosplash -r "copy_the_command_string_here";
One of the limitation is that you need to run your 4000 function calls in a row.
I like approach proposed by Magla, but given the constrains stated in your comment to it, it can be improved to still run single function in one matlab session.
Idea is to pipe your inputs and outputs. For inputs, you can check if certain input file exists, if it does, read input for your function from it, do work, write output to another file to signal script/function processing results that it matlab function is done and is waiting for the next input.
It is very straightforwad to implement using disk files, with some effort it is probably possible to do through memory disk (i.e., open input/output fiels in RAM).
function pipeConnection(numIterations,inputFile,outputFile)
for i=1:numIterations
while(!isfile(inputFile))
sleep(50);
end;
% Read inputs
output = YourFunction(x,y,z);
% Write output to file, go to next iteration
end;
return;
If number of iterations is unknown when you start, you can also encode exit conditions in input file rather than specifying number of iterations right away.
If you're starting up MATLAB from the command line with the -r option in the way you describe, then it will always start a new instance as you describe. I don't believe there's a way around this.
If you are calling MATLAB from a C/C++ application, MATLAB provides the MATLAB engine interface, which would connect to any running instance of MATLAB.
Otherwise the MATLAB Automation Server interface that you mention is the right way to go. If you're finding it complicated, I would suggest posting a separate question detailing what you've tried and what difficulties you're having.
For completeness, I'll mention that MATLAB also has an undocumented interface that can be called directly from Java - however, as it's undocumented it's very difficult to get right, and is subject to change across versions so you shouldn't rely on it.
Edit: As of R2014b, MATLAB makes available the MATLAB Engine for Python, via which you can automate MATLAB from a Python script. And as of R2016b, there is also the MATLAB Engine for Java. If anyone was previously considering the undocumented Java techniques mentioned above, this would now be the way to go.
I have checked the documents on Mathworks about command
system
I still do not fully grasp the idea of this command. It seems that this command is designed for call external programms, such Excel, Word, R, etc.
Is there any other purposes of using this command? If I do not grasp its essential idea yet.
system
is used for executing OS commands
to call Excel, Word, etc you may be better off using f.e.
actxserver()
In general you seem to have grasped the command in its entirety, it provides the facility to call external commands of all sorts, including operating system commands and other applications on the same (or indeed, different) computers. I suggest that you learn more about it by using it and waste no more time reading answers like this one on SO.
When you have more specific and more detailed questions, ask them.
EDIT in response to comment
Yes, you certainly can run an R program using the system command. For example, if you have a program called myRprogram.exe and if your path is set properly the Matlab command
system('myRprogram.exe')
should run your R program.
If what you mean is 'can I run an R program which I write in Matlab and send to the R run-time system at run-time' then the answer is (probably, I'm not an R expert) yes too. You should be able to write something like:
system('R set.seed(1); num=50; w = rnorm(num+1,0,1)')
So, if you can type and execute an R program from the command line, you can build and execute it inside a Matlab program.
NOTE: I am not an R programmer, and I make no claim that the string inside the call to system is a valid way to run R at the command line. If anyone reading this knows better, please feel free to edit or to write a better answer.