I have to make a total amount of simulations (e.g. NUM)
with Matlab witch I run with the following line
./run.sh -d num
repeatedly, where num ( NUM = integer*num) indicates the number of new Matlabs.
The same could happen with any other computing program in C or NS, etc.
The problem is that the computer where these processes run only admits a maximum number of Matlabs, e.g. MAX.
I would like to keep track of the number of Matlabs running and when the maximum number is reduced to (MAX-num), because 'num' has already finished, then run the next 'num' simulations of Matlab until the total NUM of simulations is completed.
To know the number of running processes I run the following script
ps axu |grep plopez|grep simulacion|grep MATLAB|awk '{ $2} END{print NR}'
which gives the total Matlabs running in this moment.
How could I make the whole set of simulations just from a single script?
The quick and dirty way would be to divide your set of simulations into NUM subsets, then you can work sequentially on these subsets.
I used the following script to process some images. The folder structure was like this:
CASE
20p
40p
60p
80p
100p
So I entered the CASE folder in the Terminal and ran my script. This called sequentially matlab and ran the script "processImages" to produce some extract data.
The
list=$(ls | grep p)
is really some quick and dirty line of code. However, it worked in my case and might give you the hint to solve the problem.
Have a very nice day.
#!/bin/bash
list=$(ls | grep p)
echo $list
for d in $list
do
echo $d
cd $d
matlab -nodisplay -nosplash -nodesktop -r "processImages; exit"
cd ..
done
Related
Below is a script in which I sum elements of a vector.
My problem here is that whenever I run the script, Matlab prints everything onto the command window. That is, it prints the comments, and everything else.
I only wanted to print the sum as an output, that is why I did not put ; after y = sum(x).
Can somebody help me prevent Matlab from printing the comments and contents of the script, except the ones I only want to print? Below is the script
%simple script
%this script sums the number of elements in a vector.
x = 1:2:10;
y = sum(x)
Here is the output in command window
Apparently echo on has been set in your MATLAB. You need to turn it off:
>> echo off
I am trying to write a batch script which can call and run a MATLAB script in the following manner:
matlab -r plotFunction(a,b); quit %here, a=1:10 and b=1:10
matlab -r plotFunction(a,b); quit %in 2nd instance a=11:20, b=11:20
matlab -r plotFunction(a,b); quit %in 3rd instance a=21:30, b=21:30
and so on.
That is, each time a new instance of MATLAB opens up, calls the function plotFunction which performs plotting a 100 times and then the program (MATLAB) quits. Subsequent to this, another instance of the program opens, performs plotting a 100 times again (corresponding to a=11:20 and b=11:20) and quits again. And so forth. How to put this in a loop?
The batch_job toolbox does this for you.
You can pass variables defined in the Windows command prompt to MATLAB line this:
set AMIN='1'
set BMIN='1'
set AMAX='10'
set BMAX='10'
matlab -r "disp(str2double(%AMIN%):str2double(%AMAX%)),disp(str2double(%BMIN%):str2double(%BMAX%)); input('press a key to quit'); quit"
Edit:
This can be improved like this,
set AMIN=1
set BMIN=1
set AMAX=10
set BMAX=10
set MATPROG=^
arange=(%AMIN%:%AMAX%),^
brange=(%BMIN%:%BMAX%),^
[x,y]=meshgrid(arange,brange),^
aplusb=x+y,^
plot3(x,y,aplusb),^
input('press a key to quit'),^
quit
matlab -r "%MATPROG%"
Note that ^ is the batch file line continuation character.
This change will make it easier to convert to loops in the batch file, although I don't understand why you don't create the loops in a MATLAB function and call that instead to keep the batch files as simple as possible.
I have the following problem:
I have written six scripts matlab. In the first script, the user must enter the variable n_strati (a number between 1 and 5). The first script allows you to make choices about computing models based on variables known or unknown , and runs them to n_strato=1. The second third fourth and fifth script follow the same procedure , respectively, for the layers 2-3-4-5, but in which the input parameters (not intended as a value) are different. For example:
for Strato1 performs calculations knowing the input variables A B E (and not C D F) , for Strato2 performs calculations knowing A C E (and not B D F) , for Strato3 knowing the variables B D F (and not A C E).
The sixth takes all the variables of the previous scripts and processes them to obtain the final result.
The first five scripts save the data using the command:
save Strato1 alpha beta gamma
% etc.
and the sixth script them "stores" with the command:
load Strato1
load Strato2
% etc.
But I have to make sure that:
if n_strati==1
I enter the data and choose the models in script1 jumping scripts 2-3-4-5 and proceeding with the final calculation through the script 6 .
if n_strati==2
I enter the data and choose the models for Strato1 in script1 and Strato2 in script2 jumping scripts 3-4-5 and proceeding with the final calculation through the script 6 .
and so on.
I wanted to know: how can I do?
Thank you for your cooperation.
The best way would be avoiding scripts and using functions. Even if you succeed in your plot of using multiple scripts, the code will be a big mess, hard to debug, etc. So the answer is simple:
Say NO to scripts!
It is as easy as adding a signature and declaring your input and output.
I have a large series of netcdf files representing daily snapshots of data. I am hoping to hook these up to a software which is asking me to add to the namelist the maximum and minimum values for a variable in the files. How can I enquire about the maximum and minimum values stored in a variable?
My variable is depth (here is an excerpt from an ncdump for an idea of the size of that variable)
...
dimensions:
z = 40 ;
lat = 224 ;
lon = 198 ;
time = 1 ;
variables:
float depth(z, lat, lon) ;
depth:long_name = "cell centre depth" ;
depth:units = "m" ;
...
I'm still a beginner at handling these files, and have been using NCO operators and/or matlab for netcdf handling to date - is there an easy way to perform this max min enquiry using either of these tools?
Before now I have had netcdfs where the value range was helpfully displayed in the attributes or it has been a sufficiently small amount of data to be displayed easily with a simple ncdump -v look at the values or storing the variable in matlab which auto displays the max min, but now I have too many values to use these quick and dirty methods.
Any help is greatfully received.
All the best,
Bex
One NCO method would be to use the ncrng command, which is simply a "filter" for a longer ncap2 command:
zender#roulee:~/nco/data$ ncrng three_dmn_rec_var in.nc
1.000000 to 80.000000
So, it's a three word command. Documentation on filters is here.
If you have a newer version of MATLAB, try using the ncread function.
% Update with your filename and variable name below.
% This reads in the full variable into MATLAB
variableData = ncread(filename,varname);
% Query max and min values
minValue = min(variableData(:))
maxValue = max(variableData(:))
% you could also write this information back to the file for future reference.
% see https://www.unidata.ucar.edu/software/netcdf/docs/netcdf/Attribute-Conventions.html
ncwriteatt(filename, varname, 'valid_range', [minValue, maxValue]);
% check result
ncdisp(filename, varname);
You could add two additional loops outside, one for looping through all your files and another for looping through all the variables in a file (look at ncinfo) to automate the whole thing.
The CDO method would be
cdo vertmax -fldmax in.nc max.nc
cdo vertmin -fldmin in.nc min.nc
The advantage is that you can calculate min/max just over x-y space (fldmax/fldmin), vertically (vertmax/min) or over time (timmax/min), or a combination of the three.
To dump the values from the netcdf to ascii you can use ncks
ncks -s '%13.9f\n' -C -H -v depth max.nc
To construct a namelist therefore you could for example write
echo min=`ncks -s '%13.9f\n' -C -H -v depth min.nc` >> namelist.txt
echo max=`ncks -s '%13.9f\n' -C -H -v depth max.nc` >> namelist.txt
I have Matlab .m script that sets and trains Neural network ("nn") using Matlab's Neural network toolbox. The script launches some GUI that shows trainig progress etc. The training of nn usually takes long time.
I'm doing these experiments on computer with 64 processor cores. I want to train several networks at the same time without having to run multiple Matlab sessions.
So I want to:
Start training of neural network
Modify script that creates network to create different one
Start training of modified network
Modify script to create yet another network...
Repeat steps 1-4 several times
The problem is that when I run the scrip it blocks Matlab terminal so I cannot do anything else until the script executes its last command - and that takes long. How can I run all those computations in parallel? I do have Matlab parallel toolbox.
EDIT: Matlab bug??
Update: This problem seems to happen only on R2012a, looks like fixed on R2012b.
There is very strange error when I try command sequence recommended in Edric's answer.
Here is my code:
>> job = batch(c, #nn, 1, {A(:, 1:end -1), A(:, end)});
>> wait(job);
>> r = fetchOutputs(job)
Error using parallel.Job/fetchOutputs (line 677)
An error occurred during execution of Task with ID 1.
Caused by:
Error using nntraintool (line 35)
Java is not available.
Here are the lines 27-37 of nntraintool (part of Matlab's Neural networks toolkit) where error originated:
if ~usejava('swing')
if (nargin == 1) && strcmp(command,'check')
result = false;
result2 = false;
return
else
disp('java used');
error(message('nnet:Java:NotAvailable'));
end
end
So it looks like the problem is that GUI (because Swing is not available) cannot be used when job is executed using batch command. The strange thing is that the nn function does not launch any GUI in it's current form. The error is caused by train that launches GUI by default but in nn I have switched that off:
net.trainParam.showWindow = false;
net = train(net, X, y);
More interestingly if the same nn function is launched normally (>> nn(A(:, 1:end -1), A(:, end));) it never enters the outer if-then statement of nntraintool on line 27 (I have checked that using debugger). So using the same function, the same arguments expression ~usejava('swing') evaluates to 0 when command is launched normally but to 1 when launched using batch.
What do you think about this? It looks like ugly Matlab or Neural networks toolbox bug :(((
With Parallel Computing Toolbox, you can run up to 12 'local workers' to execute your scripts (to run more than that, you'd need to purchase additional MATLAB Distributed Computing Server licences). Given your workflow, the best thing might be to use the BATCH command to submit a series of non-interactive jobs. Note that you will not be able to see any GUI from the workers. You might do something like this (using R2012a+ syntax):
c = parcluster('local'); % get the 'local' cluster object
job = batch(c, 'myNNscript'); % submit script for execution
% now edit 'myNNscript'
job2 = batch(c, 'myNNscript'); % submit script for execution
...
wait(job); load(job) % get the results
Note that the BATCH command automatically attaches a copy of the script to run to the job, so that you are free to make changes to it after submission.