How to combine outputs of multiple tasks performed using Matlab SGE? - matlab

Question

I have the following batch file launching some m-files (main.m and f.m which are scripts) 4 times (4 tasks).
#$ -S /bin/bash
#$ -l h_vmem=2G
#$ -l tmem=2G
#$ -cwd
#$ -j y
#Run 4 tasks where each task has a different $SGE_TASK_ID ranging from 1 to 4
#$ -t 1-4
#$ -N example
#Output the Task ID
echo "Task ID is $SGE_TASK_ID"
cat main.m f.m | matlab -nodisplay -nodesktop -nojvm -nosplash
At the end I obtain 4 outputs that are example.o[...].1,example.o[...].2, example.o[...].3, example.o[...].4. Each of them looks like
...
Task ID is ...
< M A T L A B (R) >
...
>> >> >> >> >> >> >> >> >> >> >>
output =
4.0234 -3.4763
How can I combine these 4 outputs in a matrix 4x2 and save it?

Answer 1

You should save the relevant output from within f.m using MATLAB's save or something similar.
If you use the -r flag to call main and f from the command line you can add a variable which will contain the task ID and you can then access that from within f.m
matlab -nodisplay -nodesktop -nojvm -nosplash -r "main; ID = $SGE_TASK_ID; f; exit"
Then within f.m
% You theoretically generate some numeric result
result = rand(1, 2);
filename = sprintf('Result.%d.mat', ID);
save(filename, 'result')
This will save Result.0.mat, Result.1.mat etc.
Alternately, you could modify f.m such that it loads the data from the file, appends to it, and re-saves it every time
result = rand(1,2);
filename = 'Results.mat';
% If this is the first task, then create a new file, otherwise append to the old
if ID == 1
data = result;
else
tmp = load(filename, '-mat');
data = tmp.data;
data(ID,:) = result;
end
save(filename, 'data')