I am trying to get Matlab to execute a number of scripts as individual batch jobs. Each script loads some data from excel sheets and implements a neural network. The neural network uses parfor loops internally for parameter tuning.
When I run the batch job on my local machine it works fine. My Matlab code looks like
job1 = batch('Historical1Step',...
'Profile', 'local',...
'Matlabpool', 3,...
'CaptureDiary',true,...
'CurrentDirectory', '.');
try
job1Results = fetchOutputs(job1);
catch err
delete(job1);
rethrow(err);
end
delete(job1);
and the diary output I get is
--- Start Diary ---
Analysing data for stock BAX
num_its =
2
100%[============================
100%[===================================================]
--- End Diary ---
However, when I change from the 'local' config to my server config I get
--- Start Diary ---
--- End Diary ---
Error using parallel.Job/fetchOutputs (line 869)
An error occurred during execution of Task with ID 1.
Error in SOExample (line 14)
job1Results = fetchOutputs(job1);
Caused by:
Index exceeds matrix dimensions.
I am assuming the problem is something to do with the visibility of my functions/data on the workers, but I have tried every combination of the 'FileDependencies' and 'PathDependencies' options I can think of within the batch function to no avail.
Any help would be much appreciated, and apologies in advance if I have done something monumentally stupid without realising it!
EDIT-
The error stack is as follows:
Index exceeds matrix dimensions.
Error in Historical1Step (line 13)
Error in parallel.internal.cluster/executeScript (line 22)
eval(['iClearAndSetCallerWorkspace(workspaceIn);' scriptName]);
Error in parallel.internal.evaluator/evaluateWithNoErrors (line 14)
[out{1:nOut}] = feval(fcn, args{:});
Error in parallel.internal.evaluator/CJSStreamingEvaluator/evaluate (line 31)
[out, errOut] = parallel.internal.evaluator.evaluateWithNoErrors( fcn, nOut, args );
Error in dctEvaluateTask>iEvaluateTask/nEvaluateTask (line 281)
[output, errOutput, cellTextOutput{end+1}] = evaluator.evaluate(fcn, nOut, args);
Error in dctEvaluateTask>iEvaluateTask (line 141)
nEvaluateTask();
Error in dctEvaluateTask (line 57)
[resultsFcn, taskPostFcn, taskEvaluatedOK] = iEvaluateTask(job, task, runprop);
Error in distcomp_evaluate_filetask_core>iDoTask (line 149)
dctEvaluateTask(postFcns, finishFcn);
Error in distcomp_evaluate_filetask_core (line 48)
iDoTask(handlers, postFcns);
Error using parallel.Job/fetchOutputs (line 869)
An error occurred during execution of Task with ID 1.
Error in SOExample (line 14)
job1Results = fetchOutputs(job1);
Caused by:
Index exceeds matrix dimensions.
The file 'Historical1Step' is the script I am trying to run. The first lines (until the code falls over) are:
wrkDir = 'V:\Individual\SOFNN'; % this is where the files are on cluster headnode
wrkFldr = [wrkDir '\ExcelSheets\1-stepAhead\']; % location of excel sheets
%%
folder = dir(wrkFldr);
isub = [folder(:).isdir]; % data is stored in sub-directory based on stock symbol
stockNames = {folder(isub).name}'; % extract stock names from names of sub-dirs
stockNames(ismember(stockNames,{'.','..'})) = []; % remove names '.' and '..'
for i = 1:1 % this loop should read in data for stock i from correct sub-dir
close all;
clc;
sym = stockNames{i};
disp(['Analysing data for stock ' sym]);
fldrName = strcat(wrkFldr,'\', sym, '\');
end % added for completion
In your code, you're using a mapped-drive letter on the workers. Typically, workers cannot see mapped-drive letters because of the way the processes are launched. Try using a UNC path instead. A little more info in the documentaton here : http://www.mathworks.com/help/distcomp/troubleshooting-and-debugging.html
Related
https://www.mathworks.com/help/optim/examples/banana-function-minimization.html
fun = #(x)(100*(x(2) - x(1)^2)^2 + (1 - x(1))^2);
options = optimset('OutputFcn',#bananaout,'Display','off');
x0 = [-1.9,2];
[x,fval,eflag,output] = fminsearch(fun,x0,options);
title 'Rosenbrock solution via fminsearch'
Fcount = output.funcCount;
disp(['Number of function evaluations for fminsearch was ',num2str(Fcount)])
disp(['Number of solver iterations for fminsearch was ',num2str(output.iterations)])
What is #bananaout here?
This is giving me the following error,
??? Error using ==> feval
Attempt to execute SCRIPT bananaout as a function:
C:\Users\admin\Desktop\bananaout.m
Error in ==> callAllOptimOutputFcns at 12
stop(i) = feval(OutputFcn{i},xOutputfcn,optimValues,state,varargin{:});
Error in ==> fminsearch>callOutputAndPlotFcns at 464
stop = callAllOptimOutputFcns(outputfcn,xOutputfcn,optimValues,state,varargin{:})
|| stop;
Error in ==> fminsearch at 199
[xOutputfcn, optimValues, stop] =
callOutputAndPlotFcns(outputfcn,plotfcns,v(:,1),xOutputfcn,'init',itercount, ...
Error in ==> test_optim at 9
[x,fval,eflag,output] = fminsearch(fun,x0,options)
As per the doc, Output Functions are called by the optimizer at every time step, enabling you to do things like plot the progress of the optimization.
In your case you get an error because bananaout seems to be a script when it needs to be a function (with specific inputs - see the doc for their details). Did you happen to save the example code in a script called bananaout? If so, rename the script.
You can see a list of all m-code that you have that are called bananaout by executing the following:
>> which bananaout -all
One of them will be the function that the example should be calling, while another will be the one that you have created and need to rename/remove.
i wrote that code
clear all;
clc;
addpath('C:\Users\John\Documents\MATLAB\code for yannis\anger(W)\');
h1 = dir('C:\Users\John\Documents\MATLAB\code for yannis\anger(W)\');
for i=3:numel(h1)
%disp(h1(i,1).name);
%disp(k);
three(h1(i,1).name);
end
and the three function is
function three(filename)
%disp(filename);
q = char(39);
filename = strcat(q,filename,q)
%disp(filename);
load(filename);
And i get that error:
Error using load
Unable to read file '03a01WaM.mat': No such file or directory.
Error in three (line 7)
load(filename);
Error in run_three (line 13)
three(h1(i,1).name);
i also wrote exist('03a01WaM.mat') and the function return 2
Does anyone has an idea, what am i doing wrong?
There are multiple issues with your code.
addpath is simply unnessecary.
You are using relative path, but not cd. You have to use the full path to access the files.
You are adding a apostrophe to the filename.
Correct code would be:
directory='C:\Users\John\Documents\MATLAB\code for yannis\anger(W)\'; %'
h1 = dir(directory);
for i=3:numel(h1)
filename=fullfile(directory,h1(i,1).name);
load(filename);
end
I have this code: http://pastebin.com/E70c4UYY
When running, I get the following error:
Error using Diffusivity.getParams
Too many output arguments.
Error in Diffusivity.D_BA (line 63)
[sigmaA, epsK_A] = Diffusivity.getParams(specieA);
Error in Diffusivity.D_Amix (line 95)
Dam = fractionsArray(j) / Diffusivity.D_BA(specieA, fractionsArrayNames_cellstr{j}, T, P);
I do not understand how it can give an error, since in the code, on line 63 and 64 I have specified that there are two outputs?
I fixed it by changing line 9 from [results] = getParams(specie) to [sigma, epsK] = getParams(specie) and then just deleting line 54.
So I have a script that I run j times. The following line produces an error:
[evec(:,:,j), eval(:,:,j)] = eig(T(:,:,j));
Error using mupadmex
Error in MuPAD command: Cannot compute the explicit
representation of the eigenvalues; use
'numeric::eigenvectors'. [linalg::eigenvectors]
Error in sym/mupadmexnout (line 2080)
out = mupadmex(fcn,args{:});
Error in sym/eig (line 68)
[V,D,p] = mupadmexnout('symobj::eigenvectors',A);
Error in SingleLayer (line 86)
[evec(:,:,j), eval(:,:,j)] = eig(T(:,:,j));
That is the error message I get. Am I correct creating 3 "sheets" (not sure how to describe it) like this. I
i'm completely new to matlab and this is my first question.
I found a program like this
x = inputdlg('foo');
x = str2num(x{1})
and trying to make some gui from it, put this line to callback function of push button:
x=get(handles.edit1, 'String')
x=str2num(x{1})
and it works, but not after i add this the same thing with different variable
y=get(handles.edit2, 'String')
y=str2num(y{1})
command window said
Cell contents reference from a non-cell array object.
Error in regresilinear>pushbutton1_Callback (line 128)
x=str2num(x{1})
Error in gui_mainfcn (line 96)
feval(varargin{:});
Error in regresilinear (line 42)
gui_mainfcn(gui_State, varargin{:});
Error in
#(hObject,eventdata)regresilinear('pushbutton1_Callback',hObject,eventdata,guidata(hObject))
Error while evaluating uicontrol Callback
I found out that the output from command window is different when it's running and not with the same input.
When it got errors:
x =
0 1 2 3
when not (the first time)
x =
'0 1 2 3'
It doesn't give any error if i delete the str2num line.
I hope somebody can help fix the problem.
Start with a clear workspace and
x=get(handles.edit1, 'String');
x=str2num(x);
or better:
x=str2num(get(handles.edit1, 'String'));
{} are used for accessing the elements of a cell array. You are probably trying to use it on a string and that is why you are getting that error.