Running out of memory when running a PARFOR loop in MATLAB - matlab

Question

Some colleagues and I are running out of memory running the following function on a cluster, where F_scattered is a collection of interpolants generated using scatteredInterpolant. Can anyone suggest steps we can take to make the parfor loop less memory-intensive? I’m concerned that the code as written may be sending all of F_scattered to each worker, but I'm not certain how to check if that’s happening. If it helps, there’s more context at the bottom of this message. Thank you in advance for your help.
function [V1q,V2q] = interpolate(F_gridded,X_gridded,F_scattered,X_scattered)
w = F_gridded(X_gridded);
k = fix(w);
w = w-k;
[n,m] = size(F_scattered);
assert(n==2)
F1 = cell(m,1);
F2 = cell(m,1);
parfor j=1:m
kj = k==j;
X = X_scattered(kj,:);
if j<m
W = w(kj);
F1{j} = (1-W).*F_scattered{1,j}(X)+W.*F_scattered{1,j+1}(X);
F2{j} = (1-W).*F_scattered{2,j}(X)+W.*F_scattered{2,j+1}(X);
else
F1{j} = F_scattered{1,j}(X);
F2{j} = F_scattered{2,j}(X);
end
end
Vq = nan(size(w));
for j=1:m
kj = k==j;
V1q(kj) = F1{j};
V2q(kj) = F2{j};
end
end
Additional context is as follows. Some colleagues and I are attempting a collocation on a seven-dimensional grid using an endogenous grid method. The relevant details are as follows:
• Our dimensions are (b1,dstate,omega,u,b2,b3,eps_pi). b1, b2, b3 and eps are continuous. dstate, omega and u are discrete. eps_pi is scattered due to the endogenous grid algorithm, but the grids for a other variables were built from vectors using ndgrid.
• Because scatteredInterpolant will at most accept three dimensional arguments, we construct our interpolants by “slicing” our collocation grid in (b1,dstate,omega,u), then running scatteredInterpolant on the (b2,b3,eps_pi) values associated with each slice.
• Since b1 is the only continuous variable in (b1,dstate,omega,u), we then interpolate by (i) finding the two slices that bracket the query values for (b1,dstate,omega,u), then (ii) running the scattered interpolants associated with those slices on the query values for (b2,b3,eps_pi), and then (iii) taking an appropriately weighted average of the outputs.
Here are the key parts of the code in relatively minimal form:
%%% setting up the grid vectors
b1_t_grid = linspace(0,1,8);
dstate_t_grid = [-1:12];
omega_t_grid = [-2,2];
u_t_grid = [0,1];
b2_t_grid = linspace(0,1,5);
b3_t_grid = linspace(0,1,5);
quasidiff_t_grid = linspace(-9*0.05*4*2,9*0.05*4*2,8); % ``target’’ variable for the endogenous grid method
%%% separating sliced dimensions from the others
gridded_dims_TSnow = [numel(b1_t_grid),numel(dstate_t_grid),numel(omega_t_grid),numel(u_t_grid)];
n_gridded_substates_TSnow = prod(gridded_dims_TSnow);
scattered_dims_TSnow = [numel(b2_t_grid),numel(b3_t_grid),numel(quasidiff_t_grid)];
n_scattered_substates_TSnow = prod(scattered_dims_TSnow);
[b1_ts_TSnow,dstate_ts_TSnow,omega_ts_TSnow,u_ts_TSnow,b2_ts_TSnow,b3_ts_TSnow,quasidiff_ts_TSnow] = ndgrid(b1_t_grid,dstate_t_grid,omega_t_grid,u_t_grid,b2_t_grid,b3_t_grid,quasidiff_t_grid);
b1_ts_TSnow = reshape(b1_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
dstate_ts_TSnow = reshape(dstate_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
omega_ts_TSnow = reshape(omega_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
u_ts_TSnow = reshape(u_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
b2_ts_TSnow = reshape(b2_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
b3_ts_TSnow = reshape(b3_ts_TSnow ,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
quasidiff_ts_TSnow = reshape(quasidiff_ts_TSnow,n_gridded_substates_TSnow,n_scattered_substates_TSnow);
grid_size_TSnow = numel(b1_ts_TSnow);
grid_dims_TSnow = size(b1_ts_TSnow);
%%% initial guess on the dimension to which we’ll ultimately apply the endogenous grid algorithm:
Gamma = #(yhat,omega) 0.05*(yhat-omega) + 0.10*(max(0,yhat-omega)).^2;
CP = #(omega,u) ((omega == 2).*(u == 1) - (omega == -2).*(u == 0))*0.05*4;
CP_ts_TSnow = CP(omega_ts_TSnow,u_ts_TSnow);
yhat_ts_TSnow_UNCnow = (ergoprob_L*omega_L + (1-ergoprob_L)*omega_H)*ones(grid_dims_TSnow);
eps_pi_ts_TSnow = quasidiff_ts_TSnow - Gamma(yhat_ts_TSnow_UNCnow,omega_ts_TSnow) - CP_ts_TSnow;
%%% pre-compute some stuff that will be useful for interpolation
substate_finder_TSnow = griddedInterpolant({b1_t_grid,dstate_t_grid,omega_t_grid,u_t_grid},reshape(1:n_gridded_substates_TSnow,gridded_dims_TSnow));
yhat_t_TSnow_fxns = cell(1,n_gridded_substates_TSnow); pihat_t_TSnow_fxns = cell(1,n_gridded_substates_TSnow);
parfor iii=1:n_gridded_substates_TSnow
yhat_t_TSnow_fxns{iii} = scatteredInterpolant(b2_ts_TSnow(iii,:)',b3_ts_TSnow(iii,:)',eps_pi_ts_TSnow(iii,:)', yhat_ts_TSnow_UNCnow(iii,:)');
pihat_t_TSnow_fxns {iii} = scatteredInterpolant(b2_ts_TSnow(iii,:)',b3_ts_TSnow(iii,:)',eps_pi_ts_TSnow(iii,:)',pihat_ts_TSnow_UNCnow(iii,:)');
end
%%% example of an interpolation, given an arbitrary grid of query points (b1s,dstates,omegas,us,b2s,b3s,epses)
yhats = NaN(size(b1s));
pihats= NaN(size(b1s));
[yhats(:),pihats(:)] = interpolate(substate_finder_TSnow ,[b1s(:),dstates(:), omegas(:),us(:)],...
[yhat_t_TSnow_fxns;pihat_t_TSnow_fxns],[b2s(:),b3(:),epses (:)]);