Updating histogram in a for-loop without growing y-data - matlab

Question

I have had zero luck finding this elsewhere on the site, so here's my problem. I loop through about a thousand mat files, each with about 10,000 points of data. I'm trying to create an overall histogram of this data, but it's not very feasible to concatenate all this data to give to hist.
I was hoping to be able to create an N and Bin variable each loop using hist (y), then N and Bin would be recalculated on the next loop iteration by using hist(y_new). And so on and so on. That way the source data doesn't grow and when the loop finally ends, I can just use bar(). If this method wouldn't work, then I am very open-minded to other solutions.
Also, it is probably not safe to assume that the x data will remain constant throughout each iteration. I'm using 2012a.
Thanks for any help!!

Answer 1

I think the best solution here is to loop through your files twice: once to set the bins and once to do the histogram. But, if this is impossible in your case, here's a one shot solution that requires you to set the bin width beforehand.
clear; close all;
rng('default') % for reproducibility
% make example data
N = 10; % number of data files
M = 5; % length of data files
xs = cell(1,N);
for i = 1:N
xs{i} = trnd(1,1,M);
end
% parameters
width = 2;
% main
for i = 1:length(xs)
x = xs{i}; % "load data"
range = [min(x) max(x)];
binsPos = 0:width:range(2)+width;
binsNeg = fliplr( 0:-width:range(1)-width );
newBins = [binsNeg(1:end-1) binsPos];
newCounts = histc(x, newBins);
newCounts(end) = []; % last bin should always be zero, see help histc
if i == 1
counts = newCounts;
bins = newBins;
else
% combine new and old counts
allBins = min(bins(1), newBins(1)) : width : max(bins(end), newBins(end));
allCounts = zeros(1,length(allBins)-1);
allCounts(find(allBins==bins(1)) : find(allBins==bins(end-1))) = counts;
allCounts(find(allBins==newBins(1)) : find(allBins==newBins(end-1))) = ...
allCounts(find(allBins==newBins(1)) : find(allBins==newBins(end-1))) + newCounts;
bins = allBins;
counts = allCounts;
end
end
% check
figure
bar(bins(1:end-1) + width/2, counts)
xFull = [xs{:}];
[fullCounts] = histc(xFull, bins);
fullCounts(end) = [];
figure
bar(bins(1:end-1) + width/2, fullCounts)