Foreach loop problems in MATLAB - matlab

I have the following piece of code:
for query = queryFiles
queryImage = imread(strcat('Queries/', query));
queryImage = im2single(rgb2gray(queryImage));
[qf,qd] = vl_covdet(queryImage, opts{:}) ;
for databaseEntry = databaseFiles
entryImage = imread(databaseEntry.name);
entryImage = im2single(rgb2gray(entryImage));
[df,dd] = vl_covdet(entryImage, opts{:}) ;
[matches, H] = matchFeatures(qf,qf,df,dd) ;
result = [result; query, databaseEntry, length(matches)];
end
end
It is my understanding that it should work as a Java/C++ for(query:queryFiles), however the query appears to be a copy of the queryFiles. How do I iterate through this vector normally?
I managed to sort the problem out. It was mainly to my MATLAB ignorance. I wasn't aware of cell arrays and that's the reason I had this problem. That and the required transposition.

From your code it appears that queryFiles is a numeric vector. Maybe it's a column vector? In that case you should convert it into a row:
for query = queryFiles.'
This is because the for loop in Matlab picks a column at each iteration. If your vector is a single column, it picks the whole vector in just one iteration.

In MATLAB, the for construct expects a row vector as input:
for ii = 1:5
will work (loops 5 times with ii = 1, 2, ...)
x = 1:5;
for ii = x
works the same way
However, when you have something other than a row vector, you would simply get a copy (or a column of data at a time).
To help you better, you need to tell us what the data type of queryFiles is. I am guessing it might be a cell array of strings since you are concatenating with a file path (look at fullfile function for the "right" way to do this). If so, then a "safe" approach is:
for ii = 1:numel(queryFiles)
query = queryFiles{ii}; % or queryFiles(ii)
It is often helpful to know what loop number you are in, and in this case ii provides that count for you. This approach is robust even when you don't know ahead of time what the shape of queryFiles is.

Here is how you can loop over all elements in queryFiles, this works for scalars, row vectors, column vectors and even high dimensional matrices:
for query = queryFiles(:)'
% Do stuff
end

Is queryFiles a cell array? The safest way to do this is to use an index:
for i = 1:numel(queryFiles)
query = queryFiles{i};
...
end

Related

loop and concatenate variable in workspace

I am trying to concatenate matrices station_1, station_2,.....station_10 from my matlab workspace and trying to concatenate all stations automatically using a loop and not calling them one by one like this
cat(1,station_1,station_2,station_3,station_4... ,station_5,station_6,station_7,station_8... ,station_9,station_10 )
Any ideas?
the code below is what i was trying to improve
for jj= 1 : 10 T= cat(1,eval(['station_', num2str(jj)])); MegaMat = cat(1,T) end
Reading your code I think at the end of your loop you will have T = station_10.
If you want to concatenate all of them you would do
T = []
for jj= 1:10
T = cat(1, T, eval(['station_', num2str(jj)]));
end
MegaMat = T;
Using eval is not a good practice. Instead of creating station_1 to station_10 you could create a cell array
station{1} = ...
station{2} = ...
Then you could iterate like
T = []
for jj = 1:length(station)
T = cat(1, T, station{jj});
end
If the number of arrays is big this will be slow due to memory reallocation and copy. In that case is more efficient to initialize T as a matrix of the final dimension and write slices.
Appendix:
There is an interesting notation trick pointed by #Cris Luengo in the comments, that is when you have a cell array station, use the notation [station{:}], I have to admit, this notation is new to me. The only caveat is that if you set the items station{i} = ... then you will have the matrices concatenated horizontally rather than vertically.
The answer from #Mateo V, is also good, probably with leas overhead since it calls eval only once. That approach can be refined giving a one linear solution, and to be honest It felt not very unreadable.
MegaMat = eval(['cat(1', num2str(1:10, ', station_%d'), ')']);
No need for loops:
str = num2str(1:10, 'station_%i,'); % returns string 'station_1, station_2, ..., station_10,'
str = str(1:end-1); % remove last comma
eval(['MegaMat = cat(1, ', str, ');'])

Converting mixed empty/non-empty cells into a numeric matrix

I am working on a code to extract my AR(1)-GARCH(1) parameter, which I estimated using an AR(1)-GJR(1,1) model to individual matrices so that I can use them as variables in my calculations. As I have 16 time series variables, I combine the code with a loop in the following way:
for i=1:nIndices
AA_ARCH(:,i) = cell2mat(fit{i}.Variance.ARCH)';
end;
My problem is that for some variables is are no for AA_ARCH(:,i) the dimension is lower than nIndices. Naturally, when I try to export the estimates in the loop which specified the dimension of (:,i) and nIndices matlab reports a dimension mismatch. I would like to tell Matlab to replace the NaN with 0 instead of leaving the spot empty so that it is able to produce a (1,nIndices) matrix from AA_ARCH.
I thought of something like the this:
fit{i}.Variance.Leverage(isnan(fit{i}.Variance.Leverage))=0
but I wasn't able to combine this part with the previous code.
I would be very happy about any hints!
Best, Carolin
UPDATE:
Here is a fully a runnable version of my code which produces my problem. Notice that the code produces a dimension mismatch error because there is no ARCH and GARCH estimate in the fit.gjr(1,1) for time series 1. For these missing values I would like to have 0 as a placeholder in the extracted matrix.
returns = randn(2,750)';
T = size(returns,1);
nIndices = 2;
model = arima('AR', NaN, 'Variance', gjr(1,1));
residuals = NaN(T, nIndices);
variances = NaN(T, nIndices);
fit = cell(nIndices,1);
options = optimset('fmincon');
options = optimset(options, 'Display' , 'off', 'Diagnostics', 'off', ...
'Algorithm', 'sqp', 'TolCon' , 1e-7);
for i = 1:nIndices
fit{i} = estimate(model, returns(:,i), 'print', false, 'options', options);
[residuals(:,i), variances(:,i)] = infer(fit{i}, returns(:,i));
end
for i=1:nIndices
AA_beta(:,i) = cell2mat(fit{i}.AR)';
AA_GARCH(:,i) = cell2mat(fit{i}.Variance.GARCH)';
AA_ARCH(:,i) = cell2mat(fit{i}.Variance.ARCH)';
AA_Leverage(:,i) = cell2mat(fit{i}.Variance.Leverage)';
end;
I have some general things to say about the code, but first a solution to your problem:
You can put a simple if/else structure in your loop to handle the case of an empty array:
for ind1=1:nIndices
AA_beta(:,ind1) = cell2mat(fit{ind1}.AR)'; %//'
%// GARCH
if isempty(cell2mat(fit{ind1}.Variance.GARCH)') %//'
AA_GARCH(1,ind1) = 0;
else
AA_GARCH(:,ind1) = cell2mat(fit{ind1}.Variance.GARCH)'; %//'
end
%// ARCH (same exact code, should probably be exported to a function)
if isempty(cell2mat(fit{ind1}.Variance.ARCH)') %//'
AA_ARCH(1,ind1) = 0;
else
AA_ARCH(:,ind1) = cell2mat(fit{ind1}.Variance.ARCH)'; %//'
end
AA_Leverage(:,ind1) = cell2mat(fit{ind1}.Variance.Leverage)'; %//'
end;
Side note: I initially tried something like this: soz = #(A)isempty(A)*0+~isempty(A)*A; as an inline replacement for the if/else, but it turns out that MATLAB doesn't handle [] + 0 the way I wanted (it results in [] instead of 0; unlike other languages like JS).
As for the other things I have to say:
I am a firm supporter of the notion that one shouldn't use i,j as loop indices, as this may cause compatibility problems in some cases where complex numbers are involved (e.g. if you loop index is i then 1*i now refers to the loop index instead of to the square root of -1).
Part of your problem was that the arrays you were writing into weren't preallocated - which also means the correct datatype was unknown to MATLAB at the time of their creation. Besides the obvious performance hit this entails, it could also result in errors like the one you encountered here. If, for example, you used cells for AA_beta etc. then they could contain empty values, which you could later replace with whichever placeholder your heart desired using a combination of cellfun and isempty. Bottom line: lint (aka the colorful square on the top right of the editor window) is your friend - don't ignore it :)

Sum of Data(end) in a cell array of timeseries

Given the code below:
% Generate some random data
n = 10;
A = cell(n, 1);
for i=1:n
A{i} = timeseries;
A{i}.Data = rand(100, 1);
A{i}.Time = 1:100;
end
I would like to make the sum of Data(end) without explicitly writing a for loop. Is there a smart way to select Data(end) in all cells in a single line? A{:}.Data(end) does not work.
You can do it with cellfun but that is essentially just a for loop wrapped up:
cellfun(#(x) x.Data(end), A)
I prefer Dan's answer, but for reference, I'll post an alternative using arrayfun. This is also just a for loop wrapped up to save keystrokes, but not necessarily time.
sum(arrayfun(#(n) A{n}.Data(end), 1:numel(A)))
You can also extract all of the Data fields into a single matrix, which might be worth it if you're planning on doing multiple operations on it:
A2 = [A{:}];
A3 = [A2.Data];
sum(A3(end,:))

copy certain rows in a new matrix within a loop

I'm trying to split a matrix in smaller matrices depending on one characteristic (i use 'if').
for jj = 1:length(FailureHoopUP_sorted)
if FailureHoopUP_sorted(jj,1)==20
FailureHoopUP_20(jj,:) = FailureHoopUP_sorted(jj,:);
elseif FailureHoopUP_sorted(jj,1)==30
FailureHoopUP_30(jj,:) = FailureHoopUP_sorted(jj,:);
else
FailureHoopUP_40(jj,:) = FailureHoopUP_sorted(jj,:);
end
end
The problem I have is that there are rows of zeroes that get in between the rows with data in the new created matrices.
I was wondering how i could avoid this?
Thank you for your help.
You don't need a loop, you can use logical indexing. For example:
FailureHoopUP_20=FailureHoopUP_sorted(FailureHoopUP_sorted(:,1)==20,:)
...
...
This should also solve the zeros issue (that happens because you keep the original index jj that is related to the length of FailureHoopUP_sorted).

Bucketing Algorithm

I've got some code that works, but is a bit of a bottleneck, and I'm stuck trying to figure out how to speed it up. It's in a loop, and I can't figure how to vectorize it.
I've got a 2D array, vals, that represents timeseries data. Rows are dates, columns are different series. I'm trying to bucket the data by months to perform various operations on it (sum, mean, etc). Here is my current code:
allDts; %Dates/times for vals. Size is [size(vals, 1), 1]
vals;
[Y M] = datevec(allDts);
fomDates = unique(datenum(Y, M, 1)); %first of the month dates
[Y M] = datevec(fomDates);
nextFomDates = datenum(Y, M, DateUtil.monthLength(Y, M)+1);
newVals = nan(length(fomDates), size(vals, 2)); %preallocate for speed
for k = 1:length(fomDates);
This next line is the bottleneck because I call it so many times.(looping)
idx = (allDts >= fomDates(k)) & (allDts < nextFomDates(k));
bucketed = vals(idx, :);
newVals(k, :) = nansum(bucketed);
end %for
Any Ideas? Thanks in advance.
That's a difficult problem to vectorize. I can suggest a way to do it using CELLFUN, but I can't guarantee that it will be faster for your problem (you would have to time it yourself on the specific data sets you are using). As discussed in this other SO question, vectorizing doesn't always work faster than for loops. It can be very problem-specific which is the best option. With that disclaimer, I'll suggest two solutions for you to try: a CELLFUN version and a modification of your for-loop version that may run faster.
CELLFUN SOLUTION:
[Y,M] = datevec(allDts);
monthStart = datenum(Y,M,1); % Start date of each month
[monthStart,sortIndex] = sort(monthStart); % Sort the start dates
[uniqueStarts,uniqueIndex] = unique(monthStart); % Get unique start dates
valCell = mat2cell(vals(sortIndex,:),diff([0 uniqueIndex]));
newVals = cellfun(#nansum,valCell,'UniformOutput',false);
The call to MAT2CELL groups the rows of vals that have the same start date together into cells of a cell array valCell. The variable newVals will be a cell array of length numel(uniqueStarts), where each cell will contain the result of performing nansum on the corresponding cell of valCell.
FOR-LOOP SOLUTION:
[Y,M] = datevec(allDts);
monthStart = datenum(Y,M,1); % Start date of each month
[monthStart,sortIndex] = sort(monthStart); % Sort the start dates
[uniqueStarts,uniqueIndex] = unique(monthStart); % Get unique start dates
vals = vals(sortIndex,:); % Sort the values according to start date
nMonths = numel(uniqueStarts);
uniqueIndex = [0 uniqueIndex];
newVals = nan(nMonths,size(vals,2)); % Preallocate
for iMonth = 1:nMonths,
index = (uniqueIndex(iMonth)+1):uniqueIndex(iMonth+1);
newVals(iMonth,:) = nansum(vals(index,:));
end
If all you need to do is form the sum or mean on rows of a matrix, where the rows are summed depending upon another variable (date) then use my consolidator function. It is designed to do exactly this operation, reducing data based on the values of an indicator series. (Actually, consolidator can also work on n-d data, and with a tolerance, but all you need to do is pass it the month and year information.)
Find consolidator on the file exchange on Matlab Central