Related
So my problem is, I'd like to do this without the for loop. Geting the prod() of multiple vectors but of different lengths.
I am dealing with rays intersecting voxels. I typically have 1e6 rays and 1e5 voxels, but this can vary.
intxRays is a list of rays that have intersected voxels.
gainList is a one dimensional vector, each element has a value that corresponds to a specific ray voxel intersection calculated previously (actually with the help of you lovely lot here).
rayIntxStart and rayIntxEnd are vectors of indices for, where in the gainlist array, each ray's corresponding values start and end (they're all in order).
Here is the code and some examples and expected outputs.
gainSum = zeros(1, 5);
% only interested in the intx uniques
intxSegCtr = 1;
% loop through all of the unique segments
for rayCtr = 1:max(intxRays)
if rayCtr == intxRays(intxSegCtr)
startInd = rayIntxStart(intxSegCtr);
endInd = rayIntxEnd(intxSegCtr);
% find which rows correspoond to those segements
gainVals = gainList(startInd:endInd);
gainProd = prod(gainVals);
% get the product of the gains for those voxels
gainSumIdx = intxRays(intxSegCtr);
gainSum(gainSumIdx) = gainProd;
% increment counter
intxSegCtr = intxSegCtr + 1;
end
end
Example data for five rays and nine voxels. Assume the voxel gain array looked like this (for simplicity) for nine voxels (used in previous step).
voxelGains = 10:10:90;
Now say rays 1 and 3 don't hit anything, ray 2 hits voxels 1 and 2, ray 4 hits voxels 2:7
and ray 5 hits voxels 6:9
intxRays = [2, 4, 5];
gainList = [10, 20, 20, 30, 40, 50, 60, 70, 60 70, 80, 90];
rayIntxStart = [1, 3, 9];
rayIntxEnd = [2, 8, 12];
For these numbers the above code would give as a result:
gainSum = [0, 200, 0, 5.0400e+09, 3.024e+07];
I hope this all makes sense.
When I developed it I was using far smaller ray and voxel numbers and it worked fine. As I'm moving up though, the major bottleneck in my code is this loop. Actually just the gainVals and gainProd assignment is like 80% and 15% of my runtime on their own.
This is the only method I can find that works, padding and the like won't work due to the sizes involved.
Is there a way to get the value I want, without this loop?
Many thanks!
ok this is a very small performance boost, but it might help. for testing the matrix way without the loop a bigger data sample is needed.
These are 3 soultions, your original, an optimized and the optimized way as a oneliner. could you please try if this is already doing something for you?
clear all
% orignial loop through all Rays
intxRays = [2, 4, 5];
gainList = [10, 20, 20, 30, 40, 50, 60, 70, 60 70, 80, 90];
rayIntxStart = [1, 3, 9];
rayIntxEnd = [2, 8, 12];
gainSum = zeros(1, 5);
tic
% only interested in the intx uniques
intxSegCtr = 1;
% loop through all of the unique segments
for rayCtr = 1:max(intxRays)
if rayCtr == intxRays(intxSegCtr)
startInd = rayIntxStart(intxSegCtr);
endInd = rayIntxEnd(intxSegCtr);
% find which rows correspoond to those segements
gainVals = gainList(startInd:endInd);
gainProd = prod(gainVals);
% get the product of the gains for those voxels
gainSumIdx = intxRays(intxSegCtr);
gainSum(gainSumIdx) = gainProd;
% increment counter
intxSegCtr = intxSegCtr + 1;
end
end
toc
clear all
%loop insted of every single one to max just through the intxRays
intxRays = [2, 4, 5];
gainList = [10, 20, 20, 30, 40, 50, 60, 70, 60 70, 80, 90];
rayIntxStart = [1, 3, 9];
rayIntxEnd = [2, 8, 12];
gainSum = zeros(1, 5);
tic
for rayCtr=1:length(intxRays)
%no if as you just go through them
%intxRays(rayCtr) is the corresponding element
startInd = rayIntxStart(rayCtr);
endInd = rayIntxEnd(rayCtr);
% find which rows correspoond to those segements
gainVals = gainList(startInd:endInd);
gainProd = prod(gainVals);
% get the product of the gains for those voxels and set them to the ray
gainSum(intxRays(rayCtr)) = gainProd;
end
%disp(gainSum);
toc
clear all
%same as above, but down to 1 line so no additional values are generated
intxRays = [2, 4, 5];
gainList = [10, 20, 20, 30, 40, 50, 60, 70, 60 70, 80, 90];
rayIntxStart = [1, 3, 9];
rayIntxEnd = [2, 8, 12];
gainSum = zeros(1, 5);
tic
for rayCtr=1:length(intxRays)
gainSum(intxRays(rayCtr))=prod(gainList(rayIntxStart(rayCtr):rayIntxEnd(rayCtr)));
end
toc
I have data set in refxy size 500 x 3 where each row represents one location such that 1st, 2nd and 3rd columns have x coordinate, y coordinate and the weight v1 of the location. Using following code, I divide this area into 14 x 18 grid and I find how many points in each grid which gives in output blockNums_v1.
function gridcounttest
load refxy
x = refxy(:,1);
y = refxy(:,2);
v1 = refxy(:,3);
nBinsX = 14 ;
nBinsY = 18 ;
xg = linspace( 0, 700, nBinsX+1 ) ;
yg = linspace( 0, 900, nBinsY+1 ) ;
nCells = nBinsX * nBinsY ;
xId = sum( bsxfun( #ge, x, xg(1:end-1) ), 2 ) ;
yId = sum( bsxfun( #ge, y, yg(1:end-1) ), 2 ) ;
cellId = nBinsY * (xId - 1) + yId ;
blockNums_v1 = accumarray( cellId, 1, [nCells, 1] )
blockSum_v1 = accumarray( cellId, v1, [nCells, 1] )
blockMean_v2 = accumarray( cellId, v1, [nCells, 1], #mean )
Can someone please help me to identify which points, probably the row numbers, are included in each grid? e.g., if grid number 10 has 3 points which are in row 23, 51 and 432. This code gives output 3 but NOT 23, 51, 432 which I need to get now :)
Thanks !!!
You can find the rows of points which are in a grid element by
rowsOfValuesInGridpoint10 = find(cellId == 10);
The problem is that the output is not uniform, when you want to find the points of several grid elements.
You could store the rows in Matlab-cells, e.g.
for i=1:nCells
rowsInThisElement{i} = find(cellId == i);
end
rowsInThisElement{10}
I have been trying to construct a structure for wind data in matlab using the struct function:
struct(fieldname1,value1,fieldname2,value2,......).
I have wind speed and direction at different heights such as 40,50,80,90 meters. The problem is I don't understand how to represent the 'value', for my big data.
wind_data=struct(ws40,[],ws50,[],ws80,[],ws90,[],wd40,[],wd50,[],wd80,[],wd90,[])
ws=wind speed. wd=wind direction and each of them are vectors.
You can allocate structs by hand:
wind_data.ws40 = [1, 2, 3];
wind_data.wd40 = [4, 5, 6];
wind_data.ws50 = [11, 22, 33];
wind_data.wd50 = [44, 55, 66];
or dynamically:
heights = [40, 50, 80, 90];
ws = round(10*rand(4,3));
wd = round(10*rand(4,3));
for hh = 1:numel(heights)
wind_data.( [ 'ws' num2str(heights(hh)) ] ) = ws(hh,:)
wind_data.( [ 'wd' num2str(heights(hh)) ] ) = wd(hh,:)
end
or directly allocate them, you have to put the fieldname in '' as Ed Smith already said:
heights = [40, 50, 80, 90];
ws = round(10*rand(4,3));
wd = round(10*rand(4,3));
wind_data = struct('ws40', ws(1,:), ...
'wd40', wd(1,:), ...
'ws50', ws(2,:), ...
'wd50', wd(2,:) );
I have 2 (not very small) 3-dimension structs with matrices as fields:
sz1 = 200;
sz2 = 9;
sz3 = [20, 40, 80, 160, 320, 640, 1280, 2560, 5120]
% actually the structs have 12 fields, each has size 200x9x5120
mat.p1(sz1, sz2, sz3(sz2)) = uint8(0);
mat.p2(sz1, sz2, sz3(sz2)) = uint8(0);
mat.p3(sz1, sz2, sz3(sz2)) = 0;
mat.p4(sz1, sz2, sz3(sz2)) = 0;
old_mat.p1(sz1, sz2, sz3(sz2)) = uint8(0);
old_mat.p2(sz1, sz2, sz3(sz2)) = uint8(0);
old_mat.p3(sz1, sz2, sz3(sz2)) = 0;
old_mat.p4(sz1, sz2, sz3(sz2)) = 0;
And i need to write a reset function, where i will re-assign the values to 3 of 4 (actually 10 of 12) fields in both matrices as follows:
for i = 1:sz1
for j = 1:sz2
for k = 1:sz3(j)
mat.p1(i,j,k) = uint8(255);
mat.p3(i,j,k) = -1;
mat.p4(i,j,k) = 0.01;
old_mat.p1(i,j,k) = uint8(255);
old_mat.p3(i,j,k) = -1;
old_mat.p4(i,j,k) = 0.01;
end
end
end
Note that actually what i need in the matrices is the same as the reset function, means i only need the 5120th index of the 3rd-dimension when 2nd-dimension is 9, if 2nd-dimension = 4, i only need up to 160 indices at 3rd-dimension etc.
The questions are:
Is there anyway to assign values to the fields at the same time (not 1 line 1 field) since i actually have to do with 10 fields?
Is there anyway to avoid the for-loops? I tried like this:
mat.p1(:) = uint8(255);
mat.p3(:) = -1;
mat.p4(:) = 0.01;
old_mat.p1(:) = uint8(255);
old_mat.p3(:) = -1;
old_mat.p4(:) = 0.01;
but here all the matrices are filled with max 3rd-dimension = 5120 so i expect someone can show me how to use the vectorizing function like arrayfun, bsxfun, cellfun etc., which can apply for just the "half-cubic" like the for-loops above.
UPDATE:
Thanks horchler for the video, it seems the problem with big size (in bytes) of the matrices is solved when i change the matrix of struct to a struct with matrices as fields. This also clears the timing problem even with nested for-loops. So i updated the questions and the input as well, please see above.
Yes, I think that using a structure of arrays will work better for you here. You should be able to allocate using zeros just as if mat.p1, mat.p2, etc. were regular arrays. I'd do something like this (note that you didn't indicate any values for mat.p2) using a single for loop:
sz1 = 200;
sz2 = 9;
sz3 = [20, 40, 80, 160, 320, 640, 1280, 2560, 5120];
% Be careful with this form of pre-allocation if your sz arrays change
% Clear your struct or make sure to use the code in a function
mat.p1(sz1,sz2,sz3(sz2)) = uint8(0);
mat.p3(sz1,sz2,sz3(sz2)) = 0;
mat.p4(sz1,sz2,sz3(sz2)) = 0;
for i = 1:sz2
mat.p1(:,i,1:sz3(i)) = uint8(255);
mat.p3(:,i,1:sz3(i)) = -1;
mat.p4(:,i,1:sz3(i)) = 0.01;
end
old_mat.p1 = mat.p1;
old_mat.p3 = mat.p3;
old_mat.p4 = mat.p4;
Alternatively, you could do something like this:
sz1 = 200;
sz2 = 9;
sz3 = [20, 40, 80, 160, 320, 640, 1280, 2560, 5120];
mat = struct('p1',zeros(sz1,sz2,sz3(sz2),'uint8'),...
'p3',zeros(sz1,sz2,sz3(sz2)),...
'p4',zeros(sz1,sz2,sz3(sz2)));
for i = 1:sz2
mat.p1(:,i,1:sz3(i)) = uint8(255);
mat.p3(:,i,1:sz3(i)) = -1;
mat.p4(:,i,1:sz3(i)) = 0.01;
end
old_mat = struct('p1',mat.p1,'p3',mat.p3,'p4',mat.p4);
I'm trying to vectorize the following code:
% code before
% code before
% a lot of code before we got to the current comment
%
% houghMatrix holds some values
for i=1:n
for j=1:m
for k = 1:maximalRadius
% get the maximal threshold
if houghMatrix(i,j,k) > getMaximalThreshold(k)
lhs = [j i k];
% verify that the new circle is not listed
isCircleExist = verifyCircleExists(circles,lhs,circleCounter);
% not listed - then we put it in the circles vector
if isCircleExist == 0
circles(circleCounter,:) = [j i k];
fprintf('Circle % d: % d, % d, % d \n', circleCounter, j, i, k);
circleCounter = circleCounter + 1;
end
end
end
end
end
Using tic tac I got the below outputs :
>> x = findCircles(ii);
Circle 1: 38, 38, 35
Circle 2: 89, 51, 34
Circle 3: 72, 66, 11
Circle 4: 33, 75, 30
Circle 5: 90, 81, 31
Circle 6: 54, 96, 26
Elapsed time is 3.111176 seconds.
>> x = findCircles(ii);
Circle 1: 38, 38, 35
Circle 2: 89, 51, 34
Circle 3: 72, 66, 11
Circle 4: 33, 75, 30
Circle 5: 90, 81, 31
Circle 6: 54, 96, 26
Elapsed time is 3.105642 seconds.
>> x = findCircles(ii);
Circle 1: 38, 38, 35
Circle 2: 89, 51, 34
Circle 3: 72, 66, 11
Circle 4: 33, 75, 30
Circle 5: 90, 81, 31
Circle 6: 54, 96, 26
Elapsed time is 3.135818 seconds.
Meaning - average of 3.1 seconds .
I tried to vectorize the code , but the problem is that I need to use
the index i,j,k in the body of the inner for (the 3rd for) .
Any suggestions how to vectorize the code would be greatly appreciated
Thanks
EDIT :
% -- function [circleExists] = verifyCircleExists(circles,lhs,total) --
%
%
function [circleExists] = verifyCircleExists(circles,lhs,total)
MINIMUM_ALLOWED_THRESHOLD = 2;
circleExists = 0;
for index = 1:total-1
rhs = circles(index,:);
absExpr = abs(lhs - rhs);
maxValue = max( absExpr );
if maxValue <= MINIMUM_ALLOWED_THRESHOLD + 1
circleExists = 1;
break
end
end
end
Heres what I think you want to do: For each valid coordinate triplet, you want to check whether there has been a nearby triplet already, otherwise, you add it to the list. This operation can be fully vectorized if there's no possibility of "chaining", i.e. if each cluster of possible candidate voxels can only accomodate one center. In this case, you simply use:
%# create a vector of thresholds
maximalThreshold = getMaximalThreshold(1:maximalRadius);
%# make it 1-by-1-by-3
maximalThreshold = reshape(maximalThreshold,1,1,[]);
%# create a binary array the size of houghMatrix with 1's
%# wherever we have a candidate circle center
validClusters = bsxfun(#gt, houghMatrix, maximalThreshold);
%# get the centroids of all valid clusters
stats = regionprops(validClusters,'Centroid');
%# collect centroids, round to get integer pixel values
circles = round(cat(1,stats.Centroid));
Alteratively, if you want to follow your scheme of selecting valid circles, you can get the ijk indices from validClusters as follows:
[potentialCircles(:,1),potentialCircles(:,2), potentialCircles(:,3)]= ...
sub2ind(size(houghMatrix),find(validClusters));
nPotentialCircles = size(potentialCircles,1);
for iTest = 2:nPotentialCircles
absDiff = abs(bsxfun(#minus,potentialCircles(1:iTest-1,:),potentialCircles(iTest,:)));
if any(absDiff(:) <= MINIMUM_ALLOWED_THRESHOLD + 1)
%# mask the potential circle
potentialCircles(iTest,:) = NaN;
end
end
circles = potentialCircles(isfinite(potentialCircles(:,1)),:);