I am using the following decision tree for detecting ROIs in an image: http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3097782/figure/F7/. The first part uses a 32x32 window to scan the entire image using the code shown below. Can anyone tell me how to calculate the average intensity of the pixels within this window? Thanks
N = 32;
info = repmat(struct, ceil(size(M, 1) / N), ceil(size(M, 2) / N));
for row = 1:N:size(M, 1)%loop through each pixel in the image matrix
for col = 1:N:size(M, 2)
r = (row - 1) / N + 1;
c = (col - 1) / N + 1;
imgWindow = M(row:min(end,row+N-1), col:min(end,col+N-1));
largest = max(imgWindow(:));
[rLarg, cLarg] = find(imgWindow == largest, 1, 'first');
window(r, c).largest = largest;
window(r, c).row = rLarg + row - 1;
window(r, c).col = cLarg + col - 1;
end
end
The mean function computes averages
average = mean(imgWindow(:));
You can then save each window average in a similar fashion to what you already did for the window maximum:
window(r, c).average = average;
Related
I have an implementation of a convolution neural network in MATLAB (from the open source DeepLearnToolbox). The following code finds the convolution of different weights and parameters:
z = z + convn(net.layers{l - 1}.a{i}, net.layers{l}.k{i}{j}, 'valid');
To update the tool, I have implemented my own fixed-point scheme based convolution using the following code:
function result = convolution(image, kernal)
% find dimensions of output
row = size(image,1) - size(kernal,1) + 1;
col = size(image,2) - size(kernal,2) + 1;
zdim = size(image,3);
%create output matrix
output = zeros(row, col);
% flip the kernal
kernal_flipped = fliplr(flipud(kernal));
%find rows and col of kernal for loop iteration
row_ker = size(kernal_flipped,1);
col_ker = size(kernal_flipped,2);
for k = 1 : zdim
for i = 0 : row-1
for j = 0 : col-1
sum = fi(0,1,8,7);
prod = fi(0,1,8,7);
for k_row = 1 : row_ker
for k_col = 1 : col_ker
a = image(k_row+i, k_col+j, k);
b = kernal_flipped(k_row,k_col);
prod = a * b;
% convert to fixed point
prod = fi((product/16384), 1, 8, 7);
sum = fi((sum + prod), 1, 8, 7);
end
end
output(i+1, j+1, k) = sum;
end
end
end
result = output;
end
The problem is that when I use my convolution implementation in the bigger application, it is super slow.
Any suggestions how to improve its execution time?
MATLAB doesn't support fixed point 2D convolution, but knowing that convolution can be written as matrix multiplication and that MATLAB has support for fixed point matrix multiplication you can use im2col to convert the image into column format and multiply it by the kernel to convolve them.
row = size(image,1) - size(kernal,1) + 1;
col = size(image,2) - size(kernal,2) + 1;
zdim = size(image,3);
output = zeros(row, col);
kernal_flipped = fliplr(flipud(kernal));
fi_kernel = fi(kernal_flipped(:).', 1, 8, 7) / 16384;
sz = size(kernal_flipped);
sz_img = size(image);
% Use the generated indexes to convert the image into column format
idx_col = im2col(reshape(1:numel(image)/zdim,sz_img(1:2)),sz,'sliding');
image = reshape(image,[],zdim);
for k = 1:zdim
output(:,:,k) = double(fi_kernel * reshape(image(idx_col,k),size(idx_col)));
end
I have the following DP which I am applying on a binarized image (either 0 or 1) in Matlab
[x, y] = size(img);
dp = zeros(x, y);
dp(1,:) = img(1,:);
dp(:,1) = img(:,1);
for i = 2:x
for j = 2:y
if img(i, j) == 0
dp(i, j) = min([dp(i, j - 1), dp(i - 1, j), dp(i - 1, j - 1)]) + 1;
end
end
end
The code for large x and y takes a lot of time maybe because of the if condition and using for loops instead of writing vectorized code.
Can anyone optimize it.?
Or is there any approach which optimizes the above code by exploiting the fact that the matrix img contains either 0 or 1 (fewer 1s than 0s).
Also is it possible to somehow use parallel for loops to speed up.?
As far as I am aware, you cannot really speed up this computation in general. But if you know that there are only very few entries where img(i,j)==0 following approach might save you a little bit of time:
[x, y] = size(img);
dp = zeros(x, y);
dp(1,:) = img(1,:);
dp(:,1) = img(:,1);
[i, j] = find(img(2:end, 2:end) == 0); % Extract only these pixels where we actually need to do something
i = i + 1; %correct for removing the first row and column
j = j + 1;
for k = 1:numel(i);
dp(i(k), j(k)) = min([dp(i(k), j(k) - 1), dp(i(k) - 1, j(k)), dp(i(k) - 1, j(k) - 1)]) + 1;
end
I have a video and I have made a Sobel mask for it on MATLAB. Now I have to apply that Sobel mask on each frame of the video by reading each frame through for loop. The process is something like:
Step 1: Reading frame.
step 2: Converting it to grayscale using rgb2gray.
Step 3: Converting it to double.
Here, after applying the mask when I try to write the frame on the resultant video.avi file, I get the following error:
"Frames of type double must be in the range of 0 to 1"
What is wrong with my code? The code I wrote is shown below:
vid = VideoReader('me.mp4');
frames = read(vid);
total = get(vid, 'NumberOfFrames');
write = VideoWriter('me.avi');
open(write);
mask1 = [-1 -2 -1; 0 0 0; 1 2 1]; % Horizontal mask
mask2 = [-1 0 1; -2 0 2; -1 0 1]; %Vertical Mask
for k = 1 : 125
image = frames(:,:,:,k);
obj = image;
obj1 = rgb2gray(obj);
obj2=double(obj1);
for row = 2 : size(obj2, 1) - 1
for col = 2 : size(obj2, 2) - 1
c1 = obj2(row - 1, col - 1) * mask1(1 ,1);
c2 = obj2(row - 1, col) * mask1(1 ,2);
c3 = obj2(row - 1, col + 1) * mask1(1 ,3);
c4 = obj2(row, col - 1)*mask1(2, 1);
c5 = obj2(row, col)*mask1(2, 2);
c6 = obj2(row, col + 1)*mask1(2, 3);
c7 = obj2(row + 1, col - 1)*mask1(3,1);
c8 = obj2(row + 1, col)*mask1(3,2);
c9 = obj2(row + 1, col + 1)*mask1(3,3);
c11 = obj2(row - 1, col - 1)*mask2(1 , 1);
c22 = obj2(row, col - 1)*mask2(2, 1);
c33 = obj2(row + 1, col - 1)*mask2(3, 1);
c44 = obj2(row -1, col)*mask2(1, 2);
c55 = obj2(row, col)*mask2(2 , 2);
c66 = obj2(row +1, col)*mask2(2 , 3);
c77 = obj2(row - 1, col + 1)*mask2(1 , 3);
c88 = obj2(row, col +1)*mask2(2 , 3);
c99 = obj2(row + 1, col + 1)*mask2(3 , 3);
result = c1 + c2 + c3 +c4 +c5+ c6+ c7+ c8 +c9;
result2 = c11 + c22 + c33 + c44 + c55 + c66 + c77 + c88 + c99;
%result = double(result);
%result2 = double(result2);
rim1(row, col) = ((result^2+result2^2) *1/2);
rim2(row, col) = atan(result/result2);
end
end
writeVideo(write, rim2); %This line has the problem with rim2 as rim2 is the frame i'm trying to write on the video file.
end
close(write);
rim2 has range [-pi/2, pi/2] at the end, which is not compatible with the write function which expects [0,1] range.
Convert it to [0,1] range using the mat2gray function, i.e.
writeVideo(write, mat2gray(rim2));
Your code will then work as expected (confirmed on my machine).
By the way, this doesn't affect your code, but presumably you meant to do im2double(A) rather than double(A). The former produces a "proper" grayscale image in the range [0,1], whereas the latter simply converts your uint8 image in the range [0,255] to double format (i.e. [0.0, 255.0]).
The line of rim2 inside your double for loop is using atan, which will generate values that are both positive and negative - from -pi/2 to +pi/2 exactly. rim2 is expected to have values that are only between [0,1]. I can't figure out what exactly you're doing, but it looks like you're calculating the magnitude and gradient angle at each pixel location. If you want to calculate the magnitude, you have to take the square root of the result, not simply multiply by 1/2. The calculation of the gradient (... or even the whole Sobel filter calculation...) is very funny.
I'll just assume this is working for your purposes so I'm not sure how to change the output of rim2for suitable display but perhaps you could scale it to the range of [0,1] before you write the video so that it's within this range.
Something like this would work before you write the frame:
rim2 = (rim2 - min(rim2(:))) / (max(rim2(:)) - min(rim2(:)));
writeVideo(write, rim2);
The above is your typical min-max normalization that is seen in practice. Specifically, the above will ensure that the smallest value is 0 while the largest value is 1 per frame. If you want to be consistent over all frames, simply add pi/2 then divide by pi. This assumes that the minimum is -1 and the maximum is +1 over all frames however.
rim2 = (rim2 + pi/2) / pi;
writeVideo(write, rim2);
However, I suspect you want to write the magnitude to file, not the angle. Therefore, replace the video writing with rim1 as the frame to write instead of rim2, then normalize after. Make sure your gradient calculation is correct though:
rim1(row, col) = ((result^2+result2^2)^(1/2));
% or use sqrt:
% rim1(row, col) = sqrt(result^2 + result2^2);
Now write to file:
rim1 = (rim1 - min(rim1(:))) / (max(rim1(:)) - min(rim1(:)));
writeVideo(write, rim1);
However, if I can provide a method of efficiency, don't use for loops to compute the gradient and angle. Use conv2 and ensure you use the 'same' flag or imfilter from the image processing toolbox to perform the filtering for you, then calculate the gradient and angle vectorized. Also, convert to grayscale and cast your frame in one go in the main loop. I'll assume you have the image processing toolbox as having the computer vision toolbox (you have this as you're using a VideoWriter object) together with the image processing toolbox is what most people have:
vid = VideoReader('me.mp4');
frames = read(vid);
total = get(vid, 'NumberOfFrames');
write = VideoWriter('me.avi');
open(write);
mask1 = [-1 -2 -1; 0 0 0; 1 2 1]; % Horizontal mask
mask2 = [-1 0 1; -2 0 2; -1 0 1]; %Vertical Mask
for k = 1 : 125
obj2 = double(rgb2gray(frames(:,:,:,k))); % New
grad1 = imfilter(obj2, mask1); % New
grad2 = imfilter(obj2, mask2); % New
rim1 = sqrt(grad1.^2 + grad2.^2); % New
rim2 = atan2(grad1, grad2); % New
% Normalize
rim2 = (rim2 - min(rim2(:))) / (max(rim2(:)) - min(rim2(:)));
writeVideo(write, rim2);
end
close(write);
I am trying to implement a simple pixel level center-surround image enhancement. Center-surround technique makes use of statistics between the center pixel of the window and the surrounding neighborhood as a means to decide what enhancement needs to be done. In the code given below I have compared the center pixel with average of the surrounding information and based on that I switch between two cases to enhance the contrast. The code that I have written is as follows:
im = normalize8(im,1); %to set the range of pixel from 0-255
s1 = floor(K1/2); %K1 is the size of the window for surround
M = 1000; %is a constant value
out1 = padarray(im,[s1,s1],'symmetric');
out1 = CE(out1,s1,M);
out = (out1(s1+1:end-s1,s1+1:end-s1));
out = normalize8(out,0); %to set the range of pixel from 0-1
function [out] = CE(out,s,M)
B = 255;
out1 = out;
for i = s+1 : size(out,1) - s
for j = s+1 : size(out,2) - s
temp = out(i-s:i+s,j-s:j+s);
Yij = out1(i,j);
Sij = (1/(2*s+1)^2)*sum(sum(temp));
if (Yij>=Sij)
Aij = A(Yij-Sij,M);
out1(i,j) = ((B + Aij)*Yij)/(Aij+Yij);
else
Aij = A(Sij-Yij,M);
out1(i,j) = (Aij*Yij)/(Aij+B-Yij);
end
end
end
out = out1;
function [Ax] = A(x,M)
if x == 0
Ax = M;
else
Ax = M/x;
end
The code does the following things:
1) Normalize the image to 0-255 range and pad it with additional elements to perform windowing operation.
2) Calls the function CE.
3) In the function CE obtain the windowed image(temp).
4) Find the average of the window (Sij).
5) Compare the center of the window (Yij) with the average value (Sij).
6) Based on the result of comparison perform one of the two enhancement operation.
7) Finally set the range back to 0-1.
I have to run this for multiple window size (K1,K2,K3, etc.) and the images are of size 1728*2034. When the window size is selected as 100, the time consumed is very high.
Can I use vectorization at some stage to reduce the time for loops?
The profiler result (for window size 21) is as follows:
The profiler result (for window size 100) is as follows:
I have changed the code of my function and have written it without the sub-function. The code is as follows:
function [out] = CE(out,s,M)
B = 255;
Aij = zeros(1,2);
out1 = out;
n_factor = (1/(2*s+1)^2);
for i = s+1 : size(out,1) - s
for j = s+1 : size(out,2) - s
temp = out(i-s:i+s,j-s:j+s);
Yij = out1(i,j);
Sij = n_factor*sum(sum(temp));
if Yij-Sij == 0
Aij(1) = M;
Aij(2) = M;
else
Aij(1) = M/(Yij-Sij);
Aij(2) = M/(Sij-Yij);
end
if (Yij>=Sij)
out1(i,j) = ((B + Aij(1))*Yij)/(Aij(1)+Yij);
else
out1(i,j) = (Aij(2)*Yij)/(Aij(2)+B-Yij);
end
end
end
out = out1;
There is a slight improvement in the speed from 93 sec to 88 sec. Suggestions for any other improvements to my code are welcomed.
I have tried to incorporate the suggestions given to replace sliding window with convolution and then vectorize the rest of it. The code below is my implementation and I'm not getting the result expected.
function [out_im] = CE_conv(im,s,M)
B = 255;
temp = ones(2*s,2*s);
temp = temp ./ numel(temp);
out1 = conv2(im,temp,'same');
out_im = im;
Aij = im-out1; %same as Yij-Sij
Aij1 = out1-im; %same as Sij-Yij
Mij = Aij;
Mij(Aij>0) = M./Aij(Aij>0); % if Yij>Sij Mij = M/Yij-Sij;
Mij(Aij<0) = M./Aij1(Aij<0); % if Yij<Sij Mij = M/Sij-Yij;
Mij(Aij==0) = M; % if Yij-Sij == 0 Mij = M;
out_im(Aij>=0) = ((B + Mij(Aij>=0)).*im(Aij>=0))./(Mij(Aij>=0)+im(Aij>=0));
out_im(Aij<0) = (Mij(Aij<0).*im(Aij<0))./ (Mij(Aij<0)+B-im(Aij<0));
I am not able to figure out where I'm going wrong.
A detailed explanation of what I'm trying to implement is given in the following paper:
Vonikakis, Vassilios, and Ioannis Andreadis. "Multi-scale image contrast enhancement." In Control, Automation, Robotics and Vision, 2008. ICARCV 2008. 10th International Conference on, pp. 856-861. IEEE, 2008.
I've tried to see if I could get those times down by processing with colfiltand nlfilter, since both are usually much faster than for-loops for sliding window image processing.
Both worked fine for relatively small windows. For an image of 2048x2048 pixels and a window of 10x10, the solution with colfilt takes about 5 seconds (on my personal computer). With a window of 21x21 the time jumped to 27 seconds, but that is still a relative improvement on the times displayed on the question. Unfortunately I don't have enough memory to colfilt using windows of 100x100, but the solution with nlfilter works, though taking about 120 seconds.
Here the code
Solution with colfilt:
function outval = enhancematrix(inputmatrix,M,B)
%Inputmatrix is a 2D matrix or column vector, outval is a 1D row vector.
% If inputmatrix is made of integers...
inputmatrix = double(inputmatrix);
%1. Compute S and Y
normFactor = 1 / (size(inputmatrix,1) + 1).^2; %Size of column.
S = normFactor*sum(inputmatrix,1); % Sum over the columns.
Y = inputmatrix(ceil(size(inputmatrix,1)/2),:); % Center row.
% So far we have all S and Y, one value per column.
%2. Compute A(abs(Y-S))
A = Afunc(abs(S-Y),M);
% And all A: one value per column.
%3. The tricky part. If Y(i)-S(i) > 0 do something.
doPositive = (Y > S);
doNegative = ~doPositive;
outval = zeros(1,size(inputmatrix,2));
outval(doPositive) = (B + A(doPositive) .* Y(doPositive)) ./ (A(doPositive) + Y(doPositive));
outval(doNegative) = (A(doNegative) .* Y(doNegative)) ./ (A(doNegative) + B - Y(doNegative));
end
function out = Afunc(x,M)
% Input x is a row vector. Output is another row vector.
out = x;
out(x == 0) = M;
out(x ~= 0) = M./x(x ~= 0);
end
And to call it, simply do:
M = 1000; B = 255; enhancenow = #(x) enhancematrix(x,M,B);
w = 21 % windowsize
result = colfilt(inputImage,[w w],'sliding',enhancenow);
Solution with nlfilter:
function outval = enhanceimagecontrast(neighbourhood,M,B)
%1. Compute S and Y
normFactor = 1 / (length(neighbourhood) + 1).^2;
S = normFactor*sum(neighbourhood(:));
Y = neighbourhood(ceil(size(neighbourhood,1)/2),ceil(size(neighbourhood,2)/2));
%2. Compute A(abs(Y-S))
test = (Y>=S);
A = Afunc(abs(Y-S),M);
%3. Return outval
if test
outval = ((B + A) * Y) / (A + Y);
else
outval = (A * Y) / (A + B - Y);
end
function aval = Afunc(x,M)
if (x == 0)
aval = M;
else
aval = M/x;
end
And to call it, simply do:
M = 1000; B = 255; enhancenow = #(x) enhanceimagecontrast(x,M,B);
w = 21 % windowsize
result = nlfilter(inputImage,[w w], enhancenow);
I didn't spend much time checking that everything is 100% correct, but I did see some nice contrast enhancement (hair looks particularly nice).
This answer is the implementation that was suggested by Peter. I debugged the implementation and presenting the final working version of the fast implementation.
function [out_im] = CE_conv(im,s,M)
B = 255;
im = ( im - min(im(:)) ) ./ ( max(im(:)) - min(im(:)) )*255;
h = ones(s,s)./(s*s);
out1 = imfilter(im,h,'conv');
out_im = im;
Aij = im-out1; %same as Yij-Sij
Aij1 = out1-im; %same as Sij-Yij
Mij = Aij;
Mij(Aij>0) = M./Aij(Aij>0); % if Yij>Sij Mij = M/(Yij-Sij);
Mij(Aij<0) = M./Aij1(Aij<0); % if Yij<Sij Mij = M/(Sij-Yij);
Mij(Aij==0) = M; % if Yij-Sij == 0 Mij = M;
out_im(Aij>=0) = ((B + Mij(Aij>=0)).*im(Aij>=0))./(Mij(Aij>=0)+im(Aij>=0));
out_im(Aij<0) = (Mij(Aij<0).*im(Aij<0))./ (Mij(Aij<0)+B-im(Aij<0));
out_im = ( out_im - min(out_im(:)) ) ./ ( max(out_im(:)) - min(out_im(:)) );
To call this use the following code
I = imread('pout.tif');
w_size = 51;
M = 4000;
output = CE_conv(I(:,:,1),w_size,M);
The output for the 'pout.tif' image is given below
The execution time for Bigger image and with 100*100 block size is around 5 secs with this implementation.
I am working with simulation of wireless sensor networks in matlab.
I have a 200*200 by field in which 100 sensor nodes have been plotted randomly. Each node has an associated load value with it. I have to place charging stations in this field. I am trying to divide this square recursively as long as I do not found a small sub-square in which I can place only one charging station. Here is the code I wrote to divide the square recursively and count number of stations that can be placed in a subsquare:
%Inputs to the function
%numstations - No. of stations to be placed = 10
%boundCoords - A 2*2 matrix with min and max coordinates of square . e.g [0 0;200 200]
% sensors - A 100*3 matrix for nodes with 1st column as randomly generated 100 x-coordinates,
%second column as randomly generated 100 y-coordinates,
%third column as corresponding load of each node (can be random)
function stationPoss = deploy(numStations, boundCoords)
global sensors;
centerCoord = mean(boundCoords, 1);
numSensors = size(sensors, 1);
sumQuadLoad = zeros(1, 4);
for i = 1:numSensors
if sensors(i, 1) < boundCoords(2, 1) && sensors(i, 2) < boundCoords(2, 2)...
&& sensors(i, 1) > boundCoords(1, 1) && sensors(i, 2) > boundCoords(1, 2)
isIn34Quads = sensors(i, 1) > centerCoord(1); % N
isIn24Quads = sensors(i, 2) > centerCoord(2);
biQuadIndex = [isIn34Quads, isIn24Quads];
quadIndex = bi2de(biQuadIndex) + 1;
sumQuadLoad(quadIndex) = sumQuadLoad(quadIndex) + sensors(i, 3);
end
end
if numStations == 1
[maxQuadLoad, quad] = max(sumQuadLoad); %#ok<ASGLU>
delta = (centerCoord - boundCoords(1, :)) .* de2bi(quad - 1);
assoQuadCoords = [boundCoords(1, :); centerCoord] + repmat(delta, 2, 1);
stationPoss = mean(assoQuadCoords, 1);
else
sumLoad = sum(sumQuadLoad);
quadNumStations = zeros(1, 4);
for i = 1:3
if sumQuadLoad(i) == 0
quadNumStations(i) = 0;
else
quadNumStations(i) = floor(numStations * sumQuadLoad(i) / sumLoad);
end
end
quadNumStations(4) = numStations - sum(quadNumStations);
stationPoss = zeros(numStations, 2);
for i = 1:4
delta = (centerCoord - boundCoords(1, :)) .* de2bi(i - 1);
newBoundCoords = [boundCoords(1, :); centerCoord] + repmat(delta, 2, 1);
if quadNumStations(i) ~= 0
indexRange = sum(quadNumStations(1:i-1)) + (1:quadNumStations(i));
stationPoss(indexRange, :) = deploy(quadNumStations(i), newBoundCoords);
end
end
end
The problem is while trying to run this code with numStations=2 it works fine and with numStations=3 it sometimes crashes. For numStation > 3 it almost always crashes.
I tried to come up with a non-recursive way to write this function but wasn't able to.
Will anyone please help me to figure out the crash problem or in writing non recursive solution to the above function. I have already tried increasing the recursion limit.