I have a matrix X that represents an image that was affected by noise. I also have a boolean matrix M that represents which pixels were affected by noise. What I want to do is to set every 'corrupted' pixel to the mean of its eight neighboring pixels.
Corrupted pixels are guaranteed to always be surrounded by uncorrupted ones, and also none of the pixels on the borders of the image are corrupted. What function can I used to write a vectorised version of this?
For your situation, this should perform quite fast
fixed = conv2 (image, [1 1 1; 1 0 1; 1 1 1]/8, "same")
# mask is a logical matrix for the corrupted pixels
image(mask) = fixed(mask)
Explanation: a mean filter is done with the conv2 function. To calculate the average of a pixel and its neighbors, the kernel used is ones (3) / 9 which means that 1/9 of each pixel value is used to calculate the new value. Since you don't want to count the center pixel in the average, you make its value 0 (in the kernel), and the others to 1/8.
This is probably not the most effective solution, but it should work.
N = size(M, 1);
target_ind = find(M);
offset = [-N-1, -N, -N+1, -1, 0, 1, N-1, N, N+1];
area_ind = bsxfun(#plus, offset, target_ind);
X(target_ind) = median(X(area_ind), 2);
Since all corrupted pixels are guaranteed to be surrounded by pixels, we can rather easily compute the linear indices of each corrupted pixel's neighbors. Here I've assumed that X is a grayscale image.
If I has more than one channel, then we could loop over each channel and add an offset to
target_ind and area_ind each time:
for i = 1:size(X, 3)
chan_offset = (i - 1)*size(X, 1)*size(X, 2) % Add the number of elements in previous channels to get indices in the current channel
X(target_ind + chan_offset) = median(X(area_ind + chan_offset), 2);
end
Related
I'm trying to apply bare-bones image processing to images like this: My for-loop does exactly what I want it to: it allows me to find the pixels of highest intensity, and also remember the coordinates of that pixel. However, the code breaks whenever it encounters a multiple of rows – which in this case is equal to 18.
For example, the length of this image (rows * columns of image) is 414. So there are 414/18 = 23 cases where the program fails (i.e., the number of columns).
Perhaps there is a better way to accomplish my goal, but this is the only way I could think of sorting an image by pixel intensity while also knowing the coordinates of each pixel. Happy to take suggestions of alternative code, but it'd be great if someone had an idea of how to handle the cases where mod(x,18) = 0 (i.e., when the index of the vector is divisible by the total # of rows).
image = imread('test.tif'); % feed program an image
image_vector = image(:); % vectorize image
[sortMax,sortIndex] = sort(image_vector, 'descend'); % sort vector so
%that highest intensity pixels are at top
max_sort = [];
[rows,cols] = size(image);
for i=1:length(image_vector)
x = mod(sortIndex(i,1),rows); % retrieve original coordinates
% of pixels from matrix "image"
y = floor(sortIndex(i,1)/rows) +1;
if image(x,y) > 0.5 * max % filter out background noise
max_sort(i,:) = [x,y];
else
continue
end
end
You know that MATLAB indexing starts at 1, because you do +1 when you compute y. But you forgot to subtract 1 from the index first. Here is the correct computation:
index = sortIndex(i,1) - 1;
x = mod(index,rows) + 1;
y = floor(index/rows) + 1;
This computation is performed by the function ind2sub, which I recommend you use.
Edit: Actually, ind2sub does the equivalent of:
x = rem(sortIndex(i,1) - 1, rows) + 1;
y = (sortIndex(i,1) - x) / rows + 1;
(you can see this by typing edit ind2sub. rem and mod are the same for positive inputs, so x is computed identically. But for computing y they avoid the floor, I guess it is slightly more efficient.
Note also that
image(x,y)
is the same as
image(sortIndex(i,1))
That is, you can use the linear index directly to index into the two-dimensional array.
I have a MxNx3 matrix that represents an RGB image. I am trying to retrieve, for each pixel, the maximum among R, G and B. This would be made easy by using a for loop, which I do not wish to do for performance reasons.
How could I go about doing that? My idea is to use find and max in the following way and get an MxN matrix:
maxRGB = find(max(rgbImage(i, j, :)));
But I am not sure how I could eliminate i and j.
The max function allows to specify along which dimension the maximum value is determined. The standard value is the first dimension. In your case, you'll want to calculate the maximum along the third dimension of the array:
maxValue = max(rgbImage,[],3);
Which returns a matrix of size MxN containing the maximum value of each pixel.
For example, lets take a random 3x3 RGB image. Applying the max function as above yields
rgbImage = rand(3,3,3);
maxValue = max(rgbImage,[],3);
maxValue =
0.6948 0.7094 0.7655
0.6555 0.7547 0.7952
0.9502 0.3816 0.8235
These are the maximal values which were present in rgbImage at each pixel location. But, you don't know if this value was in the R, G or B pixel.
To find out, which color was maximal, you can use the second (optional) argument of max, which is the index of the found maximum:
[~,maxIndex] = max(rgbImage,[],3);
which in this small example was
maxIndex =
2 3 2
1 3 2
2 2 1
where 1 corresponds to R, 2 corresponds to G and 3 corresponds to B.
To find all pixels, in which the red component was the largest, you can use the find function (probably with 2 output arguments: row and column)
[xRed,yRed] = find(maxIndex == 1)
xRed =
2
3
yRed =
1
3
So for the pixels at (2,1) and at (3,3) the red component was the largest. This is exactly what the matrix maxIndex also shows us.
I am implementing a simple algorithm to do in-painting on a "damaged" image. I have a predefined mask that specifies the area which needs to be fixed. My strategy is to start at the border of the masked area and in-paint each pixel with the central mean of its neighboring non-zero pixels, repeating until there's no unknown pixels left.
function R = inPainting(I, mask)
H = [1 2 1; 2 0 2; 1 2 1];
R = I;
n = 1;
[row,col,~] = find(~mask); %Find zeros in mask (area to be inpainted)
unknown = horzcat(row, col)';
while size(unknown,2) > 0
new_unknown = [];
new_R = R;
for u = unknown
r = u(1);
c = u(2);
nb = R(max((r-n), 1):min((r+n), end), max((c-n),1):min((c+n),end));
nz = nb~=0;
nzs = sum(nz(:));
if nzs ~= 0 %We have non-zero neighbouring pixels. In-paint with average.
new_R(r,c) = sum(nb(:)) / nzs;
else
new_unknown = horzcat(new_unknown, u);
end
end
unknown = new_unknown;
R = new_R;
end
This works well, but it's not very efficient. Is it possible to vectorize such an approach, using mostly matrix operations? Does someone know of a more efficient way to implement this algorithm?
If I understand your problem statement, you are given a mask and you wish to fill in these pixels in this mask with the mean of the neighbourhood pixels that surround each pixel in the mask. Another constraint is that the image is defined such that any pixels that belong to the mask in the same spatial locations are zero in this mask. You are starting from the border of the mask and are propagating information towards the innards of the mask. Given this algorithm, there is unfortunately no way you can do this with standard filtering techniques as the current time step is dependent on the previous time step.
Image filtering mechanisms, like imfilter or conv2 can't work here because of this dependency.
As such, what I can do is help you speed up what is going on inside your loop and hopefully this will give you some speed up overall. I'm going to introduce you to a function called im2col. This is from the image processing toolbox, and given that you can use imfilter, we can use this function.
im2col creates a 2D matrix such that each column is a pixel neighbourhood unrolled into a single vector. How it works is that each pixel neighbourhood in column major order is grabbed, so we get a pixel neighbourhood at the top left corner of the image, then move down one row, and another row and we keep going until we reach the last row. We then move one column over and repeat the same process. For each pixel neighbourhood that we have, it gets unrolled into a single vector, and the output would be a MN x K matrix where you have a neighbourhood size of M x N for each pixel neighbourhood and there are K neighbourhoods.
Therefore, at each iteration of your loop, we can unroll the current inpainted image's pixel neighbourhoods into single vectors, determine which pixel neighborhoods are non-zero and from there, determine how many zero values there are for each of these selected pixel neighbourhood. After, we compute the mean for these non-zero columns disregarding the zero elements. Once we're done, we update the image and move to the next iteration.
What we're going to need to do first is pad the image with a 1 pixel border so that we're able to grab neighbourhoods that extend beyond the borders of the image. You can use padarray, also from the image processing toolbox.
Therefore, we can simply do this:
function R = inPainting(I, mask)
R = double(I); %// For precision
n = 1;
%// Change - column major indices
unknown = find(~mask); %Find zeros in mask (area to be inpainted)
%// Until we have searched all unknown pixels
while numel(unknown) ~= 0
new_R = R;
%// Change - take image at current iteration and
%// create columns of pixel neighbourhoods
padR = padarray(new_R, [n n], 'replicate');
cols = im2col(padR, [2*n+1 2*n+1], 'sliding');
%// Change - Access the right pixel neighbourhoods
%// denoted by unknown
nb = cols(:,unknown);
%// Get total sum of each neighbourhood
nbSum = sum(nb, 1);
%// Get total number of non-zero elements per pixel neighbourhood
nzs = sum(nb ~= 0, 1);
%// Replace the right pixels in the image with the mean
new_R(unknown(nzs ~= 0)) = nbSum(nzs ~= 0) ./ nzs(nzs ~= 0);
%// Find new unknown pixels to look at
unknown = unknown(nzs == 0);
%// Update image for next iteration
R = new_R;
end
%// Cast back to the right type
R = cast(R, class(I));
I have an image include 4 values {3,-3,1,-1} as figure
Let call the index of pixel that its values equals 1 or -1 pixel in contour. These pixels will create a contour that surrounds the yellow color (-3). Now, I want to find all index pixels in the contour and plus padding position inward and outward contour. As the red color, padding is set 1, hence, the index of these pixels include pixel in the contour {1,-1} and padding index as the red color. In that task, I want to find all pixel indices. How to implement that idea in matlab code. This is my code to find the index in the contour
%% Let define the image I
idx=find(I==1|I==-1);
padding=1;
%%Continue
Update: My expected result as the above figure in white region. Hence, the indices are such as 13,14,15,..21,24,...
UPDATE
Firstly, thank Andrew and Rayryeng for your answer. I would like to extedn my problem. As the above description, the contour is created by {1,-1}. Now, I want to ignore 1 and -1, so the image I only has {3,-3}. And I defined the contour is pixel in the edge of {3,-3} such as figure. Keep the same idea of padding and pixel index. How to find the indices of pixels in contour and near contour (call narrow band of contour)(expected result is white color)
Not too difficult you are on the right track. If you have the image processing toolbox, I recommend taking a look at morphological operators. Specifically you want to use imdilate my code has all the details you need.
%rather than using find, we create a binary mask. Its not the indicies of
%the matching elements as find gives. its is 1/true if the value matches the
%criteria, and 0/false otherwise.
mask = (im=1 | im=-1);
%create a 3x3 rectangle structuring element. We use a 3x3 because we want
%to expand the image by one pixel. basically the structring element (Strel)
%is our kernal, if you know image processing this is the same thing.
%a = [0 0 0 0;
% 0 1 1 1;
% 0 1 1 1;
% 0 1 1 1];
%our kernal is center at 2,2 (for this example) which are these elements
% 0 0 0 of a think a(1:3,1:3) now what the dialate operation
% 0 1 1 says is, if the majority of these pixels are ones... they
% 0 1 1 should probabaly all be ones so all those 0s will become ones
%the size of the kernal 3x3 ensures we are only growing our image one
%pixel, hope that makes sense
se = strel('square',3);
%now we dilate, or 'expand' our mask with our structuring element
expanded_mask = imdilate(mask,se);
%if you still want the indicies you can use find on our expanded mask
idx = find(expanded_mask==1);
EDIT: without morphological operations/image processing toolbox
This method uses lots of for loops, so it isn't the fastest, and doens't do error checking, but it will work. My dilate function says if the majority of the pixels are ones make them all ones.
function expanded_mask=DilateBinaryImage(bin_im, kernal_size)
[max_row,max_col] = size(bin_im);
%since we are opening the mask (only adding 1s), we can start off with the
%same values of the mask, and simply add extra 1's as needed
expanded_mask = bin_im;
%we don't want to go off the edge of our image with this kernal
%so we offset it a bit
kern_padding = floor(kernal_size/2);
%this ignores the edges
for (curr_row=kern_padding+1:1:max_row - kern_padding)
for (curr_col=kern_padding+1:1:max_col - kern_padding)
%we do 2 sums, one for rows, one for columns
num_ones = sum(sum(bin_im(curr_row-kern_padding:curr_row+kern_padding,curr_col-kern_padding:curr_col+kern_padding)));
%if the majority of vlaues are 1, we use floor to help with corner
%cases
if (num_ones >= floor((kernal_size*kernal_size)/2))
%make all the values one
expanded_mask(curr_row-kern_padding:curr_row+kern_padding,curr_col-kern_padding:curr_col+kern_padding) = 1;
end
end
end
end
and then called it like this
kernal_size= 3;
mask = (I==1 | I==-1);
expanded_mask = DilateBinaryImage(mask, kernal_size);
idx = find(expanded_mask==1);
my dilate function doesn't work at the edges of the binary image. it just copies them exactly.
Lets say your image is N-by-M pixles. In MATLAB arrays are stored in column order (see http://www.mathworks.com/help/matlab/math/matrix-indexing.html for more information). You can use I in column format as follows. First you contour pixels are given by
idx=find(I(:)==1|I(:)==-1);
Now, if you wish to pad downward and upward it is quite simple:
idx_up=idx - padding;
idx_up = idx_up(idx_up>0);
idx_down=idx + padding;
idx_down = idx_down(idx_down<=N*M);
Note that idx_up and idx_down will also contain contour pixels.
Similarly you can pad to the left\right:
idx_left=idx - padding*N;
idx_left = idx_left(idx_left>0);
idx_right=idx + padding*N;
idx_right = idx_right(idx_right<=N*M);
And combine the overall pixels:
PaddedContour = false(N,M);
PaddedContour(unique([idx;idx_up;idx_down;idx_left;idx_right])) = true;
I wonder whether it would be possible to extract only hands from a video with matlab. In the video hands perform some gesture. Because first frames are only background I tried in this way:
readerObj = VideoReader('VideoWithHands.mp4');
nFrames = readerObj.NumberOfFrames;
fr = get(readerObj, 'FrameRate');
writerObj = VideoWriter('Hands.mp4', 'MPEG-4');
set(writerObj, 'FrameRate', fr);
open(writerObj);
bg = read(readerObj, 1); %background
for k = 1 : nFrames
frame = read(readerObj, k);
hands = imabsdiff(frame,bg);
writeVideo(writerObj,hands);
end
close(writerObj);
But I realized that colors of the hands are not "real" and they are transparent. Is there a better way to extract them from video keeping colors and opacity level exploiting the first frames (background)?
EDIT: Well, I have found a good setting for vision.ForegroundDetector object, now hands are white logical regions but when I try to visualize them with:
videoSource = vision.VideoFileReader('VideoWithHands.mp4', 'VideoOutputDataType', 'uint8');
detector = vision.ForegroundDetector('NumTrainingFrames', 46, 'InitialVariance', 4000, 'MinimumBackgroundRatio', 0.2);
videoplayer = vision.VideoPlayer();
hands = uint8(zeros(720,1280,3));
while ~isDone(videoSource)
frame = step(videoSource);
fgMask = step(detector, frame);
[m,n] = find(fgMask);
a = [m n];
if isempty(a)==true
hands(:,:,:) = uint8(zeros(720,1280,3));
else
hands(m,n,1) = frame(m,n,1);
hands(m,n,2) = frame(m,n,2);
hands(m,n,3) = frame(m,n,3);
end
step(videoplayer, hands)
end
release(videoplayer)
release(videoSource)
or put them into a videofile with:
eaderObj = VideoReader('Video 9.mp4');
nFrames = readerObj.NumberOfFrames;
fr = get(readerObj, 'FrameRate');
writerObj = VideoWriter('hands.mp4', 'MPEG-4');
set(writerObj, 'FrameRate', fr);
detector = vision.ForegroundDetector('NumTrainingFrames', 46, 'InitialVariance', 4000, 'MinimumBackgroundRatio', 0.2);
open(writerObj);
bg = read(readerObj, 1);
frame = uint8(zeros(size(bg)));
for k = 1 : nFrames
frame = read(readerObj, k);
fgMask = step(detector, frame);
[m,n] = find(fgMask);
hands = uint8(zeros(720,1280));
if isempty([m n]) == true
hands(:,:) = uint8(zeros(720,1280));
else
hands(m,n) = frame(m,n);
end
writeVideo(writerObj,mani);
end
close(writerObj);
...my PC crashes. Some suggestion?
So you're trying to cancel out the background, making it black, right?
The easiest way to do this should be to filter it, you can do that by comparing your difference data to a threshold value and then using the result as indices to set a custom background.
filtered = imabsdiff(frame,bg);
bgindex = find( filtered < 10 );
frame(bgindex) = custombackground(bgindex);
where custombackground is whatever image file you want to put into the background. If you want it to be just black or white, use 0 or 255 instead of custombackground(bgindex). Note that the numbers depend on your video data's format and could be inaccurate (except 0, this one should always be right). If too much gets filtered out, lower the 10 above, if too much remains unfiltered, increase the 10.
At the end, you write your altered frame back into the video, so it just replaces the hands variable in your code.
Also, depending on your format, you might have to do the comparison across RGB values. This is slightly more complicated as it involves checking 3 values at the same time and doing some magic with the indices. This is the RGB version (works with anything containing 3 color bands):
filtered = imabsdiff(frame,bg); % differences at each pixel in each color band
totalfiltered = sum(filtered,3); % sums up the differences
% in each color band (RGB)
bgindex = find( totalfiltered < 10 ); % extracts indices of pixels
% with color close to bg
allind = sub2ind( [numel(totalfiltered),3] , repmat(bgindex,1,3) , ...
repmat(1:3,numel(bgindex),1) ); % index magic
frame(allind) = custombackground(allind); % copy custom background into frame
EDIT :
Here's a detailed explanation of the index magic.
Let's assume a 50x50 image. Say the pixel at row 2, column 5 is found to be background, then bgindex will contain the number 202 (linear index corresponding to [2,5] = (5-1)*50+2 ). What we need is a set of 3 indices corresponding to the matrix coordinates [2,5,1], [2,5,2] and [2,5,3]. That way, we can change all 3 color bands corresponding to that pixel. To make calculations easier, this approach actually assumes linear indexing for the image and thus converts it to a 2500x1 image. Then it expands the 3 color bands, creating a 2500x3 matrix. We now construct the indices [202,1], [202,2] and [202,3] instead.
To do that, we first construct a matrix of indices by repeating our values. repmat does this for us, it creates the matrices [202 202 202] and [1 2 3]. If there were more pixels in bgindex, the first matrix would contain more rows, each repeating the linear pixel coordinates 3 times. The second matrix would contain additional [1 2 3] rows. The first argument to sub2ind is the size of the matrix, in this case, 2500x3, so we calculate the number of pixels with numel applied to the sum vector (which collapses the image's 3 bands into 1 value and thus has 1 value per pixel) and add a static 3 in the second dimension.
sub2ind now takes each element from the first matrix as a row index, each corresponding element from the second matrix as a column index and converts them to linear indices into a matrix of the size we determined earlier. In our example, this results in the indices [202 2702 5202]. sub2ind preserves the shape of the inputs, so if we had 10 background pixels, this result would have the size 10x3. But since linear indexing doesn't care about the shape of the index matrix, it just takes all of those values.
To confirm this is correct, let's revert the values in the example. The original image data would have the size 50x50x3. For an NxMxP matrix, a linear index to the subscript [n m p] can be calculated as ind = (p-1)*M*N + (m-1)*N + n. Using our values, we get the following:
[2 5 1] => 202
[2 5 2] => 2702
[2 5 3] => 5202
ind2sub confirms this.
Yes, there is a better way. The computer vision system toolbox includes a vision.ForegroundDetector object that does what you need. It implements the Gaussian Mixture Model algorithm for background subtraction.