Data: Say I have a 2000 rows by 500 column matrix (image)
What I need: Compute the FFT of 64 rows by 10 column chunks of above data. In other words, I want to compute the FFT of 64X10 window that is run across the entire data matrix. The FFT result is used to compute a scalar value (say peak amplitude frequency) which is used to create a new "FFT value" image.
Now, I need the final FFT image to be the same size as the original data (2000 X 500).
What is the fastest way to accomplish this in MATLAB? I am currently using for loops which is relatively slow. Also I use interpolation to size up the final image to the original data size.
As #EitanT pointed out, you can use blockproc for batch block processing of an image J. However you should define your function handle as
fun = #(block_struct) fft2(block_struct.data);
B = blockproc(J, [64 10], fun);
For a [2000 x 500] matrix this will give you a [2000 x 500] output of complex Fourier values, evaluated at sub-sampled pixel locations with a local support (size of the input to FFT) of [64 x 10]. Now, to replace those values with a single, e.g. with the peak log-magnitude, you can further specify
fun = #(block_struct) max(max(log(abs(fft2(block_struct.data)))));
B = blockproc(J, [64 10], fun);
The output then is a [2000/64 x 500/10] output of block-patch values, which you can resize by nearest-neighbor interpolation (or something else for smoother versions) to the desired [2000 x 500] original size
C = imresize(B, [2000 500], 'nearest');
I can include a real image example if it will further help.
Update: To get overlapping blocks you can use the 'Bordersize' option of blockproc by setting the overlap [V H] such that the final windows of size [M + 2*V, N + 2*H] will still be [64, 10] in size. Example:
fun = #(block_struct) log(abs(fft2(block_struct.data)));
V = 16; H = 3; % overlap values
overlap = [V H];
M = 32; N = 4; % non-overlapping values
B1 = blockproc(J, [M N], fun, 'BorderSize', overlap); % final windows are 64 x 10
However, this will work with keeping the full Fourier response, not the single-value version with max(max()) above.
See also this post for filtering using blockproc:Dealing with “Really Big” Images: Block Processing.
If you want to apply the same function (in your case, the 2-D Fourier transform) on individual distinct blocks in a larger matrix, you can do that with the blkproc function, which is replaced in newer MATLAB releases by blockproc.
However, I infer that you wish to apply apply fft2 on overlapping blocks in a "sliding window" fashion. For this purpose you can use colfilt with the 'sliding' option. Note that the function that we're applying on each block is the fft:
block_size = [64, 10];
temp_size = 5 * block_size;
col_func = #(x)cellfun(#(y)max(max(abs(fft2(y)))), num2cell(x, 1), 'Un', 0);
B = colfilt(A, block_size, 10 * block_size, 'sliding', col_func);
How does this work? colfilt processes the matrix A by rearranging each "sliding" block into a separate column of a new temporary matrix, and then applying the col_func to this new matrix. col_func in turn restores each column into the original block and applies fft2 on it, returning the largest amplitude value for each column.
Important things to note:
Since this mentioned temporary matrix includes all possible "sliding" blocks, memory could be a limitation. Therefore, in order to use less memory in calculations, colfilt breaks up the original matrix A into sub-matrices of temp_size, and performs calculations separately on each. The resulting matrix B is still the same, of course.
Each element in the resulting matrix B is computed from the corresponding block neighborhood. The larger your image is, the more blocks you will need to process, so the computation time will increase geometrically. I believe that you'll have to wait quite a bit until MATLAB finishes processing all sliding windows on your 2000-by-500 matrix.
Related
I am writing a MATLAB script that uses the medfilt1 function. Here is an example using an order of 100:
median_filter_results = medfilt1(my_data, 100);
When trying to export the MATLAB code via codegen, an error message states that medfilt1 is not supported. Looking on the MATLAB documentation website, I can tell that it is not there, while medfilt2 is. This makes me think that the function is probably rather easy to reproduce.
When reading this post, the authors make this comment:
You can use the median() function. Then you just have to put that inside a for loop, which is extremely trivial.
However, I am not entirely sure I know what that means since the median function returns back one number vs a vector of the medfilt1 function. Wikipedia goes a bit further, where they show a sliding window, through which one could use the median function. However, I am not entirelly too certain that this is what MATLAB is doing.
How can I rewrite the medfilt1 function (vector of data and order of 100) in a codegen safe way?
Here is an implementation using sliding window of median in a for loop:
Implementing a sliding window is simple.
There is a small complication regarding the margins.
The implementation pads the margins with zeros (default padding of medfilt1).
Here is the implementation and a test:
n = 100;
%Test using an array of random elements.
A = rand(1, 1000);
B = my_medfilt1(A, n);
%Reference for testing
refB = medfilt1(A, n);
%Display 1 if result of my_medfilt1 is the same as medfilt1
is_equal = all(B == refB)
function y = my_medfilt1(x, n)
%Perform one dimensional median filter in a loop.
%Assume x is one dimensional row vector.
if size(x, 1) > 1
error('x must be a row vector')
end
y = zeros(1, length(x)); %Initialize space for storing resut
%Add n/2 zeros from each side of x (this is the default padding of medfilt1.
x = padarray(x, [0, floor(n/2)], 0, 'both');
%Sliding window
for i = 1:length(y)
y(i) = median(x(i:i+n-1));
end
end
If the 2d filter is supported, you could repurpose it.
x=rand(100,1);
y1=medfilt1(x,11);
y2=medfilt2(x,[11,1]);
all(y1==y2)
Otherwise, read up what a median filter does. It replaces the element with the median of it and it's surrounding neighbors. Size of the neighborhood is your parameter n.
I need to extract image patches of size s x s x 3 around specified 2D locations from an image (3 channels).
How can I do this efficiently without a for loop? I know I can extract one patch around (x,y) location as:
apatch = I(y-s/2:y+s/2, x-s/2:x+s/2, :)
How can I do this for many patches? I know I can use MATLAB's function blockproc but I can't specify the locations.
You can use im2col from the image processing toolbox to transform each pixel neighbourhood into a single column. The pixel neighbourhoods are selected such that each block is chose on a column-basis, which means that the blocks are constructed by traversing down the rows first, then proceeding to the next column and getting the neighbourhoods there.
You call im2col this way:
B = im2col(A, [M N]);
I'm assuming you'll want sliding / overlapping neighbourhoods and not distinct neighbourhoods, which are what is normally used when performing any kind of image filtering. A is your image and you want to find M x N pixel neighbourhoods transformed as columns. B would be the output where each neighbourhood is a single column and horizontally-tiled together. However, you'll probably want to handle the case where you want to grab pixel neighbourhoods along the borders of the image. In this case, you'll want to pad the image first. We're going to assume that M and N are odd to allow the padding to be easier. Specifically, you want to be sure that there are floor(M/2) rows padded on top of the image as well as the bottom as well as floor(N/2) columns padded to the left of the image as well as the right. As such, we should pad A first by using padarray. Let's assume that the border pixels will be replicated, which means that the padded rows and columns will simply be those grabbed from the top or bottom row, or the left and right column, depending on where we need to pad. Therefore:
Apad = padarray(A, floor([M N]/2), 'replicate');
For the next part, if you want to choose specify neighbourhoods, you can use sub2ind to convert your 2D co-ordinates into linear indices so you can select the right columns to get the right pixel blocks. However, because you have a colour image, you'll want to perform im2col on each colour channel. Unfortunately, im2col only works on grayscale images, and so you'd have to repeat this for each channel in your image.
As such, to get ready for patch sampling, do something like this:
B = arrayfun(#(x) im2col(Apad(:,:,x), [M N]), 1:size(A,3), 'uni', 0);
B = cat(3, B{:});
The above code will create a 3D version of im2col, where each 3D slice would be what im2col produces for each colour channel. Now, we can use sub2ind to convert your (x,y) co-ordinates into linear indices so that we can choose which pixel neighbourhoods we want. Therefore, assuming your positions are stored in vectors x and y, you would do something like this:
%// Generate linear indices
ind = sub2ind([size(A,1) size(A,2)], y, x);
%// Select neighbourhoods
%// Should be shaped as a MN x len(ind) x 3 matrix
neigh = B(:,ind,:);
%// Create cell arrays for each patch
patches = arrayfun(#(x) reshape(B(:,x,:), [M N 3]), 1:numel(ind), 'uni', 0);
patches will be a cell array where each element contains your desired patch at each location of (x,y) that you specify. Therefore, patches{1} would be the patch located at (x(1), y(1)), patches{2} would be the patch located at (x(2), y(2)), etc. For your copying and pasting pleasure, this is what we have:
%// Define image, M and N here
%//...
%//...
Apad = padarray(A, floor([M N]/2), 'replicate');
B = arrayfun(#(x) im2col(Apad(:,:,x), [M N]), 1:size(A,3), 'uni', 0);
B = cat(3, B{:});
ind = sub2ind([size(A,1) size(A,2)], y, x);
neigh = B(:,ind,:);
patches = arrayfun(#(x) reshape(neigh(:,x,:), [M N 3]), 1:numel(ind), 'uni', 0);
As unexpected as this may seem, but for me the naive for-loop is actually the fastest. This might depend on your version of MATLAB though, as with newer versions they keep on improving the JIT compiler.
Common data:
A = rand(30, 30, 3); % Image
I = [5,2,3,21,24]; % I = y
J = [3,7,5,20,22]; % J = x
s = 3; % Block size
Naive approach: (faster than im2col and arrayfun!)
Patches = cell(size(I));
steps = -(s-1)/2:(s-1)/2;
for k = 1:numel(Patches);
Patches{k} = A(I(k)+steps, ...
J(k)+steps, ...
:);
end
Approach using arrayfun: (slower than the loop)
steps = -(s-1)/2:(s-1)/2;
Patches = arrayfun(#(ii,jj) A(ii+steps,jj+steps,:), I, J, 'UniformOutput', false);
I have a randomly defined H matrix of size 600 x 10. Each element in this matrix H can be represented as H(k,t). I obtained a speech spectrogram S which is 600 x 597. I obtained it using Mel features, so it should be 40 x 611 but then I used a frame stacking concept in which I stacked 15 frames together. Therefore it gave me (40x15) x (611-15+1) which is 600 x 597.
Now I want to obtain an output matrix Y which is given by the equation based on convolution Y(k,t) = ∑ H(k,τ)S(k,t-τ). The sum goes from τ=0 to τ=Lh-1. Lh in this case would be 597.
I don't know how to obtain Y. Also, my doubt is the indexing into both H and S when computing the convolution. Specifically, for Y(1,1), we have:
Y(1,1) = H(1,0)S(1,1) + H(1,1)S(1,0) + H(1,2)S(1,-1) + H(1,3)S(1,-2) + ...
Now, there is no such thing as negative indices in MATLAB - for example, S(1,-1) S(1,-2) and so on. So, what type of convolution should I use to obtain Y? I tried using conv2 or fftfilt but I think that will not give me Y because Y must also be the size of S.
That's very easy. That's a convolution on a 2D signal only being applied to 1 dimension. If we assume that the variable k is used to access the rows and t is used to access the columns, you can consider each row of H and S as separate signals where each row of S is a 1D signal and each row of H is a convolution kernel.
There are two ways you can approach this problem.
Time domain
If you want to stick with time domain, the easiest thing would be to loop over each row of the output, find the convolution of each pair of rows of S and H and store the output in the corresponding output row. From what I can tell, there is no utility that can convolve in one dimension only given an N-D signal.... unless you go into frequency domain stuff, but let's leave that for later.
Something like:
Y = zeros(size(S));
for idx = 1 : size(Y,1)
Y(idx,:) = conv(S(idx,:), H(idx,:), 'same');
end
For each row of the output, we perform a row-wise convolution with a row of S and a row of H. I use the 'same' flag because the output should be the same size as a row of S... which is the bigger row.
Frequency domain
You can also perform the same computation in frequency domain. If you know anything about the properties of convolution and the Fourier Transform, you know that convolution in time domain is multiplication in the frequency domain. You take the Fourier Transform of both signals, multiply them element-wise, then take the Inverse Fourier Transform back.
However, you need to keep the following intricacies in mind:
Performing a full convolution means that the final length of the output signal is length(A)+length(B)-1, assuming A and B are 1D signals. Therefore, you need to make sure that both A and B are zero-padded so that they both match the same size. The reason why you make sure that the signals are the same size is to allow for the multiplication operation to work.
Once you multiply the signals in the frequency domain then take the inverse, you will see that each row of Y is the full length of the convolution. To ensure that you get an output that is the same size as the input, you need to trim off some points at the beginning and at the end. Specifically, since each kernel / column length of H is 10, you would have to remove the first 5 and last 5 points of each signal in the output to match what you get in the for loop code.
Usually after the inverse Fourier Transform, there are some residual complex coefficients due to the nature of the FFT algorithm. It's good practice to use real to remove the complex valued parts of the results.
Putting all of this theory together, this is what the code would look like:
%// Define zero-padded H and S matrices
%// Rows are the same, but columns must be padded to match point #1
H2 = zeros(size(H,1), size(H,2)+size(S,2)-1);
S2 = zeros(size(S,1), size(H,2)+size(S,2)-1);
%// Place H and S at the beginning and leave the rest of the columns zero
H2(:,1:size(H,2)) = H;
S2(:,1:size(S,2)) = S;
%// Perform Fourier Transform on each row separately of padded matrices
Hfft = fft(H2, [], 2);
Sfft = fft(S2, [], 2);
%// Perform convolution
Yfft = Hfft .* Sfft;
%// Take inverse Fourier Transform and convert to real
Y2 = real(ifft(Yfft, [], 2));
%// Trim off unnecessary values
Y2 = Y2(:,size(H,2)/2 + 1 : end - size(H,2)/2 + 1);
Y2 should be the convolved result and should match Y in the previous for loop code.
Comparison between them both
If you actually want to compare them, we can. What we'll need to do first is define H and S. To reconstruct what I did, I generated random values with a known seed:
rng(123);
H = rand(600,10);
S = rand(600,597);
Once we run the above code for both the time domain version and frequency domain version, let's see how they match up in the command prompt. Let's show the first 5 rows and 5 columns:
>> format long g;
>> Y(1:5,1:5)
ans =
1.63740867892464 1.94924208172753 2.38365646354643 2.05455605619097 2.21772526557861
2.04478411247085 2.15915645246324 2.13672842742653 2.07661341840867 2.61567534623066
0.987777477630861 1.3969752201781 2.46239452105228 3.07699790208937 3.04588738611503
1.36555260994797 1.48506871890027 1.69896157726456 1.82433906982894 1.62526864072424
1.52085236885395 2.53506897420001 2.36780282057747 2.22335617436888 3.04025523335182
>> Y2(1:5,1:5)
ans =
1.63740867892464 1.94924208172753 2.38365646354643 2.05455605619097 2.21772526557861
2.04478411247085 2.15915645246324 2.13672842742653 2.07661341840867 2.61567534623066
0.987777477630861 1.3969752201781 2.46239452105228 3.07699790208937 3.04588738611503
1.36555260994797 1.48506871890027 1.69896157726456 1.82433906982894 1.62526864072424
1.52085236885395 2.53506897420001 2.36780282057747 2.22335617436888 3.04025523335182
Looks good to me! As another measure, let's figure out what the largest difference is between one value in Y and a corresponding value in Y2:
>> max(abs(Y(:) - Y2(:)))
ans =
5.32907051820075e-15
That's saying that the max error seen between both outputs is in the order of 10-15. I'd say that's pretty good.
I am brand new to MATLAB but am trying to do some image compression code for grayscale images.
Questions
How can I use SVD to trim off low-valued eigenvalues to reconstruct a compressed image?
Work/Attempts so far
My code so far is:
B=imread('images1.jpeg');
B=rgb2gray(B);
doubleB=double(B);
%read the image and store it as matrix B, convert the image to a grayscale
photo and convert the matrix to a class 'double' for values 0-255
[U,S,V]=svd(doubleB);
This allows me to successfully decompose the image matrix with eigenvalues stored in variable S.
How do I truncate S (which is 167x301, class double)? Let's say of the 167 eigenvalues I want to take only the top 100 (or any n really), how do I do that and reconstruct the compressed image?
Updated code/thoughts
Instead of putting a bunch of code in the comments section, this is the current draft I have. I have been able to successfully create the compressed image by manually changing N, but I would like to do 2 additional things:
1- Show a pannel of images for various compressions (i/e, run a loop for N = 5,10,25, etc.)
2- Somehow calculate the difference (error) between each image and the original and graph it.
I am horrible with understanding loops and output, but this is what I have tried:
B=imread('images1.jpeg');
B=rgb2gray(B);
doubleB=im2double(B);%
%read the image and store it as matrix B, convert the image to a grayscale
%photo and convert the image to a class 'double'
[U,S,V]=svd(doubleB);
C=S;
for N=[5,10,25,50,100]
C(N+1:end,:)=0;
C(:,N+1:end)=0;
D=U*C*V';
%Use singular value decomposition on the image doubleB, create a new matrix
%C (for Compression diagonal) and zero out all entries above N, (which in
%this case is 100). Then construct a new image, D, by using the new
%diagonal matrix C.
imshow(D);
error=C-D;
end
Obviously there are some errors because I don't get multiple pictures or know how to "graph" the error matrix
Although this question is old, it has helped me a lot to understand SVD. I have modified the code you have written in your question to make it work.
I believe you might have solved the problem, however just for the future reference for anyone visiting this page, I am including the complete code here with the output images and graph.
Below is the code:
close all
clear all
clc
%reading and converting the image
inImage=imread('fruits.jpg');
inImage=rgb2gray(inImage);
inImageD=double(inImage);
% decomposing the image using singular value decomposition
[U,S,V]=svd(inImageD);
% Using different number of singular values (diagonal of S) to compress and
% reconstruct the image
dispEr = [];
numSVals = [];
for N=5:25:300
% store the singular values in a temporary var
C = S;
% discard the diagonal values not required for compression
C(N+1:end,:)=0;
C(:,N+1:end)=0;
% Construct an Image using the selected singular values
D=U*C*V';
% display and compute error
figure;
buffer = sprintf('Image output using %d singular values', N)
imshow(uint8(D));
title(buffer);
error=sum(sum((inImageD-D).^2));
% store vals for display
dispEr = [dispEr; error];
numSVals = [numSVals; N];
end
% dislay the error graph
figure;
title('Error in compression');
plot(numSVals, dispEr);
grid on
xlabel('Number of Singular Values used');
ylabel('Error between compress and original image');
Applying this to the following image:
Gives the following result with only first 5 Singular Values,
with first 30 Singular Values,
and the first 55 Singular Values,
The change in error with increasing number of singular values can be seen in the graph below.
Here you can notice the graph is showing that using approximately 200 first singular values yields to approximately zero error.
Just to start, I assume you're aware that the SVD is really not the best tool to decorrelate the pixels in a single image. But it is good practice.
OK, so we know that B = U*S*V'. And we know S is diagonal, and sorted by magnitude. So by using only the top few values of S, you'll get an approximation of your image. Let's say C=U*S2*V', where S2 is your modified S. The sizes of U and V haven't changed, so the easiest thing to do for now is to zero the elements of S that you don't want to use, and run the reconstruction. (Easiest way to do this: S2=S; S2(N+1:end, :) = 0; S2(:, N+1:end) = 0;).
Now for the compression part. U is full, and so is V, so no matter what happens to S2, your data volume doesn't change. But look at what happens to U*S2. (Plot the image). If you kept N singular values in S2, then only the first N rows of S2 are nonzero. Compression! Except you still have to deal with V. You can't use the same trick after you've already done (U*S2), since more of U*S2 is nonzero than S2 was by itself. How can we use S2 on both sides? Well, it's diagonal, so use D=sqrt(S2), and now C=U*D*D*V'. So now U*D has only N nonzero rows, and D*V' has only N nonzero columns. Transmit only those quantities, and you can reconstruct C, which is approximately like B.
For example, here's a 512 x 512 B&W image of Lena:
We compute the SVD of Lena. Choosing the singular values above 1% of the maximum singular value, we are left with just 53 singular values. Reconstructing Lena with these singular values and the corresponding (left and right) singular vectors, we obtain a low-rank approximation of Lena:
Instead of storing 512 * 512 = 262144 values (each taking 8 bits), we can store 2 x (512 x 53) + 53 = 54325 values, which is approximately 20% of the original size. This is one example of how SVD can be used to do lossy image compression.
Here's the MATLAB code:
% open Lena image and convert from uint8 to double
Lena = double(imread('LenaBW.bmp'));
% perform SVD on Lena
[U,S,V] = svd(Lena);
% extract singular values
singvals = diag(S);
% find out where to truncate the U, S, V matrices
indices = find(singvals >= 0.01 * singvals(1));
% reduce SVD matrices
U_red = U(:,indices);
S_red = S(indices,indices);
V_red = V(:,indices);
% construct low-rank approximation of Lena
Lena_red = U_red * S_red * V_red';
% print results to command window
r = num2str(length(indices));
m = num2str(length(singvals));
disp(['Low-rank approximation used ',r,' of ',m,' singular values']);
% save reduced Lena
imwrite(uint8(Lena_red),'Reduced Lena.bmp');
taking the first n max number of eigenvalues and their corresponding eigenvectors may solve your problem.For PCA, the original data multiplied by the first ascending eigenvectors will construct your image by n x d where d represents the number of eigenvectors.
I have a severely unbalanced data set. I want to perform a uniform resampling with 200% of the original data set size.
the resample function seems cannot perform as I expected. Anyone knows any toolbox or function can perform this? Thanks.
If you want to randomly resample with replacement from a data set of size N, you can use randi(N,1,N*2) to return a vector of size N*2 of random integers between 1 and N. Then use that vector to index into your original matrix. For example,
N = 100;
data = rand(1,N); % This simulates your original data set
idx = randi(N, 1, N*2);
newData = data(idx);