Creating Filter's Laplacian Matrix and Solving the Linear Equation for Image Filtering - matlab

Question

I have an optimization problem to solve in order to filter an image.
I created a Linear Equation of the problem which deals with Sparse Matrices.
At first I will show the problem.
First, the Laplacian (Adjacency) matrix of the problem:
The matrix Dx / Dy is the forward difference operator -> Hence its transpose is the backward difference operator.
The matrix Ax / Ay is diagonal matrix with weights which are function of the gradient of the image (Point wise, namely the value depends only on the gradient on that pixel by itself).
The weights are:
Where Ix(i) is the horizontal gradient of the input image at the i-th pixel (When you vectorize the input image).
Assuming input Image G -> g = vec(G) = G(:).
I want to find and image U -> u = vec(U) = U(:) s.t.:
My questions are:
How can I build the matrices Dx / Dy / Ax / Ay effectively (They are all sparse)?
By setting M = (I + \lambda * {L}_{g}), Is there an optimized way to create M directly?
What would be the best way to solve this linear problem in MATLAB? Is there a way to by pass memory limitations (Namely, dealing with large images and still be able to solve it)?
Is there an Open Source library to solve it under limited memory resources? Any library with MATLAB API?
Thank You.

Answer 1

Given your comments, let's answer each question in a synopsis and go from there:
I will answer that question below using sparse and other related functions
Using (1), we can definitely build M in an optimized way.
Simply put, the \ operator is the best thing to use when solving an inverse. MathWorks have spent so much time trying to optimize it, and it pretty much uses LAPACK and BLAS under the hood, that you would be insane not to use it. The only time you wouldn't be able to use it is answered in (4).
There are some MATLAB scripts that can handle solving the matrix iteratively, like the Successive Overrelaxation technique, but you should only use those if your run out of memory (i.e. if \ doesn't give you an answer). With the sparse representation of the matrices, this shouldn't (hopefully) happen, so let's avoid using those functions for now.
Going back to your question, we can produce a sparse representation of L_g very nicely. Given the definition of Dx and Dy, we can use the sparse version of the eye command called speye. Therefore, Dx and Dy can be calculated by Dx = diff(speye(size(inputImage))); As an example, this is what would be produced if you tried doing this on a 7 x 5 image.
>> diff(speye(7,5))
ans =
(1,1) -1
(1,2) 1
(2,2) -1
(2,3) 1
(3,3) -1
(3,4) 1
(4,4) -1
(4,5) 1
(5,5) -1
As you can see, we are referencing only non-zero entries. Row 1, column 1 has a coefficient of -1, row 1, column 2 has a coefficient of 1 and so on. As for your Ax and Ay, that's also very easy to do. We have a diagonal matrix and we can set each of the entries manually. All we would do is specify a set of row indices, column indices, and what the values are at each point. Therefore, we can do that by:
inputImage = im2double(inputImage); %//Important
rows = 1 : numel(inputImage); %// Assuming a 2D matrix
cols = rows; % // Row and column indices are the same
valuesDx = exp(-(gradX(rows).^2 / 2*sigma*sigma ));
valuesDy = exp(-(gradY(rows).^2 / 2*sigma*sigma ));
The reason for the first call is because we want to make sure that the pixels are in double precision, as finding the inverse in MATLAB requires that you do this. It also ensures we don't overflow the type as we are normalizing the intensities between 0 and 1. You may have to adjust your standard deviation to reflect this. Now we just need to construct our Ax and Ay matrices, and let's put it together with Dx and Dy:
numberElements = numel(inputImage);
Ax = sparse(rows, cols, valuesDx, numberElements, numberElements);
Ay = sparse(rows, cols, valuesDy, numberElements, numberElements);
identity = speye(numberElements, numberElements);
Dx = diff(identity);
Dy = Dx.'; %// Transpose
The reason why I'm transposing Dx to get Dy is because the difference operator in the vertical direction should simply be the transpose (makes sense to me). These should all be sparse representations of each of the matrices you want. Matrix operations can also be performed on sparse matrices, including multiplication and the inverse. As such:
Lg = Dx.' * Ax * Dx + Dy.' * Ay * Dy;
You can now solve for u via:
u = (identity + lambda*Lg) \ g;
This assumes that g is structured with your pixels in your image in column-major format. The way I sampled the pixels to build Ax and Ay naturally follows this. As such, do g = inputImage(:);, assuming that we have converted to double and normalized between 0 and 1.
When you finally solve for u, you can reshape it back to an image by doing:
u = reshape(u, size(inputImage, 1), size(inputImage, 2));
u may also be sparse, so if you want the original image back, cast it using full():
u = full(u);
Hope this helps!