How to compute Gibbs energy in 2D image from a clear example - matlab

Question

I have an interesting question about matrix. In Gibbs distribution, a Gibbs energy U(x) can be compute as
which is a sum of clique potentials Vc(x) over all possible cliques C (right image). A clique c is defined as a subset of sites in S (are neighborhood of x- blue color pixels are neighbor of yellow pixel in left figure) in which every pair of distinct sites are neighbors, except for single-site cliques
In which V(x) is computed as following:
My goal is how to compute U(x). In my above example, x={1,2}. However, I have some stuck at pixel at corner of image that only has less than 8 neighbors (a pixel often has 8 neighbor pixels in normal case). To solve it, I added zeros at corner of edge by convolution image with an mask. But I think it maybe affect to U(x). From above definition, could you help me find U(x=1) and U(x=2) when i consider pixels at corner of image (gray color)? I tried to solve it by below solution but I think my solution is so long and I am not sure is whether correct or not.
This is what I do
Imlabel =[ 1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 2 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 2 2 2 1 1 1 1 1;
1 1 2 2 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 2 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1];
mask=[ 0 0 0;0 1 0;0 0 0];
Imlabel=conv2(Imlabel,mask); % To solve pixel in corner
num_class=2; % x={1,2}
beta=1;
for label=1:num_class
for i=2:size(Imlabel,1)-1
for j=2:size(Imlabel,2)-1
sum_V=0;
%North, south, east and west neighbors
if(label==Imlabel(i-1,j)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
%% Diagonal pixels
if(label==Imlabel(i-1,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i-1,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
%% Save
U(i-1,j-1,label)=sum_V; %% Because Index is extended before
end
end
end
Update: Normalization
Currently, based on my understand at here. I computed normalized term as well as gibbs distribution as
P=zeros(size(U));
Z=sum(exp(-U),3);
for i=1:num_class
P(:,:,i)=exp(-U(:,:,i))./Z;
end

Answer 1

Your explanation makes sense as well as the code. However, you can totally get this vectorized by being smart about the kinds of functions you want to use.
For the interested readers, given your source code, what you want to do is the following (in pseudocode):
Create a matrix U that is size(Imlabel,1) x size(Imlabel,2) x num_class large. Each slice of this matrix will determine which the Gibbs energy costs for the 8 neighbouring pixels at each coordinate (i,j) in this slice.
For each class label x...
a. For each pixel in the image (i,j)...
Find those locations defined in an 8-pixel neighbourhood of (i,j) that are equal to x and set them to beta
Find those locations defined in an 8-pixel neighbourhood of (i,j) that are not equal to x and set them to -beta
Compute the sum of all of these beta and -beta values within this 8-pixel neighbourhood and set this at location U(i,j,x)
Therefore, what I would do is create a 3D matrix of costs... let's call this C, where each slice y has the same dimensions as size(Imlabel,1) x size(Imlabel,2) but each location (i,j) in a slice will be beta if Imlabel(i,j) == y and -beta otherwise. Once you do this, you can perform an N-D convolution of this matrix, but using a 2D kernel so that you can compute the summation of the 8-pixel neighbourhoods of each slice separately.
Today's magic menu consists of bsxfun, convn and a side-order of permute to sweeten the pot.
To get the cost matrix C, you can do this. This is assuming Imlabel is not padded with zeros along the boundaries:
Imlabel =[ 1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 2 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 2 2 2 1 1 1 1 1;
1 1 2 2 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 2 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1];
C = double(bsxfun(#eq, Imlabel, permute(1:num_class, [1 3 2])));
C(C == 0) = -beta;
C(C == 1) = beta;
The permute function is used to create a single 3D vector that goes from 1 up to as many classes as you have defined. The reason why this is required is because bsxfun does what is known as broadcasting. What will happen in this case is given your matrix Imlabel, using the 3D vector will create a 3D cost matrix in conjunction with the eq or equals function. Each slice of this cost matrix will give you the locations of where it is either equal to the label in question, or it isn't.
You can verify that what we have for the locations matrix that is output from bsxfun is correct:
>> C = double(bsxfun(#eq, Imlabel, permute(1:num_class, [1 3 2])))
C(:,:,1) =
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 0 1 1
1 1 1 1 1 1 1 1 1 1
1 1 0 0 0 1 1 1 1 1
1 1 0 0 1 1 1 1 1 1
1 1 1 1 1 1 1 1 0 1
1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 1
C(:,:,2) =
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 1 0 0
0 0 0 0 0 0 0 0 0 0
0 0 1 1 1 0 0 0 0 0
0 0 1 1 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
As you can see, for each slice, we can see which locations shared that label that is enumerated by that slice and those that don't share the same label. Once we have these locations, we need to cast this to double so that we can modify this to be a cost matrix where -beta is a location that isn't equal to a label for a particular slice and beta is. That is done by the two assignment statements after the bsxfun call.
Once you have this cost matrix, you can pad the boundaries like you did before, but make sure that the boundaries are set to a cost of -beta as you have done in your code:
mask = zeros(3,3); mask(2,2) = 1;
Cpad = convn(C, mask);
Cpad(Cpad == 0) = -beta;
The first line of code denotes the [0 0 0; 0 1 0; 0 0 0] mask that you defined in your code. Now to pad each slice independently to have a border of zeroes, we can use convn to help us do that.
Now that we have that set up, all you have to do is perform the calculation of Gibbs' energy independently per slice:
mask2 = ones(3,3); mask2(2,2) = 0;
out = convn(Cpad, mask2, 'valid');
The first line of code denotes a mask where every value is 1 except for the middle, which is 0 and it is a 3 x 3 mask. The reason why you want to do this is because this replaces the if/else statements in your double for loop logic. What you are doing is essentially a convolution where you are adding up all of the 8 neighbours except for the middle itself. This can be accomplished by using a mask that has all 1s except for the central pixel and you set that to 0.
Next, we use convn to compute the Gibbs' energy per slice independently, but using the 'valid' flag so that we can remove the zero-padded border we included at the beginning.
Now, if you take a look at out, it compares with what you have with U, but there is a slight shift because of the way you are indexing into U. Nevertheless, with the above output of out, you can verify that what we have is correct and this will handle the border cases nicely:
>> out
out(:,:,1) =
-2 2 2 2 2 2 2 2 2 -2
2 8 8 8 8 8 6 6 6 2
2 8 8 8 8 8 6 8 6 2
2 6 4 2 4 6 6 6 6 2
2 4 2 0 4 6 8 8 8 2
2 4 2 0 2 6 8 6 6 0
2 6 4 4 6 8 8 6 8 0
2 8 8 8 8 8 8 6 6 0
-2 2 2 2 2 2 2 2 2 -2
out(:,:,2) =
-8 -8 -8 -8 -8 -8 -8 -8 -8 -8
-8 -8 -8 -8 -8 -8 -6 -6 -6 -8
-8 -8 -8 -8 -8 -8 -6 -8 -6 -8
-8 -6 -4 -2 -4 -6 -6 -6 -6 -8
-8 -4 -2 0 -4 -6 -8 -8 -8 -8
-8 -4 -2 0 -2 -6 -8 -6 -6 -6
-8 -6 -4 -4 -6 -8 -8 -6 -8 -6
-8 -8 -8 -8 -8 -8 -8 -6 -6 -6
-8 -8 -8 -8 -8 -8 -8 -8 -8 -8
For example, if we took a look at the top left corner of the first slice, we see that the east, south and south east corners have the same label of 1, and so since beta = 1, our sum would be 3, but then there are the other 5 directions that we didn't consider, and in your code, you set those to -beta and so 3 - 5 = -2.
Let's also take a look at the 6th row, 4th column and let's take a look at the second label or slice. This means that any cardinal directions that are equal to label 2, we should sum these with beta / 1 while anything that isn't equal to label 2 is -beta / -1. As you can see, there are exactly 4 labels of 1 and 4 labels of 2, and so these should cancel out and give us a 0.
You can verify that what we did for all of the other locations in this matrix result in the right calculations.
The full code is simply:
Imlabel =[ 1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 2 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 2 2 2 1 1 1 1 1;
1 1 2 2 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 2 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1];
C = double(bsxfun(#eq, Imlabel, permute(1:num_class, [1 3 2])));
C(C == 0) = -beta;
C(C == 1) = beta;
mask = zeros(3,3); mask(2,2) = 1;
Cpad = convn(C, mask);
Cpad(Cpad == 0) = -beta;
mask2 = ones(3,3); mask2(2,2) = 0;
out = convn(Cpad, mask2, 'valid');
In the case of timing, we can check to see how fast the above approach is compared to your original loop code. I used timeit to help facilitate that.
Here's the script I set up that sets up all of the relevant variables, and measures the time taken with your code and mine:
function test_clique_timing
% Setup
Imlabel_orig =[ 1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 2 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1;
1 1 2 2 2 1 1 1 1 1;
1 1 2 2 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 2 1;
1 1 1 1 1 1 1 1 1 1;
1 1 1 1 1 1 1 1 1 1];
num_class=2; % x={1,2}
beta = 1;
function test_orig
mask=[ 0 0 0;0 1 0;0 0 0];
Imlabel=conv2(Imlabel_orig,mask); % To solve pixel in corner
beta=1;
for label=1:num_class
for i=2:size(Imlabel,1)-1
for j=2:size(Imlabel,2)-1
sum_V=0;
%North, south, east and west neighbors
if(label==Imlabel(i-1,j)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
%% Diagonal pixels
if(label==Imlabel(i-1,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i-1,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j+1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
if(label==Imlabel(i+1,j-1)) sum_V=sum_V+beta;
else sum_V=sum_V-beta;end
%% Save
U(i-1,j-1,label)=sum_V; %% Because Index is extended before
end
end
end
end
function test_conv
C = double(bsxfun(#eq, Imlabel_orig, permute(1:num_class, [1 3 2])));
C(C == 0) = -beta;
C(C == 1) = beta;
mask = zeros(3,3); mask(2,2) = 1;
Cpad = convn(C, mask);
Cpad(Cpad == 0) = -beta;
mask2 = ones(3,3); mask2(2,2) = 0;
out = convn(Cpad, mask2, 'valid');
end
time1 = timeit(#test_orig);
time2 = timeit(#test_conv);
fprintf('Loop code time: %f seconds\n', time1);
fprintf('Vectorized code time: %f seconds\n', time2);
fprintf('Speedup factor: %f', time1/time2);
end
Running the above script, I get these for times:
Loop code time: 0.000049 seconds
Vectorized code time: 0.000060 seconds
Speedup factor: 0.816667
This is done in seconds and I did this using MATLAB R2013a under Mac OS Mavericks 10.10.5. The speedup doesn't look that great. In fact it's a factor less than 1, which is terrible. However, what you have shown is such a small dataset. We should try with a larger dataset and see if this still holds. Let's make the matrix 2500 x 2500, with 10 class labels. I'm going to replace the Imlabel matrix with:
rng(123); %// Set seed for reproducibility
num_class = 10;
Imlabel_orig = randi(10,2500,2500);
This produces random integers from 1 to 10 in a 2500 x 2500 matrix. Doing this and running the code again gives:
Loop code time: 17.553669 seconds
Vectorized code time: 3.321950 seconds
Speedup factor: 5.284146
Yeah... that's much better. On the 2500 x 2500 matrix of 10 labels, it took roughly 17.5 seconds to compute, whereas the the vectorized code with bsxfun / convn / permute outperforms your original loop code by a factor of almost 5.2x.
As for the normalization term to factor into your Gibbs' energy calculation, you currently have this:
P=zeros(size(U));
Z=sum(exp(-U),3);
for i=1:num_class
P(:,:,i)=exp(-U(:,:,i))./Z;
end
If you take advantage of bsxfun, you can simply do this:
Z = sum(exp(-out),3)); %// out is U in my code
P = bsxfun(#rdivide, exp(-out), Z);
This achieves the same thing your code does. For each slice in P, we find the exp and negate the slice as input, and divide each term by Z. For bsxfun, the Z matrix gets replicated for as many slices as there are in out and does the element-wise division much like the loop code does.