3d patches from a 3d matrix - matlab

Question

I have a 3d matrix (3x3x3), and I need to extract 3d patches (2x2x2) and transform them in vectors.
In 2d, simply:
I=randi(5,3,3);
2d_patches=im2col(I,[2 2],'sliding');
What about 3d?
I=randi(5,3,3,3);
3d_patches= ???
im2col just works in 2d. In 3d I should recombine the vectors 1 and 7, 2 and 8, ...
Is there any fast function for this task?

Answer 1

I do not believe that there is any built-in way to do this. If you need it to be fast, it should be fairly simple to write your own mex-function in c and call it from Matlab.
Here is my (quick and dirty) solution:
im3col.c:
#include <mex.h>
void mexFunction( int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[] )
{
const mxArray *I = prhs[0];
double *indata = mxGetPr(I);
double *patchSize = mxGetPr(prhs[1]);
const int *size = mxGetDimensions(I);
int J = (int)patchSize[0], K = (int)patchSize[1], H = (int)patchSize[2];
int M = size[0], N = size[1], P = size[2];
int numPatches = (M - J + 1)*(N - K + 1)*(P - H + 1);
int out_rows = J*K*H, out_cols = numPatches;
mxArray *out = mxCreateDoubleMatrix( out_rows, out_cols, mxREAL );
double *outdata = mxGetPr(out);
int patch = 0;
for( int h_offset = 0; h_offset < P-H+1; h_offset++ ){
for( int k_offset = 0; k_offset < N-K+1; k_offset++ ){
for( int j_offset = 0; j_offset < M-J+1; j_offset++ ){
int row = 0;
for( int h = 0; h < H; h++ ){
for( int k = 0; k < K; k++ ){
for( int j = 0; j < J; j++ ){
outdata[patch*out_rows + row] =
indata[ (j_offset+j) + (k_offset+k)*M + (h_offset+h)*M*N ];
++row;
}}}
++patch;
}}}
plhs[0] = out;
}
Compile:
>> mex -O CFLAGS="\$CFLAGS -std=c99 -Wall" im3col.c
Test:
>> A(:,:,1) = [1,4,7;2,5,8;3,6,9]; A(:,:,2) = [10,13,16;11,14,17;12,15,18];
>> B = im3col(A, [2,2,1])
B =
1 2 4 5 10 11 13 14
2 3 5 6 11 12 14 15
4 5 7 8 13 14 16 17
5 6 8 9 14 15 17 18
>> A(:,:,1),A(:,:,2)
ans =
1 4 7
2 5 8
3 6 9
ans =
10 13 16
11 14 17
12 15 18

Answer 2

Here is the other direction:
(It is pretty slow and there is definitely a faster way)
function [img] = patch2im_2d_time(patch, size_img, size_patch, size_skip, border)
Nx = size_img(1);
Ny = size_img(2);
Nt = size_img(5);
psz1 = size_patch(1);
psz2 = size_patch(2);
psz3 = size_patch(3);
%Extract blocks. One could save a lot here.
patches = reshape(patch, [psz1 psz2 psz3 size(patch,2)]);
c = 1;
img2 = zeros(squeeze(size_img));
%Count for each pixel how many times we added smth to it.
add_count = zeros(size_img);
%The first three loops, loop through all the pixels in the image
for d=1:Nt-psz3+1
for j=1:Nx-psz2+1
for i=1:Ny-psz1+1
%Here we get the next patch. The next patch is always
%the patch that has the pixel at i,j,d at its top front corner.
current_patch = patches(:,:,:,c);
%counter for the next patch
c = c + 1;
%In this loop we add the patch values of each pixel in the
%patch to the image. i,j,d is the base. We add the offset
%ii jj and dd to it. This iteration takes psz^3 many
%iterations.
for dd=1:psz3
for ii=1:psz2
for jj=1:psz1
img2(i+ii-1,j+jj-1,d+dd-1) = img2(i+ii-1,j+jj-1,d+dd-1) + current_patch(ii,jj,dd);
add_count(i+ii-1,j+jj-1,d+dd-1) = add_count(i+ii-1,j+jj-1,d+dd-1) + 1;
end
end
end
end
end
end
img = flipud(rot90(img2 ./ add_count,1));
end
Remember that MATLAB uses col major.

Answer 3

%One possible way to use matlab to call im2col and reshape twice
%N = [row, col, num_frames]
[x_height, ~, num_frames] = size(N);
patchSize = 16;
patchTemporal = 10;
N = reshape(N, x_height, []);
N = im2col(N, [patchSize, patchSize], 'distinct');
N = reshape(N, [], num_frames);
N = im2col(N, [patchSize^2, patchTemporal], 'distinct');
% N = [patchSize^2 *patchTemporal x numPatches]

Answer 4

hi guys what about this solution. To obtain 3x3x3 ROIs from I suggest :
blkSize=3; % should be a odd value like 3,5,7,etc
r=floor(blkSize/2);
k=1;
for sliceNo=(r+1):(size(I,3)-r)
img= I(:,:,sliceNo-r:sliceNo+r);
noPix=(size(img,1)-2*r)*(size(img,2)-2*r);
neiblk=zeros(blkSize^3,noPix);
for blk=1:blkSize
neiblk(blkSize^2*(blk-1)+1:blkSize^2*blk,:)=im2col(img(:,:,blk),...
[blkSize,blkSize],'sliding');
end
ROIs(:,noPix*(k-1)+1:noPix*k)=neiblk;
k=k+1;
end