filter detection - matlab

Question

I have to determine, which filter was used on a random picture - is there a common way to detect the right one (gaussian, prewitt, sobel, average, ...) or would it be clever to code some sort of 'brute force'-detection ?
I tried to find it with Matlab, but I have no glue how to search more efficiently. At the moment it`s like finding a needle in a Haystack. I also thought of using some bash-script and imagemagick, but this would be to resource hungry.
I tought this wouldn't be a problem, but it's very time-consuming to guess a filter and try it like this
f = fspecial('gaussian', [3 3], 1);
res = imfilter(orginal, f);
corr2(res, pic);

Answer 1

Let f be the original image, g be the filtered one, and h the filter applied to f, so that:
f * h = g
Passing that to the frequency domain:
F.H = G, so H = G/F
The problem is that inverting F is VERY sensitive to noise.
How to implement that in MATLAB:
close all;
f = imread('cameraman.tif');
[x,y] = size(f);
figure,imshow(f);
h = fspecial('motion', 20, 40); % abitrary filter just for testing the algorithm
F = fft2(f);
H = fft2(h,x,y);
G = F.*H;
g = ifft2(G); % the filtered image
figure, imshow(g/max(g(:)));
% Inverting the original image
epsilon = 10^(-10);
small_values = find(abs(F)<epsilon);
F(small_values) = epsilon;
F_i = ones(x,y)./F;
H_calculated = G.*F_i;
h_calculated = ifft2(H_calculated);
% remove really small values to try to infer the original size of h
r = sum(h_calculated,1)<epsilon;
c = sum(h_calculated,2)<epsilon;
h_real = h_calculated(~r,~c);
% Calculate error
% redo the filtering with the found filter
figure,g_comp = ifft2(fft2(f).*fft2(h_real,x,y));
imshow(g_comp/max(g_comp(:)));
rmse = sqrt(mean(mean((double(g_comp) - double(g)).^2,2),1))
edit: Just to explain the epsilon part:
It can be that some values in F are zero, or very close to zero. If we try to invert F with these small values, we would have problems with infinity. The easy way to solve that is to truncate every value in F that is smaller than an arbitrarily small limit, epsilon on the code.
Mathematically, what was done is this:
For all F < epsilon, F = epsilon