I'm currently working on program and I need to automatic perform motion un-blurring on a picture. Currently, I make a for loop for LEN and THETA, guessing from LEN 0:50 and THETA from 1:180. There are plenty of motion un-blurred pictures produce in this way - some correct and some are wrong. Now here is my problem: How do I actually determine which set of parameters yields the one most closest to original photo?
I'm thinking of using pixel comparison. Any idea on this?
Here's a pictorial example of what I generated:
http://dl.dropboxusercontent.com/u/81112742/Capture.JPG
If you have access to the original clean image, I would compute the Peak Signal to Noise Ratio (PSNR) for all of the images you have generated, then choose the one with the highest PSNR. Amro posted a very nice post on how to compute this for images and can be found here: https://stackoverflow.com/a/16265510/3250829
However, for self-containment, I'll post the code to do it here. Supposing that your original image is stored in the variable I, and supposing your reconstructed (un-blurred) image is stored in the variable K. Therefore, to calculate the PSNR, you would need to calculate the Mean Squared Error first, then use it to calculate the PSNR. In other words:
mse = mean(mean((im2double(I) - im2double(K)).^2, 1), 2);
psnr = 10 * log10(1 ./ mean(mse,3));
The equations for MSE and PSNR are:
Source: Wikipedia
As such, to use this in your code, your for loops should look something like this:
psnr_max = -realmax;
for LEN = 0 : 50
for THETA = 1 : 180
%// Unblur the image
%//...
%//...
%// Compute PSNR
mse = mean(mean((im2double(I) - im2double(K)).^2, 1), 2);
psnr = 10 * log10(1 ./ mean(mse,3));
if (psnr > psnr_max) %// Get largest PSNR and get the
LEN_final = LEN; %// parameters that made this so
THETA_final = THETA;
psnr_max = psnr;
end
end
end
This loop will go through each pair of LEN and THETA, and LEN_final, THETA_final will be those parameters that gave you the best reconstruction (un-blurring) of the image.
Related
I have written the following matlab code for producing a noisy image corrupted by Cauchy noise but when I compute the psnr value of the noisy image using psnr function in matlab, it returns a negetive value while according to the paper that I have implemented its results, the psnr value for the cameraman test image of size 256 by 256 and 0.02 of Cauchy noise must be around 19. I appreciate it if someone could please help me to solve this problem.
refimg = im2double(imread('cameraman.png')); % original image
img_height = size(refimg,1);
img_width = size(refimg,2);
refimg = refimg(1:img_height,1:img_width);
rng(0);
r1 = random('Normal',0, 1,[img_height img_width]);
r2 = random('Normal',0, 1,[img_height img_width]);
n = 0.02; % the noise level
u0 = refimg + n.*(r1./r2);
figure(1); imshow(u0);
PSNR_noisy = psnr(refimg,u0)
As Luis Mendo nicely stated, you cannot compute the power of noise taken from a Cauchy distribution. This is because the distribution doesn't have a mean or a variance. You can get infinitely large (or small) values from this distribution. Because of this lack of mean and variance, it doesn't make sense as a noise model for an image. However, if we were to clamp the values of the image after applying noise, then we would be able to sensibly compute statistics.
This is OP's code, slightly simplified and amended to clamp the image values to the [0,1] range after adding the noise:
refimg = im2double(imread('cameraman.tif')); % original image
sz = size(refimg);
rng(0);
r1 = randn(sz); % (using randn because I don't have the statistics toolbox)
r2 = randn(sz);
n = 0.02; % the noise level
u0 = refimg + n.*(r1./r2);
u0 = min(u0,1); % clamp large values to 1
u0 = max(u0,0); % clamp small values to 0
PSNR_noisy = psnr(u0,refimg)
The output now is 19.1419.
Note that the function psnr expects the reference image to be the second input, even though the switch doesn't affect the output in this case.
Please read my opinion here on using PSNR: https://www.crisluengo.net/archives/490/
The psnr function outputs the signal-to-noise ratio in decibels (that is, logarithtmic units). This is stated in the documentation (but I think it should be indicated more clearly, perhaps in the description section). Also, a quick look at psnr's code confirms that the outputs are given in decibels:
peaksnr = 10*log10(peakval.^2/err);
snr = 10*log10(mean(ref(:).^2)/err);
Therefore the result can be negative, meaning there is more noise power than signal power.
Note, however, that the concept of noise power cannot be applied to a Cauchy distribution. So I’m not sure it makes sense to use psnr in this case.
I am working on a project involving video motion magnification algorithms. Currently I am trying to understand phase based motion magnification using a riesz pyramid. My main source of information is this document:
Riesz Pyramids for Fast Phase-Based Video Magnification
\
I have performed the following steps to attempt to reproduce some of the results in the paper:
Decompose an image into multiple scales using the provided matlab code for the riesz pyramid
Generate the images Riesz1 and Riesz2 by convolving one subband of the pyramid with [-0.5, 0, 0.5] and [-0.5, 0, 0.5]' using the approximate riesz transform introduced in the paper.
Determine the dominant local orientation in every pixel of the subband by calculating atan(R2/R1). This calculation is derived from equation 3 in the paper.
Steer the transform to the dominant local orientation and calculate the resulting quadrature pair
Use the quadrature pair to generate a complex number (I + iQ) whose phase I then used to determine the local phase in the specific pixel.
This is the Matlab code I created:
%Generate a circle image
img = zeros(512, 512);
img(:) = 128;
rad = 180;
for i = size(img, 1)/2 - rad : size(img,1)/2 + rad
for j = size(img, 2)/2 - rad : size(img,2)/2 + rad
deltaX = abs(size(img, 1)/2 - i);
deltaY = abs(size(img, 2)/2 - j);
if (sqrt(deltaX^2+deltaY^2) <= rad)
img(i, j) = 255;
end
end
end
%build riesz pyramid
[pyr, pind] = buildNewPyr(img);
%extract band2 from pyramid (no orientation information yet)
I = pyrBand(pyr,pind,3);
%convolve band2 with approximate riesz filter for first quadrature pair
%element
R1 = conv2(I, [0.5, 0, -0.5], 'same');
%convolve band2 with approximate riesz filter (rotated by 90°) for second
%quadrature pair element
R2 = conv2(I, [0.5, 0, -0.5]', 'same');
% show the resulting image containing orientation information!
% imshow(band2_r2, []);
%To extract the phase, we have to steer the pyramid to its dominant local
%orientation. Orientation is calculated as atan(R2/R1)
theta = atan(R2./R1);
theta(isnan(theta) | isinf(theta)) = 0;
%imshow(theta, []);
% create quadrature pair
Q = zeros(size(theta, 1), size(theta, 2));
for i = 1:size(theta, 1)
for j = 1:size(theta, 1)
if theta(i, j) ~= 0
%create rotation matrix
rot_mat = ...
[cos(theta(i, j)), sin(theta(i, j));...
-sin(theta(i, j)) cos(theta(i, j))];
%steer to dominant local orientation(theta) and set Q
resultPair = rot_mat*[R1(i, j), R2(i,j)]';
Q(i,j) = resultPair(1);
end
end
end
% create amplitude and phase image
A = abs(complex(I, Q));
Phi = angle(complex(I, Q));
The generated images look like this:
Now my questions:
When calculating theta using atan(R2/R1) I get a lot of artifacts in the result (see image "dominant orientation"). Is there something obvious I miss here/do wrong?
Assuming what my results are correct thus far. To magnify motion I need to not only be able to determine the local phase, but also to change it. I seem to miss something obvious, but how would I go about that? Do I need to somehow change the phase of the pyramid subband pixels and then collapse the pyramid? If yes, how?
I am (obviously) quite new to this topic and only have a rudimentary understanding of image processing. I would be very thankful for any answer, be it a solution to my problems or just a referral to an other useful source of information.
Sincerely
I have got a functioning implementation of this algorithm. Here are the steps I took to successfully motion-magnify a video using this method.
These steps should be applied to each channel of a video sequence that you (I have tried it for RGB video, you could probably get away with doing it for just luminance, in a YUV video).
Create an image pyramid of each frame. The original paper has a recommended pyramid structure to allow greater magnification values, although it works fairly well with a Laplacian pyramid.
For each pyramid level of each video channel, calculate the Riesz transform (see The Riesz transform and simultaneous representations of phase, energy and orientation in spatial vision for an overview of the transform, and see the paper in the original question for an efficient approximate implementation).
Using the Riesz transform, calculate the local amplitude, orientation and phase for each pixel of each pyramid level of each video frame. The following Matlab code will calculate the local orientation, phase and amplitude of a (double format) image using the approximate Riesz transform:
function [orientation, phase, amplitude] = riesz(image)
[imHeight, imWidth] = size(image);
%approx riesz, from Riesz Pyramids for Fast Phase-Based Video Magnification
dxkernel = zeros(size(image));
dxkernel(1, 2)=-0.5;
dxkernel(1,imWidth) = 0.5;
dykernel = zeros(size(image));
dykernel(2, 1) = -0.5;
dykernel(imHeight, 1) = 0.5;
R1 = ifft2(fft2(image) .* fft2(dxkernel));
R2 = ifft2(fft2(image) .* fft2(dykernel));
orientation = zeros(imHeight, imWidth);
phase = zeros(imHeight, imWidth);
orientation = (atan2(-R2, R1));
phase = ((unwrap(atan2(sqrt(R1.^2 + R2.^2) , image))));
amplitude = sqrt(image.^2 + R1.^2 + R2.^2);
end
For each pyramid level, temporally filter the phase values of each pixel using a bandpass filter set to a frequency appropriate for the motion that you wish to magnify. Note that this removes the DC component from the phase value.
Calculate the amplified phase value by
amplifiedPhase = phase + (requiredGain * filteredPhase);
Use the amplified phase to calculate new pixel values for each pyramid level by
amplifiedSequence = amplitude .* cos(amplifiedPhase);
Collapse the pyramids to generate the new, amplified video channel.
Recombine your amplified channels into a new video frame.
There are some other steps in the original paper to improve noise performance, but the sequence above produces motion amplified video quite nicely.
I have fully implemented the methodology of Riesz Pyramids for fast phase based video motion magnification. I felt the papers did not clearly describe the appropriate steps required to correctly filter phase. It is important to realise that mulitple mathematically correct expressions for phase and orientation may in fact not be suitable using MATLAB's acos(), asin() and atan() functions. Here is my implementation:
% R1, R2 are Riesz transforms of the image I and Q is the Quadrature pair
Q = sqrt((R1.^2) + (R2.^2));
phase = atan2(Q,I);
The phase should be wrapped to be between -pi and +pi, i.e. if phase is greater than +pi, phase = phase - 2*pi, if phase is smaller than -pi,
phase = phase + 2*pi.
amplitude = sqrt((I.^2) + (R1.^2) + (R2.^2));
Furthermore, it is essential that the change in phase between successive frames is filtered, not the phase directly.
phase_diff = phase(t+1) - phase(t);
This quantity "phase_diff" is temporally filtered, and amplified by multiplication with an amplification factor. The filtered, amplified, variation in phase is then used to phase shift the input.
magnified output = amplitude.*cos(phase_diff_filtered_amplified + original phase).
whilst DrMcCleod's answer didn't directly provide a solution, it seems he was on the right track.
The complex representation of the image can be constructed from the input and the quadrature pair with
complexImg = complex(I, Q);
A phase shifted reconstruction of the image can then be generated by multiplying the complex representation with e^(-i*shift) which removes the complex part of the representation and results in the original image plus the introduced phase shift.
reconstructed = complexImg*exp(-sqrt(-1) * shift);
I'll have to experiment a little, but this seems to produce the expected results.
Thanks for the help!
I have implemented otsu's threshold implementation which segments an image into foreground and background image.The output of my implementation seems off from the desired one. Any thoughts on it? Thanks in advance! Would really appreciate it someone can tell me how do i resolve the issue.
My output:
For image 1-
For image 2-
My Code:
im1=imread('D:\root-image.pgm');
% im1=rgb2gray(im1);
[n,m]=size(im1);
hst=imhist(im1);
mu=zeros(255,1);
N=0;
for i=1:255
N=N+hst(i);
end
% The total mean level of the original image
for i=1:255
mu(i)=mu(i)+((i.*hst(i))./N);
end
for T=1:254
qb=0;
muT=0;
qo=0;
for i=1:T
qb=qb+(hst(i)./N); % probability of class occurence (background)
m=m+((i.*hst(i))./N);% probability of class mean (background)
end
for i=T+1:255
qo=qo+(hst(i)./N);% probability of class occurence (object)
end
sigma(T)=((mu(T)-(qb*muT))^2)/(qb*qo);
end
[Y,T] = max(sigma);
[n,m]=size(im1);
for i=1:n
for j=1:m
if im1(i,j)>T
im(i,j)=1;
else
im(i,j)=0;
end
end
end
figure(1);
subplot(1,2,1);
imshow(im1);
% subplot(1,3,2);
% imhist(im1);
subplot(1,2,2);
imshow(im);
There are a few issues with your code, and I'll outline where it's wrong:
I'm simply nitpicking, but you can count the total number of pixels (N) by using numel... it's just cleaner :)
In your original code, you are checking for the right threshold between 1 and 254. You really should be checking from 0 to 255, as there are 256 possible intensities in your image.
You also need to change your sigma declaration so that there are 256 elements, not 255. Remember, there are 256 possible intensities in your image.
Within your for loop for checking each intensity, when you are calculating the probability of class occurrences, you need to check for intensity 0 as well. Because of the fact that MATLAB starts indexing arrays at 1, you'll need to offset your access index so that you're starting at 1.
Your definition of the variance between the object and background is slightly off. You need to also calculate the probability of the class mean for the object as well. You can check the code for more details.
Your probability of class mean definitions are slightly inaccurate. You need to divide by qb and qo, not N.
You are using m to calculate then mean when you should be storing it in muT.
Finally, when you find the maximum of the variance between the object and background, you need to subtract by 1, as this will provide an intensity between 0 and 255.
As such, this is what your code looks like. Note that I have omitted the code that thresholds your image. I'm only providing the code that calculates the threshold for your image.
hst=imhist(im1);
sigma = zeros(256,1); %// Change
N = numel(im1); %// Change
for T=0:255 %// Change
qb=0;
muT=0;
qo=0;
muQ=0; %// Change
for i=0:T %// Change
qb=qb+(hst(i+1)./N); % probability of class occurence (background)
end
for i=T+1:255
qo=qo+(hst(i+1)./N);% probability of class occurence (object)
end
for i=0:T%// Change
muT=muT+((i.*hst(i+1))./qb);% probability of class mean (background)
end
for i=T+1:255 %// Change
muQ=muQ+((i.*hst(i+1))./qo);% probability of class mean (object)
end
sigma(T+1) = qb*qo*((muT-muQ)^2); %// Change
end
[Y,T] = max(sigma);
T = T-1; %// Change - For 0 to 255
This code should now work. I ran this code with my own implementation of Otsu, and I get the same calculated threshold. To be honest, I find this code to be rather inefficient due to the many for loops. What I would personally do is vectorize it, but I'll leave that to you as a learning exercise :)
Edit
OK, I'll give in. Here's some code that I wrote for Otsu that is more vectorized. This essentially performs what you are doing above, but in a more vectorized manner. You're more than welcome to use it for your own purposes, but do cite me if you intend to use it of course :)
%// Function that performs Otsu's adaptive bimodal thresholding
%// Written by Raymond Phan - Version 1.0
%// Input - im - Grayscale image
%// Output - out - Thresholded image via Otsu
function [out] = otsu(im)
%// Compute histogram
J = imhist(im);
%// Total number of intensities
L = length(J);
%// Some pre-processing stuff
%// Determine total number of pixels
num_pixels = sum(J(:));
%// Determine PDF
pdf = J / num_pixels;
%// Storing between-class variances for each intensity
s_b = zeros(1,L);
for idx = 1 : L
%// Calculate w_0
w_0 = sum(pdf(1:idx));
%// Calculate w_1
w_1 = 1 - w_0;
%// Calculate u_0
u_0 = sum((0:idx-1)'.*pdf(1:idx)) / w_0;
%// Calculate u_1
u_1 = sum((idx:L-1)'.*pdf(idx+1:L)) / w_1;
% // Calculate \sigma_b^{2}
s_b(idx) = w_0*w_1*((u_1 - u_0)^2);
end
%// Find intensity that provided the biggest variance
[max_s_b T] = max(s_b);
T = T - 1; %// Must subtract by 1, since the index starts from 1
%// Now create an output image that thresholds the input image
out = im >= T;
end
Edits by Divakar
Divakar (thanks!) has created vectorized code to replace the loop portion of the above function code and this essentially gets rid of the pre-allocation for s_b:
w_0 = cumsum(pdf);
w_1 = 1 - w_0;
u_0 = cumsum((0:L-1)'.*pdf)./w_0;
u_1 = flipud([0 ; cumsum((L-1:-1:1)'.*pdf((L:-1:2)))])./w_1;
s_b = w_0.*w_1.*((u_1 - u_0).^2);
I need to compute a moving average over a data series, within a for loop. I have to get the moving average over N=9 days. The array I'm computing in is 4 series of 365 values (M), which itself are mean values of another set of data. I want to plot the mean values of my data with the moving average in one plot.
I googled a bit about moving averages and the "conv" command and found something which i tried implementing in my code.:
hold on
for ii=1:4;
M=mean(C{ii},2)
wts = [1/24;repmat(1/12,11,1);1/24];
Ms=conv(M,wts,'valid')
plot(M)
plot(Ms,'r')
end
hold off
So basically, I compute my mean and plot it with a (wrong) moving average. I picked the "wts" value right off the mathworks site, so that is incorrect. (source: http://www.mathworks.nl/help/econ/moving-average-trend-estimation.html) My problem though, is that I do not understand what this "wts" is. Could anyone explain? If it has something to do with the weights of the values: that is invalid in this case. All values are weighted the same.
And if I am doing this entirely wrong, could I get some help with it?
My sincerest thanks.
There are two more alternatives:
1) filter
From the doc:
You can use filter to find a running average without using a for loop.
This example finds the running average of a 16-element vector, using a
window size of 5.
data = [1:0.2:4]'; %'
windowSize = 5;
filter(ones(1,windowSize)/windowSize,1,data)
2) smooth as part of the Curve Fitting Toolbox (which is available in most cases)
From the doc:
yy = smooth(y) smooths the data in the column vector y using a moving
average filter. Results are returned in the column vector yy. The
default span for the moving average is 5.
%// Create noisy data with outliers:
x = 15*rand(150,1);
y = sin(x) + 0.5*(rand(size(x))-0.5);
y(ceil(length(x)*rand(2,1))) = 3;
%// Smooth the data using the loess and rloess methods with a span of 10%:
yy1 = smooth(x,y,0.1,'loess');
yy2 = smooth(x,y,0.1,'rloess');
In 2016 MATLAB added the movmean function that calculates a moving average:
N = 9;
M_moving_average = movmean(M,N)
Using conv is an excellent way to implement a moving average. In the code you are using, wts is how much you are weighing each value (as you guessed). the sum of that vector should always be equal to one. If you wish to weight each value evenly and do a size N moving filter then you would want to do
N = 7;
wts = ones(N,1)/N;
sum(wts) % result = 1
Using the 'valid' argument in conv will result in having fewer values in Ms than you have in M. Use 'same' if you don't mind the effects of zero padding. If you have the signal processing toolbox you can use cconv if you want to try a circular moving average. Something like
N = 7;
wts = ones(N,1)/N;
cconv(x,wts,N);
should work.
You should read the conv and cconv documentation for more information if you haven't already.
I would use this:
% does moving average on signal x, window size is w
function y = movingAverage(x, w)
k = ones(1, w) / w
y = conv(x, k, 'same');
end
ripped straight from here.
To comment on your current implementation. wts is the weighting vector, which from the Mathworks, is a 13 point average, with special attention on the first and last point of weightings half of the rest.
I have been working on extracting the peak values from a graph (see previous question) which looks like this:
but I have noticed that for some of the xcorr graphs I have been working on the values do not turn out as expected and they normally turn out looking more like this:
and this:
Instead of trying to pick the peak values like the code was doing in the first figure, how would I go about trying to pick the values where the downward slope momentarily evens itself out (as shown in Figure 3)?
When I try and run the code in its current state on data like the ones shown in Figure 2 & 3, I do not get any useful data back in return.
I think I need an if statement or similar in the 'find extrema points' section but I'm not sure whether that is correct or not. My .M code for the function looks like this so far:
[inputname, pathname] = uigetfile('*.wav', 'Select WAV-file');
thumb1 = inputname; %# Get filename information
fprintf('\n%s is being turned into a 30s thumbnail...\n', thumb1);
fprintf('Please wait..!\n\n');
%# load the signal
[y, fs, nb] = wavread(thumb1);
y = mean(y,2); %# stereo, take avrg of 2 channels
%# Calculate frame energy
fWidth = round(fs*1); %# 10ms
numFrames = floor(length(y)/fWidth);
energy = zeros(1,numFrames);
for f=1:numFrames
energy(f) = sum( y((f-1)*fWidth+1:f*fWidth).^2 );
end
%# smooth the signal (moving average with window size = 1% * length of data)
WINDOW_SIZE = round(length(energy) * 0.01); %# 200
XX = filtfilt(ones(1,WINDOW_SIZE)/WINDOW_SIZE, 1, energy);
%# auto-correlation
[r,lags] = xcorr(XX, 'biased');
%# find extrema points
dr = diff(r);
eIdx = find(dr(1:end-1) .* dr(2:end) <= 0) + 1;
[~,loc] = sort(r(eIdx), 'descend');
loc = loc(1:min(3,end)); %# take the highest 3 values
inf=lags( eIdx(loc) );
thumb=max(inf);
startrecord=round((thumb/1)*fs);
endrecord=round(((thumb+30)/1)*fs);
wavwrite(y(startrecord:endrecord), fs, nb, 'Temp1');
fprintf('The thumbnail of %s has been created.\n\n', thumb1);
Sorry that it all looks so messy, but I wanted to get some visual examples in!
Those are inflexion points, and if you take the first derivative of your data you will see those as big deviations from a pretty constant slope.
Play with a boxcar average to do minimal smoothing and with the first derivative to find those inflexion points.
I believe that it will be easier to find peaks in the negative value of the second derivative. Constant or nearly constant slope will result in 2nd-der =0; anything else, including your actual peaks and inflection points, will have a non-zero 2nd-der. By finding peaks in the negative values, you'll only get the positive peaks, not the negative peaks.