I am working on video stabilisation ( making shaky videos non-shaky) using matlab.
One of the steps is to find a smooth camera path given the unstable camera path.
The unstable camera path is one which gives the jittering or shake to the video.
I have camera path specified using camera position which is a 3d-data.
camera path - (cx,cy,cz);
As i plot in matlab, i can visually see the shakiness of the camera motion.
So now i require a least squares fitting to be done on the camera path specified by(cx,cy,cz);
I came across polyfit() which does fitting for 2-dimensional data.
But what i need is a 3-d smooth curve fit to the shaky curve.
Thanks in advance.
Couldn't you just fit three separate 1d curves for cx(t), cy(t), cz(t)?
BTW: I think what you need is a Kalman filter, not a polynomial fit to the camera path. But I'm not sure if matlab has builtin support for that.
Approach using least square fit:
t = (1:0.1:5)';
% model
px = [ 5 2 1 ];
x = polyval(px,t);
py = [ -2 1 1 ];
y = polyval(py,t);
pz = [ 1 20 1 ];
z = polyval(pz,t);
% plot model
figure
plot3(x,y,z)
hold all
% simulate measurement
xMeasured = x+2*(rand(length(x),1)-0.5);
yMeasured = y+2*(rand(length(y),1)-0.5);
zMeasured = z+2*(rand(length(z),1)-0.5);
% plot simulated measurements
plot3(xMeasured, yMeasured, zMeasured,'or')
hold off
grid on
% least squares fit
A = [t.^2, t, t./t];
pxEstimated = A\xMeasured;
pyEstimated = A\yMeasured;
pzEstimated = A\zMeasured;
Let me be grateful to stackoverflow.com first of all and then my thanks to zellus and nikie who had started me thinking about the problem more. So now I have reached the solution which follows zellus approach and as nikie pointed out I used parameter 't' .
cx, cy,cz are the coordinates in 3d space and in my case they are all 343x1 doubles
My final code is shown below which fits the 3d data set:
t = linspace(1,343,343)';
load cx.mat;
load cy.mat;
load cz.mat;
plot3(cx, cy, cz,'r'),title('source Camera Path');
hold all
A = [t.^2, t, t./t];
fx = A\cx;
fy = A\cy;
fz = A\cz;
Xev = polyval(fx,t);
Yev = polyval(fy,t);
Zev = polyval(fz,t);
plot3(Xev,Yev,Zev,'+b'),title('Fitting Line');
I look forward to more interesting discussions on StackOverflow with great helpful people.
Related
My binary image has rectangular rotated objects of known size on it. I'd like to get the object inclination using axis-aligned bounding box that MATLAB's regionprops returns. What are my suggestions:
Let bounding box width be W, side of rectangle be C and inclination alpha
Then
Using Weierstrass substitution
After some simplification:
Solving the equation for tan(alpha/2) with
For any nonzero inclination discriminant is positive.
Logic seems to be OK, so as math. Could you please point where I make a mistake, or what is a better way to get inclination?
Here is corresponding MATLAB code:
img = false(25,25);
img(5:16,5:16) = true;
rot_img = imrotate(img, 30, 'crop');
props = regionprops(bwlabel(rot_img),'BoundingBox');
bbox = cat(1,props.BoundingBox);
w = bbox(3);
h = 12;
a = -1*(1+w/h); b = 2; c = 1 - w/h;
D = b^2 - 4*a*c;
alpha = 2*atand((-b + sqrt(D))/(2*a));
%alpha = 25.5288
EDIT Thank you for trigonometry hints. They significantly simplify the calculations, but they give wrong answer. As I now understand, the question is asked in wrong way. The thing I really need is finding inclination of short lines (10-50 pixels) with high accuracy (+/- 0.5 deg), the lines' position is out of interest.
The approach used in the question and answers show better accuracy for long lines, for c = 100 error is less than 0.1 degree. That means we're into rasterization error here, and need subpixel accuracy. At the moment I have only one algorithm that solves the problem - Radon transform, but I hope you can recommend something else.
p = bwperim(rot_img);
theta=0:0.1:179.9;
[R,xp] = radon(p,theta); %Radon transform of contours
a=imregionalmax(R,true(3,3)); %Regional maxima of the transform
[r,c]=find(a); idx=sub2ind(size(a),r,c); maxvals=R(idx);
[val,midx]=sort(maxvals,'descend'); %Choose 4 highest maxima
mean(rem(theta(c(midx(1:4))),90)) %And average corresponding angles
%29.85
If rectangle is square:
w/c=sin(a)+cos(a)
(w/c)^2=1+sin(2a)
sin(2a)=(w/c)^2-1
a=0.5*arcsin((w/c)^2-1)
May be use regionprops function with 'Orientation' option...
I have this 3D image generated from the simple code below.
% Input Image size
imageSizeY = 200;
imageSizeX = 120;
imageSizeZ = 100;
%# create coordinates
[rowsInImage, columnsInImage, pagesInImage] = meshgrid(1:imageSizeY, 1:imageSizeX, 1:imageSizeZ);
%# get coordinate array of vertices
vertexCoords = [rowsInImage(:), columnsInImage(:), pagesInImage(:)];
centerY = imageSizeY/2;
centerX = imageSizeX/2;
centerZ = imageSizeZ/2;
radius = 28;
%# calculate distance from center of the cube
sphereVoxels = (rowsInImage - centerY).^2 + (columnsInImage - centerX).^2 + (pagesInImage - centerZ).^2 <= radius.^2;
%# Now, display it using an isosurface and a patch
fv = isosurface(sphereVoxels,0);
patch(fv,'FaceColor',[0 0 .7],'EdgeColor',[0 0 1]); title('Binary volume of a sphere');
view(45,45);
axis equal;
grid on;
xlabel('x-axis [pixels]'); ylabel('y-axis [pixels]'); zlabel('z-axis [pixels]')
I have tried plotting the image with isosurface and some other volume visualization tools, but there remains quite a few surprises for me from the plots.
The code has been written to conform to the image coordinate system (eg. see: vertexCoords) which is a left-handed coordinate system I presume. Nonetheless, the image is displayed in the Cartesian (right-handed) coordinate system. I have tried to see this displayed as the figure below, but that’s simply not happening.
I am wondering if the visualization functions have been written to display the image the way they do.
Image coordinate system:
Going forward, there are other aspects of the code I am to write for example if I have an input image sphereVoxels as in above, in addition to visualizing it, I would want to find north, south east, west, top and bottom locations in the image, as well as number and count the coordinates of the vertices, plus more.
I foresee this would likely become confusing for me if I don’t stick to one coordinate system, and considering that the visualization tools predominantly use the right-hand coordinate system, I would want to stick with that from the onset. However, I really do not know how to go about this.
Right-hand coordinate system:
Any suggestions to get through this?
When you call meshgrid, the dimensions x and y axes are switched (contrary to ndgrid). For example, in your case, it means that rowsInImage is a [120x100x200] = [x,y,z] array and not a [100x120x200] = [y,x,z] array even if meshgrid was called with arguments in the y,x,z order. I would change those two lines to be in the classical x,y,z order :
[columnsInImage, rowsInImage, pagesInImage] = meshgrid(1:imageSizeX, 1:imageSizeY, 1:imageSizeZ);
vertexCoords = [columnsInImage(:), rowsInImage(:), pagesInImage(:)];
This question is based on a modified Matlab code from the online documentation for the optical flow system objects in version 2015a as appears in opticalFlowLK class
clc; clearvars; close all;
inputVid = VideoReader('viptraffic.avi');
opticFlow = opticalFlowLKDoG('NumFrames',3);
inputVid.currentTime = 2;
k = 1;
while inputVid.currentTime<=2 + 1/inputVid.FrameRate
frameRGB{k} = readFrame(inputVid);
frameGray{k} = rgb2gray(frameRGB{k});
flow{k} = estimateFlow(opticFlow,frameGray{k});
k = k+1;
end
By looking at flow{2}.Vx and flow{2}.Vy I get the motion maps U and V that describe the motion from frameGray{1} to frameGray{2}.
Iwant to use flow{2}.Vx and flow{2}.Vy directly on the data in frameGray{1} in order to warp frameGray{1} to appear visually similar to frameGray{2}.
I tried this code:
[x, y] = meshgrid(1:size(frameGray{1},2), 1:size(frameGray{1},1));
frameGray1Warped = interp2(double(frameGray{1}) , x-flow{2}.Vx , y-flow{2}.Vy);
But it doesn't seem to do much at all except ruin the image quality (but the objects don't display any real motion towards their locations in frameGray{2}.
I added 3 images showing the 2 original frames followed by frame 1 warped using the motion field to appear similar to frame 2:
It can be seen easily that frame 1 warped to 2 is essentially frame 1 with degraded quality but the cars haven't moved at all. That is - the location of the cars is the same: look at the car closest to the camera with respect to the road separation line near it; it's virtually the same in frame 1 and frame 1 warped to 2, but is quite different in frame 2.
I am working on an application where I need to rectify an image taken from a mobile camera platform. The platform measures roll, pitch and yaw angles, and I want to make it look like the image is taken from directly above, by some sort of transform from this information.
In other words, I want a perfect square lying flat on the ground, photographed from afar with some camera orientation, to be transformed, so that the square is perfectly symmetrical afterwards.
I have been trying to do this through OpenCV(C++) and Matlab, but I seem to be missing something fundamental about how this is done.
In Matlab, I have tried the following:
%% Transform perspective
img = imread('my_favourite_image.jpg');
R = R_z(yaw_angle)*R_y(pitch_angle)*R_x(roll_angle);
tform = projective2d(R);
outputImage = imwarp(img,tform);
figure(1), imshow(outputImage);
Where R_z/y/x are the standard rotational matrices (implemented with degrees).
For some yaw-rotation, it all works just fine:
R = R_z(10)*R_y(0)*R_x(0);
Which gives the result:
If I try to rotate the image by the same amount about the X- or Y- axes, I get results like this:
R = R_z(10)*R_y(0)*R_x(10);
However, if I rotate by 10 degrees, divided by some huge number, it starts to look OK. But then again, this is a result that has no research value what so ever:
R = R_z(10)*R_y(0)*R_x(10/1000);
Can someone please help me understand why rotating about the X- or Y-axes makes the transformation go wild? Is there any way of solving this without dividing by some random number and other magic tricks? Is this maybe something that can be solved using Euler parameters of some sort? Any help will be highly appreciated!
Update: Full setup and measurements
For completeness, the full test code and initial image has been added, as well as the platforms Euler angles:
Code:
%% Transform perspective
function [] = main()
img = imread('some_image.jpg');
R = R_z(0)*R_y(0)*R_x(10);
tform = projective2d(R);
outputImage = imwarp(img,tform);
figure(1), imshow(outputImage);
end
%% Matrix for Yaw-rotation about the Z-axis
function [R] = R_z(psi)
R = [cosd(psi) -sind(psi) 0;
sind(psi) cosd(psi) 0;
0 0 1];
end
%% Matrix for Pitch-rotation about the Y-axis
function [R] = R_y(theta)
R = [cosd(theta) 0 sind(theta);
0 1 0 ;
-sind(theta) 0 cosd(theta) ];
end
%% Matrix for Roll-rotation about the X-axis
function [R] = R_x(phi)
R = [1 0 0;
0 cosd(phi) -sind(phi);
0 sind(phi) cosd(phi)];
end
The initial image:
Camera platform measurements in the BODY coordinate frame:
Roll: -10
Pitch: -30
Yaw: 166 (angular deviation from north)
From what I understand the Yaw-angle is not directly relevant to the transformation. I might, however, be wrong about this.
Additional info:
I would like specify that the environment in which the setup will be used contains no lines (oceanic photo) that can reliably used as a reference (the horizon will usually not be in the picture). Also the square in the initial image is merely used as a measure to see if the transformation is correct, and will not be there in a real scenario.
So, this is what I ended up doing: I figured that unless you are actually dealing with 3D images, rectifying the perspective of a photo is a 2D operation. With this in mind, I replaced the z-axis values of the transformation matrix with zeros and ones, and applied a 2D Affine transformation to the image.
Rotation of the initial image (see initial post) with measured Roll = -10 and Pitch = -30 was done in the following manner:
R_rotation = R_y(-60)*R_x(10);
R_2d = [ R_rot(1,1) R_rot(1,2) 0;
R_rot(2,1) R_rot(2,2) 0;
0 0 1 ]
This implies a rotation of the camera platform to a virtual camera orientation where the camera is placed above the scene, pointing straight downwards. Note the values used for roll and pitch in the matrix above.
Additionally, if rotating the image so that is aligned with the platform heading, a rotation about the z-axis might be added, giving:
R_rotation = R_y(-60)*R_x(10)*R_z(some_heading);
R_2d = [ R_rot(1,1) R_rot(1,2) 0;
R_rot(2,1) R_rot(2,2) 0;
0 0 1 ]
Note that this does not change the actual image - it only rotates it.
As a result, the initial image rotated about the Y- and X-axes looks like:
The full code for doing this transformation, as displayed above, was:
% Load image
img = imread('initial_image.jpg');
% Full rotation matrix. Z-axis included, but not used.
R_rot = R_y(-60)*R_x(10)*R_z(0);
% Strip the values related to the Z-axis from R_rot
R_2d = [ R_rot(1,1) R_rot(1,2) 0;
R_rot(2,1) R_rot(2,2) 0;
0 0 1 ];
% Generate transformation matrix, and warp (matlab syntax)
tform = affine2d(R_2d);
outputImage = imwarp(img,tform);
% Display image
figure(1), imshow(outputImage);
%*** Rotation Matrix Functions ***%
%% Matrix for Yaw-rotation about the Z-axis
function [R] = R_z(psi)
R = [cosd(psi) -sind(psi) 0;
sind(psi) cosd(psi) 0;
0 0 1];
end
%% Matrix for Pitch-rotation about the Y-axis
function [R] = R_y(theta)
R = [cosd(theta) 0 sind(theta);
0 1 0 ;
-sind(theta) 0 cosd(theta) ];
end
%% Matrix for Roll-rotation about the X-axis
function [R] = R_x(phi)
R = [1 0 0;
0 cosd(phi) -sind(phi);
0 sind(phi) cosd(phi)];
end
Thank you for the support, I hope this helps someone!
I think you can derive transformation this way:
1) Let you have four 3d-points A(-1,-1,0), B(1,-1,0), C(1,1,0) and D(-1,1,0). You can take any 4 noncollinear points. They not related to image.
2) You have transformation matrix, so you can set your camera by multiplying points coords by transformation matrix. And you'll get 3d coords relative to camera position/direction.
3) You need to get projection of your points to screen plane. The simpliest way is to use ortographic projection (simply ignore depth coordinate). On this stage you've got 2D projections of transformed points.
4) Once you have 2 sets of 2D points coordinates (the set from step 1 without 3-rd coordinate and the set from step 3), you can compute homography matrix in standard way.
5) Apply inverse homograhy transformation to your image.
You need to estimate a homography. For an off-the-shelf Matlab solution, see function vgg_H_from_x_lin.m from http://www.robots.ox.ac.uk/~vgg/hzbook/code/ .
For the theory dig into a Computer Vision textbook, such as the one available freely at http://szeliski.org/Book/ or in Chapter 3 of http://programmingcomputervision.com/downloads/ProgrammingComputerVision_CCdraft.pdf
Maybe my answer is not correct due to my mis-understanding of the camera parameters, but I was wondering whether the Yaw/Pitch/Roll is relative to the position of your object. I used the formula of general rotations, and my code is below (the rotation functions R_x, R_y, and R_z were copied from yours, I didn't paste them here)
close all
file='http://i.stack.imgur.com/m5e01.jpg'; % original image
I=imread(file);
R_rot = R_x(-10)*R_y(-30)*R_z(166);
R_rot = inv(R_rot);
R_2d = [ R_rot(1,1) R_rot(1,2) 0;
R_rot(2,1) R_rot(2,2) 0;
0 0 1 ];
T = maketform('affine',R_2d);
transformedI = imtransform(I,T);
figure, imshow(I), figure, imshow(transformedI)
The result:
This indicates that you still need some rotation operation to get the 'correct' alignment in your mind (but probably not necessary the correct position in the camera's mind).
So I change R_rot = inv(R_rot); to R_rot = inv(R_rot)*R_x(-5)*R_y(25)*R_z(180);, and now it gave me:
Looks better like what you want.
Thanks.
Below is an arbitrary hand-drawn Intensity profile of a line in an image:
The task is to draw the line. The profile can be approximated to an arc of a circle or ellipse.
This I am doing for camera calibration. Since I do not have the actual industrial camera, I am trying to simulate the correction needed for calibration.
The question can be rephrased as I want pixel values which will follow a plot similar to the above. I want to do this using program (Preferably using opencv) and not manually enter these values because I have thousands of pixels in the line.
An algorithm/pseudo code will suffice. Also please note that I do not have any actual Intensity profile, otherwise I would have read those values.
When will you encounter such situation ?
Suppose you take a picture (assuming complete white) from a Camera, your object being placed on table, and camera just above it in vertical direction. The light coming on the center of the picture vertically downward from the camera will be stronger in intensity as compared to the light reflecting at the edges. You measure pixel values across any line in the Image, you will find intensity curve like shown above. Since I dont have camera for the time being, I want to emulate this situation. How to achieve this?
This is not exactly image processing, rather image generation... but anyways.
Since you want an arc, we still need three points on that arc, lets take the first, middle and last point (key characteristics in my opinion):
N = 100; % number of pixels
x1 = 1;
x2 = floor(N/2);
x3 = N;
y1 = 242;
y2 = 255;
y3 = 242;
and now draw a circle arc that contains these points.
This problem is already discussed here for matlab: http://www.mathworks.nl/matlabcentral/newsreader/view_thread/297070
x21 = x2-x1; y21 = y2-y1;
x31 = x3-x1; y31 = y3-y1;
h21 = x21^2+y21^2; h31 = x31^2+y31^2;
d = 2*(x21*y31-x31*y21);
a = x1+(h21*y31-h31*y21)/d; % circle center x
b = y1-(h21*x31-h31*x21)/d; % circle center y
r = sqrt(h21*h31*((x3-x2)^2+(y3-y2)^2))/abs(d); % circle radius
If you assume the middle value is always larger (and thus it's the upper part of the circle you'll have to plot), you can draw this with:
x = x1:x3;
y = b+sqrt(r^2-(x-a).^ 2);
plot(x,y);
you can adjust the visible window with
xlim([1 N]);
ylim([200 260]);
which gives me the following result: