Finding the angle between the view vector and the surface normal can be beneficial in getting the visible surfaces since we use it to conduct back face culling techniques and obtaining contours, crease edges of the object.
To obtain the visible surfaces I use the back face culling code below:
N = normals(vertex,faces);
BC = barycenter(vertex,faces);
back_facing = sum(N.*bsxfun(#minus,BC,campos),2)<=0
t.FaceVertexCData = 1*(sum(N.*bsxfun(#minus,BC,campos),2)<=0)
t.FaceVertexCData(sum(N.*bsxfun(#minus,BC,campos),2)>0) = nan;
faces1=faces(t.FaceVertexCData(:)==1,:);
facesv=sort(unique(faces1(:)));
How does one obtain the angle?
r=(sum(N.*bsxfun(#minus,BC,campos),2))
rr=bsxfun(#minus,BC,campos);
V_mag= sqrt(rr(:,1).^2+rr(:,2).^2+rr(:,3).^2);
N_mag= sqrt(N(:,1).^2+N(:,2).^2+N(:,3).^2);
for i = 1:(size(r,1))
A(i)=acosd(r(i)/(N_mag(i).*V_mag(i)));
end
This is what I have done thus far. I am not sure if it is correct and code is slow.
Related
I have this 3D image generated from the simple code below.
% Input Image size
imageSizeY = 200;
imageSizeX = 120;
imageSizeZ = 100;
%# create coordinates
[rowsInImage, columnsInImage, pagesInImage] = meshgrid(1:imageSizeY, 1:imageSizeX, 1:imageSizeZ);
%# get coordinate array of vertices
vertexCoords = [rowsInImage(:), columnsInImage(:), pagesInImage(:)];
centerY = imageSizeY/2;
centerX = imageSizeX/2;
centerZ = imageSizeZ/2;
radius = 28;
%# calculate distance from center of the cube
sphereVoxels = (rowsInImage - centerY).^2 + (columnsInImage - centerX).^2 + (pagesInImage - centerZ).^2 <= radius.^2;
%# Now, display it using an isosurface and a patch
fv = isosurface(sphereVoxels,0);
patch(fv,'FaceColor',[0 0 .7],'EdgeColor',[0 0 1]); title('Binary volume of a sphere');
view(45,45);
axis equal;
grid on;
xlabel('x-axis [pixels]'); ylabel('y-axis [pixels]'); zlabel('z-axis [pixels]')
I have tried plotting the image with isosurface and some other volume visualization tools, but there remains quite a few surprises for me from the plots.
The code has been written to conform to the image coordinate system (eg. see: vertexCoords) which is a left-handed coordinate system I presume. Nonetheless, the image is displayed in the Cartesian (right-handed) coordinate system. I have tried to see this displayed as the figure below, but that’s simply not happening.
I am wondering if the visualization functions have been written to display the image the way they do.
Image coordinate system:
Going forward, there are other aspects of the code I am to write for example if I have an input image sphereVoxels as in above, in addition to visualizing it, I would want to find north, south east, west, top and bottom locations in the image, as well as number and count the coordinates of the vertices, plus more.
I foresee this would likely become confusing for me if I don’t stick to one coordinate system, and considering that the visualization tools predominantly use the right-hand coordinate system, I would want to stick with that from the onset. However, I really do not know how to go about this.
Right-hand coordinate system:
Any suggestions to get through this?
When you call meshgrid, the dimensions x and y axes are switched (contrary to ndgrid). For example, in your case, it means that rowsInImage is a [120x100x200] = [x,y,z] array and not a [100x120x200] = [y,x,z] array even if meshgrid was called with arguments in the y,x,z order. I would change those two lines to be in the classical x,y,z order :
[columnsInImage, rowsInImage, pagesInImage] = meshgrid(1:imageSizeX, 1:imageSizeY, 1:imageSizeZ);
vertexCoords = [columnsInImage(:), rowsInImage(:), pagesInImage(:)];
Calibration:
I have calibrated the camera using this vision toolbox in Matlab. I used checkerboard images to do so. After calibration I get the cameraParams
which contains:
Camera Extrinsics
RotationMatrices: [3x3x18 double]
TranslationVectors: [18x3 double]
and
Camera Intrinsics
IntrinsicMatrix: [3x3 double]
FocalLength: [1.0446e+03 1.0428e+03]
PrincipalPoint: [604.1474 359.7477]
Skew: 3.5436
Aim:
I have recorded trajectories of some objects in motion using this camera. Each object corresponds to a single point in a frame. Now, I want to project the points such that I get a top-view.
Note all these points I wish to transform are are the on the same plane.
ex: [xcor_i,ycor_i ]
-101.7000 -77.4040
-102.4200 -77.4040
KEYPOINT: This plane is perpendicular to one of images of checkerboard used for calibration. For that image(below), I know the height of origin of the checkerboard of from ground(193.040 cm). And the plane to project the points on is parallel to the ground and perpendicular to this image.
Code
(Ref:https://stackoverflow.com/a/27260492/3646408 and answer by #Dima below):
function generate_homographic_matrix()
%% Calibrate camera
% Define images to process
path=['.' filesep 'Images' filesep];
list_imgs=dir([path '*.jpg']);
list_imgs_path=strcat(path,{list_imgs.name});
% Detect checkerboards in images
[imagePoints, boardSize, imagesUsed] = detectCheckerboardPoints(list_imgs_path);
imageFileNames = list_imgs_path(imagesUsed);
% Generate world coordinates of the corners of the squares
squareSize = 27; % in units of 'mm'
worldPoints = generateCheckerboardPoints(boardSize, squareSize);
% Calibrate the camera
[cameraParams, imagesUsed, estimationErrors] = estimateCameraParameters(imagePoints, worldPoints, ...
'EstimateSkew', true, 'EstimateTangentialDistortion', true, ...
'NumRadialDistortionCoefficients', 3, 'WorldUnits', 'mm');
%% Compute homography for peripendicular plane to checkerboard
% Detect the checkerboard
im=imread(['.' filesep 'Images' filesep 'exp_19.jpg']); %exp_19.jpg is the checkerboard orthogonal to the floor
[imagePoints, boardSize] = detectCheckerboardPoints(im);
% Compute rotation and translation of the camera.
[Rc, Tc] = extrinsics(imagePoints, worldPoints, cameraParams);
% Rc(rotation of the calibration view w.r.t the camera) = [x y z])
%then the floor has rotation Rf = [z x -y].(Normal vector of the floor goes up.)
Rf=[Rc(:,3),Rc(:,1),Rc(:,2)*-1];
% Translate it to the floor
H=452;%distance btw origin and floor
Fc = Rc * [0; H; 0];
Tc = Tc + Fc';
% Combine rotation and translation into one matrix:
Rf(3, :) = Tc;
% Compute the homography between the checkerboard and the image plane:
H = Rf * cameraParams.IntrinsicMatrix;
save('homographic_matrix.mat','H')
end
%% Transform points
function [x_transf,y_transf] =transform_points(xcor_i,ycor_i)
% creates a projective2D object and then transforms the points forward to
% get a top-view
% xcor_i and ycor_i are 1d vectors comprising of the x-coordinates and
% y-coordinates of trajectories.
data=load('homographic_matrix.mat');
homo_matrix=data.H;
tform=projective2d(inv(homo_matrix));
[x_transf,y_transf] = transformPointsForward(tform,xcor_i,ycor_i);
end
Quoting text from OReilly Learning OpenCV Pg 412:
"Once we have the homography matrix and the height parameter set as we wish, we could
then remove the chessboard and drive the cart around, making a bird’s-eye view video
of the path..."
This what I essentially wish to achieve.
Abhishek,
I don't entirely understand what you are trying to do. Are your points on a plane, and are you trying to create a bird's eye view of that plane?
If so, then you need to know the extrinsics, R and t, describing the relationship between that plane and the camera. One way to get R and t is to place a checkerboard on the plane, and then use the extrinsics function.
After that, you can follow the directions in the question you cited to get the homography. Once you have the homography, you can create a projective2D object, and use its transformPointsForward method to transform your points.
Since you have the size of squares on the grid, then given 2 points that you know are connected by an edge of size E (in real world units), you can calculate their 3D position.
Taking the camera intrinsic matrix K and the 3D position C and the camera orientation matrix R, you can calculate a ray to each of the points p by doing:
D = R^T * K^-1 * p
Each 3D point is defined as:
P = C + t*D
and you have the constraint that ||P1-P2|| = E
then it's a matter of solving for t1,t2 and finding the 3D position of the two points.
In order to create a top view, you can take the 3D points and project them using a camera model for that top view to generate a new image.
If all your points are on a single plane, it's enough to calculate the position of 3 points, and you can extrapolate the rest.
If your points are located on a plane that you know one coordinate of, you can do it simply for each point. For example, if you know that your camera is located at height h=C.z, and you want to find the 3D location of points in the frame, given that they are on the floor (z=0), then all you have to do is calculate the direction D as above, and then:
t=abs( (h-0)/D.z )
The 0 represent the height of the plane. Substitute for any other value for other planes.
Now that you have the value of t, you can calculate the 3D position of each point: P=C+t*D.
Then, to create a top view, create a new camera position and rotation to match your required projection, and you can project each point onto this camera's image plane.
If you want a full image, you can interpolate positions and fill in the blanks where no feature point was present.
For more details, you can always read: http://www.robots.ox.ac.uk/~vgg/hzbook/index.html
I am trying to perform human body animation using translation and orientation data I am given. I have a set of rigid body segments made using patch all centered at (0,0,0) to represent the human body and translated accordingly. I have set up a hierarchy for each of them and performed a transformation matrix for each rigid body segment. The limb segments begin to offset one another and give problems. For example, the rigid body of the arm moves as if it does not have a relative point of origin even though it follows the proper motion. The motion is akin to moving the rigid body from the patch center of gravity? Whereas it is supposed to move with one end being fixed while the other end follows translation data. Can someone let me know what it is that I am doing wrong? Layout of my code is:
% Body segment lengths
xlength = somevalue
ylength = somevalue
zlength = somevalue
% Translation data
Xdata
Ydata
Zdata
% Orientation data
Yaw = rotation about z axis
Pitch = rotation about x axis
Roll = rotation about y axis
Vertices = [xlength*ones(8,1),ylength*ones(8,1),zlength*ones(8,1)]...
.*[-0.5,-0.5,-0.5;
0.5,-0.5,-0.5;
-0.5,0.5,-0.5;
-0.5,-0.5,0.5;
0.5,0.5,-0.5;
-0.5,0.5,0.5;
0.5,-0.5,0.5;
0.5,0.5,0.5];
% Create patches
for i = 1:6
% create faces for patches
end
% create axes
ax = axes(...)
% draw patches
bodysegmentPatch = patch(patchxdata,patchydata,patchzdata)
% create hierarchy using hgtransform
pelvis = hgtransform('Parent',ax);
trunk = hgtransform('Parent',pelvis);
head = hgtransform('Parent',trunk);
leftupperarm = hgtransform('Parent',trunk);
leftforearm = hgtransform('Parent',leftupperarm);
rightupperarm = hgtransform('Parent',trunk);
rightforearm = hgtransform('Parent',rightupperarm);
leftthigh = hgtransform('Parent',pelvis);
leftcalf = hgtransform('Parent',leftthigh);
rightthigh = hgtransform('Parent',pelvis);
rightcalf = hgtransform('Parent',rightthigh);
% set patches to hierarchy
set(pelvisPatch,'Parent',pelvis)
% Animation loop
for i = 1:n
% translation of body segment
bodysegmentT = makehgtform('translate',[x(i) y(i) z(i)]);
% rotation of body segment
bodysegmentR = makehgtform('yrotate',Roll(i),'xrotate',Pitch(i),'zrotate',Yaw(i));
% Create transform matrices
set(pelvis,'Matrix',pelvisR);
set(trunk,'Matrix',trunkR*pelvisR);
set(leftupperarm,'Matrix',leftupperarmT*leftupperarmR*trunkR*pelvisR);
drawnow
end
I'm not sure what your problem is exactly, but without looking at your code too closely my guess is it is likely one of two common mistakes while doing these skeleton transformations.
Your matrix transformations are in the wrong order. Remember that A = A*B is not the same thing as A = B*A. When you do this kind of transformation stack this order is very important.
The objects are rotating around the wrong point. Usually they rotate around the origin. So if you want something to rotate around the center of the object you will have to translate the image to the origin, rotate the image, translate the image back to the original location.
Don't give up! These transformations can be tricky, and often times it may look totally chaotic while in reality the code is really close to being correct.
I have an inverted pendulum video here which is 33 second length. The objective is to plot a red point in the center of moving rectangle part of the pendulum and to plot a line along the black stick calculating its angle for every frame.
I have handled the video frame by frame. Then I have used Object Detection In A Cluttered Scene Using Point Feature Matching. It would be good if I had access to the matching point's indexes and then I would easily calculate the angle.
I have thought that I can get the moving rectangle part's region and seek the similar regions in the next frames. But this solution seems too local.
I do not know which techniques to apply.
clear all;
clc;
hVideoFileReader = vision.VideoFileReader;
hVideoPlayer = vision.VideoPlayer;
hVideoFileReader.Filename = 'inverted-pendulum.avi';
hVideoFileReader.VideoOutputDataType = 'single';
while ~isDone(hVideoFileReader)
grayFrame = rgb2gray(step(hVideoFileReader));
frame = step(hVideoFileReader);
if isFirstFrame
part = grayFrame(202:266,202:282); % #moving part's region
isFirstFrame = false;
subplot(1,2,1);
imshow(part);
end
partPoints = detectSURFFeatures(part);
grayFramePoints = detectSURFFeatures(grayFrame);
hold on;
subplot(1,2,1), plot(partPoints .selectStrongest(10));
subplot(1,2,2), imshow(grayFrame);
subplot(1,2,2), plot(grayFramePoints .selectStrongest(20));
frame2 = pointPendulumCenter(frame);
frame3 = plotLineAlongStick(frame2);
step(hVideoPlayer, frame3);
hold off;
end
release(hVideoFileReader);
release(hVideoPlayer);
%% #Function to find the moving part's center point and plot a red dot on it.
function f = pointPendulumCenter(frame)
end
%% #Function to plot a red line along the stick after calculating the angle of it.
function f = plotLineAlongStick(frame)
end
It would make the problem much easier if your camera did not move. If you take your video with a stationary camera (e.g. mounted on a tripod) then you can use vision.ForegroundDetector to segment out moving objects from the static background.
Below is an arbitrary hand-drawn Intensity profile of a line in an image:
The task is to draw the line. The profile can be approximated to an arc of a circle or ellipse.
This I am doing for camera calibration. Since I do not have the actual industrial camera, I am trying to simulate the correction needed for calibration.
The question can be rephrased as I want pixel values which will follow a plot similar to the above. I want to do this using program (Preferably using opencv) and not manually enter these values because I have thousands of pixels in the line.
An algorithm/pseudo code will suffice. Also please note that I do not have any actual Intensity profile, otherwise I would have read those values.
When will you encounter such situation ?
Suppose you take a picture (assuming complete white) from a Camera, your object being placed on table, and camera just above it in vertical direction. The light coming on the center of the picture vertically downward from the camera will be stronger in intensity as compared to the light reflecting at the edges. You measure pixel values across any line in the Image, you will find intensity curve like shown above. Since I dont have camera for the time being, I want to emulate this situation. How to achieve this?
This is not exactly image processing, rather image generation... but anyways.
Since you want an arc, we still need three points on that arc, lets take the first, middle and last point (key characteristics in my opinion):
N = 100; % number of pixels
x1 = 1;
x2 = floor(N/2);
x3 = N;
y1 = 242;
y2 = 255;
y3 = 242;
and now draw a circle arc that contains these points.
This problem is already discussed here for matlab: http://www.mathworks.nl/matlabcentral/newsreader/view_thread/297070
x21 = x2-x1; y21 = y2-y1;
x31 = x3-x1; y31 = y3-y1;
h21 = x21^2+y21^2; h31 = x31^2+y31^2;
d = 2*(x21*y31-x31*y21);
a = x1+(h21*y31-h31*y21)/d; % circle center x
b = y1-(h21*x31-h31*x21)/d; % circle center y
r = sqrt(h21*h31*((x3-x2)^2+(y3-y2)^2))/abs(d); % circle radius
If you assume the middle value is always larger (and thus it's the upper part of the circle you'll have to plot), you can draw this with:
x = x1:x3;
y = b+sqrt(r^2-(x-a).^ 2);
plot(x,y);
you can adjust the visible window with
xlim([1 N]);
ylim([200 260]);
which gives me the following result: