I have to implement a basic tracking program in MATLAB that, given a set of frames from a videogame, it analyzes each one of them and then creates a bounding box around each object. I've used the function regionprops in order to obtain the coordinates of the bounding boxes for each object, and visualized them using the function rectangle, as follows:
for i = 1:size( frames,2 )
CC{1,i} = findConnectedComponents( frames{1,i} );
stats{1,i} = regionprops( 'struct',CC{1,i},'BoundingBox','Centroid' );
imshow( frames{1,i} ),hold on
for j = 1:size(stats{1,i},1)
r = rectangle( 'Position',stats{1,i}(j).BoundingBox );
r.FaceColor = [0 0.5 0.5 0.45];
end
end
This works just fine, but I'd like to go one step further and be able to differenciate static objects from moving objects. I thought of using the centroid to see, for each object, if it is different in each frame (which would mean that the object is moving), but in each image I have a different number of objects.
For example, if I am trying this on Space Invaders, when you kill an alien it disappears, so the number of objects is reduced. Also each projectile is a separate object and there could be a different number of projectiles in different moments of the game.
So my question is, how could I classify the objects based on wether they move or not, and paint them with two different colors?
In the case of consistent background, using optical flow is ideal for you.
The basic idea is pretty simple, consider subtracting two consecutive frames, and use this to get flow vector of the objects that moved between frames.
You can look at Lucas–Kanade method
and Horn–Schunck method.
Here is a link for matlab implementation of the same.
Related
i want to create a shader that can cover a surface with "circles" from many random positions.
the circles keep growing until all surface covered with them.
here my first try with amplify shader editor.
the problem is i don't know how make this shader that create array of "point maker" with random positions.also i want to controll circles with
c# example:
point_maker = new point_maker[10];
point_maker[1].position = Vector2.one;
point_maker[1].scale = 1;
and etc ...
Heads-up: That's probably not the way to do what you're looking for, as every pixel in your shader would need to loop over all your input points, while each of those pixels will only be covered by one at most. It's a classic case of embracing the benefits of the parallel nature of shaders. (The keyword for me here is 'random', as in 'random looking').
There's 2 distinct problems here: generating circles, and masking them.
I would go onto generating a grid out of your input space (most likely your UV coordinates so I'll assume that from here), by taking the fractional part of the coords scaled by some value: UV (usually) go between 0 and 1, so if you want 100 circles you'd multiply the coord by 10. You now have a grid of 100 pieces of UVs, where you can do something similar to what you have to generate the circle (tip: dot product a vector on itself gives the square distance, which is much cheaper to compute).
You want some randomness, so you need to add some offset to the center of the circle. You need some sort of random number (there might be some in ASE I can't remember, or make one your own - there's plenty of that you look online) that is unique per cell of the grid. To do this you'd input the remainder of your frac() as value to your hash/random method. You also need to limit that offset depending on the radius of the circle so it doesn't touch the sides of the cell. You can overlay more than one layer of circles if you want more coverage as well.
Second step is to figure out if you want to display those circles at all, and for this you could make the drawing conditional to the distance from the center of the circle to an input coordinate you provide to the shader, by some threshold. (it doesn't have to be an 'if' condition per se, it could be clamping the value to the bg color or something)
I'm making a lot of assumptions on what you want to do here, and if you have stronger conditions on the point distribution you might be better off rendering quads to a render texture for example, but that's a whole other topic :)
Which Matlab functions or examples should be used to (1) track distance from moving object to stereo (binocular) cameras, and (2) track centroid (X,Y,Z) of moving objects, ideally in the range of 0.6m to 6m. from cameras?
I've used the Matlab example that uses the PeopleDetector function, but this becomes inaccurate when a person is within 2m. because it begins clipping heads and legs.
The first thing that you need deal with, is in how detect the object of interest (I suppose you have resolved this issue). There are a lot of approaches of how to detect moving objects. If your cameras will stand in a fix position you can work only with one camera and use some background subtraction to get the objects that appear in the scene (Some info here). If your cameras are are moving, I think the best approach is to work with optical flow of the two cameras (instead to use a previous frame to get the flow map, the stereo pair images are used to get the optical flow map in each fame).
In MatLab, there is an option called disparity computation, this could help you to try to detect the objects in scene, after this you need to add a stage to extract the objects of your interest, you can use some thresholds. Once you have the desired objects, you need to put them in a binary mask. In this mask you can use some image momentum (Check this and this) extractor to calculate the centroids. If the images in the binary mask look noissy you can use some morphological operations to improve the reults (watch this).
I need some help, I have to make a project about leaves.
I want to make it by MATLAB.
my input is an image of one leaf (with a white background) and I need to know two things about the leaf:
1) find the lobed leaf (the pixels of each lobed leaf):
Lay the leaf on a table or work space where you can examine it.
Look at the leaf you are trying to identify. If the leaf looks like it has fingers, these are considered lobes. There can be
anywhere from two to many lobes on a leaf.
Distinguish pinnate leaves from palmate leaves by looking at the veins on the underside of the leaf. If the veins all come from
the same place at the base of the leaf it is considered palmately
lobed. If they are formed at various places on the leaf from one
centre line, the leaf is pinnately lobed.
Identify the type of leaf by using a leaf dictionary.
2) find approximately the number of bumps of the leaf:
in other words, find the "swollen points" of each leaf.
these are examples of leaves:
I've found some leaves examples in here.
Here is my attempt to solve the problem.
In the images that I've found, the background is completely black. If it is not so in your images, you should use Otsu's thresholding method.
I assumed that there can be only 3 types of leaves, according to your image:
The idea is to do blob analysis. I use the morphological operation of opening, to separate the leaves. If there is only one blob after the opening, I assume it is not compound. If the leaves are not compound, I analyze the solidity of the blobs. Non-solid enough means they are lobed.
Here are some examples:
function IdentifyLeaf(dirName,fileName)
figure();
im = imread(fullfile(dirName,fileName));
subplot(1,3,1); imshow(im);
% thresh = graythresh( im(:,:,2));
imBw = im(:,:,2) > 0;
subplot(1,3,2);imshow(imBw);
radiusOfStrel = round( size(im,1)/20 ) ;
imBwOpened = imopen(imBw,strel('disk',radiusOfStrel));
subplot(1,3,3);imshow(imBwOpened);
rpOpened = regionprops(imBwOpened,'Area');
if numel(rpOpened)>1
title('Pinnately Compound');
else
rp = regionprops(imBw,'Area','Solidity');
%Leave only largest blob
area = [rp.Area];
[~,maxIndex] = max(area);
rp = rp(maxIndex);
if rp.Solidity < 0.9
title('Pinnately Lobed');
else
title('Pinnately Veined');
end
end
end
I would approach this problem by converting it from 2d to 1d by scanning in a vector the perimeter of the leaf using "right hand on the wall" -algorithm.
From that data, I presume, one can find a dominant axis of symmetry (e.g. fitting a line); the distance of the perimeter would be calculated from that axis and then one could simply use a threshold+filtering to find local maxima and minima to reveal the number lobes/fingers... The histogram of distance could differentiate between pinnately lobed and pinnately compound leaves.
Another single metrics to check the curvature of the perimeter (from two extreme points) would be http://en.wikipedia.org/wiki/Sinuosity
Recognizing veins is unfortunately a complete different topic.
I am currently trying to reconstruct a 3D trajectory of a falling object like a ball or a rock out of a sequence of images taken from an iPhone video.
Where should I start looking? I know I have to calibrate the camera (I think I'll use the matlab calibration toolbox by Jean-Yves Bouguet) and then find the vanishing point from the same sequence, but then I'm really stuck.
read this: http://www.cs.auckland.ac.nz/courses/compsci773s1c/lectures/773-GG/lectA-773.htm
it explains 3d reconstruction using two cameras. Now for a simple summary, look at the figure from that site:
You only know pr/pl, the image points. By tracing a line from their respective focal points Or/Ol you get two lines (Pr/Pl) that both contain the point P. Because you know the 2 cameras origin and orientation, you can construct 3d equations for these lines. Their intersection is thus the 3d point, voila, it's that simple.
But when you discard one camera (let's say the left one), you only know for sure the line Pr. What's missing is depth. Luckily you know the radius of your ball, this extra information can give you the missing depth information. see next figure (don't mind my paint skills):
Now you know the depth using the intercept theorem
I see one last issue: the shape of ball changes when projected under an angle (ie not perpendicular on your capture plane). However you do know the angle, so compensation is possible, but I leave that up to you :p
edit: #ripkars' comment (comment box was too small)
1) ok
2) aha, the correspondence problem :D Typically solved by correlation analysis or matching features (mostly matching followed by tracking in a video). (other methods exist too)
I haven't used the image/vision toolbox myself, but there should definitely be some things to help you on the way.
3) = calibration of your cameras. Normally you should only do this once, when installing the cameras (and every other time you change their relative pose)
4) yes, just put the Longuet-Higgins equation to work, ie: solve
P = C1 + mu1*R1*K1^(-1)*p1
P = C2 + mu2*R2*K2^(-1)*p2
with
P = 3D point to find
C = camera center (vector)
R = rotation matrix expressing the orientation of the first camera in the world frame.
K = calibration matrix of the camera (containing internal parameters of the camera, not to be confused with the external parameters contained by R and C)
p1 and p2 = the image points
mu = parameter expressing the position of P on the projection line from camera center C to P (if i'm correct R*K^-1*p expresses a line equation/vector pointing from C to P)
these are 6 equations containing 5 unknowns: mu1, mu2 and P
edit: #ripkars' comment (comment box too small once again)
The only computer vison library that pops up in my mind is OpenCV (http://opencv.willowgarage.com/wiki ). But that's a C library, not matlab... I guess google is your friend ;)
About the calibration: yes, if those two images contain enough information to match some features. If you change the relative pose of the cameras, you'll have to recalibrate of course.
The choice of the world frame is arbitrary; it only becomes important when you want to analyze the retrieved 3d data afterwards: for example you could align one of the world planes with the plane of motion -> simplified motion equation if you want to fit one.
This world frame is just a reference frame, changeable with a 'change of reference frame transformation' (translation and/or rotation transformation)
Unless you have a stereo camera, you will never be able to know the position for sure, even with calibrated camera. Because you don't know whether the ball is small and close or large and far away.
There are other methods with single camera, based on series of images with different focus. But I doubt that you can control the camera of your cell phone in that way.
Edit(1):
as #GuntherStruyf points out correctly, you can know the position if one of your inputs is the size of the ball.
I am making a script which can generate multiple objects in Pyglet. In this example (see link below) there are two pyramids in 3d space, but every triangle is recalculated in every frame. My aim is to make a swarm with a large number of pyramids flying around, but i cant seem to figure out how to implement vertex lists in a batch. (assuming this is the fastest way to do it).
Do they need to be indexed for example? (batch.add_indexed(...) )
A standard seems to be:
batch1 = pyglet.graphics.Batch()
then add vertices to batch1. and finally:
def on_draw():
batch1.draw()
So how to do the intermediate step, where pyramids are added to vertex lists? A final question: when would you suggest to use multiple batches?
Thank you!
apfz
http://www.2shared.com/file/iXq7AOvg/pyramid_move.html
Just have a look at pyglet.sprite.Sprite._create_vertex_list for inspiration. There, the vertices for simple sprites (QUADS) are generated and added to a batch.
def _create_vertex_list(self):
if self._subpixel:
vertex_format = 'v2f/%s' % self._usage
else:
vertex_format = 'v2i/%s' % self._usage
if self._batch is None:
self._vertex_list = graphics.vertex_list(4,
vertex_format,
'c4B', ('t3f', self._texture.tex_coords))
else:
self._vertex_list = self._batch.add(4, GL_QUADS, self._group,
vertex_format,
'c4B', ('t3f', self._texture.tex_coords))
self._update_position()
self._update_color()
So the required function is Batch.add(vertex_list). Your vertices should only be recalculated if your pyramid changes it's position and not at every draw call. Instead of v2f you need to use v3f for 3D-coordinates and of course you need GL_TRIANGLES instead of GL_QUADS. Here is an example of a torus rendered with pyglet.