I have two sets of 2D Points (shown in images below).
And I would like to find some high confidence correspondence between these dots.
These dots are extracted feature points from 2 camera images from different angles. Two images are relatively well rectified, though not perfect. However, there will be distortion/warp caused by depth in the scene, the number of points might not be the same, there might be outliers, etc.
One approach could be using a sliding window that contains multiple dots and try block matching. But that might be kind of slow. I feel like there should be a relatively straight forward solution to this problem.
For example, this paper might be addressing a similar problem.
You can use each dot/point in one of the images, and search for its "neighbors" in the other image.
Just a few days ago someone asked a similar question here, and got a very sophisticated (accepted) answer:
How to calculate the nearest neighbors using weka from the command line?
But maybe your problem is so common in image processing that there are even better solutions, but you might try this one (implemented in java).
Related
Given are two monochromatic images of same size. Both are prealigned/anchored to one common point. Some points of the original image did move to a new position in the new image, but not in a linear fashion.
Below you see a picture of an overlay of the original (red) and transformed image (green). What I am looking for now is a measure of "how much did the "individual" points shift".
At first I thought of a simple average correlation of the whole matrix or some kind of phase correlation, but I was wondering whether there is a better way of doing so.
I already found that link, but it didn't help that much. Currently I implement this in Matlab, but this shouldn't be the point I guess.
Update For clarity: I have hundreds of these image pairs and I want to compare each pair how similar they are. It doesn't have to be the most fancy algorithm, rather easy to implement and yielding in a good estimate on similarity.
An unorthodox approach uses RASL to align an image pair. A python implementation is here: https://github.com/welch/rasl and it also
provides a link to the RASL authors' original MATLAB implementation.
You can give RASL a pair of related images, and it will solve for the
transformation (scaling, rotation, translation, you choose) that best
overlays the pixels in the images. A transformation parameter vector
is found for each image, and the difference in parameters tells how "far apart" they are (in terms of transform parameters)
This is not the intended use of
RASL, which is designed to align large collections of related images while being indifferent to changes in alignment and illumination. But I just tried it out on a pair of jittered images and it worked quickly and well.
I may add a shell command that explicitly does this (I'm the author of the python implementation) if I receive encouragement :) (today, you'd need to write a few lines of python to load your images and return the resulting alignment difference).
You can try using Optical Flow. http://www.mathworks.com/discovery/optical-flow.html .
It is usually used to measure the movement of objects from frame T to frame T+1, but you can also use it in your case. You would get a map that tells you the "offset" each point in Image1 moved to Image2.
Then, if you want a metric that gives you a "distance" between the images, you can perhaps average the pixel values or something similar.
I'm working on MATLAB on some regions inside an image. I'm at a point in which I would like to be able to separate regions which exhibit some kind of regularity (e.g., being circle-ish or square-ish) from regions which does not resemble any known figure and which for my application are mere noise. I'll illustrate this using a descriptive MS Paint image:
Is there any tool that, most of the times (or even less, I know this can't be 100/100) will recognize the red thing as being different?
I'll deal with many shapes in a single image, so I don't mind if I carry on some red monsters along the way, as long as the majority of them is kicked out. Of course I know the indices of these regions, so I can manipulate them in MATLAB.
Many algorithms come to mind, e.g., getting the boundary and checking for its regularity/the number of times it changes curvature/..., checking for variations in vertical length through different columns (nearly 0 for the linear feature, really high for the red stuff), ...
However I was hoping in some help from a tool out there. It doesn't matter if this tool won't cover all cases (for example, will kick out circles), I've been very broad to get the maximum number of inputs from you guys - any tool will be inspiring and helpful (and, however, we can't expect a perfect answer for the deeper question - recognizing regular shapes - which seems more like a AI field of research). I also think that, while being broad, this is totally non-subjective so should fit in SO. Thank you.
Side note 1: I'll deal mostly with elongated, extended features like the top-right one, so circles are not that relevant.
Side note 2: To be 100% clear, I would need something (be it an already existant tool, or some ideas pointed out by you) that acts on the indices of the shapes, in terms of rows-columns into the original image, or on the boundary of the shape itself.
Side note 3: Apart from tools/suggestions/ideas, you are welcomed to write down some lines of code ;) I'm getting the regions as connected components from bwconncomp.
I had to solve a similar problem recently that involved counting the number of indentations on blobs within in an image (basically, the connected components returned by bwconncomp). The method I used was to look at curvature changes along the boundary calculated via the FFT. In your case, the red blobs would have a large number of curvature variations, whereas the black regions would not. It's a pretty easy calculation and relatively fast. The code is on github here:
https://github.com/mjsottile/blobdents
The file of interest is src/countindents.m. A short description of the approach is here:
http://arxiv.org/abs/1501.07692
I went for the easier road as suggested by #Mikhail in comments.
I found out regionprops has a really helpful tool called Solidity. Quoting docs,
Returns a scalar specifying the proportion of the pixels in the convex hull that are also in the region. Computed as Area/ConvexArea.
Convex hull is defined as the smallest convex polygon that can contain the region. So Solidity goes up to 1 if the shape is kind of regular and has no convexity changes; down to 0 for my red shape, which leaves space between itself and the convex polygon.
Of course it never reaches 0, lowest value should belong to a kind of +-shaped sign.
I am attempting to do some face recognition and hallucination experiments and in order to get the best results, I first need to ensure all the facial images are aligned. I am using several thousand images for experimenting.
I have been scouring the Internet for past few days and have found many different programs which claim to do so, however due to Matlabs poor backwards compatibility, many of the programs no longer work. I have tried several different programs which don't run as they are calling onto Matlab functions which have since been removed.
The closest I found was using the SIFT algorithm, code found here
http://people.csail.mit.edu/celiu/ECCV2008/
Which does help align the images, but unfortunately it also downsamples the image, so the result ends up quite blurry looking which would have a negative effect on any experiments I ran.
Does anyone have any Matlab code samples or be able to point me in the right direction to code that actually aligns faces in a database.
Any help would be much appreciated.
You can find this recent work on Face Detection, Pose Estimation and Landmark Localization in the Wild. It has a working Matlab implementation and it is quite a good method.
Once you identify keypoints on all your faces you can morph them into a single reference and work from there.
The easiest way it with PCA and the eigen vector. To found X and Y most representative data. So you'll get the direction of the face.
You can found explication in this document : PCA Aligment
Do you need to detect the faces first, or are they already cropped? If you need to detect the faces, you can use vision.CascadeObjectDetector object in the Computer Vision System Toolbox.
To align the faces you can try the imregister function in the Image Processing Toolbox. Alternatively, you can use a feature-based approach. The Computer Vision System Toolbox includes a number of interest point detectors, feature descriptors, and a matchFeatures function to match the descriptors between a pair of images. You can then use the estimateGeometricTransform function to estimate an affine or even a projective transformation between two images. See this example for details.
I am going through feature detection algorithms and a lot of things seems to be unclear. The original paper is quite complicated to understand for beginners in image processing. Shall be glad if these are answered
What are the features which are being detected by SURF and SIFT?
Is it necessary that these have to be computed on gray scale images?
What does the term "descriptor" mean in simple words.
Generally,how many features are selected/extracted?Is there a criteria for that?
What does the size of Hessian matrix determine?
What is the size of the features being detected?It is said that the size of a feature is the size of the blob.So, if size of image is M*N so will there be M*N n umber of features?
These questions may seem too trivial, but please help..
I will try to give an intuitive answer to some of your questions, I don't know answers to all.
(You didn't specify which paper you are reading)
What are the features and how many features are being detected by SURF
and SIFT?
Normally features are any part in an image around which you selected a small block. You move that block by a small distance in all directions. If you find considerable variations between the one you selected and its surroundings, it is considered as a feature. Suppose you moved your camera a little bit to take the image, still you will detect this feature. That is their importance. Normally best example of such a feature is corners in the image. Even edges are not so good features. When you move your block along the edge lines, you don't find any variation, right?
Check this image to understand what I said , only at the corner you get considerable variation while moving the patches, in other two cases you won't get much.
Image link : http://www.mathworks.in/help/images/analyzing-images.html
A very good explanation is given here : http://aishack.in/tutorials/features-what-are-they/
This the basic idea and the algorithms you mentioned make this more robust to several variations and solve many issues. (You can refer their papers for more details)
Is it necessary that these have to be computed on gray scale images?
I think so. Anyway OpenCV works on grayscale images
What does the term "descriptor" mean in simple words?
Suppose you found features in one image, say image of a building. Now you took another image of same building but from a slightly different direction. You found features in the second image also. But how can you match these features. Say feature 1 in image 1 match to which feature in image 2 ? (As a human, you can do easily, right ? This corner of building in first image corresponds to this corner in second image, so and so. Very easy).
Feature is just giving you pixel location. You need more information about that point to match it with others. So you have to describe the feature. And this description is called "descriptors". To describe this features, algorithms are there and you can see it SIFT paper.
Check this link also : http://aishack.in/tutorials/sift-scale-invariant-feature-transform-introduction/
Generally,how many features are selected/extracted?Is there a criteria
for that?
During processing you can see applying different thresholds, removing weak keypoints etc. It is all part of plan. You need to understand algorithm to understand these things. Yes, you can specify these threshold and other parameters (in OpenCV) or you can leave it as default. If you check for SIFT in OpenCV docs, you can see function parameters to specify number of features, number of octave layers, edge threshold etc.
What does the size of Hessian matrix determine?
That I don't know exactly, just it is a threshold for keypoint detector. Check OpenCV docs : http://docs.opencv.org/modules/nonfree/doc/feature_detection.html#double%20hessianThreshold
I'm trying to write a code The helps me in my biology work.
Concept of code is to analyze a video file of contracting cells in a tissue
Example 1
Example 2: youtube.com/watch?v=uG_WOdGw6Rk
And plot out the following:
Count of beats per min.
Strenght of Beat
Regularity of beating
And so i wrote a Matlab code that would loop through a video and compare each frame vs the one that follow it, and see if there was any changes in frames and plot these changes on a curve.
Example of My code Results
Core of Current code i wrote:
for i=2:totalframes
compared=read(vidObj,i);
ref=rgb2gray(compared);%% convert to gray
level=graythresh(ref);%% calculate threshold
compared=im2bw(compared,level);%% convert to binary
differ=sum(sum(imabsdiff(vid,compared))); %% get sum of difference between 2 frames
if (differ ~=0) && (any(amp==differ)==0) %%0 is = no change happened so i dont wana record that !
amp(end+1)=differ; % save difference to array amp wi
time(end+1)=i/framerate; %save to time array with sec's, used another array so i can filter both later.
vid=compared; %% save current frame as refrence to compare the next frame against.
end
end
figure,plot(amp,time);
=====================
So thats my code, but is there a way i can improve it so i can get better results ?
because i get fealing that imabsdiff is not exactly what i should use because my video contain alot of noise and that affect my results alot, and i think all my amp data is actually faked !
Also i actually can only extract beating rate out of this, by counting peaks, but how can i improve my code to be able to get all required data out of it ??
thanks also really appreciate your help, this is a small portion of code, if u need more info please let me know.
thanks
You say you are trying to write a "simple code", but this is not really a simple problem. If you want to measure the motion accuratly, you should use an optical flow algorithm or look at the deformation field from a registration algorithm.
EDIT: As Matt is saying, and as we see from your curve, your method is suitable for extracting the number of beats and the regularity. To accuratly find the strength of the beats however, you need to calculate the movement of the cells (more movement = stronger beat). Unfortuantly, this is not straight forwards, and that is why I gave you links to two algorithms that can calculate the movement for you.
A few fairly simple things to try that might help:
I would look in detail at what your thresholding is doing, and whether that's really what you want to do. I don't know what graythresh does exactly, but it's possible it's lumping different features that you would want to distinguish into the same pixel values. Have you tried plotting the differences between images without thresholding? Or you could threshold into multiple classes, rather than just black and white.
If noise is the main problem, you could try smoothing the images before taking the difference, so that differences in noise would be evened out but differences in large features, caused by motion, would still be there.
You could try edge-detecting your images before taking the difference.
As a previous answerer mentioned, you could also look into motion-tracking and registration algorithms, which would estimate the actual motion between each image, rather than just telling you whether the images are different or not. I think this is a decent summary on Wikipedia: http://en.wikipedia.org/wiki/Video_tracking. But they can be rather complicated.
I think if all you need is to find the time and period of contractions, though, then you wouldn't necessarily need to do a detailed motion tracking or deformable registration between images. All you need to know is when they change significantly. (The "strength" of a contraction is another matter, to define that rigorously you probably would need to know the actual motion going on.)
What are the structures we see in the video? For example what is the big dark object in the lower part of the image? This object would be relativly easy to track, but would data from this object be relevant to get data about cell contraction?
Is this image from a light microscop? At what magnification? What is the scale?
From the video it looks like there are several motions and regions of motion. So should you focus on a smaller or larger area to get your measurments? Per cell contraction or region contraction? From experience I know that changing what you do at the microscope might be much better then complex image processing ;)
I had sucsess with Gunn and Nixons Dual Snake for a similar problem:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.64.6831
I placed the first aproximation in the first frame by hand and used the segmentation result as starting curv for the next frame and so on. My implementation for this is from 2000 and I only have it on paper, but if you find Gunn and Nixons paper interesting I can probably find my code and scan it.
#Matt suggested smoothing and edge detection to improve your results. This is good advise. You can combine smoothing, thresholding and edge detection in one function call, the Canny edge detector.Then you can dialate the edges to get greater overlap between frames. Little overlap will probably mean a big movement between frames. You can use this the same way as before to find the beat. You can now make a second pass and add all the dialated edge images related to one beat. This should give you an idea about the area traced out by the cells as they move trough a contraction. Maybe this can be used as a useful measure for contraction of a large cluster of cells.
I don't have access to Matlab and the Image Processing Toolbox now, so I can't give you tested code. Here are some hints: http://www.mathworks.se/help/toolbox/images/ref/edge.html , http://www.mathworks.se/help/toolbox/images/ref/imdilate.html and http://www.mathworks.se/help/toolbox/images/ref/imadd.html.