I need to compare two or more images to calculate how much a point shifted in the x and y direction. How do I go about doing this in MATLAB?
What you are looking for is an "Optical Flow" algorithm. There are many around, some faster but less accurate, some slower and more accurate.
Click here to find a MATLAB optical flow implementation (Lucas Kanade).
Gilads suggestion about a Lucas-Kanade tracker/optical flow calculator is really good, and is what I would use. It does however have the drawback of not working very well if the scene has changed too much.
If the scenes are indeed very different (say you moved and rotated the camera quite a lot) you would have to find your corresponding points in some other way. One example could be to use a SIFT descriptor to find image features in the two images and then determine which points correspond to each other. If you know the camera matrices of the two images then it becomes quite easy.
Related
Given are two monochromatic images of same size. Both are prealigned/anchored to one common point. Some points of the original image did move to a new position in the new image, but not in a linear fashion.
Below you see a picture of an overlay of the original (red) and transformed image (green). What I am looking for now is a measure of "how much did the "individual" points shift".
At first I thought of a simple average correlation of the whole matrix or some kind of phase correlation, but I was wondering whether there is a better way of doing so.
I already found that link, but it didn't help that much. Currently I implement this in Matlab, but this shouldn't be the point I guess.
Update For clarity: I have hundreds of these image pairs and I want to compare each pair how similar they are. It doesn't have to be the most fancy algorithm, rather easy to implement and yielding in a good estimate on similarity.
An unorthodox approach uses RASL to align an image pair. A python implementation is here: https://github.com/welch/rasl and it also
provides a link to the RASL authors' original MATLAB implementation.
You can give RASL a pair of related images, and it will solve for the
transformation (scaling, rotation, translation, you choose) that best
overlays the pixels in the images. A transformation parameter vector
is found for each image, and the difference in parameters tells how "far apart" they are (in terms of transform parameters)
This is not the intended use of
RASL, which is designed to align large collections of related images while being indifferent to changes in alignment and illumination. But I just tried it out on a pair of jittered images and it worked quickly and well.
I may add a shell command that explicitly does this (I'm the author of the python implementation) if I receive encouragement :) (today, you'd need to write a few lines of python to load your images and return the resulting alignment difference).
You can try using Optical Flow. http://www.mathworks.com/discovery/optical-flow.html .
It is usually used to measure the movement of objects from frame T to frame T+1, but you can also use it in your case. You would get a map that tells you the "offset" each point in Image1 moved to Image2.
Then, if you want a metric that gives you a "distance" between the images, you can perhaps average the pixel values or something similar.
I have a question regarding computer vision; seems to be a general question but anyways, just wondering if you might have a clue. I was wondering if there is an efficient way to distinguish texture images (or photos with repetitive patterns) between whatnot, say realistic photos? The patterns could have exact repetitions, or just have major similarity. Actually I'm trying to see given an image if, it is possible to detect it is a texture or a pattern-based image, and that in real-time maybe?
For instance these three are considered textures in our context:
http://www.bigchrisart.com/sites/default/files/video/TR_Texture_RockWall.jpg
http://www.colourbox.com/preview/4440275-144135-seamless-geometric-op-art-texture.jpg
Thank you!
I cannot open your first image. I implemented the Fourier transform on your second one, and you can see frequency responses at specific points:
You can further process the image by extract the local maximum of the magnitude, and they share the same distance to the center (zero frequency). This may be considered as repetitive patterns.
Regarding the case that patterns share major similarity instead of repetitive feature, it is hard to tell whether the frequency magnitude still has such evident response. It depends on how the pattern looks like.
Another possible approach is the auto-correlation on your image.
I am attempting to do some face recognition and hallucination experiments and in order to get the best results, I first need to ensure all the facial images are aligned. I am using several thousand images for experimenting.
I have been scouring the Internet for past few days and have found many different programs which claim to do so, however due to Matlabs poor backwards compatibility, many of the programs no longer work. I have tried several different programs which don't run as they are calling onto Matlab functions which have since been removed.
The closest I found was using the SIFT algorithm, code found here
http://people.csail.mit.edu/celiu/ECCV2008/
Which does help align the images, but unfortunately it also downsamples the image, so the result ends up quite blurry looking which would have a negative effect on any experiments I ran.
Does anyone have any Matlab code samples or be able to point me in the right direction to code that actually aligns faces in a database.
Any help would be much appreciated.
You can find this recent work on Face Detection, Pose Estimation and Landmark Localization in the Wild. It has a working Matlab implementation and it is quite a good method.
Once you identify keypoints on all your faces you can morph them into a single reference and work from there.
The easiest way it with PCA and the eigen vector. To found X and Y most representative data. So you'll get the direction of the face.
You can found explication in this document : PCA Aligment
Do you need to detect the faces first, or are they already cropped? If you need to detect the faces, you can use vision.CascadeObjectDetector object in the Computer Vision System Toolbox.
To align the faces you can try the imregister function in the Image Processing Toolbox. Alternatively, you can use a feature-based approach. The Computer Vision System Toolbox includes a number of interest point detectors, feature descriptors, and a matchFeatures function to match the descriptors between a pair of images. You can then use the estimateGeometricTransform function to estimate an affine or even a projective transformation between two images. See this example for details.
The following problem:
Given is an arbitrary polygon. It shall be covered 100% with the minimum number of circles of a given radius.
Note:
1) Naturally the circles have to overlap.
2) I try to solve the problem for ARBITRARY polygons. But also solutions for CONVEX polygons are appreciated.
3) As far as Im informed, this problem is NP-hard ( an algorithm to find the minimum size set cover for the Set-cover problem )
Choose: U = polygon and S1...Sk = circles with arbitrary centers.
My solution:
Ive already read some papers and tried a few things on my own. The most promising idea that I came up with was in fact one already indicated in Covering an arbitrary area with circles of equal radius.
So I guess it’s best I quickly try to describe my own idea and then refine my questions.
The picture gives you already a pretty good idea of what I do
IDEA and Problem Formulation
1. I approximate the circles with their corresponding hexagons and tessellate the whole R2, i.e. an sufficiently large area; keyword hexagonally closest packaging. (cyan … tessellation, red dotted, centers of the cyan hexagons)
2. I put the polygon somewhere in the middle of this tessellated area and compute the number of hexagons that are needed to cover the polygon.
In the following Im trying to minimize N, which is number ofhexagons needed to cover the polygon, by moving the polygon around step by step, after each step “counting” N.
Solving the problem:
So that’s when it gets difficult (for me). I don’t know any optimizers that solve this problem properly, since they all terminate after moving the polygon around a bit and not observing any change.
My solution is the following:
First note that this is a periodic problem:
1. The polygon can be moved in horizontal direction x with a period of 3*r (side length = radius r) of the hexagon.
2. The polygon can be moved in vertical direction y with a period of r^2+r^2-2*rrcos(2/3*pi) of the hexagon.
3. The polygon can be rotated phi with a period of 2/3*pi.
That means, one has to search a finite area of possible solutions to find the optimal solution.
So what I do is, I choose a stepsize for (x,y,phi) and simply brute force compute all possible solutions, picking out the optimum.
Refining my questions
1) Is the problem formulated intelligently? Right now im working on an algorithm that only tessellates a very small area, so that as little hexagons as possible have to be computed.
2) Is there a more intelligent optimizer to solve the problem?
3) FINALLY: I also have difficulties finding appropriate literature, since I don’t guess I don’t know the right keywords to look for. So if anybody can provide me with literature, it would also be appreciated a lot.
Actually I could go on about other things ive tried but I think no one of u guys wants to spend the whole afternoon just reading my question.
Thx in advance to everybody who takes the time to think about it.
mat
PS i implement my algorithms in matlab
I like your approach! When you mention your optimization I think a good way to go about it is by rotating the hexagonal grid and translating it till you find the least amount of circles that cover the region. You don't need to rotate 360 since the pattern is symmetric so just 360/6.
I've been working on this problem for a while and have just published a paper that contains code to solve this problem! It uses genetic algorithms and BFGS optimization. You can find a link to the paper here: https://arxiv.org/abs/2003.04839
Edit: Answer rewritten (there's no limitation that circles couldn't go outside the polygon).
You might be interested in Covering a simple polygon with circles. I think the algorithm works or is extendable also to complex polygons.
1.Inscribe the given polygon in a minimum sized rectangle
2.Cover the rectangle optimally by circles (algorithm is available)
I'm trying to write a code The helps me in my biology work.
Concept of code is to analyze a video file of contracting cells in a tissue
Example 1
Example 2: youtube.com/watch?v=uG_WOdGw6Rk
And plot out the following:
Count of beats per min.
Strenght of Beat
Regularity of beating
And so i wrote a Matlab code that would loop through a video and compare each frame vs the one that follow it, and see if there was any changes in frames and plot these changes on a curve.
Example of My code Results
Core of Current code i wrote:
for i=2:totalframes
compared=read(vidObj,i);
ref=rgb2gray(compared);%% convert to gray
level=graythresh(ref);%% calculate threshold
compared=im2bw(compared,level);%% convert to binary
differ=sum(sum(imabsdiff(vid,compared))); %% get sum of difference between 2 frames
if (differ ~=0) && (any(amp==differ)==0) %%0 is = no change happened so i dont wana record that !
amp(end+1)=differ; % save difference to array amp wi
time(end+1)=i/framerate; %save to time array with sec's, used another array so i can filter both later.
vid=compared; %% save current frame as refrence to compare the next frame against.
end
end
figure,plot(amp,time);
=====================
So thats my code, but is there a way i can improve it so i can get better results ?
because i get fealing that imabsdiff is not exactly what i should use because my video contain alot of noise and that affect my results alot, and i think all my amp data is actually faked !
Also i actually can only extract beating rate out of this, by counting peaks, but how can i improve my code to be able to get all required data out of it ??
thanks also really appreciate your help, this is a small portion of code, if u need more info please let me know.
thanks
You say you are trying to write a "simple code", but this is not really a simple problem. If you want to measure the motion accuratly, you should use an optical flow algorithm or look at the deformation field from a registration algorithm.
EDIT: As Matt is saying, and as we see from your curve, your method is suitable for extracting the number of beats and the regularity. To accuratly find the strength of the beats however, you need to calculate the movement of the cells (more movement = stronger beat). Unfortuantly, this is not straight forwards, and that is why I gave you links to two algorithms that can calculate the movement for you.
A few fairly simple things to try that might help:
I would look in detail at what your thresholding is doing, and whether that's really what you want to do. I don't know what graythresh does exactly, but it's possible it's lumping different features that you would want to distinguish into the same pixel values. Have you tried plotting the differences between images without thresholding? Or you could threshold into multiple classes, rather than just black and white.
If noise is the main problem, you could try smoothing the images before taking the difference, so that differences in noise would be evened out but differences in large features, caused by motion, would still be there.
You could try edge-detecting your images before taking the difference.
As a previous answerer mentioned, you could also look into motion-tracking and registration algorithms, which would estimate the actual motion between each image, rather than just telling you whether the images are different or not. I think this is a decent summary on Wikipedia: http://en.wikipedia.org/wiki/Video_tracking. But they can be rather complicated.
I think if all you need is to find the time and period of contractions, though, then you wouldn't necessarily need to do a detailed motion tracking or deformable registration between images. All you need to know is when they change significantly. (The "strength" of a contraction is another matter, to define that rigorously you probably would need to know the actual motion going on.)
What are the structures we see in the video? For example what is the big dark object in the lower part of the image? This object would be relativly easy to track, but would data from this object be relevant to get data about cell contraction?
Is this image from a light microscop? At what magnification? What is the scale?
From the video it looks like there are several motions and regions of motion. So should you focus on a smaller or larger area to get your measurments? Per cell contraction or region contraction? From experience I know that changing what you do at the microscope might be much better then complex image processing ;)
I had sucsess with Gunn and Nixons Dual Snake for a similar problem:
http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.64.6831
I placed the first aproximation in the first frame by hand and used the segmentation result as starting curv for the next frame and so on. My implementation for this is from 2000 and I only have it on paper, but if you find Gunn and Nixons paper interesting I can probably find my code and scan it.
#Matt suggested smoothing and edge detection to improve your results. This is good advise. You can combine smoothing, thresholding and edge detection in one function call, the Canny edge detector.Then you can dialate the edges to get greater overlap between frames. Little overlap will probably mean a big movement between frames. You can use this the same way as before to find the beat. You can now make a second pass and add all the dialated edge images related to one beat. This should give you an idea about the area traced out by the cells as they move trough a contraction. Maybe this can be used as a useful measure for contraction of a large cluster of cells.
I don't have access to Matlab and the Image Processing Toolbox now, so I can't give you tested code. Here are some hints: http://www.mathworks.se/help/toolbox/images/ref/edge.html , http://www.mathworks.se/help/toolbox/images/ref/imdilate.html and http://www.mathworks.se/help/toolbox/images/ref/imadd.html.