I've started work on an application for iOS that would recognize faces from a photo or from the iPhone / iPad camera.
Existing solutions like OpenCV and Core Image (in iOS 5.0) provide facial detection within an image, but I can't find a library or example that matches a face with a person.
Does such a means of performing facial recognition, not just detection, exist for iOS?
On iOS 5 you can use CoreImage (CIDetector, CIFeature, CIFaceFeature should be named as the relevant keywords) for that task. Check out the SquareCam example App from Apple, it includes face detection. If you're targeting older iOS versions, openCV seems to be a good approach.
http://developer.apple.com/library/ios/#samplecode/SquareCam/Introduction/Intro.html
Edit_: Argh, soory. CoreImage can only detect faces but not recognize them. But maybe you can build a solution based on CoreImage...
What OpenCV does is called "face detection." This is different than, but related to, face recognition, which is what you seem to want to do.
Face detection "detects" faces by finding the location of facial features such as the eyes, the mouth, etc. To "recognize" these faces, you then need to compare these features with a database of known faces, for which the features have already been detected.
I'm not aware of a face recognition library for iOS, and this is no easy feat. Even Apple's own iPhoto has, in my experience, very low accuracy.
However, if you only want to do face detection, or want to build your own facial recognition algorithm on top of a face detection library, iOS 5 also includes a face detection API. You can find it in the CoreImage framework.
Related
I'm using EasyAR Sense for Unity to develop an app that tracks a target image.
I'm using https://www.easyar.com/targetcode.html to test my target images, but I have not yet understood the markers requirements. Are the colors of image important? Or is it based on the recognition of outlines and edges?
Also, are there any suggested guidelines and how do they apply to random images? (just to clarify, i'm not interested in using markers but digitally drawn pictures).
EDIT: We found out that the ways in which Vuforia and EasyAR recognize theyirtargets are pretty far apart from each other. An image that would rate fairly low on Vuforia will score high on EasyAR's site and vice-versa.
As far as we know now, yes, Vuforia bases its recognition methods on high contrasts and sharp edges.
That said, Vuforia as a solution is not feasible for our purposes, as it doesn't support front facing camera. We had to look for alternate solutions and stumbled across EasyAR which seems powerful, but with a really slim documentation on the programming side, and an inexistent design guideline documentation.
As we understand chaotic patterns will be recognized the best from EasyAR's engine but it doesn't state how much chaos defines "a rich texture".
But we are in dire need of simplicity in the images we're using in the application since it's targeted towards kids with understanding disorders, and a messy approach to the images may be counterproductive.
In my experience with EasyAR and Vuforia (they detect 2D images similarly) the more complex is the image is better for recognition, for example:
Contrast between delimited color areas are detected very good.
Lines with sharp edges are detected better.
Enter vuforia and try their system of stars when checking a target, that usually says to me what images will work great on EasyAR.
GOOD DETECTION
BAD DETECTION
In my experience with EasyAR and Vuforia, color is important and Uniform images aren't suitable for targets.
Furthermore, In Vuforia:
developer panel >> Target manager >> your database >> your target >> rating,
you can see your target's rating:
I've looked into Metaio which can do Facial 3d reconstructions
video here: https://www.youtube.com/watch?v=Gq_YwW4KSjU
but I'm not looking to do that. I want to simply be able to have the user scan in a small simple object and a 3d model be created from that. I don't need it textured or anything. As far as I can tell Metaio cannot do what I'm looking for, or at least I can't find the documentation for it.
Since you are targeting mobile, you would have to take multiple pictures from different angles and use an approach used in this CSAIL paper.
Steps
For finding the keypoints, I would use FAST, or a method using the Laplacian of Gaussian. Other options include SURF and SIFT.
Once you identify the points, use triangulation to find where the points will be in 3D.
With all of the points, create a point cloud. In unity, I would recommend doing something similar to this project, which used particle systems as the points.
You now have a 3d reconstruction of the object!
Now, in implementing each of these steps, you could reinvent the wheel, or use C++ native plugins in Unity. This enables you to use OpenCV which has many of these operations already implemented (SURF, SIFT, possibly even some 3D reconstruction classes/methods, which use Stereo Calibration*).
That all being said... the Android Computer Vision Plugin(also apparently called "Starry Night") seems to have these capabilities. However, in version 1.0, only PrimeSense sensors are supported. See the description of the plugin**
Starry Night is an easy to use Unity plugin that provides high-level 3D computer vision processing functions that allow applications to interact with the real world. Version 1.0 provides SLAM (Simultaneous Localization and Mapping) functions which can be used for 3D reconstruction, augmented reality, robot controls, and many other applications. Starry Night will interface to any type of 3D sensor or stereo camera. However, version 1.0 will interface only to a PrimeSense Carmine sensor.
*Note: That tutorial is in matlab, but I think the overview section gives a good understanding of stereo calibration
**as of May 12th, 2014
I am creating one application with face processing. How may i create face animation?
Is there any library which i can use?
How to face track once capturing image of face?
Please help me
As far as I am aware there is no completely free library to track facial expressions - which i think is what you need to produce aniamtion.
However, there is a commerical library for iOS (and other platforms) here: http://www.image-metrics.com/livedriver/overview/
This is available under a trial license and also a free educational licence. I believe it will do what you want.
Your other option is to develop you own facial feature tracking system using something like OpenCV: http://opencv.org/
Thats going to be a challenge though.
Face detection is already included in CoreImage (CI), see
http://www.bobmccune.com/2012/03/22/ios-5-face-detection-with-core-image/
If you want face recognition, you have to do something on your own, but there are some tutorials available - most of them using OpenCV.
Is there any library alternative to OpenCV which detects smile.
I dont want to use OpenCV as it sometimes fails to detect faces due to background.
Any one knw other library ? other than OpenCV ?
I would recommend having a look at The Machine Perception Toolbox (MPT Library).
I had a chance to play with it a bit at an Openframeworks OpenCV workshop at Goldsmiths and there is a c++ smile detection sample available.
I imagine you can try the MPT Library for iPhone with openframeworks or simply link to the library from an iphone project.
sometimes fails to detect faces due to
background.
An ideal lighting setup will guarantee better results, but given that you want to use this on a mobile device, you must inform your users that smile detection might fail under extreme conditions (bad lighting)
HTH
How are you doing smile detection? I can't see a smile-specific Haar dataset in the default OpenCV face detection cascades. I suspect your problem is training data rather than OpenCV itself.
Egawer is a good starting point if you need a working app to begin with.
https://github.com/Atrac613/egawer-iOS
I checked the training images of smileD_haarcascade_v0.05, an found that they include the full face. So, it seems to be a "smiling face" detector rather than a smile detector alone. While this seems easier, it can also be less accurate.
The best is to create your own Haar Cascade XML file, but admittedly most of us developers don't have time for that. You can improve the results considerably by equalizing the brightness of the image.
iOS 7 now has native support of simile detection in CoreImage. Here is the API diff:
For iOS 7, Yes, now you can do it with CoreImage.
Here is the API diff in iOS 7 Beta 2:
CoreImage
CIDetector.h
Added CIDetectorEyeBlink
Added CIDetectorSmile
I hope this falls within the "programming question" category.
Im all lightheaded from Googling (and reading every post in here on the subject) on the subject "Computer Vision", but Im getting more confused than enlightened.
I have 6 abstract shapes printed on a piece of paper and I would like to have the camera on the iPhone identify these shapes (from different angles, lightning etc.).
I have used OpenCV a while back(Java) and I looked at other libraries out there. The caveat is that it seems that either they rely on a jail broken iPhone or they are so experimental and hard to use that I would probably end up using days learning libraries only to figure out they didn't work.
I have thought of taking +1000 images of my shapes and training a Haar filter. But again
if there is anything out there that is a bit easier to work with I would really appreciate the advise, suggestion of people with a bit of experience.
Thank you for any suggestion or pieces of advise you might have:)
Have a look at at OpenCV's SURF feature extraction (they also have a demo which uses it to detect objects).
Surf features are salient image features which are invariant to rotation and scale. Many algorithms detect objects by extracting such features from an image, and then use simple "bag of words" classification (comparing the set of extracted image features to the features of your "shapes". Even without referring to their spacial alignment you can have good detection rates if you only have 6 shapes).
While not a library, Chris Greening explains how iPhone Sudoku Grab does its image recognition of puzzles in his post here. He does seem to recommend OpenCV, and not just for jailbroken devices.
Also Glen Low talks a bit about how Instaviz does its shape recognition in an interview for the Mobile Orchard podcast.
I do shape recognition in my iPhone app Instaviz and the routines are actually packaged into a library I call "Recog". Only problem is that it is meant for finger or mouse gesture recognition rather than image recognition. You pass the routines a set of points representing the gesture and it tells you whether it's a square, circle etc.
I haven't yet decided on a licensing model but probably use a minimal per-seat royalty.