I need to classify pixel-wise instances in an image. Most object detection models, e.g., RetinaNet, R-CNNs, only detect bounding box. In my case the non-instance region in a bounding box can be significantly different from the instance. Even though the mask R-CNN model still does object classification based on the bounding box area. Does anybody know what model should I use? I guess Facebook's MultiPathNet probably works, but I am not using Linux. Are there any other models? Thanks a lot.
It sounds that you're looking for instance-level segmentation (as a short term for the long explanation).
Mask R-CNN sounds just right for the job.
It does instance-level segmentation based on the region proposals, not only bounding boxes.
The segmentation is a binary mask of the instance. The classification is made by a dedicated branch.
Related
I would like to detect road in outdoor scenario as seen in the picture on the left side and produce a black/white map as seen on the right side.
outdoor scenario
I've been experimenting with neural networks but the required transformation must have the same dimension of the input and output image. Also CNNs seem useless to me because i don't want to categorize what's in the picture. Should i rather try some approach without NN and use some simple filters to detect edges?
I'v been trying to do quadrangle detection and localization for weeks, my goal is to have a robust way of getting the 4 points of an quadrangle(rectangle), so I can apply projective transform to an Image then attach it to the source image. I have try the classic opencv contour method, and also using hough transform to find lines then calculate intersections, those two methods is unusable when apply it to real life images.
So I turn to CNN for help, but currently i haven't find any one try to use CNN to solve this simple problem.
My first attempt is to use state-of-art object detection and localization methods to get quadrangle's bounding box so i can narrow the search of 4 points, then use image processing & computer vision methods to further the search for 4 points. but after trying YOLOv2 and Faster-RCNN, the prediction accuracy is not ideal.
So I'm wondering if there is any idea i can do this end to end, training and feedforward all using a single neural network. it also must be able to deal with occlusion reasonably well.
Currently my idea is to remove the fc-layers and make a huge activation map that has the same width and height as the first input layer(eg. 448x448) then optimize the 4 most highly activated areas, using argmax to get the position. but this method only works for one quadrangle it doesn't work well with corner occlusions as well.
I'll be appreciated if anyone can provide any suggestions. Thanks a lot!
You are absolutely right about the first methods you mentioned. Hough transform like methods are old and not useful for images in the wild. And of course, computer vision field turned its face to object detection and recognition with rise of deep learning.
However, there is a very nice discussion came up recently.
Have we forgotten about Geometry in Computer Vision?
My suggestion would be contour detection and then apply Hough transform(use state of the art) methods to detect rectangles you want, about the occlusion, you can set parameters for Hough transform to be more forgiving for missing edge pixels with parameters.
You can for example check most recent contour detection methods as in recent CVPR paper.
I was wondering if someone can provide me a guideline to detect if a person in a picture is bald or not, or even better, how much hair s\he has.
So far I tried to detect the face and the eyes position. From that information, I roughly estimate the forehead and bald area by cutting the area above the eyes as high as some portion of the face.
Then I extract HOG features and train the system with bald and not-bald images using SVM.
Now when I'm looking at the test results, I see some pictures classified as bald but some of them actually have blonde hair or long forehead that hair is not visible after the cutting process. I'm using MATLAB for these operations.
So I know the method seems to be a bit naive, but can you suggest a way of finding out the bald area or extracting the hair, if exists. What method would be the most appropriate for that kind of problem?
very general, so answer is general unless further info provided
Use Computer Vision (e.g MATLAB Computer Vision toolkit) to detect face/head
head has analogies (for human faces), using these one can get the area of the head where hair or baldness is (it seems you already have these)
Calculate the (probabilistic color space model) range where the skin of the person lies (most peorple have similar skin collor space range)
Calculate percentage of skin versus other color (meaning hair) in that area
You have it!
To estimate a skin color model check following papers:
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.56.8637&rep=rep1&type=pdf
http://infoscience.epfl.ch/record/135966
http://www.eurasip.org/Proceedings/Eusipco/Eusipco2010/Contents/papers/1569293757.pdf
Link
If an area does not fit well with skin model it can be taken as non-skin (meaning hair, assuming no hats etc are present in samples)
Head region is very small, hence, using HOG for classification doesn't make much sense.
You can use prior information - like detect faces; baldness/hair is certain to be found on the area above the face. Also, use some denser feature descriptors.
You are probably ending up with very sparse representation or equivalently less information because of which your classifier is not able to classify correctly.
At image i need find "table" - simple rectangle.
Problem is with edge recognition, because potencial photos will be "dark".
I tried edge - sobel, canny, log, .... - recognition and after that Hough transformation and line finding. But this algorithms are not enough for this task.
Something what can help me:
- it is rectangle!, only in perspective view (something like fitting perspective rectangle?)
- that object MUST cover atleast for example 90% of photo (i know i need looking near photo edges)
- that rectangle have fast same color (for example wood dining table)
- i need find atleast "only" 4 corners..(but yes, better will be find the edges of that table)
I know how for example sobel, canny or log algorithms works and Hough as well. And naturally those algorithms fail at dark or non-contrast images. But is there some another method for example based at "fitting"?
Images showing photo i can get (you see it would be dark) and what i need find:
and this is really "nice" picture (without noise). I tested it on more noise pictures and the result was..simply horrible..
Result of this picture with actual algorithm log (with another ones it looks same):
I know image and edge recognition is not simple challenge but are there some new better methods or something like that what i can try to use?
In one of posts in here i found LSD algorithm. It seems very nice descripted and it seems it is recognizing really nice straight lines as well. Do you think it would be better to use it insted of the canny or sobel detection?
Another solution will be corner detection, on my sample images it works better but it recognize too much points and there will problem with time..i will need to connect all the points and "find" the table..
Another solution:
I thought about point to point mapping. That i will have some "virtual" table and try to map that table above with that "virtual" table (simple 2d square in painting :] )..But i think point to point mapping will give me big errors or it will not working.
Does someone have any advice what algorithm use to?
I tried recognize edges in FIJI and then put the edge detected image in matlab, but with hough it works bad as well..:/..
What do you think it would be best to use? In short i need find some algorithm working on non contrast, dark images.
I'd try some modified snakes algorithm:
you parameterize your rectangle with 4 points and initialize them somewhere in the image corners. Then you move the points towards image features using some optimization algorithm (e.g. gradient descent, simulated annealing, etc.).
The image features could be a combination of edge features (e.g. sobel directly or sobel of some gaussian filtered image) to be evaluated on the lines between those four points and corner features to be evaluated at those 4 points.
Additionally you can penalize unlikely rectangles (maybe depending on the angles between the points or on the distance to the image boundary).
I have several images of the pugmark with lots of irrevelant background region. I cannot do intensity based algorithms to seperate background from the foreground.
I have tried several methods. one of them is detecting object in Homogeneous Intensity image
but this is not working with rough texture images like
http://img803.imageshack.us/img803/4654/p1030076b.jpg
http://imageshack.us/a/img802/5982/cub1.jpg
http://imageshack.us/a/img42/6530/cub2.jpg
Their could be three possible methods :
1) if i can reduce the roughness factor of the image and obtain the more smoother texture i.e more flat surface.
2) if i could detect the pugmark like shape in these images by defining rough pugmark shape in the database and then removing the background to obtain image like http://i.imgur.com/W0MFYmQ.png
3) if i could detect the regions with depth and separating them from the background based on difference in their depths.
please tell if any of these methods would work and if yes then how to implement them.
I have a hunch that this problem could benefit from using polynomial texture maps.
See here: http://www.hpl.hp.com/research/ptm/
You might want to consider top-down information in the process. See, for example, this work.
Looks like you're close enough from the pugmark, so I think that you should be able to detect pugmarks using Viola Jones algorithm. Maybe a PCA-like algorithm such as Eigenface would work too, even if you're not trying to recognize a particular pugmark it still can be used to tell whether or not there is a pugmark in the image.
Have you tried edge detection on your image ? I guess it should be possible to finetune Canny edge detector thresholds in order to get rid of the noise (if it's not good enough, low pass filter your image first), then do shape recognition on what remains (you would then be in the field of geometric feature learning and structural matching) Viola Jones and possibly PCA-like algorithm would be my first try though.