I have both RGB and depth images from Kinect as png format. I'm trying to use depth data with the watershed segmentation but I don't know how to combine both data and obtain a more accurate result. I checked some papers but I didn't understand the results or couldn't find a solution written specifically for the watershed algorithm. How can I include the depth data as a reference point to the segmentation process?
I'm using MatLab's Image Processing Toolbox.
The images are from Nathan Silberman et. al.'s database on Silberman's website
An example RGB image and its corresponding depth file are shown below (note that the depth image, originally it is a binary image, is converted to uint8):
Update: I tried to create a weighted grayscale image from the RGB source together with the depth data by taking each channel (red, green, blue and depth) and calculating their weights; then including the values multiplied with their weights for every corresponding pixel. But the resulting grayscale image does not improve the result significantly. It's not that better than the solely RGB based segmentation. What else could I do if I follow this approach? Alternatively, how can I see the effects of the depth data?
unless you had an error uploading, the depth image is black and doesnt contain any depth data. Keep in mind that (dutch term) your comparing apples and pears here.
Whatershed images are not depth images, they are extractions of contour.
Then there is a next thing where you go wrong, depth images have a lower resulotion then color images. for the kinect v2 its only 512x424, and the kinect one's true depth vision is even lower then its returned bitmap size (it is a low res depth and not every pixel in is a result of a measurement, in contrast to kinect v2). But then the v2 has even better video output.
If you want better watershed of a rgb image, then average out multiple camera frames to get rid of camera noise.
PS i recomend you download the windows kinect sdk and take a look of the samples provided with it.
Related
I have a series of medical images from which I am attempting to segment out and analyze the ECG tracings in Matlab (the green, spiking line in the image below):
I have so far been successful in doing this on a small set of images using color thresholding and region properties. My problem is that almost all aspects of this feature of interest can change depending on the manufacturer of the machine used to produce the images and the behavior of the user operating it (over which I have 0 control).
Potentially differing attributes include line position in the image (which can change to be almost anywhere in the image), amplitude, frequency, and even color (which can be changed to match the color of the large white surface under the line in the above image). This makes it extremely difficult to create a robust segmentation solution for all images relying only on "simple" methods (color segmentation, region properties, edge detection etc).
Would it be straight forward to train a classifier to identify the general shape of this line and segment it out? Alternatively, is there another way to search and segment an image using prior shape information?
If you are currently applying an arbitrary threshold, you can look at various technique for dynamic thresholding (here a technique that applies the concept on edge detection).
What you could also try is to threshold on a different representation of the image, such as HSL and HSV (as I am assuming you are thresholding on the RGB values)
You may use a classifier and active contour model to segment the desired region. An example can be found here: http://pratondo.staff.telkomuniversity.ac.id/2016/01/14/robust-edge-stop-functions-for-edge-based-active-contour-models-in-medical-image-segmentation/
I have collected data using Kinect v2 sensor and I have a depth map together with its corresponding RGB image. I also calibrated the sensor and obtained the rotation and translation matrix between the Depth camera and RGB camera.
So I was able to reproject the depth values on the RGB image and they match. However, since the RGB image and the depth image are of different resolutions, there are a lot of holes in the resulting image.
So I am trying to move the other way, i.e. mapping the color onto the depth instead of depth to color.
So the first problem I am having is that the RGB image has 3 layers and I have to convert the RGB image to grayscale to do it and I am not getting the correct results.
Can this be done?
Has anyone tried this before?
Why can't you fit the Z-depth to the RGB?
To fit the low res image to the high- res should be easy, as long as both represent the same size of data (i.e. corners of both images are the same point)
It should be as easy as:
Z_interp=imresize(Zimg, [size(RGB,1) size(RGB,2)])
Now Z_interp should have the same amount of pixels as RGB
If you still want to do it the other way around, well, use the same approach:
RGB_interp=imresize(RGB, [size(Zimg,1) size(Zimg,2)])
The Image Acquisition Toolbox now officially supports Kinect v2 for Windows. You can get a point cloud out from Kinect using pcfromkinect function in the Computer Vision System Toolbox.
I have two closed curve stereo rectified edge images. Is it possible to find the disparity(along x-axis in image coordinates) between the edge images and do a 3D reconstruction since I know the camera matrix. I am using matlab for the process. And I will not be able to do a window based technique as it's a binary image since a window based technique requires texture. The question how will I compute the disparity between the edge images? The images are available in the following links. Left Edge image https://www.dropbox.com/s/g5g22f6b0vge9ct/edge_left.jpg?dl=0 Right Edge Image https://www.dropbox.com/s/wjmu3pugldzo2gw/edge_right.jpg?dl=0
For this type of images, you can easily map each edge pixel from the left image to its counterpart in the right image, and therefore calculate the disparity for those pixels as usual.
The mapping can be done in various ways, depending on how typical these images are. For example, using DTW like approach to match curvatures.
For all other pixels in the image, you just don't have any information.
#Photon: Thanks for the suggestion. I did what you suggested. I matched each edge pixel in the left and right image in a DTW like fashion. But there are some pixels whose y-pixel coordinate value differ by 1 or 2 pixels, albeit they are properly rectified. So I calculated the depth by averaging those differing(up to 2-pixel difference in y-axis) edge pixels using least squares method. But I ended getting this space curve (https://www.dropbox.com/s/xbg2q009fjji0qd/false_edge.jpg?dl=0) when they actually should have been like this (https://www.dropbox.com/s/0ib06yvzf3k9dny/true_edge.jpg?dl=0) which is obtained using RGB images.I couldn't think of any other reason why it would be the case since I compared by traversing along the 408 edge pixels.
I need to perform image smoothing. I've searched the web but I didn't find anything - every thing I tried doesn't preform like I want.
for example:
as you see there are bumps or something like stairs, so what should I do so the lines will be straight?
thanks....
If the resolution of the output image is higher than the resolution of the stairs, then you can do any number of things. To name a few.
grayscale (or binary) morphological processing using imclose
edge-enhancing smoothing
march around the edges of your objects, determine the corners in your mask, and make the image locally convex, but this will take some coding.
The Matlab File Exchange is your friend.
If the resolution of the output image is the same as the stairs, and the output is grayscale, you're pretty much constrained to spatial anti-aliasing filters.
If the resolution of the output image is the same as the stairs, and the output is binary, you can't do anything, obviously.
I've scanned an old photo with paper texture pattern and I would like to remove the texture as much as possible without lowering the image quality. Is there a way, probably using Image Processing toolbox in MATLAB?
I've tried to apply FFT transformation (using Photoshop plugin), but I couldn't find any clear white spots to be paint over. Probably the pattern is not so regular for this method?
You can see the sample below. If you need the full image I can upload it somewhere.
Unfortunately, you're pretty much stuck in the spatial domain, as the pattern isn't really repetitive enough for Fourier analysis to be of use.
As #Jonas and #michid have pointed out, filtering will help you with a problem like this. With filtering, you face a trade-off between the amount of detail you want to keep and the amount of noise (or unwanted image components) you want to remove. For example, the median filter used by #Jonas removes the paper texture completely (even the round scratch near the bottom edge of the image) but it also removes all texture within the eyes, hair, face and background (although we don't really care about the background so much, it's the foreground that matters). You'll also see a slight decrease in image contrast, which is usually undesirable. This gives the image an artificial look.
Here's how I would handle this problem:
Detect the paper texture pattern:
Apply Gaussian blur to the image (use a large kernel to make sure that all the paper texture information is destroyed
Calculate the image difference between the blurred and original images
EDIT 2 Apply Gaussian blur to the difference image (use a small 3x3 kernel)
Threshold the above pattern using an empirically-determined threshold. This yields a binary image that can be used as a mask.
Use median filtering (as mentioned by #Jonas) to replace only the parts of the image that correspond to the paper pattern.
Paper texture pattern (before thresholding):
You want as little actual image information to be present in the above image. You'll see that you can very faintly make out the edge of the face (this isn't good, but it's the best I have time for). You also want this paper texture image to be as even as possible (so that thresholding gives equal results across the image). Again, the right hand side of the image above is slightly darker, meaning that thresholding it well will be difficult.
Final image:
The result isn't perfect, but it has completely removed the highly-visible paper texture pattern while preserving more high-frequency content than the simpler filtering approaches.
EDIT
The filled-in areas are typically plain-colored and thus stand out a bit if you look at the image very closely. You could also try adding some low-strength zero-mean Gaussian noise to the filled-in areas to make them look more realistic. You'd have to pick the noise variance to match the background. Determining it empirically may be good enough.
Here's the processed image with the noise added:
Note that the parts where the paper pattern was removed are more difficult to see because the added Gaussian noise is masking them. I used the same Gaussian distribution for the entire image but if you want to be more sophisticated you can use different distributions for the face, background, etc.
A median filter can help you a bit:
img = imread('http://i.stack.imgur.com/JzJMS.jpg');
%# convert rgb to grayscale
img = rgb2gray(img);
%# apply median filter
fimg = medfilt2(img,[15 15]);
%# show
imshow(fimg,[])
Note that you may want to pad the image first to avoid edge effects.
EDIT: A smaller filter kernel than [15 15] will preserve image texture better, but will leave more visible traces of the filtering.
Well i have tried out a different approach using Anisotropc diffusion using the 2nd coefficient that operates on wider areas
Here is the output i got:
From what i can See from the Picture, the Noise has a relatively high Frequency Compared to the image itself. So applying a low Pass filter should work. Have a look at the Power spectrum abs(fft(...)) to determine the cutoff Frequency.