I have been trying to separate the human body in an image from the background, but all the methods I have seen don't seem to work very well for me.
I have collected the following images;
The image of the background
The image of the background with the person in it.
Now I want to cut out the person from the background.
I tried subtracting the image of the background from the image with the person using res = cv2.subtract(background, foreground) (I am new to image processing).
Background subtraction methods in opencv like cv2.BackgroundSubtractorMOG2() and cv2.BackgroundSubtractorMOG2() only works with videos or image sequence and contour detection methods I have seen are only for solid shapes.
And grabCut doesn't quite work well for me because I would like to automate the process.
Given the images I have (Image of the background and image of the background with the person in it), is there a method of cutting the person out from the background?
I wouldn't recommend a neural net for this problem. That's a lot of work for something like this where you have a known background. I'll walk through the steps I took to do the background segmentation on this image.
First I shifted into the LAB color space to get some light-resistant channels to work with. I did a simple subtractions of foreground and background and combined the a and b channels.
You can see that there is still significant color change in the background even with a less light-sensitive color channel. This is likely due to the auto white balance on the camera, you can see that some of the background colors change when you step into view.
The next step I took was thresholding off of this image. The optimal threshold values may not always be the same, you'll have to adjust to a range that works well for your set of photos.
I used openCV's findContours function to get the segmentation points of each blob and I filtered the available contours by size. I set a size threshold of 15000. For reference, the person in the image had a pixel area of 27551.
Then it's just a matter of cropping out the contour.
This technique works for any good thresholding strategy. If you can improve the consistency of your pictures by turning off auto settings and ensure good contrast of the person against the wall then you can use simpler thresholding strategies and get good results.
Just for fun:
Edit:
I forgot to add in the code I used:
import cv2
import numpy as np
# rescale values
def rescale(img, orig, new):
img = np.divide(img, orig);
img = np.multiply(img, new);
img = img.astype(np.uint8);
return img;
# get abs(diff) of all hue values
def diff(bg, fg):
# do both sides
lh = bg - fg;
rh = fg - bg;
# pick minimum # this works because of uint wrapping
low = np.minimum(lh, rh);
return low;
# load image
bg = cv2.imread("back.jpg");
fg = cv2.imread("person.jpg");
fg_original = fg.copy();
# blur
bg = cv2.blur(bg,(5,5));
fg = cv2.blur(fg,(5,5));
# convert to lab
bg_lab = cv2.cvtColor(bg, cv2.COLOR_BGR2LAB);
fg_lab = cv2.cvtColor(fg, cv2.COLOR_BGR2LAB);
bl, ba, bb = cv2.split(bg_lab);
fl, fa, fb = cv2.split(fg_lab);
# subtract
d_b = diff(bb, fb);
d_a = diff(ba, fa);
# rescale for contrast
d_b = rescale(d_b, np.max(d_b), 255);
d_a = rescale(d_a, np.max(d_a), 255);
# combine
combined = np.maximum(d_b, d_a);
# threshold
# check your threshold range, this will work for
# this image, but may not work for others
# in general: having a strong contrast with the wall makes this easier
thresh = cv2.inRange(combined, 70, 255);
# opening and closing
kernel = np.ones((3,3), np.uint8);
# closing
thresh = cv2.dilate(thresh, kernel, iterations = 2);
thresh = cv2.erode(thresh, kernel, iterations = 2);
# opening
thresh = cv2.erode(thresh, kernel, iterations = 2);
thresh = cv2.dilate(thresh, kernel, iterations = 3);
# contours
_, contours, _ = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE);
# filter contours by size
big_cntrs = [];
marked = fg_original.copy();
for contour in contours:
area = cv2.contourArea(contour);
if area > 15000:
print(area);
big_cntrs.append(contour);
cv2.drawContours(marked, big_cntrs, -1, (0, 255, 0), 3);
# create a mask of the contoured image
mask = np.zeros_like(fb);
mask = cv2.drawContours(mask, big_cntrs, -1, 255, -1);
# erode mask slightly (boundary pixels on wall get color shifted)
mask = cv2.erode(mask, kernel, iterations = 1);
# crop out
out = np.zeros_like(fg_original) # Extract out the object and place into output image
out[mask == 255] = fg_original[mask == 255];
# show
cv2.imshow("combined", combined);
cv2.imshow("thresh", thresh);
cv2.imshow("marked", marked);
# cv2.imshow("masked", mask);
cv2.imshow("out", out);
cv2.waitKey(0);
Since it is very easy to find dataset consist a lot of human body, I suggest you to implement neural network segmentation tecniques to extract human body perfectly. Please check this link to see similar example.
Related
Since image processing and computer vision aren't of my field of study I'm having difficulty finding an algorithm that can identify the positions of rectangles of known size and proportion in certain images to automate a process.
These are grayscale images containing some circles and only one or none white rectangle with rounded edges, as can be seen in the example figure below.
3 different imagens with the "same" retangle to be found
Thank you
Try OpenCV, it stands for Open Computer Vision. It’s free. This is written in Python.
import cv2
import numpy as np
img = cv2.imread("C:\\Users\\mogar\\Desktop\\temp.jpg")
grayImage = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret, thresh = cv2.threshold(grayImage, 160, 255, cv2.THRESH_BINARY)
kernel = np.ones((5,5),np.uint8)
thresh = cv2.erode(thresh,kernel,iterations = 1)
#thresh = np.invert(thresh)
cv2.imshow("Threholded Image", thresh)
cv2.waitKey(0) & 0xFF == ord('q')
cv2.destroyAllWindows()
_, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)
for cnts in contours:
rect = cv2.minAreaRect(cnts)
box = cv2.boxPoints(rect)
box = np.int0(box)
(x,y),(w,h),angle = rect
w = int(w)
h = int(h)
area = w*h
if area > 10000 and area < 100000:
print("Area Check", area)
cv2.drawContours(img, [box], 0, (0,0,255), 5)
small = cv2.resize(img, (0,0), fx=0.3, fy=0.3)
cv2.imshow("Contours", small)
cv2.waitKey(0) & 0xFF == ord('q')
cv2.destroyAllWindows()
You may need to adjust some of the threshold values and area values so that you'll be enclosing only the rectangles. You'll notice the rectangles are not fully enclosed right now, that is literally because the text is getting in the way and cutting the rectangles in half. If you had a clean image; I'm sure this would work great. If you have any questions please don't hesitate the ask, but it may take some time before I can answer.
I've tried an image-editing-effect which should recolor a picture with little black dots, however it only works for certain images and I honestly don't know why. Any ideas?
#url = member.avatar_url
#print(url)
#response = requests.get(url=url, stream=True).raw
#imag = Image.open(response)
imag = Image.open("unknown.png")
#out = Image.new('I', imag.size)
i = 0
width, height = imag.size
for x in range(width):
i+=1
for y in range(height):
if i ==5:
# changes every 5th pixel to a certain brightness value
r,g,b,a = imag.getpixel((x,y))
print(imag.getpixel((x,y)))
brightness = int(sum([r,g,b])/3)
print(brightness)
imag.putpixel((x, y), (brightness,brightness,brightness,255))
i= 0
else:
i += 1
imag.putpixel((x,y),(255,255,255,255))
imag.save("test.png")
The comments are what I would've used if my tests had worked. Using local pngs also don't work all the time.
Your image that doesn't work doesn't have an alpha channel but your code assumes it does. Try forcing in an alpha channel on opening like this:
imag = Image.open("unknown.png").convert('RGBA')
See also What's the difference between a "P" and "L" mode image in PIL?
A couple of other ideas too:
looping over images with Python for loops is slow and inefficient - in general, try to find a vectorised Numpy alternative
you have an alpha channel but set it to 255 (i.e. opaque) everywhere, so in reality, you may as well not have it and save roughly 1/4 of the file size
your output image is RGB with all 3 components set identically - that is really a greyscale image, so you could create it as such and your output file will be 1/3 the size
So, here is an alternative rendition:
#!/usr/bin/env python3
from PIL import Image
import numpy as np
# Load image and ensure neither palette nor alpha
im = Image.open('paddington.png').convert('RGB')
# Make into Numpy array
na = np.array(im)
# Calculate greyscale image as mean of R, G and B channels
grey = np.mean(na, axis=-1).astype(np.uint8)
# Make white output image
out = np.full(grey.shape, 255, dtype=np.uint8)
# Copy across selected pixels
out[1::6, 1::4] = grey[1::6, 1::4]
out[3::6, 0::4] = grey[3::6, 0::4]
out[5::6, 2::4] = grey[5::6, 2::4]
# Revert to PIL Image
Image.fromarray(out).save('result.png')
That transforms this:
into this:
If you accept calculating the greyscale with the normal method, rather than averaging R, G and B, you could change to this:
im = Image.open('paddington.png').convert('L')
and remove the line that does the averaging:
grey = np.mean(na, axis=-1).astype(np.uint8)
I try to remove the white annotations of this image (the numbers and arrows), as well as the black grid, with MATLAB:
I tried to compute, for each pixel, the mode of neighbors, but this process is very slow and I get poor results.
How can I obtain an image like this one?
Thank you for your time.
The general name for such a task is inpainting. If you search for that you will find better methods than what I'm showing here. This is no more than a proof of concept. I'm using DIPimage 3 (because I'm an author and it's easy for me to use).
First we need to create a mask for the regions that we want to remove (inpaint). It is easy to find pixels where all three channels have a high value (white) or a low value (black):
img = readim('https://i.stack.imgur.com/16r9N.png');
% Find a mask for the areas to remove
whitemask = min(img,'tensor') > 50;
blackmask = max(img,'tensor') < 30;
mask = whitemask | blackmask;
This mask doesn't capture all of the black grid, if we increase the threshold we will also remove the dark region of sea off the coast of Spain. And it also captures the white outline of the coasts. We can do a little bit better than this with some additional filtering:
% Find a mask for the areas to remove
whitemask = min(img,'tensor') > 50;
whitemask = whitemask - pathopening(whitemask,50);
blackmask = max(img,'tensor');
blackmask2 = blackmask < 80;
blackmask2 = blackmask2 - areaopening(blackmask2,6);
blackmask = blackmask < 30 | blackmask2;
mask = whitemask | blackmask;
This produces the following mask:
Still far from perfect, but a good start for our proof of concept.
One simple inpainting method uses normalized convolution: using the inverse of the mask we made, convolve the image multiplied by the mask, and convolve the mask separately. The ratio of these two results is a smoothed image that doesn't take the masked pixels into account. Finally, we replace the pixels in the original image under the mask with the values from this normalized convolution:
% Solution 1: normalized convolution
smooth = gaussf(img * ~mask, 2) / gaussf(~mask, 2);
img(mask) = smooth(mask);
An alternative solution applies a closing on the image multiplied by the mask (note that this multiplication makes the pixels we don't want completely black; the closing will spread the surrounding colors over the black areas):
% Solution 2: morphology
smooth = iterate('closing',img * ~mask, 13);
img(mask) = smooth(mask);
i am working on a research about the swimming of fishes using analysis of videos, then i need to be carefully with the images (obtained from video frames) with emphasis in the tail.
The images are in High-Resolution and the software that i customize works with binary images, because is easy to use maths operations on this.
For obten this binary images i use 2 methods:
1)Convert the image to gray, invert the colors,later to bw and finally to binary with a treshold that give me images like this, with almost nothing of noise. The images sometimes loss a bit of area and doesn't is very exactly with the tail(now i need more acurracy for determinate the amplitude of tail moves)
image 1
2)i use this code, for cut the border that increase the threshold, this give me a good image of the edge, but i dont know like joint these point and smooth the image, or fitting binary images, the app fitting of matlab 2012Rb doesn't give me a good graph and i don't have access to the toolboxs of matlab.
s4 = imread('arecorte.bmp');
A=[90 90 1110 550]
s5=imcrop(s4,A)
E = edge(s5,'canny',0.59);
image2
My question is that
how i can fit the binary image or joint the points and smooth without disturb the tail?
Or how i can use the edge of the image 2 to increase the acurracy of the image 1?
i will upload a image in the comments that give me the idea of the method 2), because i can't post more links, please remember that i am working with iterations and i can't work frame by frame.
Note: If i ask this is because i am in a dead point and i don't have the resources to pay to someone for do this, until this moment i was able to write the code but in this final problem i can't alone.
I think you should use connected component labling and discard the small labels and than extract the labels boundary to get the pixels of each part
the code:
clear all
% Read image
I = imread('fish.jpg');
% You don't need to do it you haef allready a bw image
Ibw = rgb2gray(I);
Ibw(Ibw < 100) = 0;
% Find size of image
[row,col] = size(Ibw);
% Find connceted components
CC = bwconncomp(Ibw,8);
% Find area of the compoennts
stats = regionprops(CC,'Area','PixelIdxList');
areas = [stats.Area];
% Sort the areas
[val,index] = sort(areas,'descend');
% Take the two largest comonents ids and create filterd image
IbwFilterd = zeros(row,col);
IbwFilterd(stats(index(1,1)).PixelIdxList) = 1;
IbwFilterd(stats(index(1,2)).PixelIdxList) = 1;
imshow(IbwFilterd);
% Find the pixels of the border of the main component and tail
boundries = bwboundaries(IbwFilterd);
yCorrdainteOfMainFishBody = boundries{1}(:,1);
xCorrdainteOfMainFishBody = boundries{1}(:,2);
linearCorrdMainFishBody = sub2ind([row,col],yCorrdainteOfMainFishBody,xCorrdainteOfMainFishBody);
yCorrdainteOfTailFishBody = boundries{2}(:,1);
xCorrdainteOfTailFishBody = boundries{2}(:,2);
linearCorrdTailFishBody = sub2ind([row,col],yCorrdainteOfTailFishBody,xCorrdainteOfTailFishBody);
% For visoulaztion put color for the boundries
IFinal = zeros(row,col,3);
IFinalChannel = zeros(row,col);
IFinal(:,:,1) = IFinalChannel;
IFinalChannel(linearCorrdMainFishBody) = 255;
IFinal(:,:,2) = IFinalChannel;
IFinalChannel = zeros(row,col);
IFinalChannel(linearCorrdTailFishBody) = 125;
IFinal(:,:,3) = IFinalChannel;
imshow(IFinal);
The final image:
How should I change the background image color to black or change the RGB values become black color background. I want to take the original leaf image only.
Leaf image
In order to change the background color to black you'll need the following:
calculate the background mask by using a threshold
The threshold can be either find automatically, by using the function graythresh, or manually, by looking at the image histogram.
perform thresholding by using the value from stage 1 in order to find the foreground mask. Also, pick the largest connected component and perform noise cleaning (imclose operation).
calculate the BG from the FG, and zero out the corresponding locations in the original input image.
Code example:
I = imread('YaEwk.jpg');
%converts to hsv colorspace, and takes the 3rd dimension. normlizes it.
im = rgb2hsv(I);
im = mat2gray(im(:,:,3));
%determines a threshold to distinguish between the leaf and its surroundings.
T = graythresh(im);
%defines FG as all the values below the threshold
%Also, keeps just the biggest connected component and perform noise
%reduction.
FG = im < T;
FG = bwareafilt(FG,1);
FG = imclose(FG,strel('disk',2));
%defines the background as the opposite of the foreground
BG = ~FG;
I(repmat(BG,1,1,3)) = 0;
%smooth the output
I(:,:,1) = medfilt2(I(:,:,1));
I(:,:,2) = medfilt2(I(:,:,2));
I(:,:,3) = medfilt2(I(:,:,3));
Result: