Curious about how to read and edit a picture's pixels on the iPhone. Am I better of using an array of points with colours?
I want to do things like.. if a CGPoint intersects with a "brown" spot on the picture, set the colour of all brown pixels in a radius to white. More questions to come, but this is a start.
Cheers
The picture data is available to you as precisely that -- a two-dimensional array of pixels, each pixel being represented by a 32 bit integer. For each of the color components (red, green, blue, and alpga) there is an 8 bit value. The ordering of these 8-bit-wide values within the 32 bit integer varies with the format of the picture data. The apple doc about all this is really good. While there is some attractive Apple stuff using CGDataProviderCopyData to give you a pointer into the actual data storage of a UIImage, in practice this can be a headache, because the format of that internal storage can vary widely from one image to the next. In practice, most people doing image processing seem to use this approach:
CGImageRef image = [UIImage CGImage];
NSUInteger width = CGImageGetWidth(image);
NSUInteger height = CGImageGetHeight(image);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
unsigned char *rawData_ = malloc(height * width * 4);
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel_ * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height));
CGContextRelease(context);
// rawData contains image data in the RGBA8888 format.
// for any pixel at coordinate x,y -- the value is
//
int pixelIndex = (bytesPerRow * y) + x * bytesPerPixel;
unsigned char red = rawData[pixelIndex];
green = rawData[pixelIndex + 1];
blue = rawData[pixelIndex + 2];
alpha = rawData[pixelIndex + 3];
Related
I currently have some image data in a C array, which contains RGBA data.
float array[length][4]
I am trying to get this to a UIImage, which it looks like these are initialized with files, NSData, and URLs. Since the other two methods are slow, I am most interested in the NSData approach.
I can get all of these values into an NSArray like so:
for (i=0; i<image.size.width * image.size.height; i++){
replace = [UIColor colorWithRed:array[i][0] green:array[i][1] blue:array[i][2] alpha:array[i][3]];
[output replaceObjectAtIndex:i withObject:replace];
}
So, I have a NSArray full of objects that are a UIColor. I have tried many methods, but how do I convert this to a UIImage?
I think it would be straight forward. A function sorta like imageWithData:data R:0 B:1 G:2 A:3 length:length width:width length:length would be nice, but there is no function as far as I can tell.
imageWithData: is meant for image data in a standard image file format, e.g. a PNG or JPEG file that you have in memory. It's not suitable for creating images from raw data.
For that, you would typically create a bitmap graphics context, passing your array, pixel format, size, etc. to the CGBitmapContextCreate function. When you've created a bitmap context, you can create an image from it using CGBitmapContextCreateImage, which gives you a CGImageRef that you can pass to the UIImage method imageWithCGImage:.
Here's a basic example that creates a tiny 1×2 pixel image with one red pixel and one green pixel. It just uses hard-coded pixel values that are meant to show the order of the color components, normally, you would get this data from somewhere else of course:
size_t width = 2;
size_t height = 1;
size_t bytesPerPixel = 4;
//4 bytes per pixel (R, G, B, A) = 8 bytes for a 1x2 pixel image:
unsigned char rawData[8] = {255, 0, 0, 255, //red
0, 255, 0, 255}; //green
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
size_t bytesPerRow = bytesPerPixel * width;
size_t bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast);
CGImageRef cgImage = CGBitmapContextCreateImage(context);
//This is your image:
UIImage *image = [UIImage imageWithCGImage:cgImage];
//Don't forget to clean up:
CGImageRelease(cgImage);
CGColorSpaceRelease(colorSpace);
CGContextRelease(context);
Q: I'm looking to use the iPhone camera to take a photo and then replace the green screen in that photo with another photo.
What's the best way to dive into this? I couldn't find many resources online.
Thanks in advance!
Conceptually, all that you need to do is loop through the pixel data of the photo taken by the phone, and for each pixel that is not within a certain range of green, copy the pixel into the same location on your background image.
Here is an example I modified from keremic's answer to another stackoverflow question.
NOTE: This is untested and just intended to give you an idea of a technique that will work
//Get data into C array
CGImageRef image = [UIImage CGImage];
NSUInteger width = CGImageGetWidth(image);
NSUInteger height = CGImageGetHeight(image);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel_ * width;
NSUInteger bitsPerComponent = 8;
unsigned char *data = malloc(height * width * bytesPerPixel);
// you will need to copy your background image into resulting_image_data.
// which I am not showing here
unsigned char *resulting_image_data = malloc(height * width * bytesPerPixel);
CGContextRef context = CGBitmapContextCreate(rawData, width, height, bitsPerComponent, bytesPerRow, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height));
CGContextRelease(context);
//loop through each pixel
for(int row = 0; row < height; row++){
for(int col = 0; col < width*bytesPerPixel; col=col+4){
red = data[col];
green = data[col + 1];
blue = data[col + 2];
alpha = data[col + 3];
// if the pixel is within a shade of green
if(!(green > 250 && red < 10 && blue < 10)){
//copy them over to the background image
resulting_image_data[row*col] = red;
resulting_image_data[row*col+1] = green;
resulting_image_data[row*col+2] = blue;
resulting_image_data[row*col+3] = alpha;
}
}
}
//covert resulting_image_data into a UIImage
Have a look at compiling OpenCV for iPhone - not an easy task, but it gives you access to a whole library of really great image processing tools.
I'm using openCV for an app I'm developing at the moment (not all that dissimilar to yours) - for what you're trying to do, openCV would be a great solution, although it requires a bit of learning etc. Once you've got OpenCV working, the actual task of removing green shouldn't be too hard.
Edit: This link will be a helpful resource if you do decide to use OpenCV: Compiling OpenCV for iOS
I have looked through replacing colors in an image but cannot get it to work how i need because I am trying to do it with every color but one, as well as transparency.
what I am looking for is a way to take in an image and split out a color (say all the pure black) from that image. Then take that split out portion and make a new image with a transparent background and the split out portion.
(here is just an example of the idea, say i want to take a screenshot of this page. make every other color but pure black be transparent, and save that new image to the library, or put it into a UIImageView)
i have looked in to CGImageCreateWithMaskingColors but cant seem to do what I need with the transparent portion, and I dont really understand the colorMasking input other than you can provide it with a {Rmin,Rmax,Gmin,Gmax,Bmin,Bmax} color mask but when I do, it colors everything. any ideas or input would be great.
Sounds like you're going to have to get access to the underlying bytes and write code to process them directly. You can use CGImageGetDataProvider() to get access to the data of an image, but there's no guarantee that the format will be something you know how to handle. Alternately you can create a new CGContextRef using a specific format you know how to handle, then draw the original image into your new context, then process the underlying data. Here's a quick attempt at doing what you want (uncompiled):
- (UIImage *)imageWithBlackPixels:(UIImage *)image {
CGImageRef cgImage = image.CGImage;
// create a premultiplied ARGB context with 32bpp
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
size_t width = CGImageGetWidth(cgImage);
size_t height = CGImageGetHeight(cgImage);
size_t bpc = 8; // bits per component
size_t bpp = bpc * 4 / 8; // bytes per pixel
size_t bytesPerRow = bpp * width;
void *data = malloc(bytesPerRow * height);
CGBitmapInfo bitmapInfo = kCGImageAlphaPremultipliedFirst | kCGBitmapByteOrder32Host;
CGContextRef ctx = CGBitmapContextCreate(data, width, height, bpc, bytesPerRow, colorspace, bitmapInfo);
CGColorSpaceRelease(colorspace);
if (ctx == NULL) {
// couldn't create the context - double-check the parameters?
free(data);
return nil;
}
// draw the image into the context
CGContextDrawImage(ctx, CGRectMake(0, 0, width, height), cgImage);
// replace all non-black pixels with transparent
// preserve existing transparency on black pixels
for (size_t y = 0; y < height; y++) {
size_t rowStart = bytesPerRow * y;
for (size_t x = 0; x < width; x++) {
size_t pixelOffset = rowStart + x*bpp;
// check the RGB components of the pixel
if (data[pixelOffset+1] != 0 || data[pixelOffset+2] != 0 || data[pixelOffset+3] != 0) {
// this pixel contains non-black. zero it out
memset(&data[pixelOffset], 0, 4);
}
}
}
// create our new image and release the context data
CGImageRef newCGImage = CGBitmapContextCreateImage(ctx);
CGContextRelease(ctx);
free(data);
UIImage *newImage = [UIImage imageWithCGImage:newCGImage scale:image.scale orientation:image.imageOrientation];
CGImageRelease(newCGImage);
return newImage;
}
I have been trying to convert an array RGBA data (int bytes) into a UIImage. My code looks like as follows:
/*height and width are integers denoting the dimensions of the image*/
unsigned char *rawData = malloc(width*height*4);
for (int i=0; i<width*height; ++i)
{
rawData[4*i] = <red_val>;
rawData[4*i+1] = <green_val>;
rawData[4*i+2] = <blue_val>;
rawData[4*i+3] = 255;
}
/*I Have the correct values displayed
- ensuring the rawData is well populated*/
NSLog(#"(%i,%i,%i,%f)",rawData[0],rawData[1],rawData[2],rawData[3]/255.0f);
NSLog(#"(%i,%i,%i,%f)",rawData[4],rawData[5],rawData[6],rawData[7]/255.0f);
NSLog(#"(%i,%i,%i,%f)",rawData[8],rawData[9],rawData[10],rawData[11]/255.0f);
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL,
rawData,
width*height*4,
NULL);
int bitsPerComponent = 8;
int bitsPerPixel = 32;
int bytesPerRow = 4*width;
CGColorSpaceRef colorSpaceRef = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
CGColorRenderingIntent renderingIntent = kCGRenderingIntentDefault;
CGImageRef imageRef = CGImageCreate(width,
height,
8,
32,
4*width,colorSpaceRef,
bitmapInfo,
provider,NULL,NO,renderingIntent);
/*I get the current dimensions displayed here */
NSLog(#"width=%i, height: %i", CGImageGetWidth(imageRef),
CGImageGetHeight(imageRef) );
UIImage *newImage = [UIImage imageWithCGImage:imageRef];
/*This is where the problem lies.
The width, height displayed are of completely different dimensions
viz. the width is always zero and the height is a very huge number */
NSLog(#"resultImg width:%i, height:%i",
newImage.size.width,newImage.size.height);
return newImage;
The output image that I receive is an image of width 0, and height 1080950784 (assuming my initil height and width were 240 and 240). I have been trying to get this sorted out and have checked many related forums e.g. (link text) on how to go about it but with little success.
It turns out the problem is a pretty silly mistake that both of us overlooked. UIImage dimensions are stored as floats, not integers. :D
Try
NSLog(#"resultImg width:%f, height:%f",
newImage.size.width,newImage.size.height);
Instead. The image size has been transferred correctly.
The problem is solved now. I get the image that I want displayed but still unable to figure out why the width and height are different. Essentially nothing wrong with the program per say. The only problem being width and height.
I just independently verified this problem, I think it should be reported as a bug to Apple. +(UIImage)imageWithCGImage: doesn't properly transfer the width and height of the source CGImageRef to the UIImage.
How can I detect the difference between 2 images, creating a mask of the area that's different in order to process the area that's common to both images (gaussian blur for example)?
EDIT: I'm currently using this code to get the RGBA value of pixels:
+ (NSArray*)getRGBAsFromImage:(UIImage*)image atX:(int)xx andY:(int)yy count:(int)count
{
NSMutableArray *result = [NSMutableArray arrayWithCapacity:count];
// First get the image into your data buffer
CGImageRef imageRef = [image CGImage];
NSUInteger width = CGImageGetWidth(imageRef);
NSUInteger height = CGImageGetHeight(imageRef);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
unsigned char *rawData = malloc(height * width * 4);
NSUInteger bytesPerPixel = 4;
NSUInteger bytesPerRow = bytesPerPixel * width;
NSUInteger bitsPerComponent = 8;
CGContextRef context = CGBitmapContextCreate(rawData, width, height,
bitsPerComponent, bytesPerRow, colorSpace,
kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(colorSpace);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), imageRef);
CGContextRelease(context);
// Now your rawData contains the image data in the RGBA8888 pixel format.
int byteIndex = (bytesPerRow * yy) + xx * bytesPerPixel;
for (int ii = 0 ; ii < count ; ++ii)
{
CGFloat red = (rawData[byteIndex] * 1.0) / 255.0;
CGFloat green = (rawData[byteIndex + 1] * 1.0) / 255.0;
CGFloat blue = (rawData[byteIndex + 2] * 1.0) / 255.0;
CGFloat alpha = (rawData[byteIndex + 3] * 1.0) / 255.0;
byteIndex += 4;
UIColor *acolor = [UIColor colorWithRed:red green:green blue:blue alpha:alpha];
[result addObject:acolor];
}
free(rawData);
return result;
}
The problem is, the images are being captured from the iPhone's camera so they are not exactly the same position. I need to create areas of a couple of pixels and extracting the general color of the area (maybe by adding up the RGBA values and dividing by the number of pixels?). How could I do this and then translate it to a CGMask?
I know this is a complex question, so any help is appreciated.
Thanks.
I think the simplest way to do this would be to use a difference blend mode. The following code is based on code I use in CKImageAdditions.
+ (UIImage *) differenceOfImage:(UIImage *)top withImage:(UIImage *)bottom {
CGImageRef topRef = [top CGImage];
CGImageRef bottomRef = [bottom CGImage];
// Dimensions
CGRect bottomFrame = CGRectMake(0, 0, CGImageGetWidth(bottomRef), CGImageGetHeight(bottomRef));
CGRect topFrame = CGRectMake(0, 0, CGImageGetWidth(topRef), CGImageGetHeight(topRef));
CGRect renderFrame = CGRectIntegral(CGRectUnion(bottomFrame, topFrame));
// Create context
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
if(colorSpace == NULL) {
printf("Error allocating color space.\n");
return NULL;
}
CGContextRef context = CGBitmapContextCreate(NULL,
renderFrame.size.width,
renderFrame.size.height,
8,
renderFrame.size.width * 4,
colorSpace,
kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease(colorSpace);
if(context == NULL) {
printf("Context not created!\n");
return NULL;
}
// Draw images
CGContextSetBlendMode(context, kCGBlendModeNormal);
CGContextDrawImage(context, CGRectOffset(bottomFrame, -renderFrame.origin.x, -renderFrame.origin.y), bottomRef);
CGContextSetBlendMode(context, kCGBlendModeDifference);
CGContextDrawImage(context, CGRectOffset(topFrame, -renderFrame.origin.x, -renderFrame.origin.y), topRef);
// Create image from context
CGImageRef imageRef = CGBitmapContextCreateImage(context);
UIImage * image = [UIImage imageWithCGImage:imageRef];
CGImageRelease(imageRef);
CGContextRelease(context);
return image;
}
There are three reasons pixels will change from one iPhone photo to the next, the subject changed, the iPhone moved, and random noise. I assume for this question, you're most interested in the subject changes, and you want to process out the effects of the other two changes. I also assume the app intends the user to keep the iPhone reasonably still, so iPhone movement changes are less significant than subject changes.
To reduce the effects of random noise, just blur the image a little. A simple averaging blur, where each pixel in the resulting image is an average of the original pixel with its nearest neighbors should be sufficient to smooth out any noise in a reasonably well lit iPhone image.
To address iPhone movement, you can run a feature detection algorithm on each image (look up feature detection on Wikipedia for a start). Then calculate the transforms needed to align the least changed detected features.
Apply that transform to the blurred images, and find the difference between the images. Any pixels with a sufficient difference will become your mask. You can then process the mask to eliminate any islands of changed pixels. For example, a subject may be wearing a solid colored shirt. The subject may move from one image to the next, but the area of the solid colored shirt may overlap resulting in a mask with a hole in the middle.
In other words, this is a significant and difficult image processing problem. You won't find the answer in a stackoverflow.com post. You will find the answer in a digital image processing textbook.
Can't you just subtract pixel values from the images, and process pixels where the difference i 0?
Every pixel which does not have a suitably similar pixel in the other image within a certain radius can be deemed to be part of the mask. It's slow, (though there's not much that would be faster) but it works fairly simply.
Go through the pixels, copy the ones that are different in the lower image to a new one (not opaque).
Blur the upper one completely, then show the new one above.