I want to draw an alpha mask image in code. Right now I do:
1) Create a bitmap context using CGBitmapContextCreate with options CGColorSpaceCreateDeviceRGB and kCGImageAlphaPremultipliedFirst.
2) Then I draw into this context, using only grayscale colors like white and black.
3) Then I create a mask image from that context, using CGImageMaskCreate.
Conclusion: I waste a lot of memory! Because from my understanding, a mask image is grayscale only, right? So why create a context in ARGB in the first place.
How can I create a CGContextRef that is intended to be used for drawing a mask image? My thoughts are to use CGColorSpaceCreateDeviceGray, but here the problems start. This is the exact code how I create my ARGB bitmap context:
CGContextRef context = NULL;
CGColorSpaceRef colorSpace;
uint32_t * bitmapData;
int imageWidth = round(size.width);
int imageHeight = round(size.height);
int bitmapBytesPerRow = (imageWidth * 4);
int bitmapByteCount = (bitmapBytesPerRow * imageHeight);
colorSpace = CGColorSpaceCreateDeviceRGB();
bitmapData = malloc(bitmapByteCount);
context = CGBitmapContextCreate(bitmapData,
imageWidth,
imageHeight,
8, // bits per component
bitmapBytesPerRow,
colorSpace,
kCGImageAlphaPremultipliedFirst);
CGColorSpaceRelease(colorSpace);
I am not sure how to compute the bitmapBytesPerRow for such a context. I assume it would be just imageWidth? And what must I supply for bits per component in CGBitmapContextCreate?
There is CGColorSpaceGetNumberOfComponents() but it reports only the number of components. This does not tell me how many bytes a component has.
Also what makes me nervous is that the 4 and 8 are hard-coded in my code above. Who says it's always 4 bytes per component, and who says it's 8 bits per component? I just took this from various sample codes out there. Everyone seems to do it this way. It works. But future proof? Probably not.
You would make my day with some great answers. Thanks.
Edit: I found a code-snippet, but it is confusing:
CGColorSpaceRef colorSpace2 = CGColorSpaceCreateDeviceGray();
CGContextRef gradientBitmapContext = CGBitmapContextCreate (NULL, 1, reflectRect.size.height,8, 0, colorSpace2, kCGImageAlphaNone);
Why 0 for bytes per row? The documentation does not say you can pass 0. Looks wrong.
Those parameters are telling the system how to treat the data or memory you supply. You have created that yourself, so you know what layout you intend. What, if anything, the system might want to do with it behind the scenes is not your immediate problem.
In this case, you'll provide 8 bits per sample, with just the 1 component, and probably not want to use any row padding, in which case your bytesPerRow should indeed be the same as the image width.
Related
I am a long time reader, first time poster on StackOverflow, and must say it has been a great source of knowledge for me.
I am trying to get to know the AVFoundation framework.
What I want to do is save what the camera sees and then detect when something changes.
Here is the part where I save the image to a UIImage :
if (shouldSetBackgroundImage) {
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
// Create a bitmap graphics context with the sample buffer data
CGContextRef context = CGBitmapContextCreate(rowBase, bufferWidth,
bufferHeight, 8, bytesPerRow,
colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
// Create a Quartz image from the pixel data in the bitmap graphics context
CGImageRef quartzImage = CGBitmapContextCreateImage(context);
// Free up the context and color space
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
// Create an image object from the Quartz image
UIImage * image = [UIImage imageWithCGImage:quartzImage];
[self setBackgroundImage:image];
NSLog(#"reference image actually set");
// Release the Quartz image
CGImageRelease(quartzImage);
//Signal that the image has been saved
shouldSetBackgroundImage = NO;
}
and here is the part where I check if there is any change in the image seen by the camera :
else {
CGImageRef cgImage = [backgroundImage CGImage];
CGDataProviderRef provider = CGImageGetDataProvider(cgImage);
CFDataRef bitmapData = CGDataProviderCopyData(provider);
char* data = CFDataGetBytePtr(bitmapData);
if (data != NULL)
{
int64_t numDiffer = 0, pixelCount = 0;
NSMutableArray * pointsMutable = [NSMutableArray array];
for( int row = 0; row < bufferHeight; row += 8 ) {
for( int column = 0; column < bufferWidth; column += 8 ) {
//we get one pixel from each source (buffer and saved image)
unsigned char *pixel = rowBase + (row * bytesPerRow) + (column * BYTES_PER_PIXEL);
unsigned char *referencePixel = data + (row * bytesPerRow) + (column * BYTES_PER_PIXEL);
pixelCount++;
if ( !match(pixel, referencePixel, matchThreshold) ) {
numDiffer++;
[pointsMutable addObject:[NSValue valueWithCGPoint:CGPointMake(SCREEN_WIDTH - (column/ (float) bufferHeight)* SCREEN_WIDTH - 4.0, (row/ (float) bufferWidth)* SCREEN_HEIGHT- 4.0)]];
}
}
}
numberOfPixelsThatDiffer = numDiffer;
points = [pointsMutable copy];
}
For some reason, this doesn't work, meaning that the iPhone detects almost everything as being different from the saved image, even though I set a very low threshold for detection in the match function...
Do you have any idea of what I am doing wrong?
There are three possibilities I can think of for why you might be seeing nearly every pixel be different: colorspace conversions, incorrect mapping of pixel locations, or your thresholding being too sensitive for the actual movement of the iPhone camera. The first two aren't very likely, so I think it might be the third, but they're worth checking.
There might be some color correction going on when you place your pixels within a UIImage, then extract them later. You could try simply storing them in their native state from the buffer, then using that original buffer as the point of comparison, not the UIImage's backing data.
Also, check to make sure that your row / column arithmetic works out for the actual pixel locations in both images. Perhaps generate a difference image the absolute difference of subtracting the two images, then use a simple black / white divided area as a test image for the camera.
The most likely case is that the overall image is shifting by more than one pixel simply through the act of a human hand holding it. These whole-frame image shifts could cause almost every pixel to be different in a simple comparison. You may need to adjust your thresholding or do more intelligent motion estimation, like is used in video compression routines.
Finally, when it comes to the comparison operation, I'd recommend taking a look at OpenGL ES 2.0 shaders for performing this. You should see a huge speedup (14-28X in my benchmarks) over doing this pixel-by-pixel comparison on the CPU. I show how to do color-based thresholding using the GPU in this article, which has this iPhone sample application that tracks colored objects in real time using GLSL shaders.
Human eyes are way much different than a camera (even a very expensive one) in the way that we don't perceive minimal light changes or small motion changes. Cameras DO, they are very sensitive but not smart at all!
With your current approach (it seems you are comparing each pixel):
What would happen if the frame is shifted only 1 pixel to the right?! You can image right the result of your algorithm, right?. Humans will perceive nothing or almost nothing.
There is also the camera shutter problem: That means that every frame might not have the same amount of light. Hence, a pixel-by-pixel comparison method is too prone to fail.
You want to at least pre-process your image and extract some basic features. Maybe edges, corners, etc. OpenCV is easy for that but I am not sure that doing such a processing will be fast in the iPhone. (It depends on your image size)
Alternatively you can try the naive template matching algorithm with a template size that will be a little short than your hole view size.
Image Processing is computationally expensive so don't expect it to be fast from the first time, specially in a mobile device and even more if you don't have experience in Image Processing/Computer Vision stuff.
Hope it helps ;)
I'm generating an image using quartz2d and I want to use it as an opengl texture.
The tricky part is that I want to use as few bits per pixel as possible, so I'm creating cgContext as following:
int bitsPerComponent = 5;
int bytesPerPixel = 2;
int width = 1024;
int height = 1024;
void* imageData = malloc(width * height * bytesPerPixel);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGImageContext context = CGBitmapContextCreate(imageData, width, height, bitsPerComponent, width * bytesPerPixel, colorSpace, kCGImageAlphaNoneSkipFirst);
//draw things into context, release memory, etc.
As stated in the documentation here, this is the only supported RGB pixel format for CGBitmapContextCreate which uses 16 bits per pixel.
So now I want to upload this imageData which looks like "1 bit skipped - 5 bits red - 5 bits green - 5 bits blue" into an opengl texture. So I should do something like this:
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_SHORT_5_5_5_1, imageData);
That won't work because in this call I've specified pixel format as 5 red - 5 green - 5 blue - 1 alpha. That is wrong, but it appears that there is no format that would match core graphics output.
There are some other options like GL_UNSIGNED_SHORT_1_5_5_5_REV, but those wont work on the iphone.
I need some way to use this imageData as a texture, but I really don't want to swap bytes around manually using memset or such, because that seems terribly inefficient.
You do need to swap bits around to get it into a denser format like RGBA551 or RGB565, since as you note, CGBitmapContext does not support these formats for drawing (for simplicity and efficency's sake).
memset isn't going to do the trick, but there are "fast" conversion routines in Accelerate.framework.
See vImageConvert_ARGB8888toRGB565(…) and vImageConvert_ARGB8888toARGB1555(…), available on iOS 5 and later.
For iOS 7.0, OS X.9 and later:
vImage_CGImageFormat fmt = {
.bitsPerComponent = 5,
.bitsPerPixel = 16,
.colorSpace = NULL, // faster with CGImageGetColorSpace(cgImage) if known to be RGB
.bitmapInfo = kCGImageAlphaNoneSkipFirst | kCGBitmapByteOrder16Little // ARGB1555 little endian
};
vImage_Buffer buf;
vImageBuffer_InitWithCGImage( &buf, &fmt, NULL, cgImage, kvImageNoFlags );
...
free(buf.data);
Data is in buf.data, along with image height, width and rowBytes info. I don't recall what GL's requirements are for whether row padding is allowed. You can control that by preallocating the buf.data and buf.rowBytes fields and passing kvImageDoNotAllocate in the flags.
565_REV is kCGImageAlphaNone | kCGBitmapByteOrder16Little.
5551_REV is kCGImageAlphaNoneSkipLast | kCGBitmapByteOrder16Little
When I use the following code:
UIImage *image=[UIImage imageNamed:#"loginf1.png"];
CGImageRef rawImageRef=image.CGImage;
const float colorMasking[6] = {222, 255, 222, 255, 222, 255};
CGImageRef maskedImageRef=CGImageCreateWithMaskingColors(rawImageRef, colorMasking);
maskedImageRefis always nil. Why is this, and what can I do to correct this?
I had the same problem. The CGImageRef you are creating has only 6 bytes for each pixel with byte with no alpha channel. The masking function needs a CGImageRef with 8 bytes for each pixel, only 6 of them used, with no alpha channel. At least, I think this is what's causing it.
Anyway, fix it by creating a bitmap context, drawing your image to that bitmap context, then getting your CGImageRef from CGBitmapContextCreateImage.
The reason it is failing is because you CANNOT create a mask image with an alpha channel. Unfortunately what you are trying to is NOT possible.
The only way to use "CGImageCreateWithMaskingColors(...)" it to provide it a bitmap context WITHOUT an alpha channel. The catch 22 here is that it is NOT possible to create a bitmap context WITHOUT an alpha channel. Don't you just love Apple?
Is there a way to find out whether two CGPathRefs are intersected or not. In my case all the CGPaths are having closePath.
For example, I am having two paths. One path is the rectangle which is rotated with some angle and the other path is curved path. Two paths origin will be changing frequently. At some point they may intersect. I want to know when they are intersected. Please let me know if you have any solution.
Thanks in advance
Make one path the clipping path, draw the other path, then search for pixels that survived the clipping process:
// initialise and erase context
CGContextAddPath(context, path1);
CGContextClip(context);
// set fill colour to intersection colour
CGContextAddPath(context, path2);
CGContextFillPath(context);
// search for pixels that match intersection colour
This works because clipping = intersecting.
Don't forget that intersection depends on the definition of interiority, of which there are several. This code uses the winding-number fill rule, you might want the even odd rule or something else again. If interiority doesn't keep you up at night, then this code should be fine.
My previous answer involved drawing transparent curves to an RGBA context. This solution is superior to the old one because it is
simpler
uses a quarter of the memory as an 8bit greyscale context suffices
obviates the need for hairy, difficult-to-debug transparency code
Who could ask for more?
I guess you could ask for a complete implementation, ready to cut'n'paste, but that would spoil the fun and obfuscate an otherwise simple answer.
OLDER, HARDER TO UNDERSTAND AND LESS EFFICIENT ANSWER
Draw both CGPathRefs separately at 50% transparency into a zeroed, CGBitmapContextCreate-ed RGBA memory buffer and check for any pixel values > 128. This works on any platform that supports CoreGraphics (i.e. iOS and OSX).
In pseudocode
// zero memory
CGContextRef context;
context = CGBitmapContextCreate(memory, wide, high, 8, wide*4, CGColorSpaceCreateDeviceRGB(), kCGImageAlphaPremultipliedLast);
CGContextSetRGBFillColor(context, 1, 1, 1, 0.5); // now everything you draw will be at 50%
// draw your path 1 to context
// draw your path 2 to context
// for each pixel in memory buffer
if(*p > 128) return true; // curves intersect
else p+= 4; // keep looking
Let the resolution of the rasterised versions be your precision and choose the precision to suit your performance needs.
1) There isn't any CGPath API to do this. But, you can do the math to figure it out. Take a look at this wikipedia article on Bezier curves to see how the curves in CGPath are implemented.
2) This is going to be slow on the iPhone I would expect but you could fill both paths into a buffer in difference colors (say, red and blue, with alpha=0.5) and then iterate through the buffer to find any pixels that occur at intersections. This will be extremely slow.
For iOS, the alpha blend seems to be ignored.
Instead, you can do a color blend, which will achieve the same effect, but doesn't need alpha:
CGContextSetBlendMode(context, kCGBlendModeColorDodge);
CGFloat semiTransparent[] = { .5,.5,.5,1};
Pixels in output Image will be:
RGB = 0,0,0 = (0.0f) ... no path
RGB = 64,64,64 = (0.25f) ... one path, no intersection
RGB = 128,128,128 = (0.5f) ... two paths, intersection found
Complete code for drawing:
-(void) drawFirst:(CGPathRef) first second:(CGPathRef) second into:(CGContextRef)context
{
/** setup the context for DODGE (everything gets lighter if it overlaps) */
CGContextSetBlendMode(context, kCGBlendModeColorDodge);
CGFloat semiTransparent[] = { .5,.5,.5,1};
CGContextSetStrokeColor(context, semiTransparent);
CGContextSetFillColor(context, semiTransparent);
CGContextAddPath(context, first);
CGContextFillPath(context);
CGContextStrokePath(context);
CGContextAddPath(context, second);
CGContextFillPath(context);
CGContextStrokePath(context);
}
Complete code for checking output:
[self drawFirst:YOUR_FIRST_PATH second:YOUR_SECOND_PATH into:context];
// Now we can get a pointer to the image data associated with the bitmap
// context.
BOOL result = FALSE;
unsigned char* data = CGBitmapContextGetData (context);
if (data != NULL) {
for( int i=0; i<width; i++ )
for( int k=0; k<width; k++ )
{
//offset locates the pixel in the data from x,y.
//4 for 4 bytes of data per pixel, w is width of one row of data.
int offset = 4*((width*round(k))+round(i));
int alpha = data[offset];
int red = data[offset+1];
int green = data[offset+2];
int blue = data[offset+3];
if( red > 254 )
{
result = TRUE;
break;
}
}
And, finally, here's a slightly modified code from another SO answer ... complete code for creating an RGB space on iOS 4, iOS 5, that will support the above functions:
- (CGContextRef) createARGBBitmapContextWithFrame:(CGRect) frame
{
/** NB: this requires iOS 4 or above - it uses the auto-allocating behaviour of Apple's method, to reduce a potential memory leak in the original StackOverflow version */
CGContextRef context = NULL;
CGColorSpaceRef colorSpace;
void * bitmapData;
int bitmapByteCount;
int bitmapBytesPerRow;
// Get image width, height. We'll use the entire image.
size_t pixelsWide = frame.size.width;
size_t pixelsHigh = frame.size.height;
// Declare the number of bytes per row. Each pixel in the bitmap in this
// example is represented by 4 bytes; 8 bits each of red, green, blue, and
// alpha.
bitmapBytesPerRow = (pixelsWide * 4);
bitmapByteCount = (bitmapBytesPerRow * pixelsHigh);
// Use the generic RGB color space.
colorSpace = CGColorSpaceCreateDeviceRGB();
if (colorSpace == NULL)
{
fprintf(stderr, "Error allocating color space\n");
return NULL;
}
// Create the bitmap context. We want pre-multiplied ARGB, 8-bits
// per component. Regardless of what the source image format is
// (CMYK, Grayscale, and so on) it will be converted over to the format
// specified here by CGBitmapContextCreate.
context = CGBitmapContextCreate (NULL,
pixelsWide,
pixelsHigh,
8, // bits per component
bitmapBytesPerRow,
colorSpace,
kCGImageAlphaPremultipliedFirst
//kCGImageAlphaFirst
);
if (context == NULL)
{
fprintf (stderr, "Context not created!");
}
// Make sure and release colorspace before returning
CGColorSpaceRelease( colorSpace );
return context;
}
I have created an context like this (simplified):
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate (bitmapData,
pixWide,
pixHeigh,
8, // bits per component
bitmapBytesPerRow,
colorSpace,
kCGImageAlphaPremultipliedFirst);
Now, when I try to extract the data for the first pixel in my PNG with Alphatransparency, it has very weird alpha values. I have an simple PNG that's a square. On each edge I cut off 10x10 pixel and made them totally transparent. Alpha shouldn't be something like 153 there.
There's an kCGImageAlphaOnly declared in CGImage.h. The doc says:
kCGImageAlphaOnly There is no color
data, only an alpha channel.
Ok, so that actually sounds good, because I only need Alpha data, and nothing else. But this raises some question marks in my head. If I do habe a fully equipped PNG with a bunch of colors + alpha: Would this constant make sure that my PNG is converted to match that color space? Or would I have to provide an PNG that matches that specified color space?
Edit: I tried using kCGImageAlphaOnly, but I get this error:
<Error>: CGBitmapContextCreate: unsupported parameter combination: 8 integer bits/component; 24 bits/pixel; 0-component colorspace; kCGImageAlphaOnly; 55 bytes/row.
What may be the problem here? I specified this before:
size_t pixelsWide = CGImageGetWidth(inImage);
size_t pixelsHigh = CGImageGetHeight(inImage);
bitmapBytesPerRow = (pixelsWide * 1); // not * 4, because I just want alpha
bitmapByteCount = (bitmapBytesPerRow * pixelsHigh);
Edit: I've been reading this a minute ago:
PNG's which are added to XCode are
optimized by 'pngcrush' during
compilation. This does some
byte-swapping (from RGBA to BRGA) and
pre-multiplication of alpha.
I assume that this pre-multiplication of alpha makes trouble.
Edit: The alpha channel keeps intact after pngcrunch did the byte-swapping stuff to the PNG. Since I don't care about colors, just alpha, that pre-multiplication shouldn't be a too big problem, I think.
My PNG's have been 24bit PNG bevor I added them to Xcode.
You can't do this:
bitmapBytesPerRow = (pixelsWide * 1); // not * 4, because I just want alpha
The function you're calling will always return all the image data. The kCGImageAlphaOnly constant is used to tell YOU that an image only contains an alpha channel, no colour information.
You'll need to use pixelsWide * 4 for the bytesPerRow. Also note that the bitmapData argument to CGBitmapContextCreate() is used to provide storage space explicitly, rather than having it drawn for you.
Possibly what you want to do is this (untested code, just typed from memory):
CGImageRef image = GetMyImageFromWhereverItIs();
CGColorSpaceRef space = CGColorSpaceCreateDeviceRGB();
CGContextRef ctx = CGBitmapContextCreate( NULL, CGImageGetWidth(image),
CGImageGetHeight(image), CGImageGetBitsPerComponent(image),
CGImageGetBytesPerRow(image), space,
kCGBitmapByteOrderDefault | kCGImageAlphaLast );
CGColorSpaceRelease( space );
// now draw the image into the context
CGRect rect = CGRectMake( 0, 0, CGImageGetWidth(image), CGImageGetHeight(image) );
CGContextDrawImage( ctx, rect, image );
UInt32 * pixels = CGBitmapContextGetData( ctx );
// now we can iterate through the data & read the alpha values
int i, count = CGBitmapContextGetBytesPerRow(ctx) * CGBitmapContextGetHeight(ctx);
for ( i = 0; i < count; i++ )
{
UInt8 alpha = pixels[i] & 0x000000ff;
// do with the alpha what you will
}
Are you sure you're looking only at alpha values?
If you're expecting all the alpha components to come first, then all the red components, etc.: That's planar layout, and I don't think Quartz supports it natively—it only supports all the components together in each pixel (ARGBARGBARGBARGB…, not AAAA…RRRR…GGGG…BBBB…). So if you're just marching straight into the data treating every byte as alpha, that's your problem: you're looking at red, green, and blue components and treating them as alpha.
As for premultiplication, that doesn't affect the alpha channel, it affects the color channels. The formula for raster compositing (putting one raster image over another) is:
dst.r = src.r * src.a + dst.r * (1.0 - src.a);
dst.g = src.g * src.a + dst.g * (1.0 - src.a);
dst.b = src.b * src.a + dst.b * (1.0 - src.a);
Premultiplication cuts out the first multiplication expression:
dst.r = src.r′ + dst.r * (1.0 - src.a);
dst.g = src.g′ + dst.g * (1.0 - src.a);
dst.b = src.b′ + dst.b * (1.0 - src.a);
This works because the source color components are already multiplied by the alpha component—hence the name “premultiplied”. It doesn't need to multiply them now, because it already has the results.
It's an optimization, and presumably an important one on the iPhone (all those multiplication operations add up when you do a million or two of them). But it doesn't affect the layout of the components: interleaved remains interleaved, whether the RGB components are premultiplied or not.