How to change thin image to fat image in iPhone SDK? - iphone

Hi, is there any way to change your image thin size to fat size vice-versa in iPhone SDK?
In this application I want to provide the user a possibility to change its image from regular size to fat size by sliding the slider he can measure its size in iPhone SDK?
I think this can be worked by getting pixels of image i have tried this code to get pixels of image but it just removes colors from the image.
UIImage *image = [UIImage imageNamed:#"foo.png"];
CGImageRef imageRef = image.CGImage;
NSData *data = (NSData *)CGDataProviderCopyData(CGImageGetDataProvider(imageRef));
char *pixels = (char *)[data bytes];
// this is where you manipulate the individual pixels
// assumes a 4 byte pixel consisting of rgb and alpha
// for PNGs without transparency use i+=3 and remove int a
for(int i = 0; i < [data length]; i += 4)
{
int r = i;
int g = i+1;
int b = i+2;
int a = i+3;
pixels[r] = pixels[r]; // eg. remove red
pixels[g] = pixels[g];
pixels[b] = pixels[b];
pixels[a] = pixels[a];
}
// create a new image from the modified pixel data
size_t width = CGImageGetWidth(imageRef);
size_t height = CGImageGetHeight(imageRef);
size_t bitsPerComponent = CGImageGetBitsPerComponent(imageRef);
size_t bitsPerPixel = CGImageGetBitsPerPixel(imageRef);
size_t bytesPerRow = CGImageGetBytesPerRow(imageRef);
CGColorSpaceRef colorspace = CGColorSpaceCreateDeviceRGB();
CGBitmapInfo bitmapInfo = CGImageGetBitmapInfo(imageRef);
CGDataProviderRef provider = CGDataProviderCreateWithData(NULL, pixels, [data length], NULL);
CGImageRef newImageRef = CGImageCreate (
width,
height,
bitsPerComponent,
bitsPerPixel,
bytesPerRow,
colorspace,
bitmapInfo,
provider,
NULL,
false,
kCGRenderingIntentDefault
// the modified image
UIImage *newImage = [UIImage imageWithCGImage:newImageRef];
imgView.image = newImage;
I have also tried by stretching image from this code.
UIImage *stretchImage = [image stretchableImageWithLeftCapWidth:10 topCapHeight:10];
Can anybody help me? I didn't find any framework or SDK that gives me that kind of functionality. I have googled for long time.

As it happens, I'm working on something almost exactly like what you describe for our company right now. We're tackling faces, which is an easier problem to do realistically.
Here is a before image (of yours truly)
And here it is with a "fat face" transformation:
What I did is to import the image into an OpenGL texture, and then apply that texture to a mesh grid. I apply a set of changes to the mesh grid, squeezing certain points closer together and stretching others further apart. Getting a realistic fat face took a lot of fine-tuning, but the results are quite good, I think.
Once I have the mesh calculated, OpenGL does the "heavy lifting" of transforming the image, and very fast. After everything is set up, actually drawing the transformed image is a single call.
Here is the same image, showing the grid lines:
Fat face with grid lines http://imageshack.com/a/img404/8049/afterwithlines.jpg
It took me a couple of weeks of full-time work to get the basic mesh warping working (for a client project that fell through) and then another week or so part time to get the fat face layout fine-tuned. Getting the whole thing into a salable product is still a work in progress.
The algorithm I've come up with is fast enough to apply the fat transformation (and various others) to video input from an iOS camera at the full 30 FPS.
In order to do this sort of image processing you need to understand trig, algebra, pointer math, transformation matrixes, and have a solid understanding of how to write highly optimized code.
I'm using Apple's new Core Image face recognition code to find peoples face features in an image, and use the coordinates of the eyes and mouth as the starting point for my transformations.
Your project is much more ambitious. You would have to do some serious image processing to find all the features, trace their outlines, and then figure out how to transform the image to get a convincing fat effect. You'd need high resolution source images that were posed, lit, and shot under carefully controlled conditions in order to get good results.
It might be too much for the computing power of current iOS devices, and the kind of image recognition you'd need to do to figure out the body parts and how to transform them would be mind-bendingly difficult. You might want to take a look at the open source OpenCV project as a starting point for the image recognition part of the problem.

Related

CGContextDrawImage inconsistent byte values across devices?

I'm trying to compare two images (actually locate a smaller "sub-image" in bigger image) and I'm loading the images using the method provided below.
The code below now contains a testing for-loop which sums up all the individual byte values. What I discovered is that this sum and therefor bytes, differ, depending on which device it is being run. My question is why is that happening ?
// Black and white configuration:
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceGray();
NSUInteger bytesPerPixel = 1;
NSUInteger bitsPerComponent = 8;
CGBitmapInfo bitmapInfo = kCGBitmapByteOrderDefault;
// Image
CGImageRef imageRef = [[UIImage imageNamed:#"image.jpg"] CGImage];
NSUInteger imageWidth = CGImageGetWidth(imageRef);
NSUInteger imageHeight = CGImageGetHeight(imageRef);
NSUInteger imageSize = imageHeight * imageWidth * bytesPerPixel;
NSUInteger imageBytesPerRow = bytesPerPixel * imageWidth;
unsigned char *imageRawData = calloc(imageSize, sizeof(unsigned char));
CGContextRef imageContext = CGBitmapContextCreate(imageRawData, imageWidth, imageHeight, bitsPerComponent,
imageBytesPerRow, colorSpace, bitmapInfo);
// Draw the actual image to the bitmap context
CGContextDrawImage(imageContext, CGRectMake(0, 0, imageWidth, imageHeight), imageRef);
CGContextRelease(imageContext);
NSUInteger sum = 0;
for (int byteIndex = 0; byteIndex < imageSize; byteIndex++)
{
sum += imageRawData[byteIndex];
}
NSLog(#"Sum: %i", sum); // Output on simulator: Sum: 18492272
// Output on iPhone 3GS: Sum: 18494036
// Output on another 3GS: Sum: 18494015
// Output on iPhone 4: Sum: 18494015
free(imageRawData);
CGColorSpaceRelease(colorSpace);
Are the devices all running the same version of the OS? Another possibility (beyond colorspaces, which someone already mentioned) is that the JPG decoding libraries may be subtly different. As JPEG is a lossy image format, it's not inconceivable that different decoders would produce resulting bitmaps that were not bit-equal. It's seems reasonable to posit that, given the heavy use of images in iOS UI, that the JPG decoder is something that would be undergoing constant tuning for maximum performance.
I'd even believe it conceivable that between the same OS version running on different models of device (i.e. different processors), the results could be not bit-equal if there were multiple versions of the JPG decoder, each heavily optimized for a specific CPU, although that would not explain the difference between 2 devices of the same model, with the same OS.
You might try to re-run the experiment with an image in a lossless format.
It also may be worth pointing out that providing your own backing memory for a CGBitmapContext, but not making special allowances for word alignment is likely to lead to poor performance. For instance, you have:
NSUInteger imageBytesPerRow = bytesPerPixel * imageWidth;
If imageBytesPerRow is not a multiple of the CPU's native word length, you're going to get sub-optimal performance.
I assume the "device grey" color space varies by device. Try with a device independent color space.

iPhone and OpenCV errors using the AV SampleBuffer Output and Face Detection

Hi fellow Stack Overflow coders,
I've recently been playing around with OpenCV on the iPhone, but have run into a wall and so am turning to you for help. I am familiar with UIKit, Objective C, have used OpenCV for blob detection and face tracking with the Openframeworks C++ framework on a desktop computer.
I am currently attempting to get face tracking working with a real time video stream.
I have got face detection working from this tutorial and sample code using a single image:
http://niw.at/articles/2009/03/14/using-opencv-on-iphone/en
and I have successfully got the data from the video buffer output displaying in a window using OpenGL from a tutorial and sample code (and Apple's example) that I can't actually find the link for right now (will post as soon as I can find it).
I have managed to get edgeDetection of a video stream working with iPhone using the video buffer output by creating an IPLImage from didOutputSampleBuffer and using the standard openCVEdgeDetect and displaying on the screen.
But I can't get Face Detection working, I've tried and tried, but am stuck. Currently I'm just trying to get a single image from the sample buffer passed into the standard openCVFaceDetect methods and then displaying that image with a square (or any marker) around the face. Once I get that working I'll then attempt with a full video stream.
The function that crashes the app is being passed the IPLImage, it then does it's openCV magic and passes this image to a delegate function to be displayed etc
- (void) opencvFaceDetect:(IplImage *)aTempOverlayImage {
cvSetErrMode(CV_ErrModeParent);
IplImage *aOverlayImage = aTempOverlayImage;//[self CreateIplImageFromUIImage:imageView.image];
// Scaling down
IplImage *small_image = cvCreateImage(cvSize(aOverlayImage->width/2,aOverlayImage->height/2), IPL_DEPTH_8U, 3);
cvPyrDown(aOverlayImage, small_image,CV_GAUSSIAN_5x5);
// Load XML
NSString *path = [[NSBundle mainBundle] pathForResource:#"haarcascade_frontalface_default" ofType:#"xml"];
CvHaarClassifierCascade* cascade = (CvHaarClassifierCascade*)cvLoad([path cStringUsingEncoding:NSASCIIStringEncoding], NULL, NULL, NULL);
CvMemStorage* storage = cvCreateMemStorage(0);
// Detect faces and draw rectangle on them
CvSeq* faces = cvHaarDetectObjects(small_image, cascade, storage, 1.2f, 2, CV_HAAR_DO_CANNY_PRUNING, cvSize(20, 20));
cvReleaseImage(&small_image);
[self.delegate parseOpenCVFaces:faces];
// Create canvas to show the results
CGImageRef imageRef = [self getCGImageFromCVImage:aOverlayImage];//imageViewFromOpenCV.image.CGImage;
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef contextRef = CGBitmapContextCreate(NULL, 360, 480,
8, 480 * 4,
colorSpace, kCGImageAlphaPremultipliedLast|kCGBitmapByteOrderDefault);
CGContextDrawImage(contextRef, CGRectMake(0, 0, 360, 480), imageRef);
CGContextSetLineWidth(contextRef, 4);
CGContextSetRGBStrokeColor(contextRef, 0.0, 0.0, 1.0, 0.5);
int scale = 2;
// Draw results on the iamge
for(int i = 0; i < faces->total; i++) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
// Calc the rect of faces
CvRect cvrect = *(CvRect*)cvGetSeqElem(faces, i);
CGRect face_rect = CGContextConvertRectToDeviceSpace(contextRef, CGRectMake(cvrect.x * scale, cvrect.y * scale, cvrect.width * scale, cvrect.height * scale));
CGContextStrokeRect(contextRef, face_rect);
[pool release];
}
imageViewFromOpenCV.image = [UIImage imageWithCGImage:CGBitmapContextCreateImage(contextRef)];
[self.delegate parseOpenCVFaceImage:imageViewFromOpenCV.image];
CGContextRelease(contextRef);
CGColorSpaceRelease(colorSpace);
cvReleaseMemStorage(&storage);
cvReleaseHaarClassifierCascade(&cascade);
}
I get the error: Thread 1: Program received signal:"SIGABRT" at the line:
cvPyrDown(aOverlayImage, small_image,CV_GAUSSIAN_5x5);
I've looked at the openCV source code and it seems this code shrinks and blurs the image before it is analysed. Although I would love a better explanation from someone who has more knowledge (I've done numerous Google searches). When I comment out this code I get another similar error on the closing bracket of the function.
Am I going about this the right way? I know people have got this working and that many other people would love help with this. Any advice or help with this would be much appreciated.
thanks.

IPhone YUV channel orientation

I am grabbing the YUV channel from the IPhone in the kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange format (YUV, bi-planar).
I intend to process the y-channel, so I grab it using
CVImageBufferRef pixelBuffer = CMSampleBufferGetImageBuffer( sampleBuffer );
CVPixelBufferLockBaseAddress( pixelBuffer, 0 );
int bufferHeight = CVPixelBufferGetHeight(pixelBuffer);
int bufferWidth = CVPixelBufferGetWidth(pixelBuffer);
uint8_t *y_channel = CVPixelBufferGetBaseAddressOfPlane(pixelBuffer, 0);
The problem is that the y_channel pixels appears rotated and mirrored (I draw them on an overlay layer to see what the look like:
- (void)drawLayer:(CALayer *)layer inContext:(CGContextRef)context
{
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceGray();
CGContextRef bitmapContext = CGBitmapContextCreate(rotated,
imageSize->x,
imageSize->y,
8, // bitsPerComponent
1*imageSize->x, // bytesPerRow
colorSpace,
kCGImageAlphaNone);
CFRelease(colorSpace);
CGImageRef cgImage = CGBitmapContextCreateImage(bitmapContext);
CGContextDrawImage(context, CGRectMake(0, 0, imageSize->x/2, imageSize->y/2), cgImage);
CFRelease(cgImage);
CFRelease(bitmapContext);
}
I have considered looping through the pixels and created a fixed version of the image, but I am wondering if there is a method to get the y_channel in the correct orientation (IE: not rotated in 90 degrees) straight from the camera.
I don't believe there is a way to alter the orientation of the Y plane coming from the camera, but that shouldn't matter that much in your processing, because you should be able to work with it just fine in its native orientation. If you know that it's rotated 90 degrees, simply tweak your processing to work with it at that rotation.
Also, I believe the mirroring you see is due to your drawing into the Core Graphics coordinate space, where the origin is in the lower left. On the iPhone, CALayers that back UIViews get flipped so that the origin is in the upper left, which can cause images drawn using Quartz in these layers to be inverted.
For display, I would recommend not doing Quartz drawing like you show here, but instead use the Y channel image as an OpenGL ES texture. This will be far more performant. Also, you can simply specify the correct texture coordinates to automatically deal with any image rotation you want in a hardware-accelerated manner.
I describe how to do hardware-accelerated image processing on iOS devices using OpenGL ES 2.0 here, and I provide a sample application that does this processing using shaders and draws the result to the screen in a texture. I'm working with the BGRA colorspace in that example, but you should be able to pull out the Y channel and use it as a luminance texture in the same way.

Does CGContextDrawImage decompress PNG on the fly?

I'm trying to write an iPhone app that takes PNG tilesets and displays segments of them on-screen, and I'm trying to get it to refresh the whole screen at 20fps. Currently I'm managing about 3 or 4fps on the simulator, and 0.5 - 2fps on the device (an iPhone 3G), depending on how much stuff is on the screen.
I'm using Core Graphics at the moment and currently trying to find ways to avoid biting the bullet and refactoring in OpenGL. I've done a Shark time profile analysis on the code and about 70-80% of everything that's going on is boiling down to a function called copyImageBlockSetPNG, which is being called from within CGContextDrawImage, which itself is calling all sorts of other functions with PNG in the name. Inflate is also in there, accounting for 37% of it.
Question is, I already loaded the image into memory from a UIImage, so why does the code still care that it was a PNG? Does it not decompress into a native uncompressed format on load? Can I convert it myself? The analysis implies that it's decompressing the image every time I draw a section from it, which ends up being 30 or more times a frame.
Solution
-(CGImageRef)inflate:(CGImageRef)compressedImage
{
size_t width = CGImageGetWidth(compressedImage);
size_t height = CGImageGetHeight(compressedImage);
CGContextRef context = NULL;
CGColorSpaceRef colorSpace;
int bitmapByteCount;
int bitmapBytesPerRow;
bitmapBytesPerRow = (width * 4);
bitmapByteCount = (bitmapBytesPerRow * height);
colorSpace = CGColorSpaceCreateDeviceRGB();
context = CGBitmapContextCreate (NULL,
width,
height,
8,
bitmapBytesPerRow,
colorSpace,
kCGImageAlphaPremultipliedLast);
CGColorSpaceRelease( colorSpace );
CGContextDrawImage(context, CGRectMake(0, 0, width, height), compressedImage);
CGImageRef result = CGBitmapContextCreateImage(context);
CFRelease(context);
return result;
}
It's based on zneak's code (so he gets the big tick) but I've changed some of the parameters to CGBitmapContextCreate to stop it crashing when I feed it my PNG images.
To answer your last questions, your empirical case seems to prove they're not uncompressed once loaded.
To convert them into uncompressed data, you can draw them (once) in a CGBitmapContext and get a CGImage out of it. It should be well enough uncompressed.
Off my head, this should do it:
CGImageRef Inflate(CGImageRef compressedImage)
{
size_t width = CGImageGetWidth(compressedImage);
size_t height = CGImageGetHeight(compressedImage);
CGContextRef context = CGBitmapContextCreate(
NULL,
width,
height,
CGImageGetBitsPerComponent(compressedImage),
CGImageGetBytesPerRow(compressedImage),
CGImageGetColorSpace(compressedImage),
CGImageGetBitmapInfo(compressedImage)
);
CGContextDrawImage(context, CGRectMake(0, 0, width, height), compressedImage);
CGImageRef result = CGBitmapContextCreateImage(context);
CFRelease(context);
return result;
}
Don't forget to release the CGImage you get once you're done with it.
This question totally saved my day! thanks!! I was having this problem although I wasn't sure where the problem was. Speed up UIImage creation from SpriteSheet
I would like to add that there is another way to load the image directly decompressed, qithout having to write to a context.
NSDictionary *dict = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES]
forKey:(id)kCGImageSourceShouldCache];
NSData *imageData = [NSData dataWithContentsOfFile:#"path/to/image.png"]];
CGImageSourceRef source = CGImageSourceCreateWithData((__bridge CFDataRef)(imageData), NULL);
CGImageRef atlasCGI = CGImageSourceCreateImageAtIndex(source, 0, (__bridge CFDictionaryRef)dict);
CFRelease(source);
I believe it is a little bit faster this way. Hope it helps!

OpenGL ES (IPhone) alpha blending looks weird

I'm writing a game for IPhone in Opengl ES, and I'm experiencing a problem with alpha blending:
I'm using glBlendFunc(Gl.GL_SRC_ALPHA, Gl.GL_ONE_MINUS_SRC_ALPHA) to achieve alpha blending and trying to compose a scene with several "layers" so I can move them separately instead of having a static image. I created a preview in photoshop and then tried to achieve the same result in the iphone, but a black halo is shown when I blend a texture with semi-transparent regions.
I attached an image. In the left is the screenshot from the iphone, and in the right is what it looks like when I make the composition in photoshop. The image is composed by a gradient and a sand image with feathered edges.
Is this the expected behaviour? Is there any way I can avoid the dark borders?
Thanks.
EDIT: I'm uploading the portion of the png containing the sand. The complete png is 512x512 and has other images too.
I'm loading the image using the following code:
NSString *path = [NSString stringWithUTF8String:filePath];
NSData *texData = [[NSData alloc] initWithContentsOfFile:path];
UIImage *image = [[UIImage alloc] initWithData:texData];
if (image == nil) NSLog(#"ERROR LOADING TEXTURE IMAGE");
GLuint width = CGImageGetWidth(image.CGImage);
GLuint height = CGImageGetHeight(image.CGImage);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
void *imageData = malloc( height * width * 4 );
CGContextRef context = CGBitmapContextCreate( imageData, width, height, 8, 4 * width, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big );
CGColorSpaceRelease( colorSpace );
CGContextClearRect( context, CGRectMake( 0, 0, width, height ) );
CGContextTranslateCTM( context, 0, height - height );
CGContextDrawImage( context, CGRectMake( 0, 0, width, height ), image.CGImage );
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);
CGContextRelease(context);
free(imageData);
[image release];
[texData release];
I need to answer my own question:
I couldn't make it work using ImageIO framework so I added libpng sources to my project and loaded the image using it. It works perfect now, but had I to solve the following problem:
The image was loaded and showed fine in the simulator but was not loading at all on the real device. I found on the web that what's going on is that the pixel ordering in PNG image-format files is converted from RGBA to BGRA, and the color values are pre-multiplied by the alpha channel value as well, by a compression utility 'pngcrush' (for device-specific efficiency reasons, when programming with the UIKit interface).
The utility also renames a header of the file, making the new PNG file unusable by libpng. These changes are done automatically when PNG files are deployed onto the iPhone. While this is fine for the UIKit, libpng (and other non-Apple libraries) generally can't then read the files.
The simple solutions are:
rename your PNG files with a different extension.
for your iPhone
-device- build add the following user-defined setting:
IPHONE_OPTIMIZE_OPTIONS | -skip-PNGs
I did the second and it works perfect on simulator and device now.
Your screenshot and photoshop mockup suggest that the image's color channels are being premultiplied against the alpha channel.
I have no idea what your original source images look like but to me it looks like it is blending correctly. With the blend mode you have you're going to get muggy blends between the layers.
The photoshop version looks like you've got proper transparency for each layer, but not blending. I suppose you could experiement with glAlphaFunc if you didn't want to explicitly set the pixel alphas exactly.
--- Code relating to comment below (removing alpha pre-multiplication) ---
int pixelcount = width * height;
unsigned char* off = pixeldata;
for (int pi=0; pi<pixelcount; ++pi)
{
unsigned char alpha = off[3];
if( alpha!=255 && alpha!=0 )
{
off[0] = ((int)off[0])*255/alpha;
off[1] = ((int)off[1])*255/alpha;
off[2] = ((int)off[2])*255/alpha;
}
off += 4;
}
I am aware this post is ancient, however I had the identical problem and after attempting some of the solutions and agonising for days I discovered that you can solve the pre-multiplied RGBA png issue by using the following blending parameters;
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA);
The GL_ONE parameter replaced the GL_SRC_ALPHA parameter in my case.
I can now use my RGBA PNGs without the gray alpha edges effect which made my glyph text look nasty.
Edit:
Okay, one more thing, for fading etc (setting the alpha channel in code), you will need to pre-multiply manually when the blending is set as above, like so;
glColor4f(r*a, g*a, b*a, a);