When a PNG is added to an XCode iPhone project, the compiler optimizes it using pngcrush. Once on the device, the image's rendering performance is very fast.
My problem is that my application downloads its PNGs from an external source at runtime (from Picasa Web albums, using the Google Data APIs). Unfortunately, these images' performance is quite bad. When I do custom rendering on top of the image, it seems 100x slower than its internally stored counterparts. I strongly suspect this is because the downloaded images haven't been optimized.
Does anyone know how I can optimize an externally downloaded PNG at runtime on the iPhone? I'm hoping for a class that does this. I even considered adding pngcrush's source code to my app, which seems drastic. I haven't been able to find an decent answer myself. I'd be very grateful for any help.
Thanks!
Update:
Some folks have suggested that it may be due to the file's size, but it isn't. During my tests, I added a toggle button to switch between using the embedded version and the downloaded version of exactly the same PNG. The only difference is that the embedded one was optimized by 'pngcrush' during compilation. This does some byte-swapping (from RGBA to BRGA) and pre-multiplication of alpha. (http://iphonedevelopment.blogspot.com/2008/10/iphone-optimized-pngs.html)
Also, the performance I'm referring to isn't the downloading, but the rendering. I superimpose custom painting on top of the image (overriding the drawRect method of the UIView), and it's very choppy when the background is the downloaded version, and very smooth when it's the embedded (and therefore optimized) version. Again, it's exactly the same file. The only difference is the optimization, which I'm hoping I can perform on the image at runtime, on the device, after downloading it.
Thanks again for everyone's help!
That link you posted pretty much answers your question.
During the build process XCode pre-processes your png so it's in a format that's more friendly to the graphics chip in the iPhone.
Png's that have not been processed like this will likely use a slower rendering path, one that deals with the non-native format and the fact that the alpha must be computed separately for each color.
So you have two options;
Perform the same work that pngcrush does and swap ordering/pre-multiply alpha. The speed up may be due to one or both of these.
After you have loaded your image, you can "create" a new image from it. This new image should be in the iPhone's native format and so should perform faster. The downside is it could potentially take up a bit more memory.
E.g.
CGRect area = CGRectMake(0, 0, width, height);
CGSize size = area.size;
UIGraphicsBeginImageContext(size);
[oldImage drawInRect:area];
UIImage *newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext();
The fact that you say it "seems" 100x slower indicates that you have not performed any experimentation, but made a guess (it must be the PNG optimization), and are now going down a path based on a hunch.
You should spend time to confirm what the problem is before you try to solve it. My gut says that PNG optimization shouldn't be the issue: that mostly affects the loading of images, but once they are in memory it doesn't matter what file format they were originally in.
Anyway, you should try an A-B comparison, either get your code to load an optimized PNG from somewhere else and see how it compares, or make a test app that just does some drawing on the two PNG types. Once you've confirmed what the problem is, then you can figure out if you need to compile pngcrush into your app.
On the surface, it sounds like something else is at play here. Any additional image manipulation should only add time until it's displayed onscreen...
Would it be at all possible to get the server to gzip the images by sending the appropriate HTTP header? (If it even helps file size much, that is.)
Temporarily using the pngcrush source might be a good test as well, just to get some measurements.
Are you storing the png at the original downloaded size? If it's a large image it'll take significantly longer to render.
Well it seems that a good way to do it (since you can't run pngcrush on the iPhone and expect that to speed it up) would be to make your requests through a proxy that runs pngcrush. The proxy would have nice horse power to actually give you some gain over the 100x pain you feel.
try pincrush to trans the normal png file to the crushed png file
You say you are drawing on top of the image by overriding a UIView's drawRect: method. Are you trying to do some animation by repeatedly drawing the whole image with your custom stuff on top of it?
You might get better results if you put your custom stuff in a separate view or layer, and let the OS deal with compositing the result over the background. The OS will only update the parts of the screen that you actually change, and won't be repainting the entire image as often.
Related
I'm making a Universal App using MonoTouch, and I'm adding my Default-Portrait.png file. That file alone (a 768x1004 .png file) is adding 711k to the size of the app. My app itself is only about 7 megs, so it's adding 10% just for the splash screen.
I could easily make this thing an 80k jpg file instead of a png, but the device doesn't seem to look for a .jpg file. Does anyone have tips for reducing the size of this launch art?
At this point, I'm thinking I might just leave the launch art out and load my own jpg and display it as soon as I have the ability to. That'll keep my app size down, but it's not as nice as having the launch art scale in immediately like most apps do.
Hmmm...given the screen of the iPad and the visual quality users are expecting, I'd just leave it like that.
But if you do want to reduce the disk space, try going to Project > Edit Project Settings > Build (tab at top), and searching for a parameter called "Compress PNG Files." Make sure that's checked. It'll run the pngcrush utility before loading the file onto disk (check the size of your IPA archive after to see if it had any effect).
pngcrush is nice as well, however that will not reduce the quality of you image. If reducing the quality of the image is an option for you, then you might try this tool: http://www.punypng.com/ - or just use an image editing tool to "optimize" the image ...
I recommend pngout if you want to really squeeze those PNGs down, and this won't cost you any quality. It simply removes unnecessary metadata (like pngcrush) and uses its own compression algorithm which is compatible with the regular decompressor used in PNG (zlib). It's really slow, though.
A simpler option is to try "Save for web" in your image manipulation program of choice. Exporting from Acorn (not just the regular save) sometimes gives me vastly smaller files. This is especially true for default images which have large, uniform areas in one colour (screenshots, a small logo in the middle of a black screen).
Is there any reason why you want to reduce the file size that badly? I don't think it matters in your case. I just checked 3 of my apps and the Default.png (of various portrait/landscape varieties) is between 29KB and 422KB, so whilst yours do seem a little heavy, your still way under the 3G download limit.
Are you positive it's adding that much to the size of the app? Did you compare a before and after?
Xcode uses pngcrush on the images for you. I know because I just tried to substitue jpegs for pngs and got the following result:
So, in short, there's not a lot to be done except simplify the image beforehand. Xcode will handle the rest.
I have a large png Image that I need to Zoom&Move.
I therefore created a UIScrollView and embedded a UIImageView.
The App works fine in the simulator, but when running it on the device (8GB iPod Touch) it crashes as soon as the view is loaded.
I tried with a smaller test Image (4MB) works fine and suspect the iPod can't handle a 20MB PNG. I also tried different other formats, such as JPG (in various save patterns), but that did't help either.
Any clues how I can solve this?
Ouch, 20M is a large image. The first thought that comes to mind is can you dice up the image? I.e. instead of one image have a whole bunch of small images which together make up the larger image. Then you can load on demand the same way google maps downloads image squares.
have a look at the ScrollViewSuite Example from apple. Sounds exactly like what you are trying to do.
3_Tiling demonstrates:
How to subclass UIScrollView to add content tiling
Reusing tiles to optimize performance and memory use
Changing the resolution of the content in response to zooming
I suggest the Example Photoscroller. It demonstrates CATiledLayer which you can use to tile your image and even use smaller images as lod images. It's much smaller then the complete ScrollViewSuite example but has everything you need to do what you want. It contains only 2 classes which you should be able to use in your project with minor edits.
You might want to check the WWDC 2010 Session #104 "Desinging Apps with Scroll Views"... they handle and explain that example.
You will need to tile your image. I suggest imagemagick for that :)
I have an app that will display a bunch of images in a slideshow. Those images will be part of the bundle, thus distributed with the app.
All the images are photographs or photographic, etc.
I've read that it's preferred to use PNG as the image format, but seeing that the JPG version will be much smaller, I'd rather be using that.
Are there any guidelines which format to use and in which case?
PNG's are pixel perfect (non-lossy), and require very little extra CPU energy to display. However, large PNGs may take longer to read from storage than more compressed image formats, and thus be slower to display.
JPG's are smaller to store, but lossy (amount depends on compression level), and to display them requires a much more complicated decoding algorithm. But the typical compression and image quality is usually quite sufficient for photos.
Use JPG's for photos and for anything large, and PNG's for anything small and/or designed to be displayed "pixel perfect" (e.g. small icons) or as a part of a composited transparent overlay, etc.
Apple optimizes PNG images that are included in your iPhone app bundle. In fact, the iPhone uses a special encoding in which the color bytes are optimized for the hardware. XCode handles this special encoding for you when you build your project. So, you do see additional benefits to using PNG's on an iPhone other than their size consideration. For this reason it is definitely recommended to use PNG's for any images that appear as part of the interface (in a table view, labels, etc).
As for displaying a full screen image such as a photograph you may still reap benefits with PNG's since they are non-lossy and the visual quality should be better than a JPG not to mention resource usage with decoding the image. You may need to decrease the quality of your JPG's in order to see a real benefit in file size but then you are displaying non-optimal images.
File size is certainly a factor but there are other considerations at play as well when choosing an image format.
There is one important thing to think about with PNGs. If a PNG is included in your Xcode build it will be optimized for iOS. This is called PNG crush. If your PNG is downloaded at run time it will not be crushed. Crushed PNGs run about the same as 100% JPGs. Lower quality JPGs run better than higher quality JPGs. So from a performance standpoint from fastest to slowest it would go low quality JPG, high quality JPG, PNG Crushed, PNG.
If you need to download PNGs you should consider crushing the PNGs on the server before the download.
http://www.cocoanetics.com/2011/10/avoiding-image-decompression-sickness/
The Cocoanetics blog published a nice iOS performance benchmark of JPGs at various quality levels, and PNGs, with and without crushing.
From his conclusion:
If you absolutely need an alpha channel or have to go with PNGs then
it is advisable to install the pngcrush tool on your web server and
have it process all your PNGs. In almost all other cases high quality
JPEGs combine smaller file sizes (i.e. faster transmission) with
faster compression and rendering.
It turns out that PNGs are great for small images that you would use
for UI elements, but they are not reasonable to use for any full
screen applications like catalogues or magazines. There you would want
to choose a compression quality between 60 and 80% depending on your
source material.
In terms of getting it all to display you will want to hang onto
UIImage instances from which you have drawn once because those have a
cached uncompressed version of the file in them. And where you don’t
the visual pause for a large image to appear on screen you will have
to force decompression for a couple of images in advance. But bear in
mind that these will take large amounts of RAM and if you are
overdoing it that might cause your app to be terminated. NSCache is a
great place to place frequently used images because this automatically
takes care of evicting the images when RAM becomes scarce.
It is unfortunate that we don’t have any way to know whether or not an
image still needs decompressing or not. Also an image might have
evicted the uncompressed version without informing us as to this
effect. That might be a good Radar to raise at Apple’s bug reporting
site. But fortunately accessing the image as shown above takes no time
if the image is already decompressed. So you could just do that not
only “just in time” but also “just in case”.
Just thought I'd share a bit of decompression performance data...
I'm doing some prototyping of a 360 degree viewer - a carousel where the user can spin through a series of photos taken from different angles, to give the impression of being able to smoothly rotate an object.
I have loaded the image data into an array of NSData's to take file i/o out of the equation, but create NSImage's on the fly. Testing at near max frame rate (~25 fps) and watching in Instruments I see the app is clearly CPU-bound and there's an approximately 10% increase in CPU load showing ~275 kb png's vs. ~75 kb jpg's.
I can't say for sure but my guess is the CPU limit is just from general program execution and moving all the data around in memory, but that image decompression is done on the GPU. Either way and the JPG vs. PNG performance argument looks to favour JPG, especially when the smaller file sizes (and therefore smaller sizes of objects in memory at least in some parts of the chain) is taken into consideration.
Of course every situation is different, there's no substitute for testing...
I have found massive differences in animation performance when using jpegs vs png. For example placing three screen-sized jpegs side by side in a UIScrollView and scrolling horizontally on an iPhone4 results in lag and a thoroughly unpleasant jerky animation. With non-transparent pngs of the same dimensions the scrolling is smooth. I never use jpegs, even if the image is large.
I think if you want to use transparent, you have no choice except PNG. But, if your background is opaque already, then you may use JPG. That is the only difference I can see
'Use JPEG for photos' as mentioned in Human Interface Guidelines under section Produce artwork in the appropriate format.
What is the best way to use the custom UI graphics on the iPhone?
I have come across CGContextDrawPDFPage and Panic's Shrinkit. Should I be using PDF's to store my vector ui graphics and loading them using CGContextDrawPDFPage to draw them.
Previously I asked what way Apple store their UI graphics and was answered crushed png. The options are I can think of are below, but I'm also interested in any other techniques...
PNG (bitmapped image)
Custom UIView drawing code (generated from Opacity)
PDF (I've not used this method, is it with CGContextDrawPDFPage?)
This question is for vector graphics only (but I guess some people may only use bitmapped?). Looking for what is standard / most effective / most efficient.
Edit: Bounty added, I'm interested to hear the process of anyone who works with UI designers, or are themselves a UI designer. And pointers on resolution independence i.e. for iPad / iPhone HD future proofing.
Many thanks
Ross
I can suggest 3 different ways, 2 of which you already mentioned:
Creating custom UIView.
Drawing in a CGLayer.
loading from PDF.
Each have their advantages, depending on what you want to do:
UIView vs CGLayer
In terms of performance (for one-time drawing) and ease of use there shouldn't be much difference between the two (there are minor differences, but nothing serious). Apparently Opacity can export source code for both (I haven't personally used it). That said, there are things you should consider before choosing:
If you have a fixed image (which your question suggests so), use CGLayer. CGLayer objects will be cached on the graphics device, so re-using them is much faster. Even if the cache is cleared, you're still using the same object for redrawing, meaning there's no need for re-creating it.
On the other hand, if you need to change your drawing as the user interacts with the app, UIView could be faster, as you have the flexibility of updating just one part of the image instead of the whole view.
CGLayer is independent of the UI. So the same code works fine for Mac/iPhone/iPad, or even for saving to files.
Conclusion: Use CGLayer, unless it's a special case.
CGLayer (In code drawing) vs PDF (loading from file)
I don't have any benchmark for this, but I expect CGLayer to be slightly faster: (1) there's no need to read a file. (2) the pdf commands should be converted to system's graphic elements, which is more or less the same as creating a CGLayer. (3) I'm not sure about how pdf pages are cached, but I don't expect it to be faster than CGLayer. Anyway, all this shouldn't make much difference unless you want optimization till the last millisec. Again, the choice should be based on your use case:
CGLayer gives you more flexibility in the code. Your only access to a pdf page is through CGContextDrawPDFPage, which means even simple tasks such as scaling/transforming the drawing will be harder.
Using PDFs on the other hand, is more flexible after finishing the code. You can simply update the pdf file with a new whenever you want, load it from the web, etc. .
Creating a pdf could be easier than coding the drawing. You can use any app you want, you don't need to worry about the API and system resources. After all, the code can output a pdf file, not the other way around.
Conclusion: If you don't need to do much with the drawing (just want to show an icon or something), go with the pdf. if you need to work on it in the app, consider CGLayer.
Of course you could always mix the approaches as you see fit: e.g. Load a pdf file, put it in a CGLayer to adjust it, draw it with a UIView where you can put a badge on it!
I stumbled over this question because I have a question about PDFs too. I'm working together with a UI designer and we are succesfully using PDFs to create UI elements. For example: For a button we have 3 PDFs for ON, OFF and the shadow. I wrote a piece of code that transformes a PDF into a UIImage. It can scale the resulting image and even colorize it to have one template for many styles of buttons. It works pretty good :)
Our problem is that we can't scale up the vector graphics without quality loss. That's why we decided to use graphics that are big enough and we only have to scale them down. But I still wonder if there's a way to scale up a PDF before drawing it to a context and create a UIImage. Here's my post.
When creating an UIImage file from a .png to be displayed on a button, view/cell background, etc. for a standard Iphone application, should all of them be in powers of 2 for optimization reasons?
As others have said, no - but you should generally use images with even dimensions. This is because when views are positioned with the center property, it'll position an odd-dimensioned image at some half-pixel position. This will cause the image to appear blurry.
As long as you're aware of this it shouldn't really cause you any problems, but it's still a good idea to use even sizes just to be on the safe side.
(This applies for UIKit, not necessarily OpenGL)
Apple uses odd and arbitrary dimensions for all the images it adds to the interface on your behalf, such as system toolbar items. The best optimization you can do is anything that reduces compositing, which basically means setting the opaque property of views and layers whenever possible.
If you have the choice between a transparent png that will be composited over a static background and an opaque png with the background already included, you have a chance to optimize. When the images will be sliding around or the background will change, you have to composite, otherwise choose opaque.
Here is an article on optimization of iPhone images -- basically tells you why to use PNG files. The size shouldn't matter unless you are using OpenGLES.
No, this will have little or no benefit, I usually suffice at doing my own optimization using photoshop "Save for web or devices" option.
Please see http://iphonedevelopment.blogspot.com/2008/10/iphone-optimized-pngs.html for a detailed explanation about the iPhones pre-optimization of pngs.