I am working on an app where the customer wants to animate large images (305x332). The customer wants 50 frames in 1.75 seconds to animate in a loop. I am finding that the app is very slow with this much processing. It is slow to start, respond to touches and to shutdown. On the iPhone it self, the app will often crash or lockup the phone. See the code below. My question(s):
Am I doing something to cause the poor performance or is 50 frames too much to ask?
Is there a best practices for number of frames in animations and speed of animations?
Is there a best practices for size of images in an animation?
Please let me know. Here is the code...
NSMutableArray *tempArray = [[NSMutableArray alloc] init];
for(int i = 1; i <= 50; i++)
{
[tempArray addObject:[UIImage imageNamed:[NSString stringWithFormat:#"%#-%d-%04d.JPG",[constitution getConstitutionWord], constitution.getAnimationEnum, i]]];
}
backgroundImage.animationImages = tempArray;
[tempArray release];
backgroundImage.animationDuration = 1.75; // seconds
backgroundImage.animationRepeatCount = 0; // 0 = loops forever
[backgroundImage startAnimating];
I ran some tests a while back. Managed to max out around 40 frames of around 20K PNGs before UIImageview animation gave up and borked.
If you need more than that, you can either switch to use video or write your own animation rendering engine. The rendering engine is pretty straightforward. It would run on a timer that fetches an already loaded UIImage from the head of a queue, updates the view then releases the image. A separate thread preloads a few frames ahead at the tail end of the queue. This way, you have at most N frames in memory at any given time. You can tweak it to find the optimum balance between timer delay and number of frames to pre-load for your app.
A project I worked on used that technique to display hundreds of large images with no problems.
A few tips:
Use CALayers as much as you can.
Don't do ANY scaling when displaying each frame. Try to make each source image the exact size you want displayed.
Try not to cover the animation with other views. It'll slow things down.
You can load the image list from the bundle or by scanning a directory. The disadvantage of bundle is that the images can't be updated without updating the whole app. The disadvantage of loading from a directory is that on first startup you'll have to copy the images from the bundle to a writable location and your app size at runtime gets larger. The main benefit is that you can update the media in that directory via the net.
Sound synchronization becomes a bit dicey. You'll have to come up with your own way to designate when to start/stop sounds. If you have sound and need it to be precise (like lip-synching) going the video route might be more practical.
Good luck.
I think this is a little too much to ask of the iPhone. Decompressed your images take up about 21Mb according to my fag packet. This will likely cause your app to be terminated on the phone just through memory usage. Shifting that much data into the framebuffer is also going to cause the phone problems.
I think you need to use a more suitable animaton technology. ~30fps at 305*322 sounds like video to me. Video is compressed in such a way that you don't need to hold all of the decompressed frames in memory at once. Sadly, if you want to display video without giving over the whole screen to the built in control, you are going to have to build your own player - VLC has been ported to iPhone so could make a good starting point.
The way it was explained to me, the way OS X, including iPhone OS, renders it's view, the images would be continually stacked on each other. Something that you may want to try is a large sprite image that includes all frames, cropped to one at a time and snaps to each frame as needed. This also reduces the image file overhead and rendering speed much like CSS sprites do in web apps. This works well in the HTML5/CSS3 animations I've done on the iPhone and may have the same success in your background image animation.
Related
I'm currently trying to load a map image which is a large(16mb) .jpg file inside a scrollView so you can zoom in and out.
When I launch the app inside the simulator everything runs fine and smooth. However once I run it on my test device (iPod 4.1, iOS 6.0) the app shows the launch image and then it crashes with no error messages at all.
This is how the code currently looks like.
myImage = [[UIImageView alloc] initWithImage:[UIImage imageNamed:#"map.jpg"]];
myScrollView=[[UIScrollView alloc]initWithFrame:CGRectMake(15,120,290,365)];
myScrollView.showsVerticalScrollIndicator=YES;
myScrollView.scrollEnabled=YES;
myScrollView.maximumZoomScale = 10.0;
myScrollView.minimumZoomScale = [myScrollView frame].size.width / myImage.frame.size.width;
myScrollView.clipsToBounds = YES;
myScrollView.bounces = NO;
myScrollView.showsHorizontalScrollIndicator = NO;
myScrollView.showsVerticalScrollIndicator = NO;
[self.view addSubview:whiteFrame];
[self.view addSubview:myScrollView];
myScrollView.contentSize = CGSizeMake(myImage.frame.size.width,myImage.frame.size.height);
myScrollView.delegate = self;
myScrollView.zoomScale = [myScrollView frame].size.width / myImage.frame.size.width;
[myScrollView addSubview:myImage];
Any help is greatly appreciated.
EDIT:
I found this in the docs
If you have a very large image, or are loading image data over the web, you may want to create an incremental image source so that you can draw the image data as you accumulate it. You need to perform the following tasks to load an image incrementally from a CFData object:
Create the CFData object for accumulating the image data.
Create an incremental image source by calling the function CGImageSourceCreateIncremental.
Add image data to the CFData object.
Call the function CGImageSourceUpdateData, passing the CFData object and a Boolean value (bool data type) that specifies whether the data parameter contains the entire image, or just partial image data. In any case, the data parameter must contain all the image file data accumulated up to that point.
If you have accumulated enough image data, create an image by calling CGImageSourceCreateImageAtIndex, draw the partial image, and then release it.
Check to see if you have all the data for an image by calling the function CGImageSourceGetStatusAtIndex. If the image is complete, this function returns kCGImageStatusComplete. If the image is not complete, repeat steps 3 and 4 until it is.
Release the incremental image source.
Does anyone know about a sample code for that ?
A 16mb jpg is going to take a pretty decent chunk of RAM when uncompressed. Your app is probably crashing because it is using too much memory. You can check this by reproducing the crash and checking the device's console log in the Xcode organizer.
You will need to either:
Reduce the dimensions of the image as much as your design will permit to greatly reduce memory usage. For example, there's no reason to have a 16mb jpg if it will never be shown at full-resolution.
Chop the image up into tiles and only load the tiles currently displayed on the screen and the surrounding areas. Then load additional tiles as the user scrolls around. This is how maps apps are able to display extremely large images without running out of RAM.
Remember to also test your app on the supported device with the lowest amount of RAM. These devices will probably kill your app sooner than the newest devices.
At 16 MB, the image is too big for your iPod to handle. The app is crashing because it's loading the image directly, and asking for too much memory and the system has had to kill it.
You should create a smaller version of your image, in terms of both canvas size and image quality, for your app.
You could also incrementally load the image instead.
When you're testing on the simulator, it has full access to the gobs of memory on your computer. Way more than the 256MB available to your iPod 4 (there isn't such thing as an iPod 4.1).
Have a look at CATiledLayer
https://developer.apple.com/library/ios/documentation/GraphicsImaging/Reference/CATiledLayer_class/Introduction/Introduction.html
There is also an example Apple project using it
https://developer.apple.com/library/ios/samplecode/PhotoScroller/Introduction/Intro.html#//apple_ref/doc/uid/DTS40010080
Good luck
I hope you are not loading this image on app launch. Application launch should be as light as possible. If launch time is more than 20 secs then iOS kills the app on its own. On simulator resources are much bigger and you may not see the similar behavior as that with device. You may also want to use the lazy loading technique to load lighter image first and then load heavy image only when needed while zooming, panning etc.
Having said that, one imortant aspect of [UIImage imageNamed] is that it caches all images loaded in that way and they never get unloaded, even if you dealloc the UIImage that was created! This is good in some circumstances (e.g. smallish images in UITableView cells), but bad in others (e.g. large images).
The solution is to use [UIImage imageWithData] which does not do this caching and which unloads the data when the UIImage is dealloc'd.
Another solution to this problem consists of either loading scaled version of the image (*1) or you have option to display it in full resolution with help of CATiledLayer(*2).
[UIImage imageWithCGImage:scale:orientation:];
CATiledLayer example
I have an iPhone app that, among other things, allows users to store photos. When a new photo is added to the app's data store, I cache a thumbnail version of the image so that the photo thumbnail grids load in a reasonable amount of time.
The problem is that these thumbnails look great on a pre-Retina Display screen, but they look a little blurry on RD displays. It's not so bad that the images are unusable, but I would really like to be able to get the full benefit of Retina Display for images users saved with older versions of my app.
The problem is that re-creating all these thumbnails takes way too long. In my tests, it took about a minute and a half to re-encode a sample database to high-res thumbnails (admittedly a large one) on my iPhone 4. It will be even worse on older hardware.
How can I get around this? Doing a one-time migration seems out of the question, given the performance results above. Other options are shrinking the thumbnails lazily (i.e. as they're displayed on-screen) and then saving them to the database at that point. Screens full of old images will be sluggish the first time they're viewed, and then snappier after that.
Are there other approaches to consider? Anyone else faced this problem?
I dont like the idea that you try and convert the images.
User will quickly get impatient and say you app is buggy and takes ages to load.
I think you solve the situation without any re-processing of full sized images.
On older hardware you would not have a retina display (so no need to upsize the images). If they have a retina display then they have a fast iPhone iPod.
I would suggest you graphically solve the problem by how you display the thumbnail images. so instead of fullscreen put a border around this image and show it at its true resolution (dont upscale it). Or show 4 images where you normally show 1 (since iPhone screen is 4x the resolution).
Instead of resampling the original massive image, you could do a bicubic upsample of the thumbnail making it 4x the size. This will make it slightly blurry but it should look better than the iPhone scaling which will look really bad. The upsample would be ultra fast as its working with a small image.
I cannot help you out on upsampling but there will be some code somewhere.
Cheers, John.
Screens full of old images will be sluggish the first time they're viewed, and then snappier after that.
It doesn't have to be sluggish.
It's a bit of a pain, but you can do most of your processing in a background thread. Set the thread priority to something low (like 0.1) to avoid making the UI too slow. The easiest way to do this is to set up an NSOperation for each image you need to convert and add them to a NSOperationQueue with maxConcurrentOperationCount=1.
If writes are not atomic, in -applicationDidEnterBackground: or -applicationWillTerminate: (or in something listening for the corresponding notifications notifications), do something like [queue cancelAllOperations]; for (NSOperation * operation in queue) { [operation setThreadPriority:1]; } [queue waitUntilAllOperationsAreFinished];; you get about 10 seconds or so which should be enough for the image conversion to finish writing to disk (and thus avoid half-written files). For added protection, check [operation isCancelled] immediately before the write if it might take longer than 10 seconds. Obviously, in -applicationWillEnterForeground:, you should restart the conversion (remembering that some of the images have already been converted).
Concurrency issues are fun to track down...
(Note that [data writeToFile:path atomically:YES] isn't sufficient — it's likely to leave temporary files lying around if the app is killed during the write. I'd recommend storing thumbnails in Core Data if you can, but that might be out of the question for existing apps.)
I'm seeing my app being killed by iOS with an out of memory message, however, while tracing the progress of the app in the Allocations Instrument, I see lots of mallocs that seem to be occurring outside of the code I've written.
I'm not seeing any leaks being caught, so I assume these allocations are supposed to be there. Thing is, because I'm not sure about why they have been allocated, I'm not sure what I can do to optimize the app and prevent the OS from jettisoning my app.
Does anyone know why the memory is being allocated, or is there any way for me to find out?
Here are a couple of shots from Instruments showing the mallocs. In the second shot, all of the allocations have the same stack trace.
EDIT
I' displaying a single large image as the UIView background (1024x768), then overlaying a smaller (600px square) UIView with some custom drawing and a third UIView (550px square) over the top of those that contains two 550px square images overlayed.
I'm guessing that this is not appropriate, and there is probably a better way of achieving the composition of views I need for the app to work.
Should this be possible on the iPad?
I think there's not really much information to go on here - if you add a bit more information about what this view in your app is doing you might get some more informed suggestions.
From the screenshot, it would appears large blocks are being allocated to display an image.
Given that I'd hazard a guess that either you're trying to display some very large images, or you UIView is large, or you have more UIViews in memory that you need to display the current screen.
I guess the easiest way to track down exactly where they're coming from would be to disable the part of the application you suspect then run again and see if the allocations still occur.
EDIT
Are all the images the same size as you're displaying them? (ie. are you trying to display a 5M photo as the 1024x768 background?) If not you probably need to scale them down to the size you are display them, or at least closer.
If you're not needing transparency, make sure to make all the views opaque.
I figured out the source of the problem - I was using
[UIImage imageNamed:#'Someimage']
to load in my images. This, as I'm sure many people are aware, caches the image data. I had enough images of sufficient size to cause my app to be jettisoned.
The problem was apparent not because of the size of the image but because of both the size and number of images I was using. The lesson here is be careful with [UIImage imageNamed:].
Thanks for all of the help, chaps!
Mallocs can occur inside of other API's that your app calls (such as loading images, views, playing long sounds, etc.) You can try changing the size of your images, views, sounds and other objects by various amounts as a test, and see if the size of the malloc'd memory changes track one of the changes that you've made.
I've got an app I'm working on where we handle a LOT of images at once in a scrollview. (Here's how it looks, each blue block being in image on a scrollview expanding to the right: http://i.stack.imgur.com/o7lFx.png) So to be able to handle the large strain doing this puts on memory. So I've implemented a bunch of techniques such as reusing imageviews etc which have all worked quite successfully in keeping my memory usage down. Another thing I do is instead of keeping the actual image in memory (which I of course couldn't do for all of them because that would run out of memory very quickly) I only keep the image's filepath in memory and then read the image when the user scrolls to an area of the scroll view near that image. However, although this is memory efficient, it's causing a LOT of lag in the scrollview because of the fact that it has to constantly read images from the disk. I can't think of a good solution on how to fix this. Basically right now the app draws to the screen only the visible uiimageviews and while the user scrolls the app will look to see if it can dequeue another imageview so it doesn't have to allocate another one and at that point it reads the image into memory, but as I said it's causing the scrolling action to be very slow. Any ideas on a strategy to use to fix this? Does anyone know what the native photos app does to handle this kind of thing? Thanks so much!
I can suggest you a simple solution to balance both the memory and the computer processing. You only keep small images like thumbnails in memory and only keep about 20 of them. One project that I am doing, I keep 20 thumbnail images (100 x 100) recently accessed, which doesn't cost a lot of memory. I believe that it costs about 200 kb all the time but comparing to a general available memory. I think it is good enough.
It also depends on your use case : if user scroll really fast and you don't know when will they go. You can have even smaller images than the thumnail and when you show it on the UIImageView, you resize it to fit. When user stops scrolling for a while. You can start loading bigger images and then you have a nicer images. User may not even notice about the process
I don't think there is a solution that can be fast and using as less memory as possible. Because we have memory, maybe not big but have enough if we use it smartly.
Slow scrolling performance might mean that you're blocking the main thread while loading images. In that case, the scrolling animation won't continue until the images are loaded, which would indeed cause pretty choppy scrolling performance.
It would be better to lazily load requested images in the background, while the main thread continues to handle the scrolling animation. A library that provides this functionality (among other things) is the 'three20' library. See the Tidbits document, and scroll down to the bottom where the 'TTImageView' class is described.
I had a similar issue with a PDF viewer, The recommended way to do this is to have as low a res image as you can get away with and if you are allowing the user to blow the image up/zoom, then have two versions or three versions of that image increasing the res as you go.
Put as much code as you can get away with in the didDecelerate method (like loading in higher res images like vodkhang talks about), rather than processing loads in didScroll. Recycle Views out of scope as you have said. and beware of autoreleased Context based Image Creation functions.
Load images in on background threads intelligently (based on the scrollView Offset position and zoom level), and think about using CALayer/Tiled Layer drawing for larger images.
Three20 (an open source iOs lib) has a great Photo Viewer that can be subclassed, it has thumbnail navigation, large image paging, caching and gestures right out of the box.
My app is doing some pretty but heavy weight core animations during scrolling. Sometimes it crashes due to bad performance. So I need some way to find out if there is enough capability to make the animations, and if not, I just leave them away. Best way would be if I could ask the system how busy it is.
UPDATE: I mean especially Core Animation.
By animation, do you mean frames that play after one another (like an animated GIF) or some CoreAnimation (OpenGL) effect that is moving polygons with mapped textures around?
If it's the former, I'd really consider some way of optimizing the animation or eliminating it in all cases.
If it's the latter, I'd do some deeper digging into the source of the problem. Core Animation under normal circumstances will drop frames in order to keep from getting into situations like this in the first place.
In either case case you might consider loading the texture assets a little earlier. I have had some trouble in my apps with animation methods that take a UIImage parameter when I created the UIImage in the function call. Preloading the asset a little earlier in my code took care of the problem nicely.
As an example:
BAD
[[UIImage imageNamed:#"checkmark.png"] drawAtPoint:p];
BETTER
//declared at top of class
static UIImage *checkmark = nil;
in init:
checkmark = [UIImage imageNamed:#"checkmark.png"];
in drawRect:
[checkmark drawAtPoint:p];
You would need to adapt this technique to your particular situation. In my case, checkmark is used often, and it quite small, I don't mind dedicating the memory to it permanently.
I wonder if your crashes could be fixed by making sure the assets were ready to be used by the application.
I wouldn't do that. If your app crashes, it's to heavy. You could run your app with some instruments to see where your bottlenecks are.
So, without trying to sound too harsh, your best way is to rewrite some parts in order to make you app run on an iPhone at all times.