I have tried Instruments and Shark to profile an iPhone app, but both use a data sampling approach, by regularly taking screenshots of thread stacks.
I would prefer to have a full coverage profiling tool, which would record every single function call and the time spent in functions and their subroutines. This seems intuitively better than getting samples. Something like AQtime on Windows would be great.
So my questions are:
Can sample-based profiling be trusted and be as useful as function-call-based profiling?
Can Instruments or Shark do this type of profiling?
Are there other tools available, which would be closer to what I want?
I've found that the time profiling used by Shark is very accurate for determining what your bottlenecks are in your code. You can adjust the sampling interval to be more fine-grained by showing the Mini Config editor using Config | Show Mini Config Editor and lowering the sample time.
Instruments in Xcode 3.2 also now has a nice Time Profiler instrument, although that's Mac-only. I've found that Instruments works well for profiling, but it can drop samples if the system is under heavy load. Generally, I start with Instruments, given how easy it is to use, then move on to Shark if I need a more detailed view of what's going on.
If you really want to do function-call-based profiling, I'd look at DTrace. I've written a couple of articles about tuning Cocoa applications using DTrace here and here. The latter one even shows an example of tuning the startup time of an iPhone application using a custom DTrace script.
Unfortunately, DTrace currently does not run on the iPhone itself, but you can still gather a lot of interesting information using it by running your application in the simulator. While the exact timing information will be nowhere near what it is on the device, knowing exactly what methods are executed how many times and in what order can give some clues as to where to optimize. I use DTrace to provide a different perspective on information gathered by Shark and Instruments, and to answer specific questions about my application.
Yes, stack sampling can be counterintuitive, and yes it can be trusted. Look at this link, and the first comment.
It's not about asking "How long did this take?"
It's about asking (in a doubtful voice) "Was that nanosecond necessary?"
Related
I'm designing an iPhone application and writing a paper on it. I was wondering how I can see how much virtual/real memory my application will take in the iPhone. I am currently running it in the simulator.
No, this cannot be done accurately. You can get consistently good values for most of the APIs you use using the Allocations instrument, but there are differences in execution environments which make exact figures impossible.
Yes, you want to Profile you app. In Xcode, hold down the "Run" button and select Profile. A window should pop up asking you to select an Instrument to profile your app with. For memory usage, you can analyze your apps using either Allocations or Leaks. I would do some googling for instructions on how to use these two instruments.
What is the best and most efficient way to benchmark an iOS application? We are mainly looking to get response times for the application to communicate with our API and complete the processing of the returned data.
If you're looking for API response times, you can simply add the two lines of code that measure the times within your app (log at request start, log at request end.)
You can also look into using Apple's Instruments toolset to measure device CPU performance and leaks.
For the quickset-and-dirtiest method of measuring your performance, just NSLog at the start of the request, end of the request/start of processing, and end of processing. That will give you an idea of whether your device or the server are causing a hold-up (something I assume you're looking for). Xcode will timestamp the outputs and you can analyze them after a few trials.
Also, if you run your app in the simulator, it will not give a good representation of phone speed, as it just runs at your computer's speed, but it will give you the option of using the new Network Link Conditioner in Lion to simulate slow and sketchy network connections, so you can see how the network performance would feel out in the field - just make sure to bear in mind you won't have the comfort of the extra processing power.
Flurry provides decent, free analytics and support timed events. Take a look: http://www.flurry.com/product/analytics/index.html
This is good if you want to collect data from other folks running your app.
I have an app that I added a lot of animation to. The app also used "iPhone sleep preventer" to play silent audio. Since then, I noticed that the battery consumption increased by up to 4 times! I'd like to find a method to profile the power consumption (I think I saw an option in Instruments) to find and eliminate the offending method(s).
Where would I start looking for information like this? Currently I have the phone left on the desk for ~3 hours to record power drain over time. Is there a better method to predict when the app will run out of power if running my app continuously?
An extra side question: are the % of battery left displayed in the status bar linear or is there some non-linearity towards the end of the battery life?
Edit: I found a "power" preset in xcode>product>profile>CPU>Energy diagnostics. It doesn't seem to work perfectly, as the power consumption level is always 0/20. But it does tell me how much of the CPU time is spent on app foreground, graphics and music!
Now I dont know how the CPU power is managed, is running the CPU at 75% more power consuming than lets say 30%? Intuitively it feels like it should...
Thank you!
I'm no expert. Im fact I am only starting to power profile a iphone today, and looked upon your question here in hope off learning.
So I will share with what I've found in meanwhile.
On IOS Developer Library I have found the following:
Connect the device to your development system.
Launch Xcode or Instruments.
On the device, choose Settings > Developer and turn on power logging.
Disconnect the device and perform the desired tests.
Reconnect the device.
In Instruments, open the Energy Diagnostics template.
Choose File > Import Energy Diagnostics from Device.
And you have a report of Cpu and energy during the time of the log.
You can find this steps and many more info on this section of the IOS Dev. lib.
I am still a bit fresh on this matter, so if you find anything that you think is meaningful please post that info here.
Edit: The apple dev lib suffered some changes. Updated link
I don't have an iPhone Developer Program Account yet and will be getting one in the next couple of days. Can instruments be used with the simulator to give a rough estimate on how well my app may perform? Using instruments I checked and fixed all the leaks it was detecting, and it appears that my memory usage maxes out at about 5.77mb. Is there any other tests I could perform with instruments to judge how well my app would perform? I realize there is no way other then the actual device to get a definite answer, it would be nice to get an estimate.
Keeping memory under control is a good first step. Also, make sure you're not allocating too many objects, as all those allocations can take time (use ObjAlloc to check this; it can show you things that Leaks cannot).
Comparing performance between the Sim and Device is pretty much pointless... you have effectively unlimited memory can CPU on the desktop. Best wait until you get your app installed on an actual device.
You could test with a jailbroken device while you're waiting for your developer account. By the time you get your account, you would have time to fix any potential problems, and only need to recompile with your new credentials and deploy.
As anyone with an iPhone knows, some applications launch quickly, while others take several seconds.
What are the best techniques for ensuring an iPhone app launches and becomes usable in a snappy manner?
Apple recommends you "lazy load" every view. I.e. only load the first page on start up, and other pages only when they are navigated to.
In terms of graphics, use PNGs wherever possible as the device is heavily optimized for this format.
Also include the startup screenshot so the user knows the application is loading.
I use lots of external resources, so I use Lazy loading to get up and running quickly. This way the APP can start with the barest minimum and then load the rest while its already begun.
Made a big difference in start time
This is one of those things where there is no sure-fire path to success. Use Apple's excellent Instruments tool to monitor your application's launch. You then need to delve into the results to figure out ways to optimise the launch process.
Also try profiling with Shark, to find any performance bottlenecks.
http://developer.apple.com/iphone/library/documentation/DeveloperTools/Conceptual/SharkUserGuide/Introduction/Introduction.html