Recording User Actions using Instruments - iphone

I've tried looking for a while, but is there a way to record user actions on iphones/iphone simulators for testing with Instruments? Mainly I want to be able to target an app and record whatever user input I enter, then have that be able to play back/get data for that. Thanks!

There are 2 options in Instruments:
1) Use User Interface recorder instrument, that will record mouse move and clicks of the user and allows to replay these. (Unfortunately it seems that Phone Simulator do not expose your app accessibility elements to UI recorder).
2) Simulate interactions with Automation instrument in javascript using UIA classes - useful for writing interface tests for the app or simulating user interactions. You can read intro about using Automation in this blogpost by Alex Vollmer.

FoneMonkey is a promising third party solution, I had that working on a recent project and whilst it wasn't perfect, it does let you record user actions, play them back and so forth. Plus it seems to be open source so you can extend it yourself if required.
There is also the UIAutomation framework in iOS but I have no direct experience with it.

As Roger said, you're going to have to use the UIAutomation framework. Apple's documentation should get you where you need to go. Also, take a look at this.
You have to write the automation in JavaScript. Annoying, I know, but with enough examples in front of you you should be able to pick it up in under an hour.

Related

How can I run automated tests that support real iOS devices, take screenshots, and identify controls by ID?

I need to run some automated UI checks on our native iPhone app. I'd like to be able to do the following:
I'd like to automate a few actions and take screen shots.
I don't want the tool to identify the buttons by image or text, as my app changes images and text all the time. I'd like to identify controls by ID.
I'd like to run these tests on a real iPhone device.
Lastly, when I complete one test, I need to access to settings to modify a few changes, then relaunch app to test. (no need to resume the app)
Is there a good way to achieve these kinds of automated UI tests?
You can use the instrument that apple provides - UIAutomation
It provides all expected automation facilities. Screenshots can also be taken. Check for documentation here apple doc
Here is a nice tutorial to start with http://cocoamanifest.net/articles/2011/05/uiautomation-an-introduction.html

How To Record/Track User Actions in iPhone.....?

Is there any easy and effective way to track user activities from my iPhone app. I have to record all actions performed by user in my application without any performance/frozen problems. There should be no limits like per second hits etc.... App is global and cross platform. It is better if I can write my own custom algorithm to make this functional.
You can also use google analytics
https://developers.google.com/analytics/devguides/collection/ios/
We use a service called Flurry Analytics. It works on all mobile platforms and we have had great experiences with it. Free too!
http://www.flurry.com/flurry-analytics.html
The app can collect all kinds of information in a very versatile manner and is efficient in when it sends the data back the the server. I believe it's only when the app terminates.
On the device side it is super easy to set up and beginning tracking user events.

Simulate actual button press on mobile-safari on iPod Touch / iPhone

I am building a web-based tool for internal purposes for my company that runs on an ipod touch. It's working fine, but there are a few quirks such as not being able to auto-focus on a text field when a page loads without the user actually tapping the screen (I can "focus" the field, but the keyboard is not active). Additionally, I cannot programmatically trigger sounds to play (I am using the jPlayer library). What it seems to come down to is this:
Is there some way I can trick the browser on an ipod touch 4 to thinking the user has actually tapped a specific div on the screen? If I can do that, I can solve every other issue. Since this is for internal purposes, I am free to make any modifications needed. However, I need to able to do keep the "app" code in HTML5 and JavaScript for a myriad of reasons. Perhaps an app with a modification to safari to allow this, then I can run my site in that app?
Perhaps an app with a modification to safari to allow this, then I can run my site in that app?
Yes, you could write a really simple app with just a UIWebView in which you display your HTML5 based app. If you need extra things such as back button etc. you would have to implement that (it's also not very difficult). The UIWebView should behave mostly exactly like Safari, so it should be a de facto "app with a modification to safari".
You could then give the right element focus and call
[webView becomeFirstResponder];
The sounds could also be played programmatically by simply requesting the appropriate URL.
I think with this setup the additional effort in terms of coding beyond your existing web based tool is minimal. However, this assumes you have Xcode, know some basic Objective-C and are familiar with the procedures of ad hoc or company distribution of "real" apps.
You can try to use a timed event

Getting started with accessibility

What are the typical mistakes an iPhone developer unaware of accessibility makes that renders their app unusable by customers with common impairments?
What are the first and easiest steps to take when making an iPhone app accessible to the vision impaired (etc.)?
How do I make sure Voice Over says or does something appropriate for all my programmatically created UIButtons, UISliders, etc.?
See the following link from Apple on accessibility on iOS.
http://developer.apple.com/iphone/library/documentation/UserExperience/Conceptual/iPhoneAccessibility/Accessibility_on_iPhone/Accessibility_on_iPhone.html
I strongly suggest using voiceover to test your application after you've used the developer tools to check it's accessibility. I'm a blind iPod touch user and find that VoiceOver has a very low entry barrior, unlike windows screen reading software that requires you to memorize a bunch of keystrokes and realize that your application may be presented in an entirely different way to the user then you expect.

What does iPhone OS 3.0 need from a programming perspective?

iPhone OS 3.0 is being announced and previewed next week (March 17).
We all know the feature set users want. Copy/paste, MMS, Flash on iPhone, etc.
We'll see about those.
What I'm interested in what does the development community feel the SDK is missing, in need of, to make programming for the platform easier and more productive.
A more complete Interface Builder with support for custom palettes and all sorts of goodies like that.
Better control over the keyboard.
Better unit testing support. (Unit testing can be done, but only on the simulator, and it's very awkward to set up.)
Push notifications. Please.
A more accurate simulator, i.e. one with a more accurate set of frameworks.
The ability to easily build views like the Mail compose window.
For that matter, an in-application compose window.
A better way for apps to share data locally than by invoking URLs.
Access to the calendar, notes, mail (possibly read-only), and bookmarks (again, read-only) databases. Maybe even limited access to the iPod database—even just the ability to read song metadata and access and change the playing song would be helpful.
Some sort of middle ground between UILabel and UIWebView that allows for formatted text without a huge hassle.
More built-in toolbar icons.
The return of the "glass" button style that was in the beta SDK.
A few useful internal views, like UIProgressHUD, exposed.
And last but not least...
A pony.
An easy Javascript bookmarklet installation method for Mobile Safari. (OpenRadar: 1, 2)
UIWebView needs more of UIScrollView's properties and methods, such as contentOffset.
More configurability on some of the built-in behaviors and views, e.g. the button text on UITableViewCell's "Delete" button, or the styles and text of UIAlertSheet/UIAlertView buttons. (Some of these can be done today with undocumented calls, but I'd rather not rely on those.)
More flexibility from UINavigationController, such as the ability to push/pop views that selectively don't display the navigation bar but using the same animations and stack, or more customizability over the navigation bar button labels and behaviors.
The ability to restrict interface orientation per UIViewController, not just accept/reject changes via shouldAutorotate. E.g. I want my main content view to be autorotatable, but I want my navigation hierarchy and settings screens to always display in portrait, even if the content view was rotated to landscape.
libxml and its handy DOM XML parser instead of the SAX-based NSXMLParser.
libcurl w/SSL, or more options and functionality for NSURLConnection.
Ability to check whether a URL scheme is registered. This could be used for apps to detect whether other specific apps are installed, and enable functionality selectively, e.g. when Instapaper detects Tweetie is installed, it can offer a "Post with Tweetie" button. (Disclaimer: That was a plug. I make Instapaper.)
I'm sure I'll think of more, but overall, I'm very happy developing for the iPhone. I'm amazed at the quality and sophistication of the iPhone OS, the SDK, and the development tools given how incredibly young they all are.
I'm surprised no one has mentioned garbage collection yet. Objective-C 2.0 on the Mac supports optional garbage collection. I don't really see any reason it wouldn't work just fine on the iPhone as well and it would eliminate much of the tedium of having to explicitly release objects all over the place.
What I'm hoping most for is to allow iPhones to talk to each other either via Bluetooth or some other means. Granted, they can talk via Bonjour if they are on the same Wi-Fi network, but that's just not convenient enough in 2009. If I'm out with a friend and want to play a multi-player game we first have to find a Starbucks or whatever the heck to get on the same Wi-Fi network. Also, think of the ridiculous amount of social apps you could have if iPhones could talk to each other without needing Wi-Fi. Exchange business cards, flirt with the cute girl over there, etc.
Form a PURE programmers perspective, make XCode as helpful of an IDE as Eclipse or IntelliJ are in the Java world. There's so much time I waste on stupid stuff that the IDE could have found for me as I typed it.
I also don't understand why I can't color buttons without having to use images.
Better multitasking is absolutely key at this point. Android's got it, Palm's WebOS has it - both, it seems, in largely unrestricted and well-implemented fashion. Possibilities:
Push notifications with a good UI (message stack in addition to badging/sound/whatever - if they have to have an extra approval step so apps can't be obnoxious, so be it)
Multiple full processes (not possible with current OS, I realize, but then I've never seen a good explanation why the iPhone doesn't support virtual memory)
Smaller "background" versions of apps that can run in the background - no GUI and a significantly tighter memory constraint
A good mapping API. Let us access the Google Maps abstraction that the Maps application uses !
More Interface Builder goodness
Better simulator
Smart inbox. Incoming messages are routed to installed handlers based on type.
Synchronisation framework that simplifies syncing with desktop & Mobile Me.
Decent landscape support, without the multitude of bugs, especially for the camera picker. Better support for rotation and more control of it.
Access to EXIF data on images from the picker, so we can tell their location
Deeper access to the camera API, so that we are not rail-roaded into the standard photo taker / picker
Push notifications that can launch an application. (In lieu of full multi-tasking, which I don't think we'll get and which could be problematic.)
Better, more intuitive keyboard controls.
API for inter-application messaging.
Access to data from Calendar, iTunes, Mail, Notes and more (with user's permission)
A more accurate simulator, with, for example, ways to limit bandwidth, and use the Mac's camera to actually take a photo.
Phone-phone bluetooth for data exchange
Access to more of the views used by iPhone apps, e.g. the progress HUD, email "blobbing" mechanism for email addresses, thumbnail scrollers, HUD brought up in Photos app, and more.
Less sandboxing. It won't likely happen, but it would always be appreciated for an app to have slightly more power than they currently do (actual filesystem access, for example. even if it was read-only access, it would still allow for more interesting applications to exist).
EDIT: Also, access to the copy/paste API. But I hope that one is obvious to Apple.
My list:
More full-featured IB support as the Mac has
Inter-app Data transfer mechanism (could be C&P, but does not have to be)
Greatly improved camera API with deeper level of control and more flexibility
SDK access to bluetooth and more support for protocols
Real ObjectiveC framework around the address book like the Mac has today.
Warnings similar to the location warning when an app tries to access address book data.
I'm sure whatever they actually have prepared, there will be a few interesting twists.
Ability to send SMS messages without having to have launch the SMS client and have the user type the message.
Access to the raw camera data so that things can be done without having to take a picture and wait for it to save (like you can do with Android)
push notification so that you can launch tasks... would need to be user controllable.
A camera that can focus (I know... have to wait for the next iPhone for that... if they decide to put it in...)
A UIKit level drawing api.
We all know the feature set people want. Copy/Paste, MMS, Flash on iPhone, etc.
I would have thought those specific items were down the SO wish list (although it seems I'm wrong looking at the votes on this comment :-).
MMS is a pretty pointless app when you have eMail. Flash is not an OS issue - Flash could be delivered today.
I don't even want push notifications - they're just a patch, I want background apps. I also want fixes for all the broken APIs like Camera, video and landscape support. Support for CoreImage filters would be nice too but probably too much to wish for.
[[ABAddressBook sharedAddressBook] me] for being able to use the owner's Zip code, phone number, or whatever.
Ability to download files to local storage and sync them back to iTunes or your hard drive
Get EXIF data from photos
Pull all photos at once
Pull all contacts at once
Control screen brightness
Access to music in iPod section
Read access to email and text messages
Access to Safari cookies (so maybe I could make some kind of keep-me-logged-in app.)
fix table view in landscape mode
new camera API with direct access to the camera
distribution code signing automatically when uploading to the app store (instead of code signing in xcode)
ability to request more memory so users don't have to reboot their phones to get rid of background apps
A non-Mac based development envionment.