I'm curious about how people test their apps In general.
I recently uploaded an app and wanted to reach as many iOS' as possible so i targeted it to 3.0. I did test on iphone 3, iphone 4 and 3GS but found out that there was an obscure sizing of a button image on a specific ios version.
In general how do you guys test for different versions?
Do people actually keep 3, 3gs and 4. Then on each, do people test on the various versions of ios within each of the phones. Yuck, anything to make this easier?
For the simulator, each xcode download contains only latest os and hardware to target. It would be nice if it could keep older os versions on the simulator to test with, is this possible?
My apps are CPU demanding, so optimizing for each device is big issue for me, so I keep an 3G, 3GS, 4, a 2G iPod Touch, and an iPad around for testing.
It's not always necessary though. Most cases you probably don't have heavy, time critical code, that needs to be optimized, so an iPhone 3GS at a minimum, possibly an iPhone 4 (for high DPI testing, although the simular may be fine for this), and an iPad (if you support it) is all you need.
If you keep your device images in your XCode organizer, you can reflash your device with all the OS version you want to QA on. It really depends on what you need. Unfortunately you can't go back and find older OS versions and install them if you didn't get them before they became out of date (partly because of the checksum process that calls back to Apple after you update to a new version, that attempts to prevent downgrading or upgrading to versions with security holes).
Related
I am building an iOS application for public release, and I would like to develop it against iOS 4, as I see from the instapaper stats from November that iOS 5 has a 48% market share, which is good, but my app will probably not be used by these early adopters, and it's probably not going to be enough to force them to upgrade.
The issue is that I only have an iPhone 4S, which shipped with iOS 5. I have been unable to find an iOS build for my 4S, which upon reflection makes sense. Nontheless, I'd like to know if there is some sort of unofficial build for the 4S or some way to make the iPhone 4 version work on the 4S.
I'm pretty sure that the answer is going to end up being "Buy a 3GS/iPod touch for development". Would that necessarily be worth it or should I just tough it out? I'm pretty sure that most iOS 4 users will upgrade eventually. Does anyone have any stats on how long it took iOS 3 users to move on?
I would still suggest getting a used iPhone 3GS. Even if you are able to install iOS 4 on the 4S you still wouldn't be able to do any performance testing, but it's highly likely that someone who still uses iOS 4 has a 3GS.
For your first cut testing, you can use the simulator which, will allow you to test for iOS 4.3
Ideally you need to get hold of the appropriate devices to test your app. You say you're supporting iOS 4, but do you mean 4.2 or 4.3? If 4.2 then you'll be supporting iPhone 3G, if 4.3 then 3GS upwards.
There are testing services out there, just search "iPhone testing services". I've never used them myself, so can't vouch for any of them, but the cost may be less than investing in actual devices.
I kept a 3G and 3GS for testing, but to be honest, even though my main app supports 4.2, I rarely do any testing for older OS versions outside the simulator. I gave the 3GS to my father on condition he tested for me when asked… but of course I had to upgrade it to iOS 5 for him, so it's of little practical value, apart from testing at different screen res.
I would suggest you develop your app with minimum version 4.0 (or other) set. Then you test that on your iOS 5 iphone. I did it with my apps and it works great.
You can test on the lower version simulator or for final testing you could borrow a friend's device or similar.
As I understand (and believe), the compiler would stop you if you included features that are not 4.0 (or the set version) compatible.
Buying lower iOS is not worth it. If you can you should borrow it. In the simulator you can also make the iOS version to lower version to test. Also you can try ad hoc distribution to check how your app performs on other ios before releasing on appstore
I have the following problem, in near future I will need to write a few not very complicated apps for iPhone. I do not currently own an iPhone, so I plan to purchase one for testing purposes. In short would iPhone 4 allow me to test apps written for older ios versions? Any pitfalls? Or would I have to purchase older phones/ipad to do the testing properly?
Yes, to perform a proper test you will need to test on the individual devices you are supporting. The documentation will tell you if the framework/API you are using is supported in the older iOS versions. I'm afraid there will be some subjective answers here as I am sure some apps end up running just fine across many versions with simulator testing. There are many factors to consider when looking at backwards compatibility (desired speed, memory usage, APIs, ...). You stated your app is "not very complicated" so you may be fine. I would test your application on as many devices as possible.
I should also make the distinction that there is a difference between "will it compile" and "does it perform as expected". You can use xcode to test whether the application is compatible by targeting and older version. Whether or not the application runs as desired will be discovered when testing on the specific devices.
You can't run iOS 3 on iPhone 4, so you have to buy separate devices.
But if it is a new app I wouldn't consider to make it iOS3 compatible. Most users have upgraded to iOS4 already. And at the time your app is in the store there will be even less users running iOS3.
Sure, this will remove like 0,5% of potential customers. But you don't want to buy another device for 100$ if you can get only 10$ revenue from the 0,5% customers that run iOS3. And you have to test your whole app again, which will take a significant amount of time.
So you should consider to drop support for iOS3.
See this question: Will an application compiled for iOS 3 run on iOS 4?
It will allow you to test apps which are targeted at older iOS versions, yes.
Performance differences to older iDevices like the 1st gen iPhone / iPod touch are quite significant however. So if your apps require expensive calculations or the like, I'd recommend buying an older, used device too to get a feeling for performance on that generation of hardware.
Note that if you need to make sure the apps you are about to develop will run on very old devices / iOS versions, you maybe need to do without some cocoa classes and methods which require more recent versions of iOS
If you bought an iPhone 4 for testing purpose.. You're testing on the latest iPhone available in the market and that doesn't means your app is ready to target low-end device automatically.
Let's take a deep look: If you've designed an App that must look gorgeous and work right out of the box on iPhone 4, then you're probably making your app a Retina display compatible. That is like playing with 960x640 resolution screen.
Now, if you planned to target low-end device for eg, iPhone 3GS, then all you gotta do is to reduce the graphic assets size by half, as iPhone 3GS is 320x480
That's it! Your App is ready for the low-end device as well, in looks and feels perspective.
Now, let's look from deployment perspective. If you base SDK is 4.2.1 and you're targeting to devices which runs 3.3, then all you gotta do is to make sure you don't use latest libraries/classes which are introduced in iOS 4.2.1, as they won't be available to device which are running on iOS 3.3. If you take care of this part, your app is almost ready to ship out.
From Market perspective. I see all the great apps out there had made their deployment target as iOS 4.0 or later. That means, they are targeting to only devices which runs iOS 4.0 or later versions.
I think, you can try your build in low-end iPhone simulator after successful testing cycles ran on iPhone 4 device. :)
The iPhone is generally not backwards compatible. Sometimes methods, properties, classes etc. are added or older ones become deprecated.
For testing, i would recommend that you have several devices with different iOS Versions depending on the target iOS Version you want to release.
Also check the the iOS Reference Library: Under "Availability" for each function etc. is noted since when it is Available
I'm needing to purchase whatever devices are needed to do testing for iOS applications. My initial thoughts are to test on a first-generation iPod Touch and an iPhone 4. Testing on the old iPod Touch with 3.1 software will give a good indication on how the app will work with little memory, and the iPhone 4 will obviously be a test for the faster system.
What do you all typically test on before submitting an application?
At minimum, a device running the oldest OS that I list as my Deployment Target, the slowest device/OS combo that I have (e.g. a 3G running 4.0.x is likely slower than anything running a 3.x OS) and which the app supports, and the current/latest released OS.
A device which includes all the features that my app will support (e.g. mic, camera, gyro, GPS, etc., if needed.)
Optionally, an iPad if the app is Universal, or if not to see how bad 2X zoom looks.
Optionally, a device with a beta OS installed if it appears Apple is going to release it well before I plan on revising the app.
So far all the apps I've submitted have been compatible with iOS 3.1.x, and I test them on my old 1st gen iPod Touch. I also test on my 3.2.2 iPad and my 4.0.2 iPhone 3GS.
My guess is all of these answers are going to be along the same lines -- test with as wide of a variety of devices as you can that are compatible with your app (i.e. meet the Minimum Deployment Version).
This is especially important if you develop Universal Apps that support 3.1.x. You'll find a ton of bugs with the hardware that you will never get with the simulator.
right now, I test using a 3g running 3.1 (a popular build), a 3gs running 4.0.2 (latest), a touch running 3.1.3(basically 3.1), and an iPhone 4 with 4.0.2.
Mainly try to keep a 3.x device around for sure, and then obviously a device with the most up to date OS.
Based on AdMob reports...around 98% of users are upgraded to atleast 3.0 and those that aren't shouldn't be your client base anyway (unlikely to use new apps).
If I am creating an iPhone app, is it worth trying to make sacrifices with functionality (e.g. with MPMoviePlayer) in the application and support previous versions of the OS?
Are the majority of iPhone users running iOS 4?
Will I lose out on a big portion of the market if I only support iOS 4?
There will still be a lot of iOS 3.x users because iOS 4 cannot be installed on original iPhones and older iPod touches. And iOS 4 barely works on iPhone 3G. You can still use functionality of iOS 4 while still targeting older versions though.
I've been learning through doing user testing that a lot of people don't upgrade their apps, much less their OS. Add that to the fact that lots of devices can't run 4.0, and I think you'll limit your market if you don't support at least 3.1.x.
As to 2.x, this report says it was less than 13% 5 months ago, so it's probably safe not to support for a new app at this point.
First, you're forgetting something: You need to support 3.2 if you want your app to run on the iPad.
You don't have to "sacrifice functionality"; you just have to code your app to detect what features are available at runtime and fallback gracefully.
If your app isn't free, then the real question is "Will users who don't upgrade buy my app?", and I suspect the answer is generally no. Apparently a substantial portion of the 2.x installed base are iPod Touch users who don't want to pay for the upgrade.
The other factor is the amount of work. If you use few enough iOS 4 features that it's not much work to also make it run on 3.1.3, then supporting 3.1.3 is a no-brainer. Of course, never support an OS version you can't test.
If this is a brand new app, there's not much point in supporting older versions of the OS. It will require making sacrifices when programming the app, and over time everybody will eventually move to iOS 4.
I'm developing an app that uses iOS4 features (like MFMessageComposeViewController), but I want to support iOS 3.1.x as well. I know how to do that in code and build settings, but it's getting hard to find devices to test on--I've upgraded my phone and so have most of my friends. And the simulator with XCode 3.2 doesn't support simulating 3.1 as far as I can tell.
When my apps are raking in money hand over fist, of course I'll invest in multiple devices to test different OS and hardware, but I'm wondering what people do when they're starting out and resources are limited. For example, to test an SMS-capable device with 3.1.x, I'd need not only a spare device, but I'd have to pay for a phone service plan as well.
A couple of things I've considered in the meantime:
Install an older version of XCode and use that simulator for 3.1 testing
Buy a used iPod touch and install OS 3.1 (but couldn't test SMS/GPS)
Other ideas? I'd love to hear what other people are doing.
What I do is keep one of each model of iPhone and iPod touch and install the latest version of the OS that runs on it. So I have iPhone 2G and iPod touch 1G with OS 3.1.3; iPhone 3GS, iPhone 4, and iPod touch 2G with OS 4.0; and a WiFi iPad with OS 3.2. Only one of the phones has cell phone service.
As for my builds, I build with SDK 4.0 and set OS 3.1.3 as the earliest target. If you claim to support a certain OS, you must have tested on the device with that OS.
We have a couple of devices at our startup company, and also have very old iPods. Perhaps you can get some for very little second hand?
OffTopic: I think supporting versions old of an OS when a free update is available shouldn't be done for too long. People need a bit of "help" updating. Must admit that iOS 4 might be an exception though, my iPod doesn't live for long with it installed. :(