I've been developing for the iPhone for quite some time and I've been wondering if there's any array object that uses circular buffer in Obj-C? Like Java's Stack or List or Queue.
I've been tinkering with the NSMutableArray, testing it's limits... and it seems that after 50k simple objects inside the array - the application is significantly slowed down.
So, is there any better solution other than the NSMutableArray (which becomes very slow with huge amounts of data). If not, can anyone tell me about a way to create such an object (would that involve using chain (node) objects??).
Bottom line: Populating a UITableView from an SQLite DB directly would be smart? As it won't require memory from an array or anything, but just the queries. And SQLite is fast and not memory grinding.
Thank you very much for you time and attention,
~ Natanavra.
From what I've been thinking it seems that going for Quinn's class is the best option possibly.
I have another question - would it be faster or smarter to load everything straight from the SQLite DB instead of creating an object and pushing it into an array?
Thank you in advance,
~ Natanavra.
Apologies for tooting my own horn, but I implemented a C-based circular buffer in CHDataStructures. (Specifically, check out CHCircularBufferQueue and CHCircularBufferStack.) The project is open source and has benchmarks which demonstrate that a true circular buffer is quite fast when compared to NSMutableArray in the general case, but results will depend on your data and usage, as well as the fact that you're operating on a memory-constrained device (e.g. iPhone). Hope that helps!
If you're seeing performance issues, measure where your app is spending its time, don't just guess. Apple provides an excellent set of performance measurement tools.
It's trivial to have NSMutable array act like a stack, list, queue etc using the various insertObject:atIndex: and removeObjectAtIndex: methods. You can write your own subclasses if you want to hardwire the behavior.
I doubt the performance problems you are seeing are being caused by NSMutableArray especially if your point of reference is the much, much slower Java. The problem is most likely the iPhone itself. As noted previously, 50,000 objective-c objects is not a trivial amount of data in this context and the iPhone hardware may struggle to managed that much data.
If you need some kind of high performance array for bytes, you could use one of the core foundation arrays or roll your own in plain C and then wrap them in a custom class.
It sounds to me like you need to switch to core data so you don't have to keep all this in memory. Core data will efficiently fetch what you want only when you need it.
You can use STL classes in "Objective-C++" - which is a fancy name for Objective-C making use of C++ classes. Just name those source files that use C++ code with a ".mm" extension and you'll get the mixed runtime.
Objective-C objects are not really "simple," so 50,000 of them is going to be pretty demanding. Write your own in straight C or C++ if you want to avoid the bottlenecks and resource demands of the Objective-C runtime.
A rather lengthy and non-theoretical discussion of the overhead associated with convenience:
http://www.cocoabuilder.com/archive/cocoa/35145-nsarray-overhead-question.html#35128
And some simple math for simple people:
All it takes to make an object as opposed to a struct is a single pointer at the beginning.
Let's say that's true, and let's say we're running on a 32-bit system with 4 byte pointers.
4 bytes x 50,000 objects = 200000 bytes
That's nearly 200MB worth of extra memory that your data suddenly needs just because you used Objective-C. Now compound that with the fact that whatever NSArray you add those objects to is going to double that by keeping its own set of pointers to those objects and you've just chewed up 400MB of RAM just so you could use a couple of convenience wrappers.
Refresh my memory here... Are swap files on hard drives as fast as RAM? How much RAM is there in an iPhone? How many function calls and stack frames does it take to send an object a message? Why isn't IOKit written in Objective-C? How many of Apple's flagship applications that do a lot of DSP use AppKit? Anybody got a copy of otool they can check with? I'm seeing zero here.
Related
So, I'm making an iPhone app in Objective-C and need to use an array. Usually in Objective-C I'd use NSArray or NSMutableArray, but I'm starting to wonder if that's the best idea when simply using a basic array. Memory- and efficiency-wise, does it make more sense to use a regular C array or an Objective-C array when simply keeping a basica array of custom objects? And do you usually use C arrays or Objective-C arrays when programming in Objective-C? Thanks!
This really just depends on what you want to do with the array.
For arrays of numbers that you're going to do lots of math on, C arrays are probably a better choice (or at least a class that wraps C arrays). If you're constantly converting back and forth between scalars and objects and aren't using any of NSArray's more advanced features, all it will buy you is a lot of overhead.
However, Cocoa objects have a number of memory management invariants that you must obey or your app will go boom. C arrays and C++ vectors will blissfully ignore these requirements, meaning a lot more work for you to keep them in line. NSArray will take care of these for you. When you're dealing with Cocoa objects, use an NSArray.
Unless I've found a good reason to not use them, NSArray is my de-facto choice.
And in several years of Cocoa programming, I've never really found a good reason to not use it. The only things I've been able to come up with are when then things I want to store in an array are not objects, or I want to store objects but not have them be retained. In both of those cases, however, a simple CFMutableArrayRef with custom value callbacks does the trick.
When in rome.
Use NSArray or NSMutableArray when developing iPhone applications, as that is the idiomatic thing to do.
Unless you have a very specific reason to use plain old c arrays, stick to the idiomatic approach.
NSArray and NSMutableArray are quite efficient. If performance isn't a problem for you yet, there's no reason to contort yourself into using plain C arrays when Objective-C objects would be more natural – if performance becomes a problem later, then you can optimize it more.
Objective C arrays (NSArray, et.al.) are grossly inefficient if you are trying to use them for processing samples of real-time multi-channel audio and/or raw pixels of real-time video, bits that you need to touch on the order of many millions of times per second. Plain C arrays of data laid out in some data cache line aware manner are much more appropriate, and likely to use far less memory and battery power for any significant processing.
Note that you can't even feed NSArrays to the Accelerate framework, and most of the Core Audio, Core Graphics or Open GL APIs.
For data you seldom touch, say at a rate of only many thousands of time per second or less, Objective C arrays will help with memory management and data abstraction, probably without their time and memory penalty being easily measurable. Useful for user data, UI elements, UI tables, and other really slow stuff that happens at merely human tempos.
Although I'm in basic agreement with the preferred answer, I think it lacks the underlying criteria, which is this: When programming for iPhone, the code that you write represents merely few percent (in CPU cycles Memory, etc) of the total that is done by your app. The rest - is done by Cocoa-Touch frameworks and the OS. Usually your data structures (including arrays) will only serve intermediate needs, on your way to Cocoa API calls. These, however, only accept NSArrays, NSDictionaries etc. So if you choose to use C arrays, prepare for reconstructing NSArrays from them very soon (e.g. to display data in the UI, to query a database, or to send a network request) and this bridging back-and-forth costs a lot.
In addition, Foundation collection classes (such as NSArray) are highly optimized, and adapt to different scenarios automatically (NSArray for instance can use C arrays in its implementation, and switch them to Tree structures if the Array is big, or even as a linked-list if only few indexes are populated. You will find them vastly superior to C-arrays in the long run, as you don't need to re-architect your app every time your needs change. Apple is doing this for you.
For simplistic needs and relatively small arrays - the difference in performance (if any) is negligible.
This isn't saying C arrays have no place in iPhone development - only that their (justifiable) use is rare.
I have in my app a considerable amount of data that it needs to access, but will never be changed by the app. Currently I'm using this data in other applications in JSON files and SQL databases, but neither seems very straightforward to use in iOS.
I don't want to use CoreData, which provides tons of unnecessary functionality and complexity.
Would it be a good idea store the data in PropertyList file and build an accessor class? Are there any simple ways to incorporate SQLite without going the CoreData route?
You can only use plist if the amount of data is relatively small. Plist are entirely loaded into memory so you can only really use them if you can sustain all the objects created by the plist in memory at once for as long as you need them.
Core Data has a learning curve but in use it is usually less complex than SQL. In most cases the "simpler" SQL leads to more coding because you end up having to duplicate much of the functionality of Core Data to shoehorn the procedural SQL into the object-oriented API. You have to manually manage the memory use of all the data by tracking retention. You've write a lot of SQL code every time you want data. I've updated several apps from SQL to Core Data and in all cases the Core Data implementation was smaller and cleaner than the SQL.
Neither is the memory or processor "overhead" any larger. Core Data is highly optimized. In most cases, off the shelf Core Data is more efficient than hand tuned SQL. One minor sub optimization in SQL usually destroys any theoretical advantage it might have.
Of course, if you're already highly skilled at managing SQL in C then you personally might get the app to market more quickly by using SQL. However, if you're wondering what you should plan to use in general on on Apple Platforms, Core Data is almost always the answer and you should take the time to learn it.
You can just use SQLite directly without the overhead of Core Data using the SQLite C API.
Here is a tutorial I found on your use-case - simply loading some data from an SQLite database. Hope this helps.
Depending on the type of your data, the size and how often it changes, you may desire to just keep things simple and use a property list. Otherwise, using SQLite (documented in Jergason's answer) would be where I'd go. Though let me say that if you have a relatively small (less than a couple hundred) set of basic types (arrays, dictionaries, numbers, strings) that don't change frequently, then a property list will be a better choice in my opinion.
As an example to that, in one of my games, I create the levels from a single property list per difficulty. Since there are only a handful of levels per difficulty (99) and a small set of parameters for each (number of elements in play, their initial positions, mass, etc) then it makes sense, and I avoid having to deal with SQLite directly or worse yet, setting up and maintaining CoreData.
What do you mean by "best"? What kind of data?
If it's a bunch of objects, then JSON or (binary) plist aren't terrible formats, since you'll want the whole thing loaded in memory to walk the object graph. Compare space efficiency and loading performance to pick which one to use.
If it's a bunch of binary blobs, then store the blobs in a big file, memory-map the file (NSDataReadingMapped a.k.a. NSMappedRead), and use indexes into the blobs. iOS frameworks use a mixture of these (e.g. there are a lot of .pngs, but also "other.artwork" which just contains raw image data).
You can also use NSKeyedArchiver and friends if your classes implement the NSCoding protocol, but there's some object graph management overhead and the plist format it produces isn't exactly nice to work with.
I'm developing a game/engine for iPhone OS. It's the first time I'm using Objective-C.
I made my own binary format for geometry data and for textures I'm focusing on PVRTC. That should be the optimal approach as far as speed and space are concerned. I really want to keep loading time to a minimum and - if possible - be able to save very fast as well.
So now I'm trying to make my "Entity" stuff persistent without sacrificing performance.
First I wanted to use NSKeyedArchiver. From what I've heard, it's not very fast. Also, what I want to serialize is mostly structs made of floats with some ints and strings, so there isn't really a need for all that "object graph" overhead.
NSArchiver would have been more appropriate, but they kicked that off the iphone for some reason.
So now I'm thinking about making my own serialization scheme again. Am I wrong in thinking that NSKeyedArchiver is slow (I only read that, haven't tested it myself)? If so, what's the best way to encode/decode structs (with no pointers, mostly floats) without sacrificing speed?
Don't bother optimizing something that you haven't tested... it might be "good enough". One hint; the binary format is about 2x as fast and nearly 2x as memory efficient as the XML format.
Beyond that... well... you get into all kinds of case specific issues. Do you need all the data at once? Need it to be streamed? Arbitrary subsets? How connected is it? Will it be reconstituted into a single or many ObjC objects? Is the data pre-canned? etc.etc.etc...
I've had good results using yajl-objc for both serialization and communicating with REST-style web services; it is efficient enough for my needs and simple to use.
That being said, unless you have a ton of data NSKeyedArchiver will likely do just fine.
Ok guys I am developing an iPhone app I have a Model class which follows a Singleton design pattern.
Now I have an NSArray in it which is initialized to around some 1000 NSStrings in the init method.
Now I need to use this data in some view controller. so I import Model.h, I create an array of NSString objects in view controller & set the data to it. But now the problem is that now I have 2000 NSStrings currently allocated, which I believe is not a good thing on iPhone due to memory considerations.
releasing model object wont help because I've overrided release method to release nothing according to the pattern & I cannot change the design now because now a lot of code works on the assumption of model being a singleton.
& in future maybe the initial NSStrings may grow to 2000 or even more & then I'll have 4000 NSStrings allocated at one time ....
I am a little confused on how to go about it any suggestions
A few thousand strings take barely any memory at all. 4000 strings would take a couple of hundred kB, depending on length. (Rule of thumb here is string length + 20).
Edit: Probably more like string length + 30 or 40, actually; I'm not certain how much overhead NSArray adds.
Reedit: Given the information from the below question; you could probably get away with loading a few hundred strings at the most; just around the area you are browsing; basically turning your SQLite access into a sparse array that caches a few strings around the search area. Not, of course, that I believe it to be necessary; if the strings are location names they probably have an average byte size of 20-30 bytes; giving a (very) rough estimate of 300k of memory to keep them all in memory permanently, greatly reducing access time and giving a better user experience. The iPhone doesn't have a lot of RAM; but you can afford, at the very least a fair few megabytes; 300k isn't going to break your back.
It's difficult to offer specific suggestions without knowing more about your implementation--where do your strings come from? In general, the best performance optimization for this sort of situation is lazy loading. Here are some examples of ways to have a reduced memory footprint with different technologies if you have a table view:
Core Data: Not usually a problem, since objects are faulted and fetched automatically.
SQLite: Again, not usually a problem--you query the database every time you need a particular value (such as when a table view cell needs to display a string).
Internet: Start a request (usually via a thread) when the table view cell is visible.
XML: Trickier, but use SAX (event)-based parsing to find values instead of DOM parsing (which loads the entire document to memory).
That being said, if you made design decisions that are difficult to reverse, it may not be possible to significantly reduce your memory footprint without major refactoring.
EDIT: As per other answers, it's probably not worrying about, but if you were to optimize for memory, you would not load all the SQLite values at application launch, but instead fetch each value from SQLite in cellForRowAtIndexPath. These sorts of problems are made much easier using Core Data--I would highly recommend using Core Data instead of straight SQLite (although it sounds as if you might be too far into development to switch at this point).
If your Model object really is a singleton, then all you need to do is get your strings from the model object in the view controller and use them. I don't see why you would get duplicates.
With NSString, as long as the strings are immutable, the copy method should just retain the object to copy and return the same object to you. Also, if your strings are constant strings i.e. defined like so:
NSString* foo = #"bar";
they are actually part of the executable and will take up no extra RAM at run time.
I have an object graph in Objective-C on the iPhone platform that I wish to persist to flash when closing the app. The graph has about 100k-200k objects and contains many loops (by design). I need to be able to read/write this graph as quickly as possible.
So far I have tried using NSCoder. This not only struggles with the loops but also takes an age and a significant amount of memory to persist the graph - possibly because an XML document is used under the covers. I have also used an SQLite database but stepping through that many rows also takes a significant amount of time.
I have considered using Core-Data but fear I will suffer the same issues as SQLite or NSCoder as I believe the backing stores to core-data will work in the same way.
So is there any other way I can handle the persistence of this object graph in a lightweight way - ideally I'd like something like Java's serialization? I've been thinking of trying Tokyo Cabinet or writing the memory occupied by bunch of C structs out to disk - but that's going to be a lot of rewrite work.
I would reccomend re-writing as c structs. I know it will be a pain, but not only will it be quick to write to disk but should perform much better.
Before anyone gets upset, I am not saying people should always use structs, but there are some situations where this is actually better for performance. Especially if you pre-allocate your memory in say 20k contiguous blocks at a time (with pointers into the block), rather than creating/allocating lots of little chunks within a repeated loop.
ie if your loop continually allocates objects, that is going to slow it down. If you have preallocated 1000 structs and just have an array of pointers (or a single pointer) then this is a large magnitude faster.
(I have had situations where even my desktop mac was too slow and did not have enough memory to cope with those millions of objects being created in a row)
Rather than rolling your own, I'd highly recommend taking another look at Core Data. Core Data was designed from the ground up for persisting object graphs. An NSCoder-based archive, like the one you describe, requires you to have the entire object graph in memory and all writes are atomic. Core Data brings objects in and out of memory as needed, and can only write the part of your graph that has changed to disk (via SQLite).
If you read the Core Data Programming Guide or their tutorial guide, you can see that they've put a lot of thought into performance optimizations. If you follow Apple's recommendations (which can seem counterintuitive, like their suggestion to denormalize your data structures at some points), you can squeeze a lot more performance out of your data model than you'd expect. I've seen benchmarks where Core Data handily beat hand-tuned SQLite for data access within databases of the size you're looking at.
On the iPhone, you also have some memory advantages when using controlling the batch size of fetches and a very nice helper class in NSFetchedResultsController.
It shouldn't take that long to build up a proof-of-principle Core Data implementation of your graph to compare it to your existing data storage methods.