Hi thank you for the help in advance,
I have looked at some of the posts and I am a bit confused about the multi threading. It seems that it may be pretty easy, however I am very new to programming so I am still trying to grasp this.
These are two calls to pull data from a database, and they take forever as it is... So I'm thinking about multithreading these until I can learn how to build a core data for this. Right now i am using sqllite and the database involves 10,000 + recipes... So not lightning fast like I would like...
Please let me know what you think, and how I can make these happen maybe simultaneously? (If thats even possible)
Thank you in advance.
requestCount++;
[[DataPuller sharedDataPuller] getAllDeletedRecipeList];
[DataPuller sharedDataPuller].target = self;
requestCount++;
[[DataPuller sharedDataPuller] getAllRecipesList];
[DataPuller sharedDataPuller].target = self;
If you're doing SQLite, you might want to contemplate using FMDB which (a) gets you out of the weeds of sqlite3 calls; and (b) offers a FMDatabaseQueue which allows you to coordinate queries from multiple queues, so that the data operations don't stumble across each other.
Having said that, you suggest that you're having significant performance issues which you're hoping to solve with a shift to Core Data or going multi-threaded with SQLite. That's unlikely. Poor performance of local database operations is generally more of a matter of your application design (e.g. it's unlikely to be wise to try to retrieve the entire details for all 10,000 recipes ... you probably want to retrieve just the unique identifiers, perhaps only those required for the given screen, and then only retrieve the particulars for a given recipe at a later point as you need that). For local database interaction, you rarely have to contemplate a multithreaded implementation, but rather just design the system to retrieve the least possible information at any given point that you need for the presentation. I personally find that my database-driven apps generally only need to go multithreaded when I'm doing extensive interaction with some remote web service (in which case, it's the retrieval of server data and the parsing of that which goes on the separate thread, not necessarily the database operations themselves).
If you're having performance issues with your recipe app, I'd suggest you submit a far more detailed question, with code samples, that articulates your particular performance problem. And I wouldn't be surprised if multi-threading was not part of the solution. Likely, appropriate use of indexes and a more judicious retrieval of information at any given point might be more critical.
Get records from database in the form of pages; i.e. 20 or 50 recipes per page. Have a look on YouTube app. on iPhone or have a look on my app. HCCHelper
Related
Now this might look like a duplicate thread, but my question is that I have read a lot of questions like.. Core Data vs SQLite 3 and others but these are 2-3 years old. I have also read that FMDB was developed as core data was not supported on iOS, So it should not be used any more. And on the other hand I have read that one should not use core data as a database.
So I am seriously confused,whether I should use core data for object storage or not. I mean on what basis I should decide which to use? Are there any guidelines provided by apple or someone else.. or is it something that will come to me with time.?
Ankit,
Here's the tl;dr skinny: use Core Data.
Here's the long form:
While you could use many criteria to choose between Core Data, an ORM (FMDB) or direct sqlite calls, the real cost of this choice comes from your time to use it, Apple's support and leverage from other projects. (RESTKit, which maps REST services on to Core Data, is popular these days.)
Hence, a large percentage of the time, say 90+% (a made up stat), the answer on iOS will be to use Core Data. Why? Once you get the hang of it and build out a few little helper methods, Core Data keeps you in a consistent computing world -- the Objective-C object graph. Core Data will teach you things about how to use a dynamic language that will help every other aspect of your iOS programming. Hence, you are more productive. Don't fight the framework.
If you are bringing over a large, complex SQLite database & schema from another app, it then might be cost effective to use either FMDB or SQLite. But I doubt it. Your time writing a simple Mac-based command line app to migrate the DB to a Core Data DB is a finite and simple task. You are almost guaranteed to have to rewrite most of the business logic in Objective-C. (Yes, C++ and Objective-C++ are both good technologies. Has your database business logic really been tuned to work on a memory limited device? I didn't think so.)
Core Data gets a bum rap on performance. It is really quite fast. You just have to use it differently than you use a DB. In particular, you almost always over-fetch data from the store and then refine it using predicates directly on the various sets and arrays. On iOS devices, where the flash is surprisingly slow, this over-fetch strategy is particularly effective. You actually have a lot of RAM on these devices, use it to gain performance. (Yes, I know this is an apparent contradiction to my above knock on portable business logic. But really, code ported from a desktop or server environment has so many implicit assumptions about the speed of the disk, the amount of memory and the reality of a VM with a backing store, it just will not work well on a battery powered, memory limited device with a funky memory model. [It won't work very well on Android devices either.]) You will also denormalize your data to simplify displaying it in various iOS and Mac OS X UI widgets. There are a few applications where Core Data will be slower than an equivalent SQLite DB. Those have been detailed elsewhere. The one major claim is that tasks where IDs are defined by upstream databases hits Core Data's performance is true. But it can be somewhat mitigated by judicious indexing and over-fetching.
The thing to remember about mobile devices too is that the database size, because these are mobile devices on the leaves of the internet, is generally of modest size. Hence, performance is easier to attain. Many lessons from the world of servers may not apply to this mobile, battery powered world.
In other words, you've had to go "all in" to use Objective-C on iOS/Mac OS X, you will gain some important productivity benefits from using Core Data too.
Andrew
I recently embarked on this journey myself, and ended up trying out all three. Here's what I learned:
Raw sqlite3
Low-level, full access to database. No abstractions. Very verbose - it takes a good deal of code to do very simple things.
Core Data
Very high-level, built on abstractions, MUST use Apple's generated database. Useful for iCloud synchronization and simple iOS-only data management. Difficult and dangerous to directly access database, and should not be used for cross-platform databases. Still takes a fair amount of code to do simple things.
FMDB
High-level, very abstraction friendly but not forced. Still get full access to database if you need it. Provides an NSDictionary of the result with each item automatically typecasted to the mutable variant of the proper datatype (e.g., text columns are returned as NSMutableString). I ended up building a very simple wrapper class around it to abstract it even more, so I have a helper class with static functions like selectAllFrom:(NSString *)table where:(NSDictionary *)conditions, which returns an NSArray of NSDictionary objects. It's fantastic to be able to do things like NSArray *usersNamedJoe = [DBHelper selectAllFrom:#"user" where:#{#"name": #"Joe"}];.
Basically, while Core Data may be useful for simple iOS-only apps, anyone who's interested in using cross-platform databases should stay far, far away from it - Apple has no interest in making that easy, and it shows.
TL;DR:
Don't use raw sqlite3 unless you're doing something extremely trivial.
Core Data is fine for simple iOS-only data, if you're comfortable with being locked into it.
If you want full control over the database and you're not doing something trivial, or you're building your app for multiple platforms, FMDB is definitely the way to go.
I use FMDB for all my projects that have heavy usage of "INSERTs" and FMDB is not out of date. The last commit on Github was at last November. If you go with SQL I recommend you to use FMDB.
Core Data fits to 95% of all projects, but if it comes to optimization to run to a wall. If you want the benefits of Core Data (OOP, ...) then use it. If you have a lot of insert an deletes with "WHERE" user Sqlite (FMDB)
This POST explain the off and top site for Core Date vs. Sqlite (FMDB)
CoreData is not just an abstraction of an SQL database. CoreData is also does object graph management. CoreData can do things that FMDB simply can't do.
As always: It really depends on your use case. But in 99% of cases CoreData is the right choice.
If performance is critical, you still have to understand how a database works. But CoreData can deliver that performance if you use it the right way. But it takes some time to learn. There are many things that are trivial to do in CoreData that would be very complex to do in FMDB.
As a new SQL guy, I'm going to throw in my two cents:
In Core Data, you have a bit of "boilerplate" code that you need to put in before you can actually use your database. Your app needs at least one of these:
A persistent store coordinator
A managed object context
A managed object. This correlates to an entity, which correlates to a table if you use an SQLite database.
To take full advantage of the framework, you need to understand what role these objects play in the management of your data.
On the other hand, we have SQLite, which - in my opinion - is much easier to understand. To start, you'll need:
A database
A table or more (depending on your data)
Knowledge of SQL - a flexible language with a simplistic syntax (SELECT queries do more than what you might originally think they do)
An object through which your app communicates with SQLite.
Core Data is only an object abstractization of the SQLite3 database. That means you'll have persistant objects easy to manage for standard database operations. You can also work in an transactional mode and design your core data database structure in XCode by creating models.
If you don't whant to create manually your SQLite3 database or persistant methods use Core Data.
I know little about Core Data. Read: I've read a few tutorials, kinda understood how it works, but never tried to use it in any of my apps. That said, I'd like to know if it's worth the effort to use it in the app I'm developing. Note that I'm not asking if I should learn Core Data, but if it's worthwhile to invest time learning it for this specific app I'm making, or if I should use archiving instead, ship the app, and only then learn Core Data in my spare time.
Basically my app reads a list of items from a web service, and needs to save the last N items. The user should also be able to bookmark items, so that's another thing I should store somewhere. So, right now I'm just archiving a subarray with range 0-N of the latest items. Does it work? Yes. Is it efficient, and the best way to achieve this? That's my question for you actually.
My doubt comes from the fact that whenever I see someone asking 'Is Core Data overkill for my project?' everyone suggests to use it anyway.
If the amount of data you need to persist can easily fit into memory without degrading the apps performance then you should just archive the array and ship the app.
Core Data gives a lot of advantages when handling large and complex data sets. It gives a lot of advantages in maintaining and upgrading a shipped app. However, those advantages should not get in the way of shipping an otherwise completed app. Later, you can always write code to migrate a shipped version to a Core Data version. It's more work, but hey, at least you've got a shipped app to work on.
I've seen a lot of small startups/developers come and go and the major factor that separates the successful from non-succesful is that the successful actually ship/release product. You can spend forever polishing an app but the key thing is knowing when to say, "It's good enough" and get the thing into the user's hands.
If it's a small list and you have the data in NSDictionary or NSArray collections, use writeToFile:atomically: instead for flat file XML Plist storage.
I still think you should learn Core Data eventually, but.
I have read a lot of the MongoDB.
I like all the features it provides, but I wonder if it's possible to have it as the only database for my application, including storing sensitive information.
I know that it compromises the durability part in ACID but I will as a solution have 1 master and 2 slaves in different locations.
If I do that, is it possible to use it as the primary database, storing everything?
UPDATE:
Lets put it this way.
I really need a document storage rather than traditional dbms for be able to create my flexible application. But is MongoDB reliable enough to store customer sensitive information if I have multiple database replications and master-slave? Cause as far as I know one major downside is that it compromises the D in ACID. So I solve it with multiple databases.
Now there is not major problems such as lost of data issues?
And someone told me that with MongoDB a customer could be billed twice. Could someone enlighten this?
Yes, MongoDB can work as an application's only data store. Use replication and make sure you know about safe writes and w.
And someone told me that with MongoDB a customer could be billed
twice. Could someone enlighten this?
I'm guessing whoever told you that was talking about eventual consistency. Some NoSQL databases are eventually consistent, meaning that you could have conflicting writes. MongoDB is not, though, it is strongly consistent (just like a relational database).
Your application being flexible or not has absoutely nothing to do with wether you use "nosql", a "document db" or a proper RDBMS. Nobody using your application will care either way.
If you need flexibility while coding, you should research into frameworks, like ActiveRecord for Ruby, which can make DB-interfacing much more simple, generic and powerful. At that level, you can gain alot more than just changing the DB, and you can even become DB-agnostic, meaning you can change DB without changing any code. Indeed, I have found ActiveRecord to boost my productivity many many fold by alleviating me from tedious and error-prone "code intermixed with SQL".
Indeed, if you feel you need a schemaless database, for critical data, you are doing something wrong. You are putting your own convenience over critical needs of the projects, or in ignorance thinking you won't get into problems later. Sooner or later, lack of consistency will bite your ass, hard!
I feel you are hesistant towards RDBMS because you are not really that comfortable with all the jargons, syntax and sound CS principles.
Believe me, if you're going to create an application of any value, you are hundred times better learning SQL, ACID and good database-principles in the first place. Just read up on the basics, and build your knowledge from wherever you are now. It's the same for each and every one of us, it takes time, but you learn to do things right from the start.
Low-level tools like MongoDB and equivalent just provide you with infinitely more ammunition to shoot yourself in the foot with. They make it seem easy. In reality however, they leave the hard work for you, the coder, to deal with, while an RDBMS will handle more of the cruft for you once you grok the basics.
Why use computers at all, if you want more work, you can just go back to paper. Design will be a breeze, and implementation can be super-quick. Done. Except it won't be right of course.
In the real world, we can't afford to ignore consistency, database design and many more sound CS principles. Which is why it's a great idea to study them in the first place, and keep learning more and more.
Don't buy into the hype. You ask question about MongoDB here, but include that you really need its features. With 25 years of computer experience, I simply don't buy it. If you think creatively, an RDBMS can be made to do whatever you want it to, or a framework can be utilized to save you from errors and wasted time.
Crafting ACID properties onto MongoDB seems like more work to me, and by experience, sounds like an excercise in futility, rather than using what is already designed to suit such purposes.
Is it possible? Sure. Why not? It's possible to store everything as XML in text files if you wanted to.
Is it the best idea? It depends on what you're trying to do, the rest of your system architecture, the scale your site is intended to achieve, and so on.
I have some doubts about SQLite...
My app uses only one table (encapsulated in a data logic class) to store small data, the connection is opened in didFinishLaunchingWithOptions and finalized in applicationWillTerminate (yes, I know that in iOS4 applicationWillTerminate is only called by iSO if necessary, but I write on database at every change of data).
Now I have to add a second table (and relative class) that will store different data (no join between the two tables) and of normal size (about 1-2 rows for day).
I thought of applying the singleton pattern to share the connection and the statements, but I read that for performance reason it's recommended to use class-local variables to hold the connection.
So what's the best practice? Singleton or two open connection?
What solution would you reccommend for my situation?
Thanks
Singleton for me when stored in the same database.
Small amounts of data shouldn't be a performance bottleneck anytime soon.
Or, of course, go with CoreData. :-)
For such a simple use, a singleton is probably the right answer for the very reasons that #Eiko mentions.
However, it all begs the question: Why aren't you using Core Data?
Using SQLite correctly is actually quite hard and I've seen dozens of very talented/experienced engineers get it entirely wrong without realizing it. Worse, scaling an existing implementation is even harder. Adding concurrency is really really hard with straight SQLite (and the Core Data engineers have expended a huge amount of energy and applied a ton of expertise to support concurrency on top of SQLite correctly).
For such a simple use, Core Data won't be hard to learn and it will leave you with a much more solid and versatile code base.
iPhone 3.0 support the Core Data, it seems a great tool for developer. But SQL statement seems it is easier to get start, but core data is easier for maintaining db. But I'm considering the SQL statement, because it seems have better performance. I am consider which one is better for iPhone development, any suggestion?
The answer is actually much simpler than either of those blog posts make it appear. The rule is:
If you are developing for any OS X platform;
If you are not accessing a proprietary format; and
If your persistence file does not need to be read on a non OS-X platform
Then you should go with Core Data. It is that simple. Core Data gives you so many features and the ease of use compared to straight SQL makes the choice simple. As far as performance, that is a red herring. Where it counts, Core Data can and does easily out perform custom code accessing a SQLite database. However, performance on Cocoa Touch is actually a secondary concern.
The primary concern is memory. You have a tiny amount of memory in which to work with on Cocoa Touch and your data model can easily blow that out. Core Data solves that issue. It watches how much memory it is using and will drop objects out of memory automatically when it receives a memory warning. All of that fairly complex code you would have to write yourself if you used SQLite directly.
Less time coding your data model means you have more time making your application great.
This provides a reasonable list of the pros and cons of either approach: http://maniacdev.com/2009/09/iphone-sqlite-vs-core-data-–-which-to-choose/
The recommendation is to use Core Data as it makes things so much easier in the long term.
There is an in-depth explanation of the differences between the 2 here: http://cocoawithlove.com/2010/02/differences-between-core-data-and.html
That's a great read. The 2 approaches are different and both have their pros and cons.