Consider an iPhone application that is a catalogue of animals. The application should allow the user to add custom information for each animal -- let's say a rating (on a scale of 1 to 5), as well as some notes they can enter in about the animal. However, the user won't be able to modify the animal data itself. Assume that when the application gets updated, it should be easy for the (static) catalogue part to change, but we'd like the (dynamic) custom user information part to be retained between updates, so the user doesn't lose any of their custom information.
We'd probably want to use Core Data to build this app. Let's also say that we have a previous process already in place to read in animal data to pre-populate the backing (SQLite) store that Core Data uses. We can embed this database file into the application bundle itself, since it doesn't get modified. When a user downloads an update to the application, the new version will include the latest (static) animal catalogue database, so we don't ever have to worry about it being out of date.
But, now the tricky part: how do we store the (dynamic) user custom data in a sound manner?
My first thought is that the (dynamic) database should be stored in the Documents directory for the app, so application updates don't clobber the existing data. Am I correct?
My second thought is that since the (dynamic) user custom data database is not in the same store as the (static) animal catalogue, we can't naively make a relationship between the Rating and the Notes entities (in one database) and the Animal entity (in the other database). In this case, I would imagine one solution would be to have an "animalName" string property in the Rating/Notes entity, and match it up at runtime. Is this the best way to do it, or is there a way to "sync" two different databases in Core Data?
Here's basically how I ended up solving this.
While Amorya's and MHarrison's answers were valid, they had one assumption: that once created, not only the tables but each row in each table would always be the same.
The problem is that my process to pre-populate the "Animals" database, using existing data (that is updated periodically), creates a new database file each time. In other words, I can't rely on creating a relationship between the (static) Animal entity and a (dynamic) Rating entity in Core Data, since that entity may not exist the next time I regenerate the application. Why not? Because I have no control how Core Data is storing that relationship behind the scenes. Since it's an SQLite backing store, it's likely that it's using a table with foreign key relations. But when you regenerate the database, you can't assume anything about what values each row gets for a key. The primary key for Lion may be different the second time around, if I've added a Lemur to the list.
The only way to avoid this problem would require pre-populating the database only once, and then manually updating rows each time there's an update. However, that kind of process isn't really possible in my case.
So, what's the solution? Well, since I can't rely on the foreign key relations that Core Data makes, I have to make up my own. What I do is introduce an intermediate step in my database generation process: instead of taking my raw data (which happens to be UTF-8 text but is actually MS Word files) and creating the SQLite database with Core Data directly, I introduce an intermediary step: I convert the .txt to .xml. Why XML? Well, not because it's a silver bullet, but simply because it's a data format I can parse very easily. So what does this XML file have different? A hash value that I generate for each Animal, using MD5, that I'll assume is unique. What is the hash value for? Well, now I can create two databases: one for the "static" Animal data (for which I have a process already), and one for the "dynamic" Ratings database, which the iPhone app creates and which lives in the application's Documents directory. For each Rating, I create a pseudo-relationship with the Animal by saving the Animal entity's hash value. So every time the user brings up an Animal detail view on the iPhone, I query the "dynamic" database to find if a Rating entity exists that matches the Animal.md5Hash value.
Since I'm saving this intermediate XML data file, the next time there's an update, I can diff it against the last XML file I used to see what's changed. Now, if the name of an animal was changed -- let's say a typo was corrected -- I revert the hash value for that Animal in situ. This means that even if an Animal name is changed, I'll still be able to find a matching Rating, if it exists, in the "dynamic" database.
This solution has another nice side effect: I don't need to handle any migration issues. The "static" Animal database that ships with the app can stay embedded as an app resource. It can change all it wants. The "dynamic" Ratings database may need migration at some point, if I modify its data model to add more entities, but in effect the two data models stay totally independent.
The way I'm doing this is: ship a database of the static stuff as part of your app bundle. On app launch, check if there is a database file in Documents. If not, copy the one from the app bundle to Documents. Then open the database from Documents: this is the only one you read from and edit.
When an upgrade has happened, the new static content will need to be merged with the user's editable database. Each static item (Animal, in your case) has a field called factoryID, which is a unique identifier. On the first launch after an update, load the database from the app bundle, and iterate through each Animal. For each one, find the appropriate record in the working database, and update any fields as necessary.
There may be a quicker solution, but since the upgrade process doesn't happen too often then the time taken shouldn't be too problematic.
Storing your SQLite database in the Documents directory (NSDocumentDirectory) is certainly the way to go.
In general, you should avoid application changes that modify or delete SQL tables as much as possible (adding is ok). However, when you absolutely have to make a change in an update, something like what Amorya said would work - open up the old DB, import whatever you need into the new DB, and delete the old one.
Since it sounds like you want a static database with an "Animal" table that can't be modified, then simply replacing this table with upgrades shouldn't be an issue - as long as the ID of the entries doesn't change. The way you should store user data about animals is to create a relation with a foreign key to an animal ID for each entry the user creates. This is what you would need to migrate when an upgrade changes it.
Related
I'm migrating a CoreData model between two versions of an application. I was storing binary data as blobs in the previous version and I want to take them out of the blobs for performance. My issue is that during the migration it seems that Core Data loads everything into memory which leads to Low Memory Warnings and then to my app being killed.
Apple documentation suggests the following :
http://developer.apple.com/library/mac/documentation/Cocoa/Conceptual/CoreDataVersioning/Articles/vmCustomizingTheProcess.html#//apple_ref/doc/uid/TP40005510-SW9
However, it seems to rely on the fact that the large objects are applied different mapping.
In my case, all the objects are basically the same and the same mapping has to be applied to each of them. I don't see in this case how I could apply their technique.
How should I handle a migration with very large objects ?
I'm guessing that you have a bunch of changes you want to make in addition to pulling the data out of blobs. My suggestion is to do the migration in a few stages. I'm kind of thinking out loud here, so it might be possible to improve on this. This requires you to be using SQLite.
To make this work, you're going to have three versions of your model:
The original model
The model with the attribute removed (and possibly with a special unique ID added--see below)
The model with all of the changes you've made, including the addition of the new entity and relationships replacing the attribute
The reason to do this is that the transition from version 1 to 2 should be doable with an automatic lightweight migration. In that case Core Data doesn't need to load anything into memory--it just issues SQL statements to make the changes directly on the database.
So, you start by setting up your persistent store coordinator using the old model version. Once you've loaded the data, go through all of the objects you're migrating, extract the binary attribute, and write it to disk somehow. You can use a fetch request with batching and regular autorelease pool draining to make sure you don't use up too much memory for temporary objects. Store the data into the directory you get with NSCachesDirectory. You'll obviously want to store the data in a way that lets you relate it back to the object's managedObjectID.
Then, you shut everything down and ask Core Data to migrate the store from version 1 to version 2. See this link for details. Open up the store with version 2.
You might have to add a step where you assign some sort of unique ID to each object, because I'm not sure if Core Data maintains object IDs when it does a non-lightweight migration. If you need to do this, your version 2 model would add a new attribute to the object you're taking the binary data out of that would be either optional or have a default value set. Since lightweight migration shouldn't change the managedObjectIDs, you could at save the mapping of your new unique ID to the managedObjectIDs you saved along with the binary data two paragraphs ago.
Save the data and close the store.
Open the store and do a migration from version 2 to version 3, which should basically be the code you already had written before you posted the question. Once the store is open, add all of the objects you saved from the version 1 store and set up the relationships using the data you saved along the way.
Simple, right?
Hello fellow stackoverflow family members?
I know it is un-efficient to create one extra sqlite table in iPhone CoreData system. (X)
Currently, My app has one sqlite table but there are preset data to users to no need to waste parse time. But if I adding new entity in current structure of SQLite table, it wipes up the whole preset data. I haven't tried to use immigrate method but I don't think it wouldn't be just adding a new entity on the table. I'm thinking it also wipes up previous preset data.
Current Architecture of SQLite file.
entity : A
attributes : contains data
I want to keep A with attributes still contain data but also add new entity : B.
entity : A
attributes : previous contained data
(PLUS+)
entity : B
attributes : new data
Do I need to create extra persistence set to separately store entity B (create another SQLite file and use as storage) or Is there possible way to add entity B in current SQLite with no changes in entity A?
Thank you.
To directly answer your question: Yes, you can reference multiple SQLite files in Core Data. You'll want to use a unique persistent store for each one.
I don't understand the rest of your question though, so it may be likely that creating multiple persistent stores is not what you want at all.
I know it is in efficient to create one extra sqlite table in iPhone CoreData system.
This is untrue. Creating multiple entities (aka, tables, but that is abstracted away) can lead to much more efficient queries.
Currently, My app has one sqlite table but there is no need to change preset sqlite table but want to add some other attribute.
Just add the attribute to your entity. If your app has already been released or you want to maintain existing data, you'll have to set up a migration and/or turn on lightweight migration.
So commit with new attribute and update coredata is not valid. Because it flash off current data set and need to spend another hours to type manually.
This didn't make any sense to me. Can you clarify?
I would like to know if there is some kind of similar functionality or way to preform an "on duplicate key update" function with core-data as there is with MySQL.
What I want to do is to save an object to the database every time a user presses a button. But if the button is already pressed I want to update the row with some new values instead of adding a new row.
The only way I currently know how to do this is to read the rows from the DB, see if the row exists and then update it.. otherwise add a new row. This seems kind of bad to do this way, am I wrong?
The easiest answer to this is to run a query against the Core Data context and get the object back if it exists. This is the most efficient and least error prone solution to the problem.
You do not need to create a separate NSManagedObjectContext and attempt to deal with merge policies, that is a very inefficient and dangerous way to try and resolve such a simple issue.
Core Data handles a lot of caching for you in the background. If you are attempting to retrieve an object that you just created there is a very high probability that it is still sitting in the cache so the response to your query will be nearly instantaneous.
Note
I just went back to both of those sample projects again to file a bug against them and noticed that they have been updated and finally removed the suggestion of creating a new context. They are now using the NSUndoManager where appropriate and a single context.
EDIT
Remember that the core data framework manages persistence of your object graph. It is not an interface to a sqlite database.
Worry about your object life cycle. When do instances get created? When are they destroyed? What makes your instances unique? Using Books as an example entity, a book has an ISBN which is a unique way of identifying a title, yet many copies of each title can exist. You have two choices in your Entity model, you can create separate instances for each copy of the title or have one instance with a count attribute.
The sample projects CoreDataBooks and iPhoneCoreDataRecipes use NSUndoManager to track state changes between views.
I have an iPhone Core Data app with a pre-populated sqlite "baseline" database. Can I add a second smaller sqlite database with the same tables as my pre-populated "baseline" database but with additional / complementary data such that Core Data will happily union the data from both databases and, ultimately, present to me as if it was all a single data source?
Idea that I had is:
1) the "baseline" database never changes.
2) I can download the smaller "complementary" sqlite database for additional data as and when I need to (I'm assuming downloading sqlite database is allowed, please comment if otherwise).
3) Core Data is then able to union data from 1 & 2. I can then reference this unified data by calling my defined Core Data managed object model.
Hope this makes sense.
Thanks in advance.
Core Data is designed to handle multiple data files via the – addPersistentStoreWithType:configuration:URL:options:error: method. This will allow you to combine all of the data files together and then access them via a single NSManagedObjectContext.
Your only issue, and it may not even be an issue for you, is that the store files cannot directly reference each other. Therefore you will need to reference data between files "manually" via unique identifiers. However I suspect you are already aware of that limitation.
Manual Relationships
The idea is that when both objects in a "relationship" are in one model and one file, Core Data does its magic and handles all of the referential integrity for you. However when they are in different files and/or models this doesn't happen automatically anymore.
The solution to this issue is to use a fetched property that looks up some unique identifier to retrieve the entity (or entities) that you want to be on the other side of the relationship. This will give you a "weak" relationship between files.
One thing to note though when doing this. The fetched property does not get updated automatically when something changes. This means when data changes that would cause that relationship to change, your application will not be automatically made aware of it and you will need to request that property again to get the updated relationship information.
Hopefully that makes it a bit clearer.
Co-existance of fetched properties and relationships
They can definitely co-exist but realize that they are two separate properties. If you want your controller code to see them as one, then I would suggest building a subclass for that entity and then adding a convenience method in there that hits both the relationship and the fetched property and then rolls them up into one NSArray or NSSet before returning it back to your controller code.
You can attach the downloaded database with ATTACH DATABASE statement and operate with unions of tables.
I'm implementing a new iPhone app and am relatively new to Cocoa development overall. I am at the stage of choosing how the persistence layer of this app will work, and it looks like I'm basically choosing between Core Data and sqlite3.
The persisted models in this app are intended to have a schema that is loaded at runtime (from some kind of defn file, probably XML). By which I mean, this app is intended to have objects that are user-definable to some extent, e.g. the Customer type (which has certain built-in fields like "name" and "email") can be modified to have extra fields based on the user's specific needs (e.g. a user might want to add a "favourite fruit" field to their Customer type).
Having said that, will Core Data work for an app with a non-baked-in data model like this? I've just started playing around with the Core Data object designer thing in XCode and it seems like this thing wants to work with objects that have fixed fields that are compiled in.
I'm definitely trying to take the path of least resistance here, and I can see the benefits of using an Apple-supplied data framework, but don't want to start down that path if it's going to lock me into a data model that's defined at compile time.
The Core Data data model needs to be defined at compile time, but that does not mean you can't allow for custom fields to be added and used by end users.
It just means that you would define an entity for custom fields and create the fields as objects.
It is best to design a data model that meets your needs rather than think of how you would solve the problem in SQL.