Is it possible to load a database / entity directly into CoreData? The app relies of data, the initial data load being > 100,000 items. On an iPhone 4 it's taking about 15 minutes to complete the insertion, which is an unpleasantly long time.
Is there any way I could:
Load a pre-populated entity into CoreData
Load a pre-populated database (all entities) into CoreData
Something along these lines
to make the insertion of data significantly quicker
Thanks.
Create your database using CoreData and add it to your project bundle, then at first load, copy the database from the bundle to the file system so it will be writable, then open it from there. You may want to mark the database as don't share to iCloud depending on your needs.
Yes it is possible. The 'normal' approach to this is to pre-build the database, include it as a resource, check on startup whether you have a database in the documents folder and, if not, copy it over from the bundle.
I usually run my code to build the database on the simulator, and then go copy it from the sim folder and add to my project resources.
Copying a pre-made Core Data generated sqlite file via the application bundle is a good idea, even though I find it a bit cumbersome. This make most sense if you have a store for static data and one for dynamic data - in most cases it is feasible to copy the static one like this.
But nevertheless, for the record, 15 minutes is ridiculous. I can close to guarantee that you can cut this down to a few seconds.
I have been able to accomplish 500K inserts in less than 30 seconds on a first generation iPad. The most important optimisation techniques are:
disabling the undo manager,
using a background thread with #autoreleasepool,
saving periodically with exactly the right batch size (determine experimentally).
Related
I have a CoreData sqlite files *.sqlite *.sqlite-shm *.sqlite-wal in a zip format from my previous application version. The file is almost few hundred MBs.
In recent version, I have done some changes in the database schema i-e adding few new fields etc.
In Importing, I am simply replacing the current database by old database and making the user to restart application, which is crude. It does work however it causes 2 issues
1- It takes a lot of time on splash screen hanging the application.
2- If the database is big enough, the hanging time passes the Timeout
of Application and closes the application automatically.
What is the better way of importing database into core data saved in Documents as zip file.
First detect whether you are doing a migration. If you are, then display a ViewController with a spinner that explains what you are you doing ("please wait while we optimize the app"). When the database migration is done, the load your regular viewController.
This exact issue is discussed in this lecture: Core Data Potpourri (Paul Goracke, February 13, 2014) around 58:00. While the lecture is a bit dated (it was made before NSPersistentContainer) it is still one of the best I have ever seen and it is worthwhile to watch the entire video.
Just started to read various posts how to sync files or core data using iCloud. The app I'm currently developing stores data inside core data and filenames as references to the image files stored in documents app sandbox. So, a related file (photo) is also created in documents dir every time a user makes a record inside a database.
Everything looks fine if we would need to sync files OR core data, however I'm looking a way how to sync core data AND files. So, I'm worried about the case if new core data records will arrive earlier than image files of those records. In that case, data integrity will be broken. Actually, I would prefer all new related files would come first, and then all core data updates. Is it possible to do that?
Not really, no. You send data to the cloud, but you have no way to control when it appears on other devices. iCloud is going to bring over your managed objects whenever it feels like it, regardless of the state of the external files. The only way you could make this happen would be to find and download any external files, wait for the download to finish, and only then bring up your Core Data stack. But that would mean locking the user out of the data store until the downloads finish, which is not a good idea.
When I faced a similar situation, I handled it like this:
Initiate downloads of all the external files and bring up the Core Data stack.
Modify the getter method for the image to check whether the file exists and has been downloaded.
If yes to both, proceed normally
If no, display a "loading..." UI element. This could be a spinner or a progress indicator. Listen for a custom "download complete" notification.
Whenever an external file finishes downloading, post that "download complete" notification. Re-check the file, and if it's ready, replace the "loading..." UI with the image.
I have an app with a very large Core Data database. I have versioned it many times over the past year.
The last time I versioned the database I made one simple change to an entity: I added a new optional attribute. For some reason it would not migrate using Light-weight Migration. I found out much later that this was due to a bug in Apple's Light-weight Migration code resulting from the 'renaming identifiers' that I had needed back in another versioning.
Anyway, I digress...
Because of the bug that kept me from using Light-weight migration, I created a mapping file to help with the migration, not understanding that this would was a much heavier process and would force my users to wait while the app loaded the entire database into memory while doing the migration. It turns out that this is not really an option at all with very large databases and many of my users were unable to migrate the database at all due to memory problems, etc.
So now I want to re-release my app and clear up this problem. The trouble is, some of my users have a database that is somehow marked as being 'in the middle of migrating'. Even with my new code, which gets rid of the mapping file and supports Light-weight migration, users that are in this state, 'in the middle of a migration', don't seem to get reset.
What are my options for backing out a migration?
- I can detect that I am in this state because there is a '.myDB.sqlite.migrationdestination_41b5a6b5c6e848c462a8480cd24caef3' file in the Documents directory. Deleting this file does not clear up the migration. My guess is that the database is somehow flagged as being in this state, or is already partially migrated.
- I can detect this state and then delete the database altogether. But this forces my users to re-download their data.
Any Thoughts?
Thanks for you help.
The only thing that occurs to me would be crack open the SQL store of an affected file and look for flags or something else that might signal the db being in a transitory state. You might be able to write directly to the file and alter something.
That's really ugly problem.
this is my first question here.
Well, I am developing an iPhone application that will use coredata, but the theres not going to be an option to insert data into the database of the app.
I want to know how to pre-populate it's database.
I thought of creating a plist with all the info and make a function to put it all into the coredata database if the database is not set yet...
but I want to know if it is the right way to do so.
Thanks all.
The easiest way is to create a desktop application with the same model (the exact same model even, just link it in). Then have Interface Builder create you a default UI for it and enter the data. Far easier than doing it on a device.
Once you have the data then include the SQLite file in your bundle and since your app is read only as you claimed; point the NSPersistentStoreCoordinator at the file in your app bundle and Bob's your uncle.
Creating an app on the desktop takes about 5 minutes and will save you far more time on the data entry side.
You can pre-create the database file (sqlite file) using your device and put that in the bundle when you distribute it. When you run the app for the first time you can move that from your applications bundle you can move it to the documents directory where core data will read it.
You can also have a function that runs at the start of your applications first run where it reads from your plist and creates the appropriate objects.
I am creating an application, and am using Core Data as my backend data source.
I am wondering how I can manually add entries into a Core Data Entity which can be used to hold static data which will never change throughout the applications lifetime.
There are many solutions to this issue. First, for data entry, you can create a trivial desktop application in a couple of minutes that will let you do the data input. Since the underlying sqlite file is the same, you can then copy the file to your project.
As for accessing it, depends on your data structure. If you have entities that are both static and editable (such as a look up list that can be added to), then I suggest carrying the sqlite file in your bundle and on first launch copy it to the documents directory.
If the tables are NEVER writable then you can leave the static data in your bundle and create a second store in the documents directory that is writable. You can then bundle the two together on creation of your Core Data stack and the rest of your application will treat them as if they are a single database/object graph. The only issue with this is that you cannot have relationships across files.
I have never tried this, however I think you can populate core data with an example program which you run in the simulator which populates the database with the data.
Then you can get .sqlite database out of the simulator and into Xcode resource folder.
Every time your app starts, even first run the database will contain the objects you first populated it with in the example program.