I am retrieving 10,000 records from an Ultralite database to an array.
My query is taking 3 seconds load values to the array. This is making the UI to freeze for 3 seconds whenever i click to open the view controller.
I want to open the view controller immediately and show Activity indicator for 3 seconds while my query is executing in background.
And if possible i want to show row animation and show row count like "Number Of Products retrieved is 5045" dynamically.
Please, can anyone help me on this?
Thanks in advance.
EDIT:
NSMutableArray *customerArray = [[DB sharedInstance] LoadCustomerOverview];
The "LoadCustomerOverview" is a function which is having the select statement which retrieves 10,000 records from Ultrlite database.
The above line is taking 3 seconds. I checked this with NSLog before and after above statement. Using this "customerArray" i will fill the UITableview in my view controller,which is taking only Milli seconds to prepare cells.
Problem is with the above line.
How can i solve this problem? or any other way to improve performance?
Thanks in advance.
Accept from all other good answers, this may little different, there's (mostly) 10-11 for iPhone device and its double in iPad devices (I'm not having exact idea), number of rows visible to users in UITableView. If you really don't need to process or show all 10k records at once, I think you really don't query for all as it takes good memory and processing time as questioned by you. Instead you can fetch (query for) 1k (or even some small amount) of records at once, once you get 1k (or the amount of you want) you can query for the next and that should be in [self performSelectorInBackground:#selector(getRecordsFromDatabase) withObject:]; so your app will never freeze & user won't feel any interrupts, This scenario could only achieve if your data is persistent and having a unique row identifier key (primary key), also if you're query those data in either ascending or descending order, for other cases like, if you want any random data then this answer can't work.
Also note that, putting code in viewDidAppear & UIActivityIndicator may be your solution, but in case you'll fetching some more amount of rows that time it will interruptable for user.
I can't imagine why would you need 10000 records in an array, as this can probably be optimised.
However, to answer your question, you could move your loading method to viewDidAppear: (I'm assuming you're doing your fetching in viewDidLoad:), and bring up some sort of a progress bar before you start the load.
[NSThread detachNewThreadSelector:#selector(startBackgroundProcess:) toTarget:self withObject:YOUR_OBJECT];
-(void)startBackgroundProcess:(id)obj{
//interact with DB..
//After firing the query and get the Data
[self performSelectorOnMainThread:#selector(finishedBackgroundProcess:) withObject:YOUR_RESULT waitUntilDone:NO];
}
Solution is performSelectorInBackground which will not freeze your UI
[self performSelectorInBackground:#selector(getRecordsFromDatabase) withObject:yourObjectHereAsArgument]
Now method:
-(void)getRecordsFromDatabase
{
//retrieve records here
}
Update: Do you use Core Data ?
What if you batch ?
[fetchRequest setFetchBatchSize:20];
https://developer.apple.com/library/mac/#documentation/Cocoa/Reference/CoreDataFramework/Classes/NSFetchRequest_Class/NSFetchRequest.html
Regards,
Related
I am implementing tableView which loads data from server. I got two scenario:
Get complete data from the server and store in the array say 500 items. Now whenever I need those data I will call my array and fetch data accordingly. In this case, loading huge data from server at first time will take place.
Get required data say 15 from the server and store in the Mutable array. And, if User has scroll down ,get data from the server again and add those in mutable array and display those in the table accordingly. In this case, whenever user will scroll up and down, we have to call server like Lazy Loading.
So, which would be appropriate way to load table from server.Any Feedback will be appreciated.
I think you should be use Lazy Loading. Pull To Refresh.... tableview is available for that. every refresh get 15 records and add into your array. i think this is the best way. because if you load 500 items at a time and any user required only 5th item, so other record are not useful..so i suggested you to use 2nd way.
I think for better solution you can have a bottom cell named "Load more items", click on that cell will load next set of items, it will help both user and programmer for avoiding unwanted items.
I'm building an iPhone app which uses your current location to get a series of results from a WCF Service. I then store the data returned by the WCF using Core Data and then populate a TableView.
What I'd like to do is store the results for around 30 minutes then each time the view loads, check whether I need to refresh the data. I would also be watching the users' location so that if they move a considerable distance, the data will also update.
My problem is that I'm not sure what the best approach would be to monitor the age of the data.
Should I store an insert date with the Core Data Entity. Alternatively I could create a new entity to store a global 'last retrieved' date. Or I could just store a value in the plist.
I'm not sure which of my above suggestions would be the best, or even there are any better ones. Any help with this is much appreciated!
An insertDate attribute would probably be the best bet. Whenever your app becomes loads, you can check the change in time versus that attribute's value. If the time is over 30 minutes, you can fire your reload method, else you can take the remaining time and set up a scheduled NSTimer event that will fire the method when the time runs up. Just be sure to reset the insertDate value once you reload your data
In the model, I have two entities: Record and Category. Category is one-to-many with Record through inverse relationship. The persistent store is of SQLITE type and the db is not so small, about 23MB (17k records).
I use a list-detail design to show the records table and the detailed record view.The list viewController uses NSFetchedResultsController.
Building on the device, if I don't use setFetchBatchSize:
CoreData: annotation: sql connection fetch time: 15.8800s
CoreData: annotation: total fetch execution time: 16.9198s for 17028 rows.
OMG!
If I use setFetchBatchSize:25, everything works great again:
CoreData: annotation: sql connection fetch time: 1.1736s
CoreData: annotation: total fetch execution time: 1.1900s for 17028 rows.
Yeah, that would be great! But it is not! In the list viewController, when user taps on a record I allocate a detailed viewController and I pass the record at the indexPath in the fetchedResultsController:
- (void)tableView:(UITableView *)tableView didSelectRowAtIndexPath:(NSIndexPath *)indexPath {
Record *record = (Record *)[fetchedResultsController objectAtIndexPath:indexPath];
RecordViewController *recordViewController= [[RecordViewController alloc] init];
recordViewController.record = record;
[self.navigationController pushViewController:recordViewController animated:YES];
[recordViewController release];
}
NOW, in the detailed viewController, I have a button to set a record as favorite or not:
- (IBAction) setFavorite {
if (![record.ISFAV intValue])
[record setValue:[NSNumber numberWithInt:1] forKey:#"ISFAV"];
else
[record setValue:[NSNumber numberWithInt:0] forKey:#"ISFAV"];
###SAVE ON THE CONTEXT HERE###
}
OK, are u ready? If I tap on the first record in the list, then I add or remove it from the favorites, it happens in 0.0046 seconds, instantly! Console with SQL Debug mode shows only the UPDATE statement:
CoreData: sql: BEGIN EXCLUSIVE
CoreData: sql: UPDATE ZRECORD SET ZISFAV = ?, Z_OPT = ? WHERE Z_PK = ? AND Z_OPT = ?
CoreData: sql: COMMIT
CoreData: annotation: sql execution time: 0.0046s
If I scroll very fast the big list (and I obviously find the batch requests on the console), when I tap a record reached with many batch requests and I add\remove it from favorites, many many many many (too many! the more I scroll the more they are!) SELECT statements appears in the console before the UPDATE one. This means total execution time not acceptable (the uibutton freezes for a long time on the iphone).
What's happening? The problem is clearly related to the batched fetch requests. More fetch requests = more SELECT statements before the UPDATE statement. This is one of them:
CoreData: sql: SELECT 0, t0.Z_PK, t0.Z_OPT, t0.ZCONTENT, t0.ZCONTENT2, t0.ZISUSER, t0.ZISFAV, t0.ZTITLE, t0.ZTITLE2, t0.ZID, t0.ZAUTHOR, t0.ZCATEGORY FROM ZRECORD t0 WHERE t0.Z_PK IN (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?) ORDER BY t0.ZTITLE LIMIT 26
If I remove the setFetchBatchSize, there's no problem (but startup requires 16 seconds). It seems that when I update the property ISFAV, CoreData needs to execute again all the fetchRequests that were needed to reach that record, even if I pass that record to the detail viewController as object.
Sorry for the long post, I tried to be as clearer as possible.
Thank you very much, I'm driving myself crazy...
What's happening is the fetched results controller sees the change notification when you update the managed object, and it has to figure out what index that object was so it can tell its delegate that the object was updated. To do this, it's going through all the batch selects again until it can find the right one. I'm not sure why it can't just have this information cached, but obviously it isn't. Have you tried adding a section cache to the fetched results controller? That may possibly speed things up (depending on whether or not the fetched results controller uses that cache in this instance). You do so simply by specifying the cache name when you call -initWithFetchRequest:managedObjectContext:sectionNameKeyPath:cacheName:.
First, I would suggest to avoid showing to the user a big table with 17K records; instead you should allow the user searching for records and then selecting one of the search results. Anyway, if you want to allow the user selecting a record directly from the big table, you need to think about the fetching process.
Start checking that you have properly indexed in your Core Data model the attributes you use to setup the NSPredicate associate to your NSFetchedResultsController. Think about the size of your "working set" of records. This should be as small as possible, and is usually in the order of hundreds of records.
In your case, setting setFetchBatchSize to 25 is probably not appropriate, given that you want to allow your user browsing 17K records. Since 17000:25 = 680, you will need that many fetches to reach the latest 25 records. But fetching involves actual I/O to the underlying database to make sure that everything is always in sync with other "possible" insert/delete/update operations done by other threads.
Even if your application does not use multiple threads with Core Data, the Core Data framework must check to verify if something changed. Now, since I/O is expensive, you need a tradeoff. Setting setFetchBatchSize to 1000 will require in the worst case 17 fetches to reach the latest 1000 records (improving by a factor of 40) even though each individual fetch may take "slightly" longer.
Using the cache as suggested may provide some benefit unless other threads modify the data. Indeed, cache hits are fast, very fast. However, cache misses are extremely expensive, requiring I/O to fetch the associated data from the database. The chance of cache misses increases of course when multiple threads work simultaneously on the same database (unless these threads only read records).
Another possible solution you may want to try, involves using multiple threads. Your main thread only fetches an initial number of records and presents them to the user while another thread using a different managed object context fetches another batch of records asynchronously, in background (using a proper offset). These records are then handed over to the main thread for visualization.
One more thing. You should not use KVC to update the value of your attributes; for performance reasons it is much better to do something like
record.ISFAV = [NSNumber numberWithInt:1];
or
[record setISFAV:[NSNumber numberWithInt:1]];
Updating just a single attribute you may not notice a difference, but if you need to work with several attributes, you may start experiencing huge savings.
i read about Batch insertion and deletion of rows in the apple documentation,but didnt get a clear idea of it's use.
For example can i use it to add a few rows(animated) when the user taps on a row in the table,like a drop down list??Can somebody explain it to me or give a reference as to where i can find a tutorial??
It was pretty easy,Figured this one out on my own.All that i had to to do was modify the array from which the cells got their content,just before inserting the rows as explained in Apple's documentation.'reloadData' takes care of the rest.
One thing to note is that the logic for setting the cells content should see to it that whenever reload data is called, the cells get correct contents from the array.
HI All,
I currently have an NSFetchedResultsController setup to return all rows in a table in my core data database. This then fills up my UITableView. The trouble is this will quickly get out of hand as the rows grow in number.
How can I limit the initial query to say 20 results, then add a button somewhere to "Get More" from where we left off?
Thanks for any guidance as always
This is controlled with NSFetchRequest's -setFetchLimit: and -setFetchOffSet.
If I recall correctly, the drawback with NSFetchedResultsController is that you can't modify the fetch request after you create your NSFetchedResultsController instance. I believe this means you'll have to create a new one (instance w/new fetch request) each time you change the range you want to retrieve/display.
File an enhancement request with Apple at bugreporter.apple.com if you feel this shouldn't be the case.
To change the limit number on the fly you simply need to:
Access the fetchRequest of your NSFetchedResultsController instance, change the limit, delete the old cache if there is any and perform a new fetch.
Code:
[yourFetchedResultsController.fetchRequest setFetchLimit:50];
[NSFetchedResultsController deleteCacheWithName:"you cache name"];
[yourFetchedResultsController performFetch:nil];
fetchBatchSize only affects how many objects are fetched at a time. It will not limit number of objects in-memory concurrently so it is still possible to run out of memory. It is possible to limit the total concurrent objects with a combination of batchSize, fetchLimit, and offset but it requires deleting the cache or storing separate caches per "page", which seems un-ideal to me.
Another more hacky method to get around it is to re-create the NSFetchedResultsController, the results from the old controller will be faulted if possible, and you can start with a clean slate. Really crude, but it avoids deleting the cache.
I believe that instead of setting -setFetchLimit and limiting your NSFetchRequest (for new rows you have to create a new reqeust), set -fetchBatchSize to only control how many rows will be loaded into memory. Say, If you show 10 cells per view, set your batch size to double or so. As you scroll your view, the controller will automatically load new set into memory.