I'm trying to import a large data set (~6,000) in to my core data application. I've read the Apple document "Efficiently Importing Data" and I think I set it up correctly. The weird thing is the application isn't crashing in the simulator, although it does if I run it with the Leaks instrument, but it isn't saving all the data. Sometimes it will only save 3-4 hundred other times it will save 3-4 thousand and rarely the whole data set. I think it's probably memory leak related and I'm pretty new to using NSAutoReleasePool, any help would be much appreciated.
NSURL *url = [NSURL URLWithString:#""];
NSString *responseString = [NSString stringWithContentsOfURL:url encoding:NSASCIIStringEncoding error:nil];
if (responseString) {
NSArray *players = [responseString componentsSeparatedByString:#";"];
NSUInteger LOOP_LIMIT = 100, count = 0;
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSManagedObjectContext *context = [[AppController sharedAppController] managedObjectContext];
[context setUndoManager:nil];
for (int i=0; i<([players count] - 1); i++) {
NSArray *info = [[players objectAtIndex:i] componentsSeparatedByString:#","];
NSString *dateInfo = [info objectAtIndex:10];
NSLocale *usLocale = [[[NSLocale alloc] initWithLocaleIdentifier:#"en_US"] autorelease];
NSDateFormatter *fo = [[[NSDateFormatter alloc] init] autorelease];
[fo setDateFormat:#"MM/dd/yyyy"];
[fo setLocale:usLocale];
[fo setTimeZone:[NSTimeZone systemTimeZone]];
NSDate *dob = [fo dateFromString:dateInfo];
Players *player = [NSEntityDescription insertNewObjectForEntityForName:#"Players"
inManagedObjectContext:context];
NSNumberFormatter *f = [[[NSNumberFormatter alloc] init] autorelease];
[f setNumberStyle:NSNumberFormatterNoStyle];
player.playerID = [f numberFromString:[info objectAtIndex:0]];
player.lastName = [info objectAtIndex:1];
player.firstName = [info objectAtIndex:2];
player.position = [info objectAtIndex:4];
NSString *teamName = [info objectAtIndex:3];
NSFetchRequest *req = [[[NSFetchRequest alloc] init] autorelease];
NSEntityDescription *ent = [NSEntityDescription entityForName:#"Teams" inManagedObjectContext:context];
[req setEntity:ent];
[req setIncludesPropertyValues:NO];
NSPredicate *pre = [NSPredicate predicateWithFormat:#"team=%#", teamName];
[req setPredicate:pre];
NSArray *list = [context executeFetchRequest:req error:nil];
if ([list count]) {
Teams *team = [list objectAtIndex:0];
player.team_Players_Teams = team;
}
count++;
if (count == LOOP_LIMIT) {
[context save:nil];
[context reset];
[pool drain];
pool = [[NSAutoreleasePool alloc] init];
count = 0;
}
}
if (count != 0) {
NSLog(#"In Save Remaining");
[context save:nil];
[context reset];[pool drain];
}
I can't see anything dodgy in the code either. Definitely no errors appearing in the log?
Btw one other optimisation tip covered in the Core Data pdf for importing data is to move the predicate creation outside the loop and use substitution variables.
I can't see any obvious leaks but:
You could minimise the amount of memory used by alloc init retain-ing the NSLocale, NSDateFormatter, NSNumberFormatter and then release-ing them after the loop is complete. These don't seem to change between runs of the loop.
I don't know Core Data but where does the NSManagedObject/Player *player object get released? Is this autoreleased through Core Data?
As an aside, you can use [list lastObject] rather than [list count] and [list objectAtIndex:0] as the last two will crash if list is nil
Update based on response:
If nothing appears to make a difference then the next step is to simplify the code so as to remove any error sources.
Carry out my suggestions in #1 above to minimise the amount of code in the loop.
Check you're releasing the list object somewhere or that it is allocated as autorelease.
Remove the intermediate saving (all that code inside count == LOOP_LIMIT) and only save and drain the pool once all the arrays have been processed. You also should not need the code following on inside the if (count != 0)
Replace the error:nil statements with proper error:&&error and log errors. To log the errors do the following (apologies but the code formatting doesn't seem to work - no idea why):
NSError *error = nil; //Declared upfront
// Your streamlined code
// ....
error = nil; //Just before a fetchRequest or context:save
/* After looping through all your code now attempt to save */
if(![context save:&error]) {
NSLog(#"Failed to save to data store: %#", [error localizedDescription]);
NSArray *detailedErrors = [[error userInfo] objectForKey:NSDetailedErrorsKey];
if(detailedErrors != nil && [detailedErrors count] > 0) {
for(NSError *detailedError in detailedErrors) {
NSLog(#"DetailedError: %#", [detailedError userInfo]);
}
} else {
NSLog(#" %#", [error userInfo]);
}
}
Then check the log to see if you're getting any odd messages. You can also use similar code anywhere in Core Data that you need to check for errors (i.e. executeFetchRequest). It's worth a shot to find out what the error is here.
Related
I am trying to send a CSV file through MfMail Composer.Everything works fine but there are lot leaks while using Instruments.I am not able trace out where I went wrong.Here is my code.
-(NSData *)getCSV
{
NSManagedObjectContext *moc = [(ETAppDelegate *)[[UIApplication sharedApplication] delegate]managedObjectContext];
NSFetchRequest *request = [[[NSFetchRequest alloc]init]autorelease];
[request setEntity:[NSEntityDescription entityForName:#"Expense" inManagedObjectContext:moc]];
NSError *error = nil;
NSArray *results = [moc executeFetchRequest:request error:&error];
NSMutableArray *expensesList = [[[NSMutableArray alloc]init]autorelease];
for(int i = 0;i<[results count];i++){
NSString *category = [[results objectAtIndex:i]valueForKey:#"category"];
NSString *date = [[NSDateFormatter dateFormatterwithMediumStyle]stringFromDate:[[results objectAtIndex:i]valueForKey:#"date"]];
NSString *amount = [NSNumberFormatter localizedStringFromNumber:[[results objectAtIndex:i]valueForKey:#"amount"] numberStyle:NSNumberFormatterCurrencyStyle];
NSString *mailString = [NSString stringWithFormat:#"%#,%#,%#",category,date,amount ];
[expensesList addObject:mailString];
}
NSString *expensesString = [expensesList componentsJoinedByString:#"\n"];
NSData *expensesData = [expensesString dataUsingEncoding:NSUTF8StringEncoding];
return expensesData;
}
-(void)displayComposerSheet
{
NSData *csvFile = [self getCSV];
NSString *csvFileName = #"MyExpenses";
MFMailComposeViewController *mailController = [[MFMailComposeViewController alloc]init];
[mailController setSubject:#"List Of Expenses"];
[mailController setMessageBody:#"Expenses" isHTML:NO];
[mailController addAttachmentData:csvFile mimeType:#"text/csv" fileName:csvFileName];
[mailController setMailComposeDelegate:self];
[self presentModalViewController:mailController animated:YES];
[mailController release];
}
I can't say that I see anything in the code you've provided that should cause you to leak. Your leaks are likely taking place elsewhere.
What I do see, however, is the potential to create a lot of autoreleased objects in that loop in -getCSV, and depending on how many iterations you're performing, that could be almost as bad. The way you've written it, I see a date formatter and three strings headed for your main autorelease pool with every iteration. One suggestion is to create a date formatter outside your loop that you can reuse inside it. Another suggestion is to frame the guts of your loop with a local autorelease pool; that will prevent your main autorelease pool from becoming too large.
Apropos of nothing, you should also consider using fast enumeration.
Here's your loop with the suggestions applied:
NSDateFormatter *myDateFormatter = [NSDateFormatter dateFormatterWithMediumStyle];
NSMutableArray *expensesList = [[[NSMutableArray alloc] init] autorelease];
for (id obj in results)
{
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSString *category = [obj valueForKey:#"category"];
NSString *date = [myDateFormatter stringFromDate:[obj valueForKey:#"date"]];
NSString *amount = [NSNumberFormatter localizedStringFromNumber:[obj valueForKey:#"amount"] numberStyle:NSNumberFormatterCurrencyStyle];
NSString *mailString = [NSString stringWithFormat:#"%#,%#,%#", category, date, amount];
[expensesList addObject:mailString];
[pool release];
}
I am a newbie and I would like to store and retrieve data from a Core Data database.
I get all the data from a php file, which communicates with a SQL database. This php file returns a JSON object, which in turn is parsed by my app and then written into Core Data by doing the following:
AppDelegate.m
-(void)writeDataIntoCoreData
{
NSManagedObjectContext *context = [self managedObjectContext];
NSManagedObject *articles = [NSEntityDescription
insertNewObjectForEntityForName:#"Articles"
inManagedObjectContext:context];
for(int i= 0; i<[json count];i++){
NSDictionary *info = [json objectAtIndex:i];
NSNumberFormatter * f = [[NSNumberFormatter alloc] init];
[f setNumberStyle:NSNumberFormatterDecimalStyle];
NSNumber * myNumber = [f numberFromString:[info objectForKey:#"userID"]];
[articles setValue:myNumber forKey:#"articleID"];
[articles setValue:[info objectForKey:#"username"] forKey:#"author"];
[articles setValue:[info objectForKey:#"user_pic"] forKey:#"text"];
NSLog(#"Name = %#",[info objectForKey:#"username"]);
NSLog(#"Text = %#",[info objectForKey:#"user_pic"]);
}
NSError *error;
if (![context save:&error]) {
NSLog(#"Whoops, couldn't save: %#", [error localizedDescription]);
}
}
MainViewController.m
Here I try to retrieve all the data in the Core Data database by iterating through the fetchedObjects.
-(IBAction)showCD:(id)sender
{
NSManagedObjectContext *context = [self managedObjectContext];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription
entityForName:#"Articles" inManagedObjectContext:context];
[fetchRequest setEntity:entity];
NSError *error;
NSArray *fetchedObjects = [context executeFetchRequest:fetchRequest error:&error];
for (NSManagedObject *info in fetchedObjects) {
NSLog(#"id: %#", [info valueForKey:#"articleID"]);
NSLog(#"Name: %#", [info valueForKey:#"author"]);
NSLog(#"Text: %#", [info valueForKey:#"text"]);
}
}
The NSLog, however, only displays one object, namely the first one that was written into the database. What is wrong ?
move your
NSManagedObject *articles = [NSEntityDescription
insertNewObjectForEntityForName:#"Articles"
inManagedObjectContext:context];
into your for loop,
for(int i= 0; i<[json count];i++){
NSManagedObject *articles = [NSEntityDescription
insertNewObjectForEntityForName:#"Articles"
inManagedObjectContext:context];
NSDictionary *info = [json objectAtIndex:i];
NSNumberFormatter * f = [[NSNumberFormatter alloc] init];
[f setNumberStyle:NSNumberFormatterDecimalStyle];
NSNumber * myNumber = [f numberFromString:[info objectForKey:#"userID"]];
[articles setValue:myNumber forKey:#"articleID"];
[articles setValue:[info objectForKey:#"username"] forKey:#"author"];
[articles setValue:[info objectForKey:#"user_pic"] forKey:#"text"];
NSLog(#"Name = %#",[info objectForKey:#"username"]);
NSLog(#"Text = %#",[info objectForKey:#"user_pic"]);
}
Move the call to insertNewObjectForEntityForName:inManagedObjectContext: inside your loop. The way you're doing it now, you're creating just one managed object and then repeatedly change its attributes.
I have a program that runs perfectly fine when I run on the simulator, but it won't run from the device at all. Part of the initial run involves loading a few XML files into core data and it seems that these files are not being found when running on the device.
Here is the beginning of the routine that loads the file. Any help is much appreciated.
- (BOOL) checkForUpdate:(NSString *)entityName {
NSArray *thisObjectArray = nil;
NSDate *thisEntityDate = nil;
BOOL returnVal = NO;
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription entityForName:#"AppData" inManagedObjectContext:[self managedObjectContext]];
[fetchRequest setEntity:entity];
NSError *error;
NSArray *appDataArray = [self.managedObjectContext executeFetchRequest:fetchRequest error:&error];
[fetchRequest release];
AppData *thisAppData = [appDataArray objectAtIndex:0];
if ([entityName isEqualToString:#"Features"]) {
thisEntityDate = thisAppData.FeaturesUpdated;
}
else if ([entityName isEqualToString:#"DisplayTypes"]) {
thisEntityDate = thisAppData.DisplayTypesUpdated;
}
else if ([entityName isEqualToString:#"Sections"]) {
thisEntityDate = thisAppData.SectionsUpdated;
}
NSString *filePath = [[NSBundle mainBundle] pathForResource:entityName ofType:#"xml"];
NSData *xmlData = [[NSMutableData alloc] initWithContentsOfFile:filePath];
GDataXMLDocument *doc = [[GDataXMLDocument alloc] initWithData:xmlData options:0 error:&error];
if (doc != nil) {
// process information - this code is not being called because doc is returning nil
}
[xmlData release];
[doc release];
return returnVal;
}
Mind the case sensitivity. The device is case sensitive, whilst the simulator is not.
I have a few apps which are largely data driven, so most screens are basically composed of:
Open the screen
Download the data via an NSOperation
Display data in a UITableView
Make a selection from the UITableView
Go to new screen, and start over from step 1
I am finding that everything works in normal usage, but if the user goes away from the app for a while and then comes back, I'm getting an EXC_BAD_ACCESS error when the next NSOperation runs. This doesn't seem to matter if the user sends the app into the background or not, and it only seems to occur if there's been at least a few mins since the previous data connection was made.
I realise this must be some form of over-releasing, but I'm pretty good with my memory management and I can't see anything wrong. My data calls generally look like this:
-(void)viewDidLoad {
[super viewDidLoad];
NSOperationQueue* tmpQueue = [[NSOperationQueue alloc] init];
self.queue = tmpQueue;
[tmpQueue release];
}
-(void)loadHistory {
GetHistoryOperation* operation = [[GetHistoryOperation alloc] init];
[operation addObserver:self forKeyPath:#"isFinished" options:0 context:NULL];
[self.queue addOperation:operation];
[operation release];
}
-(void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)change context:(void *)context {
if ([keyPath isEqual:#"isFinished"] && [object isKindOfClass:[GetHistoryOperation class]]) {
GetHistoryOperation* operation = (GetHistoryOperation*)object;
if(operation.success) {
[self performSelectorOnMainThread:#selector(loadHistorySuceeded:) withObject:operation waitUntilDone:YES];
} else {
[self performSelectorOnMainThread:#selector(loadHistoryFailed:) withObject:operation waitUntilDone:YES];
}
} else {
[super observeValueForKeyPath:keyPath ofObject:object change:change context:context];
}
}
-(void)loadHistorySuceeded:(GetHistoryOperation*)operation {
if([operation.historyItems count] > 0) {
//display data here
} else {
//display no data alert
}
}
-(void)loadHistoryFailed:(GetHistoryOperation*)operation {
//show failure alert
}
And my operations generally looks something like this:
-(void)main {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
NSError* error = nil;
NSString* postData = [self postData];
NSDictionary *dictionary = [RequestHelper performPostRequest:kGetUserWalkHistoryUrl:postData:&error];
if(dictionary) {
NSNumber* isValid = [dictionary objectForKey:#"IsValid"];
if([isValid boolValue]) {
NSMutableArray* tmpDays = [[NSMutableArray alloc] init];
NSMutableDictionary* tmpWalksDictionary = [[NSMutableDictionary alloc] init];
NSDateFormatter* dateFormatter = [[NSDateFormatter alloc] init];
[dateFormatter setDateFormat:#"yyyyMMdd"];
NSArray* walksArray = [dictionary objectForKey:#"WalkHistories"];
for(NSDictionary* walkDictionary in walksArray) {
Walk* walk = [[Walk alloc] init];
walk.name = [walkDictionary objectForKey:#"WalkName"];
NSNumber* seconds = [walkDictionary objectForKey:#"TimeTaken"];
walk.seconds = [seconds longLongValue];
NSString* dateStart = [walkDictionary objectForKey:#"DateStart"];
NSString* dateEnd = [walkDictionary objectForKey:#"DateEnd"];
walk.startDate = [JSONHelper convertJSONDate:dateStart];
walk.endDate = [JSONHelper convertJSONDate:dateEnd];
NSString* dayKey = [dateFormatter stringFromDate:walk.startDate];
NSMutableArray* dayWalks = [tmpWalksDictionary objectForKey:dayKey];
if(!dayWalks) {
[tmpDays addObject:dayKey];
NSMutableArray* dayArray = [[NSMutableArray alloc] init];
[tmpWalksDictionary setObject:dayArray forKey:dayKey];
[dayArray release];
dayWalks = [tmpWalksDictionary objectForKey:dayKey];
}
[dayWalks addObject:walk];
[walk release];
}
for(NSString* dayKey in tmpDays) {
NSMutableArray* dayArray = [tmpWalksDictionary objectForKey:dayKey];
NSSortDescriptor *sortDescriptor = [[NSSortDescriptor alloc] initWithKey:#"startDate" ascending:YES];
NSArray *sortDescriptors = [NSArray arrayWithObject:sortDescriptor];
NSArray* sortedDayArray = [dayArray sortedArrayUsingDescriptors:sortDescriptors];
[sortDescriptor release];
[tmpWalksDictionary setObject:sortedDayArray forKey:dayKey];
}
NSSortDescriptor* sortDescriptor = [NSSortDescriptor sortDescriptorWithKey:nil ascending:NO selector:#selector(localizedCompare:)];
self.days = [tmpDays sortedArrayUsingDescriptors:[NSArray arrayWithObject:sortDescriptor]];
self.walks = [NSDictionary dictionaryWithDictionary:tmpWalksDictionary];
[tmpDays release];
[tmpWalksDictionary release];
[dateFormatter release];
self.success = YES;
} else {
self.success = NO;
self.errorString = [dictionary objectForKey:#"Error"];
}
if([dictionary objectForKey:#"Key"]) {
self.key = [dictionary objectForKey:#"Key"];
}
} else {
self.errorString = [error localizedDescription];
if(!self.errorString) {
self.errorString = #"Unknown Error";
}
self.success = NO;
}
[pool release];
}
-(NSString*)postData {
NSMutableString* postData = [[[NSMutableString alloc] init] autorelease];
[postData appendFormat:#"%#=%#", #"LoginKey", self.key];
return [NSString stringWithString:postData];
}
----
#implementation RequestHelper
+(NSDictionary*)performPostRequest:(NSString*)urlString:(NSString*)postData:(NSError**)error {
[UIApplication sharedApplication].networkActivityIndicatorVisible = YES;
NSURL* url = [NSURL URLWithString:[NSString stringWithFormat:#"%#/%#", kHostName, urlString]];
NSMutableURLRequest *urlRequest = [NSMutableURLRequest requestWithURL:url cachePolicy:NSURLRequestReloadIgnoringCacheData timeoutInterval:30];
[urlRequest setHTTPMethod:#"POST"];
if(postData && ![postData isEqualToString:#""]) {
NSString *postLength = [NSString stringWithFormat:#"%d", [postData length]];
[urlRequest setHTTPBody:[postData dataUsingEncoding:NSASCIIStringEncoding]];
[urlRequest setValue:postLength forHTTPHeaderField:#"Content-Length"];
[urlRequest setValue:#"application/x-www-form-urlencoded" forHTTPHeaderField:#"Content-Type"];
}
NSURLResponse *response = nil;
error = nil;
NSData *jsonData = [NSURLConnection sendSynchronousRequest:(NSURLRequest *)urlRequest returningResponse:(NSURLResponse **)&response error:(NSError **)&error];
NSString *jsonString = [[NSString alloc] initWithBytes: [jsonData bytes] length:[jsonData length] encoding:NSUTF8StringEncoding];
NSLog(#"JSON: %#",jsonString);
//parse JSON
NSDictionary *dictionary = nil;
if([jsonData length] > 0) {
dictionary = [[CJSONDeserializer deserializer] deserializeAsDictionary:jsonData error:error];
}
[UIApplication sharedApplication].networkActivityIndicatorVisible = NO;
return dictionary;
}
If I have the autorelease pool in place, the crash occurs on [pool release]. If I don't, then the crash just looks to appear in the main.m method, and I don't seem to get any useful information. It's difficult to track down when I have to wait 10 mins in between every test!
If anyone can offer any clues or directions to go, that'd be much appreciated.
It's almost certain you're overreleasing something in your code, seeing that the crash is occurring during a [pool release] (There's a autorelease pool in the main method as well).
You can find it using Xcode - use build and analyze to have the static analyser pinpoint potential problems. Run it and post the results.
try this:
http://cocoadev.com/index.pl?NSZombieEnabled
also, you should avoid:
1) calling UIKit methods from secondary threads
2) making (synchronous) url requests from the main thread.
you must be doing one in any case in RequestHelper's performPostRequest method.
My guess is this section
GetHistoryOperation* operation = (GetHistoryOperation*)object;
if(operation.success) {
[self performSelectorOnMainThread:#selector(loadHistorySuceeded:) withObject:operation waitUntilDone:YES];
} else {
[self performSelectorOnMainThread:#selector(loadHistoryFailed:) withObject:operation waitUntilDone:YES];
}
If the sleep happens at a bad point here, you have an object being passed to another thread. I'd find a way around having to pass the operation as the object.
This is a really old question, so sorry for the dredge, but there is no accepted answer.
I was also getting a EXC_BAD_ACCESS on NSOperationQueue -addOperation for seemingly no reason, and after a few days of hunting down memory leaks, and turning on all the debugger options i could find (malloc guard, zombies) and getting nothing, I found an NSLog warning that said: "[NSoperation subclass] set to IsFinished before being started by the queue."
When I modified my base operation subclass, so that its -cancel function only set (IsRunning = NO) and (IsFinished = YES) IF AND ONLY IF (IsRunning == YES), NSOperationQueue stopped crashing.
So if you're ever calling NSOperationQueue -cancelAllOperations, or you're doing that manually (i.e. for (NSOperation *op in queue.allOperations) ) double check to make sure that you don't set IsFinished on those operations in your subclass implementation.
I'm using Core Data and I need to loop thru the result of the request, create several custom objects in the loop and store them in a NSMUtableArray, so I can send it to another view to feed a UI component. This is what I'm doing:
NSMutableArray *persons = [[NSMutableArray alloc] init];
NSError *error = nil;
NSManagedObjectContext *context = [self managedObjectContext];
NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init];
NSEntityDescription *entity = [NSEntityDescription
entityForName:#"Person" inManagedObjectContext:context];
[fetchRequest setEntity:entity];
NSArray *fetchedObjects = [context executeFetchRequest:fetchRequest error:&error];
for (NSManagedObject *info in fetchedObjects) {
ToggleButtonInfo *btn = [[ToggleButtonInfo alloc] init];
NSString *personName = [NSString stringWithFormat:#"ww %#", [info valueForKey:#"name"]];
NSLog(#"pn: %#", personName);
[btn setButtonInfo:personName];
[persons addObject:btn];
}
[fetchRequest release];
return persons;
The loop is working just fine, the information is there. The problem is that I get a "EXC_BAD_ACCESS" in my component if I use:
[info valueForKey:#"name"]
if I do something like this:
[btn setButtonInfo:#"something else here"];
everything works fine. So it looks like info is been de-allocated and that is causing the error, right? I try creating the scring using stringWithFormat but it doesn't work, same error.
An ideas?
Where do you get the EXC_BAD_ACCESS? I assume it's later when you're displaying the button? -setButtonInfo: probably isn't retaining, or you're over-releasing somewhere else.
Note that you're leaking btn in this code.