I have a long running task that reads a file from the documents directory, parses it, deletes the file, then reads the next file from the doc directory, parses, deletes. I'm trying to put this code in the background but having trouble. I got so far as:
(pseudocode)
while (fileExistsInDocDirectory) {
NSOperationQueue *queue = [[NSOperationQueue alloc] init];
[queue setName:#"LongTask"];
[queue addOperationWithBlock:^{
parseSuccessful = [self doLongTask];
}];
[queue addOperationWithBlock:^{
[[NSOperationQueue mainQueue] addOperationWithBlock:^{
[self updateProgress];
}];
}];
}
It will work for one file only since I'm using FMDB my doLongTask is inserting rows into the db. So when the next queue comes in and tries to insert its data and the db is already in use, it doesn't work. Am I able to not run the 2nd queue until the first one finishes within a while loop? I saw some thing about setting up dependent queues, but wasn't sure if there was an easy way to do it without subclassing NSOperation. Thanks!
Related
I am writing an application that periodically fetches data from a web server using ASI HTTP and then processes that data to display something relevant to the user on the UI. The data is retrieved from different requests on a single server. The data itself needs to be processed in a specific order. One of the blocks of data is much bigger than the other ones.
In order not to lock the UI while the data is being processed, I have tried to use the NSOperationQueue to run the data processing on different threads. This works fine about 90% of the times. However, in the remaining 10% of the time, the biggest block of data is being processed on the main thread, which cause the UI to block for 1-2 seconds. The application contains two MKMapViews in different tabs. When both MKMapViews tabs are loaded the percentage of time the biggest block of data is being processed on the main thread increases above 50% (which seems to point to the assumption that this happens when there is more concurrent activity).
Is there a way to prevent the NSOperationQueue to run code on the main thread?
I have tried to play with the NSOperationQueue –setMaxConcurrentOperationCount:, increasing and decreasing it but there was no real change on the issue.
This is the code that starts the periodic refresh:
- (void)refreshAll{
// Create Operations
ServerRefreshOperation * smallDataProcessor1Op = [[ServerRefreshOperation alloc] initWithDelegate:_smallDataProcessor1];
ServerRefreshOperation * smallDataProcessor2Op = [[ServerRefreshOperation alloc] initWithDelegate:_smallDataProcessor2];
ServerRefreshOperation * smallDataProcessor3Op = [[ServerRefreshOperation alloc] initWithDelegate:_smallDataProcessor3];
ServerRefreshOperation * smallDataProcessor4Op = [[ServerRefreshOperation alloc] initWithDelegate:_smallDataProcessor4];
ServerRefreshOperation * smallDataProcessor5Op = [[ServerRefreshOperation alloc] initWithDelegate:_smallDataProcessor5];
ServerRefreshOperation * hugeDataProcessorOp = [[ServerRefreshOperation alloc] initWithDelegate:_hugeDataProcessor];
// Create dependency graph (for response processing)
[HugeDataProcessorOp addDependency:smallDataProcessor4Op.operation];
[smallDataProcessor5Op addDependency:smallDataProcessor4Op.operation];
[smallDataProcessor4Op addDependency:smallDataProcessor3Op.operation];
[smallDataProcessor4Op addDependency:smallDataProcessor2Op.operation];
[smallDataProcessor4Op addDependency:smallDataProcessor1Op.operation];
// Start be sending all requests to server (startAsynchronous directly calls the ASIHTTPRequest startAsynchronous method)
[smallDataProcessor1Op startAsynchronous];
[smallDataProcessor2Op startAsynchronous];
[smallDataProcessor3Op startAsynchronous];
[smallDataProcessor4Op startAsynchronous];
[smallDataProcessor5Op startAsynchronous];
[hugeDataProcessorOp startAsynchronous];
}
This is the code that sets the ASI HTTP completion block that starts the data processing:
[_request setCompletionBlock:^{
[self.delegate setResponseString:_request.responseString];
[[MyModel queue] addOperation:operation]; // operation is a NSInvocationOperation that calls the delegate parse method
}];
I have added this block of code in all NSInvocationOperation Invoked method at the entry point:
if([NSThread isMainThread]){
NSLog(#"****************************Running <operation x> on Main thread");
}
The line is printed every time the UI freezes. This shows that the whole operation is run on the main thread. It is actually always the hugeDataProcessorOp that is run on the main thread. I assume that this is because it is the operation that always receives its answer last from the server.
After much investigation in my own code, I can confirm that this was a coding error.
There was an old call remaining that did not go through the NSInvocationOperation but was calling the selector that NSInvocationOperation should have called directly (therefore not using the concurrent NSOperationQueue.
This means that the NSOperationQueue DOES NOT use the main thread (except if it is the one retrieved by +mainQueue).
Override isConcurrent on your NSOperation and return YES. According to the documentation this will cause your NSOperation to be run asynchronously.
I need in my application to download directories and their content. So I decided to implement a NSOperationQueue and I subclassed NSOperation to implement NSURLRequest etc...
The problem is I add all the operations at once and I can't figure out when all the files for one directory are downloaded in order to update the UI and enable this specific directory.
Now I have to wait that all the files from all the directories are downloaded in order to update the UI.
I already implemented key-value observing for the operationCount of the NSOperationQueue and the isFinished of the NSOperation but I don't know when a directory has all the files in it !
Do you have any idea ?
Thanks a lot
Add a "Done" NSOperation which has all other NSOperations for one directory as dependency.
Something like this:
NSInvocationOperation *doneOp = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(done:) object:nil];
NSInvocationOperation *op1 = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(doSomething:) object:nil];
[queue addOperation:op1];
[doneOp addDependency:op1];
NSInvocationOperation *op2 = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(doSomething:) object:nil];
[queue addOperation:op2];
[doneOp addDependency:op2];
NSInvocationOperation *op3 = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(doSomething:) object:nil];
[queue addOperation:op3];
[doneOp addDependency:op3];
[queue addOperation:doneOp];
doneOp will only run after op1, op2 and op3 have finished executing.
[opQueue operationCount]
Hope this helps
One approach would be to create some sort of Directory class with a properties such as loadedCount (initially 0) and fileCount (initialized to however many files are in the directory) and create a dictionary mapping each NSOperation to the appropriate Directory before adding the operation to the queue. Inside your callback for isFinished, you could pull the Directory object for the given NSOperation out of the dictionary and increment the directory.loadedCount by 1. If your directory.loadedCount == directory.fileCount, trigger an update to the UI.
You can refactor your code to avoid enqueuing all requests at once. Enqueue only requests for a single directory at a time. When operationCount reaches zero, you can be sure that all the requests either completed or failed, update the UI and enqueue the requests for the next directory. Proceed until the array of directories is empty.
The advantages are:
relative simplicity
you don't have to query the file system only to figure out what has been downloaded
if need be, you can re-enqueue failed requests without changing other logic.
I want to handle freezing my program, when it load an xml from bad address.
I tryed it with using #try and #catch, but it doesn't work.
Can I use some alternative handling?
#try{
NSString *test=[[NSString alloc] initWithContentsOfURL:[NSURL URLWithString:[NSString stringWithFormat:#"http://%#:%#",addressLabel.text,portLabel.text]] encoding:NSUTF8StringEncoding error: nil];
}
#catch (NSException *ex) {
NSLog(#"Bad IP address");
return;
}
Run your XML Parser in NSThread and use notification for errors.
initWithContentsOfURL is a synchronous call. The control will return back from the function only on complete. Try this function is a worker thread so that your main thread will not be blocked.
If you use use NSThread then you have to dive into the memory management unless you are working in XCode 4.2 and using ARC.
So there are two ways for fetching the XML from the server.
1) Use NSURLConnection to get the xml as a NSData object and when you finish loading the data you can simply use that data to initialize an NSString Object. NSURLConnection sends asynchronous call to the server so it will not freeze your view.
2) You can use NSIncovationOperation and NSQueue to fetch your XML and it will also not effect your main thread. like
-(void)myMethod{
NSString *test=[[NSString alloc] initWithContentsOfURL:[NSURL URLWithString:[NSString stringWithFormat:#"http://%#:%#",addressLabel.text,portLabel.text]] encoding:NSUTF8StringEncoding error: nil];
[self performSelectorOnMainThread:#selector(handleString:) withObject:test];
}
You can use NSInvocationOperation object as follow
NSInvocationOperation *opr = [[NSInvocationOperation alloc] initWithTarget:self selector:#selector(myMethod) object:nil];
NSOperationQueue *queue= [NSOperationQueue new];
[queue addOperation:opr];
When the perform selector will be call then you can pass that XML string object to the handleString: method. like
-(void)handleString:(NSString*)xmlString{
// Do something with string
}
I hope that it clarifies a little bit of your confusion. All this was to give you an idea how can you achieve your goal without freezing your interface i.e main thread.
regards,
Arslan
You need to launch all long time operations on a second thread to avoid blocking the main thread. Use [self performSelector:#selector(yourXmlDownloadMethod)].
In my app i have a large number , around 70000 records to load in a tableview. It takes a lot of time to load like ten minutes. Since it blocks the main UIthread, I am unable to go back or access any buttons. Is there any alternate way like using a separate thread for this purpose or any alternate approach ? Please show me some way.
Thanks,
Vinod.
Use a NSThread.
Your code will look something along the lines of:
NSThread *thread = [NSThread initWithTarget:self selector:#selector(loadData:) object:nil];
[thread start];
[thread release];
-(void) loadData:(id) obj {
NSAutoreleasePool *pool = [[NSAutoreleasePool alloc] init];
// load data
[pool release];
}
If you need to do anything on the main UI thread from the newly created thread, use the performSelectorOnMainThread:withObject method on the current object.
I am trying to create class that will handle multiple downloads at same time (I need to download a lot of small files) and I have problems with "disappearing" connections.
I have function addDonwload that adds url to list of urls to download, and checks if there is free download slot available. If there is one it starts download immediately. When one of downloads finishes, I pick first url form list and start new download.
I use NSURLConnection for downloading, here is some code
- (bool) TryDownload:(downloadInfo*)info
{
int index;
#synchronized(_asyncConnection)
{
index = [_asyncConnection indexOfObject:nullObject];
if(index != NSNotFound)
{
NSLog(#"downloading %# at index %i", info.url, index);
activeInfo[index] = info;
NSURLRequest *request = [NSURLRequest requestWithURL:info.url cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:15];
[_asyncConnection replaceObjectAtIndex:index withObject:[[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:TRUE]];
//[[_asyncConnection objectAtIndex:i] scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
return true;
}
}
return false;
}
- (void)connectionDidFinishLoading:(NSURLConnection*)connection
{
[self performSelectorOnMainThread:#selector(DownloadFinished:) withObject:connection waitUntilDone:false];
}
- (void)DownloadFinished:(id)connection
{
NSInteger index = NSNotFound;
#synchronized(_asyncConnection)
{
index = [_asyncConnection indexOfObject:(NSURLConnection*)connection];
}
[(id)activeInfo[index].delegate performSelectorInBackground:#selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
[_data[index] release];
[activeInfo[index].delegate release];
#synchronized(_asyncConnection)
{
[[_asyncConnection objectAtIndex:index] release];
[_asyncConnection replaceObjectAtIndex:index withObject:nullObject];
}
#synchronized(downloadQueue)
{
[downloadQueue removeObject:activeInfo[index]];
[self NextDownload];
}
}
- (void)NextDownload
{
NSLog(#"files remaining: %i", downloadQueue.count);
if(downloadQueue.count > 0)
{
if([self TryDownload:[downloadQueue objectAtIndex:0]])
{
[downloadQueue removeObjectAtIndex:0];
}
}
}
_asyncConnection is my array of download slots (NSURLConnections)
downloadQueue is list of urls to download
What happens is, at the beginning everything works ok, but after few downloads my connections start to disappear. Download starts but connection:didReceiveResponse: never gets called. There is one thing in output console that I don't understand I that might help a bit. Normaly there is something like
2010-01-24 21:44:17.504 appName[3057:207]
before my NSLog messages. I guess that number in square brackets is some kind of app:thread id? everything works ok while there is same number, but after some time, "NSLog(#"downloading %# at index %i", info.url, index);" messages starts having different that second number. And when that happens, I stop receiving any callbacks for that urlconnection.
This has been driving me nuts as I have strict deadlines and I can't find problem. I don't have many experiences with iphone dev and multithreaded apps. I have been trying different approaches so my code is kinda messy, but I hope you will see what I am trying to do here :)
btw is anyone of you know about existing class/lib I could use that would be helpful as well. I want parallel downloads with ability o dynamically add new files to download (so initializing downloader at the beginning with all urls is not helpful for me)
You've got a bunch of serious memory issues, and thread synchronization issues in this code.
Rather than go into them all, I'll ask the following question: You are doing this on a background thread of some kind? Why? IIRC NSURLConnection already does it's downloads on a background thread and calls your delegate on the thread that the NSURLConnection was created upon (e.g., your main thread ideally).
Suggest you step back, re-read NSURLConnection documentation and then remove your background threading code and all the complexity you've injected into this unnecessarily.
Further Suggestion: Instead of trying to maintain parallel positioning in two arrays (and some sketchy code in the above relating to that), make one array and have an object that contains both the NSURLConnection AND the object representing the result. Then you can just release the connection instance var when the connection is done. And the parent object (and thus the data) when you are done with the data.
I recommend that you take a look at this:
http://allseeing-i.com/ASIHTTPRequest/
It's a pretty sophisticated set of classes with liberal licensing terms (free too).
It may provide a lot of the functionality that you are wanting.
This snippet can be the source of the bug, you release the object pointed to by the activeInfo[index].delegate pointer right after issuing async method call on that object.
[(id)activeInfo[index].delegate performSelectorInBackground:#selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
[_data[index] release];
[activeInfo[index].delegate release];
Do you use connection:didFailWithError: ? There may be a timeout that prevents the successful download completion.
Try to get rid of the #synchronized blocks and see what happens.
The string inside the square brackets seems to be thread identifier as you guessed. So maybe you get locked in the #synchronized. Actually, I don't see a reason for switching thread - all the problematic code should run in the main thread (performSelectorOnMainThread)...
Anyhow, there is no need to use both the #synchronized and the performSelectorOnMainThread.
BTW, I didn't see the NSURLConnection *connection = [[NSURLConnection alloc] initWithRequest:request delegate:self]; line. Where do you initiate the connection?
As for the parallel downloads - I think that you can download more than one file in a time with the same code that you use here. Just create a separate connection for each download.
Consider just keeping a download queue along with a count of active connections, popping items off the top of the queue when downloads complete and a slot becomes free. You can then fire off NSURLConnection objects asynchronously and process events on the main thread.
If you find that your parallel approach prohibits doing all of the processing on the main thread, consider having intermediary manager objects between your main thread download code and NSURLConnection. Using that approach, you'd instantiate your manager and get it to use NSURLConnection synchronously on a background thread. That manager then completely deals with the downloading and passes the result back to its main thread delegate using a performSelectorOnMainThread:withObject: call. Each download is then just a case of creating a new manager object when you've a slot free and setting it going.