I am trying to download files from remote, and I now can monitor every single files download success status
[operation setCompletionBlockWithSuccess:^(AFHTTPRequestOperation *operation, id responseObject) {
But any way to monitor the whole process of downloading? How should I know all downloads are finished?
And I tried start download request with
[afhttpClient enqueueBatchOfHTTPRequestOperations:operationArray
progressBlock:progressBlock
completionBlock:completionBlock];
seems not work, so what the difference between above code and start download request with [operation start] in a loop?
#mattt (if you can see this)
You most probably have an NSArray of URL objects that you use to download images. What you need to do is to create an integer value equal to the count of your URL objects. Each time you successfully download an image or absolutely fail to download it (for instance after few timeouts or upon receiving 404 HTTP status code) you need to decrement that integer (note that it should be an atomic property, since blocks are being executed on different threads). Once the count reaches zero - all requests are finished. You can also use that integer to update a progress bar or simply notify user that "#/15 images are downloaded". Let me know if you need any other clarifications.
And unfortunately I have not worked with AFHTTPClient, so I can't tell you the difference between the two operations precisely, but contextually, first one executes all the requests almost at the same time asynchronously and the latter one uses consecutive approach, where second request will only be launched upon completion of the first one.
You can create NSOperationQueue and put all AFHTTPRequestOperation into it.
To observe the "operations" by using KVO. When the count go to zero, that is the time to say all operations completed.
If what you want is a constant progress update for each operation with bytes downloaded and total expected then I can highly recommend Peter Steinberger's AFDownloadRequestOperation.
This class derives from AFHTTPRequestOperation and adds a progressiveDownloadProgressBlock per-operation rather than just a per-operation-completion progress at the operation queue level, which is what I think you're looking for. Another great bonus is that it makes resumable/partial downloads much more accessible than in the core AFNetworking implementation.
it's this easy to use (example from the GitHub project's README.md):
[pdfRequest setProgressiveDownloadProgressBlock:^(NSInteger bytesRead,
long long totalBytesRead, long long totalBytesExpected,
long long totalBytesReadForFile, long long totalBytesExpectedToReadForFile)
{
self.downloadProgress = totalBytesReadForFile/(float)totalBytesExpectedToReadForFile;
}];
I use this in a few enterprise iOS projects to download multi-gigabyte files and I can tell you that it works great with the 1.0.1 release of AFNetworking.
Hope that helps…
Related
I am struggeling with something and I would like to ask you if you could point me in the right direction.
I have four tasks I want to complete, -one after the other.
Fetch html-code from web
Parse this code and save to core data storage
Use this data and batch save to calendar
Upload the parsed data to my own web server.
I have written all the code for this and it executes fine. However, at times it struggles as some of the code is executed before the other has finished.
Example:
func startProcess () {
fetchHTMLFromWeb()
parseHTML()
saveToCalendar()
//Sometimes uploadToWeb() starts before saveToCalendar() is finished
uploadToWeb()
}
I have tried reading up on GCD, but it is a rather complex subject and I am finding it hard to grasp it.
Can you recommend any good readups on this subject?
Thank you very much!
You can use the GCD to execute all your stuffs in the background queue.
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_DEFAULT, 0), ^{
self.startProcess();
});
with that, startProcess will start on the background queue/thread. So you can
In the fetchHtmlFromWeb method just call parseHtml(), when the fetch is ended.
hope it helps.
I'm using libspotify SDK, C library for win32.
I think to have a right setup, every session callback is registered. I don't understand why i can't receive the call for end_of_track, while music_delivery continues to be called with zero padding 22050 long frames.
I attempt to start playing first loading the track with sp_session_load; till it returns SP_ERROR_IS_LOADING I post a message on my message queue (synchronization method I've used, PostMessage win32 API) in order to reload again with same API sp_session_load. As soon as it returns SP_ERROR_OK I use the sp_session_play and the music_delivery starts immediately, with correct frames.
I don't know why at the end of track the libspotify runtime then start sending zero padded frames, instead of calling end_of_track callback.
In other conditions it works perfectly: I've used the sp_track obtained from a album browse, so the track is fully loaded at the moment I load to the current session for playing: with this track, it works fine with end_of_track called correctly. In the case with padding error, I search the track using its Spotify URI and got the results; in this case the track metadata are not still ready (at the play attempt) so I used that kind of "polling" on sp_session_load with PostMessage.
Can anybody help me?
I ran into the same problem and I think the issue was that I was consuming the data too fast without giving other threads time to do any work since I was spending all of my time in the music_delivery callback. I found that if I add some throttling and notify the main thread that it can wake up to do some processing, the extra zeros at the end of track is reduced to one delivery of 22,050 frames (or 500ms at 44.1kHz).
Here is an example of what I added to my callback, heavily borrowed from the jukebox.c example provided with the SDK:
/* Buffer 1 second of data, then notify the main thread to do some processing */
if (g_throttle > format->sample_rate) {
pthread_mutex_lock(&g_notify_mutex);
g_notify_do = 1;
pthread_cond_signal(&g_notify_cond);
pthread_mutex_unlock(&g_notify_mutex);
// Reset the throttle counter
g_throttle = 0;
return 0;
}
As I said, there was still 22,050 frames of zeros delivered before the track stopped, but I believe libspotify may purposely do this to ensure that the duration calculated by the number of frames received (song_duration_ms = total_frames_delivered / sample_rate * 1000) is greater than or equal to the duration reported by sp_track_duration. In my case, the track I was trying to stream was 172,000ms in duration, without the extra padding the duration calculated is 171,796ms, but with the padding it was 172,296ms.
Hope this helps.
I'm trying to figure out how to make MKNetworkKit working with data from stream. I can see that some data is beeing downloaded (the indicator on status bar), but I don't have any idea what happens with that data after it's actually downloaded. I put a NSLog statement inside body of connection: didReceiveData: but it's not called during streaming. Any pointers how to fix that issue ?
Edit
Sorry my question was inaccurate. I know how to stream to a file but I need to stream to memory (NSData instance preferably). Okay it seems simple again due to NSOutputStream method initWithBytes:capacity:. And my problem is here, my stream has undefined length so there would be enormous impact on memory. I don't know what to do. My perfect solution works like this. Small chunks of data from the stream are processed having been downloaded and then they are discarded.
You could use the outputStreamToBuffer:capacity: method to create the stream.
As for the buffer, you can use a circular buffer, so that you can read from it as the stream writes to it. A great implementation (and explanation) is here.
Streaming a file download is a three line magic with MKNetworkKit.
//Create a MKNetworkOperation for the remote URL.
MKNetworkOperation *op = [self operationWithURLString:remoteURL
params:nil
httpMethod:#"GET"];
// add your output stream, in this case a file
[op addDownloadStream:[NSOutputStream outputStreamToFileAtPath:filePath
append:YES]];
// enqueue the operation to a MKNetworkEngine.
[self enqueueOperation:op];
That's it.
Is it possible to know how many requests finished in ASINetworkQueue?
Question is very straight forward.. Please see below example..
Example
Suppose there are number of ASIHttpRequests in ASINetworkQueue and if all requests are running and in between I am cancelling all operations in ASINetworkQueue then how do I know that how many requests are finished before cancel ?
Please help me to solve this question or just tell me that it is possible or not to count this?
Thanks in advance..
Let networkQueue be the object of your networkQueue. You may use
[networkQueue requestsCount];
It returns the number of operations pending in Network queue and on completion of each operations it subtracts the requestCount property. Thus by subtracting it From total requests you added in network queue , you may get the number of completed requests of networkQueue
If you are using NSOperationQueue then following code can be used for getting count and to cancel all operations (requests) in the queue.
if([self.serviceQueue operationCount])
{
[self.serviceQueue cancelAllOperations];
self.serviceQueue = nil;
}
Its a very old thread but I am posting for future considerations. Please let me know if any one is facing any other issue.
I am trying to create class that will handle multiple downloads at same time (I need to download a lot of small files) and I have problems with "disappearing" connections.
I have function addDonwload that adds url to list of urls to download, and checks if there is free download slot available. If there is one it starts download immediately. When one of downloads finishes, I pick first url form list and start new download.
I use NSURLConnection for downloading, here is some code
- (bool) TryDownload:(downloadInfo*)info
{
int index;
#synchronized(_asyncConnection)
{
index = [_asyncConnection indexOfObject:nullObject];
if(index != NSNotFound)
{
NSLog(#"downloading %# at index %i", info.url, index);
activeInfo[index] = info;
NSURLRequest *request = [NSURLRequest requestWithURL:info.url cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:15];
[_asyncConnection replaceObjectAtIndex:index withObject:[[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:TRUE]];
//[[_asyncConnection objectAtIndex:i] scheduleInRunLoop:[NSRunLoop currentRunLoop] forMode:NSDefaultRunLoopMode];
return true;
}
}
return false;
}
- (void)connectionDidFinishLoading:(NSURLConnection*)connection
{
[self performSelectorOnMainThread:#selector(DownloadFinished:) withObject:connection waitUntilDone:false];
}
- (void)DownloadFinished:(id)connection
{
NSInteger index = NSNotFound;
#synchronized(_asyncConnection)
{
index = [_asyncConnection indexOfObject:(NSURLConnection*)connection];
}
[(id)activeInfo[index].delegate performSelectorInBackground:#selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
[_data[index] release];
[activeInfo[index].delegate release];
#synchronized(_asyncConnection)
{
[[_asyncConnection objectAtIndex:index] release];
[_asyncConnection replaceObjectAtIndex:index withObject:nullObject];
}
#synchronized(downloadQueue)
{
[downloadQueue removeObject:activeInfo[index]];
[self NextDownload];
}
}
- (void)NextDownload
{
NSLog(#"files remaining: %i", downloadQueue.count);
if(downloadQueue.count > 0)
{
if([self TryDownload:[downloadQueue objectAtIndex:0]])
{
[downloadQueue removeObjectAtIndex:0];
}
}
}
_asyncConnection is my array of download slots (NSURLConnections)
downloadQueue is list of urls to download
What happens is, at the beginning everything works ok, but after few downloads my connections start to disappear. Download starts but connection:didReceiveResponse: never gets called. There is one thing in output console that I don't understand I that might help a bit. Normaly there is something like
2010-01-24 21:44:17.504 appName[3057:207]
before my NSLog messages. I guess that number in square brackets is some kind of app:thread id? everything works ok while there is same number, but after some time, "NSLog(#"downloading %# at index %i", info.url, index);" messages starts having different that second number. And when that happens, I stop receiving any callbacks for that urlconnection.
This has been driving me nuts as I have strict deadlines and I can't find problem. I don't have many experiences with iphone dev and multithreaded apps. I have been trying different approaches so my code is kinda messy, but I hope you will see what I am trying to do here :)
btw is anyone of you know about existing class/lib I could use that would be helpful as well. I want parallel downloads with ability o dynamically add new files to download (so initializing downloader at the beginning with all urls is not helpful for me)
You've got a bunch of serious memory issues, and thread synchronization issues in this code.
Rather than go into them all, I'll ask the following question: You are doing this on a background thread of some kind? Why? IIRC NSURLConnection already does it's downloads on a background thread and calls your delegate on the thread that the NSURLConnection was created upon (e.g., your main thread ideally).
Suggest you step back, re-read NSURLConnection documentation and then remove your background threading code and all the complexity you've injected into this unnecessarily.
Further Suggestion: Instead of trying to maintain parallel positioning in two arrays (and some sketchy code in the above relating to that), make one array and have an object that contains both the NSURLConnection AND the object representing the result. Then you can just release the connection instance var when the connection is done. And the parent object (and thus the data) when you are done with the data.
I recommend that you take a look at this:
http://allseeing-i.com/ASIHTTPRequest/
It's a pretty sophisticated set of classes with liberal licensing terms (free too).
It may provide a lot of the functionality that you are wanting.
This snippet can be the source of the bug, you release the object pointed to by the activeInfo[index].delegate pointer right after issuing async method call on that object.
[(id)activeInfo[index].delegate performSelectorInBackground:#selector(backgroundDownloadSucceededWithData:) withObject:_data[index]];
[_data[index] release];
[activeInfo[index].delegate release];
Do you use connection:didFailWithError: ? There may be a timeout that prevents the successful download completion.
Try to get rid of the #synchronized blocks and see what happens.
The string inside the square brackets seems to be thread identifier as you guessed. So maybe you get locked in the #synchronized. Actually, I don't see a reason for switching thread - all the problematic code should run in the main thread (performSelectorOnMainThread)...
Anyhow, there is no need to use both the #synchronized and the performSelectorOnMainThread.
BTW, I didn't see the NSURLConnection *connection = [[NSURLConnection alloc] initWithRequest:request delegate:self]; line. Where do you initiate the connection?
As for the parallel downloads - I think that you can download more than one file in a time with the same code that you use here. Just create a separate connection for each download.
Consider just keeping a download queue along with a count of active connections, popping items off the top of the queue when downloads complete and a slot becomes free. You can then fire off NSURLConnection objects asynchronously and process events on the main thread.
If you find that your parallel approach prohibits doing all of the processing on the main thread, consider having intermediary manager objects between your main thread download code and NSURLConnection. Using that approach, you'd instantiate your manager and get it to use NSURLConnection synchronously on a background thread. That manager then completely deals with the downloading and passes the result back to its main thread delegate using a performSelectorOnMainThread:withObject: call. Each download is then just a case of creating a new manager object when you've a slot free and setting it going.