I want to download a large file (> 500MB) to my application from the server. I used NSURLConnection, that works well if the network is very good. but sometimes I tried to download 500MB file, but 200MB or 100MB only downloaded if the network is not very good.That means I got the connectionDidFinishLoading method when the task was not completed.Someone says set a timeout second to avoid this situation，but i set timeout second 30s，it did not work.Should I set 60s or more？ Does someone have better idea,please help me.
in connectionDidFinishLoading method every time check the length of data to download and the downloaded data.
length of the data to be download is gain by this [response expectedContentLength]; in didReceiveResponse method
You should download such big file in parts. Specify the Content-Range field in the header of your HTTP request and ask only for a small portion of the file at once. When you get all portions, you can assemble the file together.
You can set HTTP headers with NSMutableURLRequest setValue:#"0-1023/*" forHTTPHeaderField:#"Content-Range"];, this example downloads only the first kbyte of the file. See also Content-Range in http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
Related
I have a Golang web server that I have written to handle large file uploads 30GB or more. In a proof of concept using Dropzone.js I can upload files of any size with no issue as long as they are chunked.
The way DropzoneJS.js implemented this is that each chunk has items added the to the header like:
dzchunkindex: 435
dzchunksize: 10000
dztotalchunkcount: 3498274
So I receive a chunk, I create the file (if needed), write the data, and check to see if I'm on the last chunk. Then repeat as needed. Once I see I've written the last chunk I close the file.
It seems like Alamofire supports chunked uploads using its AF.Upload method.
However, how should my server know when the last chunk has been uploaded? I can certainly check this a different way. Just curious what that way should be? Ive combed over the Alamofire docs and can't find much.
I can chunk the file manually and upload it but id rather use Alamofire if possible.
Thanks,
Ed
I've developed a method that does the following steps, in this order:
1) Get a report's metadata via /gdc/md//obj/
2) From that, get the report definition and use that as payload for a call to /gdc/xtab2/executor3
3) Use the result from that call as payload for a call to /gdc/exporter/executor
4) Perform a GET on the returned URI to download the generated CSV
So this all works fine, but the problem is that I often get back a blank CSV or an incomplete CSV. My workaround has been to put a sleep() in between getting the URI back and actually calling a GET on the URI. However, as our data grows, I have to keep increasing the delay on this, and even then it is no guarantee that I got complete data. Is there a way to make sure that the report has finished exporting data to the file before calling the URI?
The problem is that export runs as asynchronous task - result on the URL returned in payload of POST to /gdc/exporter/executor (in form of /gdc/exporter/result/{project-id}/{result-id}) is available after exporter task finishes its job.
If the task has not been done yet, GET to /gdc/exporter/result/{project-id}/{result-id} should return status code 202 which means "we are still exporting, please wait".
So you should periodically poll on the result URL until it returns status 200 which will contain a payload (or 40x/50x if something wrong happened).
OK, here's a goal I've been looking for a while.
As it's known, most advertising and analytics companies use a so called "pixel" code in order to track websites views, transactions, conversion etc.
I do have a general idea on how it works, the problem is how to implement it. The tracking codes consist from few parts.
The tracking code itself.
This is the code that the users inserts on his webpage in the <head> section. The main goal of this code is to set some customer specific variables and to call the *.js file.
*.js file.
This file holds all the magic of CRUD (create/read/update/delete) cookies, track user's events and interaction with the webpage.
The pixel code.
This is an <img> tag with the src atribute pointing to an image *.gif (for example) file that takes all the parameters collected on the page, and stores them in the database.
Example:
WordPress pixel code: <img id="wpstats" src="http://stats.wordpress.com/g.gif?host=www.hostname.com&list_of_cookies_value_pairs;" alt="">
Google Analitycs:
http://www.google-analytics.com/__utm.gif?utmwv=4&utmn=769876874&etc
Now, it's obvious that the *.gif request has to reach a server side scripting language in order to read the parameters data and store them in a db.
Does anyone have an idea how to implement this in Zend?
UPDATE
Another thing I'm interested in is: How to avoid the user's browser to load the cached *.gif ? Will a random parameter value do the trick? Example: src="pixel.gif?nocache=random_number" where the nocache parameter value will be different on every request.
As Zend is built using PHP, it might be worth reading the following question and answer: Developing a tracking pixel.
In addition to this answer and as you're looking for a way of avoiding caching the tracking image, the easiest way of doing this is to append a unique/random string to it, which is generated at runtime.
For example, server-side and with the creation of each image, you might add a random URL id:
<?php
// Generate random id of min/max length
$rand_id = rand(8, 8);
// Echo the image and append a random string
echo "<img src='pixel.php?a=".$vara."&b=".$varb."&rand=".$rand_id."'>";
?>
Just adding my 2 cents to this thread because I think an important, and frequently used, option is missing: you don't necessarily need a scripting language to capture the request. A more efficient approach is to use the web server access log (like apache access log for instance) to log the request and then handle that log with whatever tools you see fit, like ELK stack for instance.
This makes serving the requests much lighter because no scripting language is loaded to prepare the response, just native apache response, which is typically much more efficient.
First of all, the *.gif doesn't need to be that file type, the only thing that is of interest is the Content-Type http header. Set that to image/gif (or any other, appropiate type) in the beginning, execute your code and render some sort of image to the response body.
Well, all of the above codes are correct and is good but to be certain, the guy above mention "g.gif"
You can just add a simple php code to write to an sql or fwrite("file.txt",$opened)
where var $opened serves as the counter++ if someone opened your mail... then save it as "g.gif"
TO DO all of this just add these:
<Files "/thisdirectory">
AddType application/x-httpd-php .gif
</Files>
to your ".htaccess" file but be sure to make a new directory for that g.gif or whatever.gif where the directory only contains g.gif and .htaccess
I'm out in the woods with this one: I have a universal, navigation-based app that displays data currently stored in a plist file. In a future release, I want to migrate the database to a JSON file on my server which the app can download to it's bundle, then parse it. Can anyone suggest a simple light-weight way of checking that the currently stored file in the bundle matches the version hosted on the server? Essentially checking for updates to the db without re-downloading the entire JSON file.
Here's a snippet of what the beginning of the JSON file currently looks like.
{
"version" : "0.2",
"description" : "1. Corrections to several entries.\n2. Added 21 new departments from Alameda & Fresno Counties.",
"counties" : {............... *Rest of the JSON file here* .....}
My idea was to store the "version" ("0.2") value to NSUserDefaults and use that value to check against the available JSON file online every time the app launches.
Am I on the right track or is there a better way of doing this altogether?
Thank you
Romeo
You can add the If-Modified-Since header to an instance of NSMutableURLRequest. If the document on the server has changed since that date, you'll get the data back. If it hasn't changed, you get a 304 Not Modified and no data.
This is much better than making a HEAD request because in the event of an updated file, you're only making one request instead of two.
Do a HEAD request (instead of a GET) and check the LastModified header. if the file has been modified since the last time you checked, download the file. Save the modified time somewhere to compare against next time.
You can set the http method on the request object like so:
[request setHTTPMethod:#"HEAD"];
Alright. Hopefully this will be my last post about the download manager I am writing in Objective-C. Everything seems to work well except the pause/resume functionality. My issue is that when a download tries to continue from where it left off, it appends the data it receives to the file, but it still seems that it's trying to download the entire file. This results in a file that is larger than the original file is supposed to be. Here is the code I am using for downloading files. Am I doing something wrong?
-(void)start:(unsigned int)fromByte {
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:[NSURL URLWithString:self.url] cachePolicy:NSURLRequestUseProtocolCachePolicy timeoutInterval:DEFAULT_TIMEOUT];
// Define the bytes we wish to download.
NSString *range = [NSString stringWithFormat:#"bytes=%i-", fromByte];
[request setValue:range forHTTPHeaderField:#"Range"];
// Data should immediately start downloading after the connection is created.
self.urlConnection = [[NSURLConnection alloc] initWithRequest:request delegate:self startImmediately:TRUE];
if (!self.urlConnection) {
#warning Handle error.
}
}
I see you are specifying the Range header in the request. First thing to check is whether the server is actually honoring the Range request, by checking the headers in the response object (which should be an NSHTTPURLResponse) in connection:didReceiveResponse: for a proper Content-Range.
I finally figured this out. It turns out that the 'getFilesizeInBytes' method I had was get the NSFileSize object from the file's attributes, but I was directly casting this to an int. This caused the number to be about 20 times larger than it should have been. I was able to fix this by using [#"" intValue]. Once this was fixed, the servers were able to give me the rest of the file starting with the correct byte. It seems that before my issue was not that the server wasn't honoring my request, but that it couldn't honor my request due to me requesting data that was well beyond the final byte of the file.
There's no support for pause/resume in NSURLConnection. You can emulate it by stopping the request, then issuing a request for the rest of the content with a Range header on resume. Some support from the HTTP server is required for that, and is not guaranteed.
Looks like lack of support at the server is what you're facing.
Most download managers, however, implement their own HTTP stack on top of sockets.