I have an iOS app that I'm migrating from the very slow and clunky SOAP to a custom data format (basically CSV with some extra bits).
My priority is getting initial data to the client as quickly as possible while letting it still load more in the background. The server side is written to continuously flush data instead of caching the response.
So I'd like to parse out every line as they arrive at the client, instead of waiting for the full response.
If I view it in a browser I get progressive loading. However, using MKNetworkKit or ASIHTTPRequest or similar, I'm only able to get the full response which takes several seconds longer.
Does anyone know what the best options could be?
NSURLconnection can do what you want. You set the delegate and use -connection:didWriteData:totalBytesWritten:expectedTotalBytes: callback to read in a chunk of the data as it's downloading.
It will be up to you to properly handle splitting up the lines and handling chunks containing partial lines.
Related
I need to send a stream of data to Play server. The length of the stream is unknown and I need to get a response every line break \n or for every several lines. Rather then wait for the whole data to be sent.
Think of the following usecase:
lets say i'm intended to write a console application, that when launched, connects to my web server, and all the user input are being sent to play on every line break, and gets responded asynchronously. All above should be performed on a single connection, i.e. I don't want to open a new connection on every request I send to Play (a good analog would be 2 processes communicating through 2 pipes).
What is the best way to achieve this?
And is it possible to achieve with a client that communicates with the server only via http (with a single http connection)?
EDIT:
my current thoughts on how to approach this are as follows:
i can define a new BodyParser[Future[String]] which is basically an Iteratee[Array[Byte],Future[String]]. while the parsing takes place, i can compute the result asynchronously and the action can return the result as ChunkedResult in the future's onComplete method.
does this sound like the right approach?
any suggestions on how to achieve this?
Maybe you should look at websockets.
Java: http://www.playframework.com/documentation/2.1-RC3/JavaWebSockets
Scala: http://www.playframework.com/documentation/2.0/ScalaWebSockets
I'm trying to stream data between two iOS NSURLConenctions (one downloading, the other uploading), but can't seem to make it work.
I have two NSURLConnections running simultaneously.
One is downloading content from a url using a GET request.
The other is uploading the same content just received, to another url, in the request body of a PUT request.
In the upload connection I'm using setHTTPBodyStream to specify a custom NSInputStream whose read method returns data previously received from the other connection.
Both NSURLConnections are scheduled in the run loops of separate background threads, so that any (possibly blocking) delegate callbacks don't mess with each other (and neither with the main thread).
So I thought it would work like this:
The upload connection calls [read:maxLength] (which I have overridden) on the input stream.
Since there's no data available yet, the read call blocks.
On another thread, [connection:didReceiveData:] is called on the delegate of the download connection.
It puts the received data in a shared buffer, thus making it available for the input stream of the upload connection.
The upload stream's read call now isn't blocked anymore, it can return a chunk of data.
Unfortunately in practice, this does not work. After the upload stream's read method blocks, the download connection's delegate methods (eg. didReceiveData) don't get called anymore. (Note that if I disable the blocking on the upload side, then didReceiveData on the download side does get called all right.)
I suspect that this has to do something with the fact that the upload input stream's read method is called not on the thread where the connection and the stream objects were created, but on some other thread (apparently created by Cocoa). As if this was some shared thread used by both NSURLConnections, so once it's blocked, all other connections stop working as well. Or something like that.
Does anyone have an idea about what's really happening?
Also, is there a way to control on which thread the request body input stream's read method gets called?
I am developing an iPhone application which streams data(e.x.ECGData like points) from a server and displays(means Plotting) it on the screen -- i.e., live streaming. For that purpose, I am using NSURLConnection.
The problem I am facing is that, since the data is coming so speedily from the server to the iPhone, the cache buffer is increasing rapidly, causing the displayed data to lag behind the actual data coming from the server. After some time, the application goes too slowly, and gets a memory warning.
So my question is, how should I handle this data coming from the server? Should I continue with NSURLConnection or go for lower level socket programming?
I propose you implement some sort of flow control:
The simplest approach is to drop data if your buffers are full. For video streams, frames can be dropped. I don't know whether the same is possible with your data.
Another approach is to switch from the event-based API of NSURLConnection (where the framework controls when you have to react) to CFSocket class where you can read data when you are ready for it. It's more low-level, requires a separate thread and some advanced logic like going to sleep when the buffer is full and being woken up when the main thread has displayed more data and made more space in the buffer. With this approach you are basically building on top of TCPs flow control mechanism.
Yet another approach would be to use another network protocol where you have more control about the amount of data being sent.
I would use ASIHttpRequest streaming. You can implement a delegate method request:didReceiveData: to get your data in chunks as it comes in, deflate it if needed pares it and display. If you need cache you can always save it to file.
I'm writing an iPhone app that requests data from a web service, and in order to get that data, I'm using NSMutableURLRequest.
The problem that I am having is that the amount of data being requested is quite large (~11Mb), and this is causing my app to be killed by the OS.
Is there any way of streaming the data in a way that will allow me to process chunks of it, or should I just split the request over several separate requests in order to prevent the memory load spiking?
Think about converting your use of NSMutableURLRequest to an NSURLConnection. That class provides a way to specify a delegate object that will receive a series of connection:didReceiveData: messages, each of which will have some chunk of data from your web server. You can implement this method in your delegate in such a way that it will process data as it becomes available while still waiting for more data from the connection.
I have a question and I am very open to suggestions (even very odd ones!)
I am writing an iPhone app, which does a request (URL with parameters) to a server. As a response, the iPhone receives XML. All is well.
Right now, I am looking to improve my application's speed by measuring the time it takes to perform certain tasks. I've found that, of all the tasks performed (downloading, XML Parsing, sending the request, handeling the request, parsing objects from XML), downloading the actual XML takes the longest.
Now, my XML files are very, very, very easy and very, very, very small. I use them primarily to read RSS-like data and show them in a UITableView.
My app works very well, and there is nothing that feels really slow, but there is one application in the App Store right now which does something very similar to my application, but is way faster and feels more 'snappy', if you know what I mean. It also has the great feature to load the headlines one by one from the RSS-feed.
Currently I'm experimenting with gzip compression of my data, but the compression only makes my data half the size and it doesn't seem to do any real good for performance. The main thing is, that the data has to be downloaded, before it gets parsed. It would be very cool to have a 'stream' of data, which is parsed as it comes in. That way, I can do two jobs almost simultaneous and load headlines one by one (making user interactivity more attractive).
Anyone has an idea of how to improve my performance? Either by great compression tips or entirely different ways to communicate with the server.. All is welcome!
UPDATE: putting the latency and responsiveness of the server aside; how could I get a source of XML to be 'streamed' to my iPhone (downloaded byte for byte) and at the same time get parsed? Right now it is a linear process of downloading -> parsing -> showing, but it could become semi-parallel by downloading & parsing at the same time (and show each item when it is done downloading, instead of loading it all at the same time in a UITableView)
Assuming you're using NSXMLParser's initWithContentsOfURL:, that's probably part of the problem. It seems like that downloads the entire contents of the URL, then hands it off to the parser to parse all at once.
Despite the fact that the NSXMLParser is an event-driven parser, it doesn't seem to support streaming data to the parser in an incremental manner. You could, of course, replace NSXMLParser with some other parsing library that handles incremental data in a more sensible way.
An alternative would be to use NSURLConnection, and create a new NSXMLParser and re-parse the data each time some data comes in, in the connection:didReceiveData: method of your NSURLConnection's delegate. You''d have to write some extra code to ignore the extra events from re-parsing the beginning of the file more than once.
This seems like it'd be more work than just grabbing some other library and adapting it, but maybe not, depending on how you're handling the downstream creation of your table data.
If NSMutableArray is the underlying data structure of your UITableView you should try
using initWithContentsOfURL. The format on the server needs to be in apples "plist" xml which is easy to generate. I'm guessing that if cocoa already has resources acquired for
processing xml it would be quicker to use them instead of creating your own xml parser instance.
How are you parsing the XML data. The need to parse XML as you receive it is the whole reason pull parsing was invented, and they have the NSXMLParser event driven parser that operates on a stream of data...
That also means compression of the data is counterproductive.
For very small amounts of XML, the issue of speed is probably about latency of connection.
Thus, other big factor could be the DNS lookup. You could do that yourself once beforehand and cache the IP, perhaps rechecking only on failure to connect to your server...
Since your xml files are very small, (just 8k or so, I imagine?) I don't think you'd see a big performance boost trying to parse them as they come in. You can use an asynchronous NSURLConnection to do that, but I can't see it helping with a small file very much. Have you considered the time it takes to generate the XML file on the server? Are you using PHP to create the XML, or accessing a static file? For testing purposes, it might be interesting to access static files and see how that compares.
I ran into a problem like this a while ago, and it turned out the mySQL database on my server was being slow, and it really had nothing to do with my app.
Also, use Instruments to look at the amount of memory that is allocated while you're downloading and parsing an XML document. Some XML parsers load the entire XML document into memory and then allow you to index randomly into the document's elements, while others provide step by step parsing only. The step-by-step method is much better for limited devices like the iPhone and it's probably what you're using, but a quick look at memory consumption should tell you.