On this post, I read about the usage of XMPP. Is this sort of thing necessary, and more importantly, my main question expanded: Can a chat server and client be built efficiently using only standard HTTP and browser technologies (such as PHP and JS, or RoR and JS, etc)? Or, is it best to stick with old protocols like XMPP find a way to integrate them with my application?
I looked into CampFire via LiveHTTPHeaders and Firebug for about 5 minutes, and it appears to use Ajax to send a request which is never answered until another chat happens. Is this just CampFire opening a new thread on the server to listen for an update and then returning a response to the request when the thread hears an update? I noticed that they're requesting on a specific port (8043 if memory serves me) which makes me think that they're doing something more complex than just what I mentioned. Also, the URL requested started with /tcp/ which I found interesting.
Note: I don't expect to ever have more than 150 users live-chatting in all the rooms combined at the same time. I understand that if I was building a hosted pay for chat service like CampFire with thousands of concurrent users, it would behoove me to invest time in researching special technologies vs trying to reinvent the wheel in a simple way in my app.
Also, if you're going to do it with server polling, how often would you personally poll to maximize response without slamming the server?
The technology is broadly called Comet, which is supposedly some hilarious pun on Ajax1.
The XmlHTTPResponse variant seems to be the most popular.
The XHR version isn't strictly polling per se; as you said, the client connects with a long timeout and the server doesn't actually send a response until there is anything to send. Once the response is sent, it drops the connection and the client reconnects. They call it long polling, because the client is initiating the connection, but it differs from classic polling in that the client doesn't constantly connect requesting new content even if nothing has changed (i.e. no "is there a message now? no? how about now? what about now?")
It's more like trying to keep a constantly dropping connection open.
Yes it can absolutely be built using standard web technologies.
1I prefer to think of Ajax as a mighty Greek warrior rather than a cleaning product, so I frown mightily upon this pun.
That would first depend on your strategy of your webserver load balancing. 150 concurrent users that publish data over a stateless medium (HTTP) is certainly efficient with the bit of scripting (client- and server side). Remember that chat applications are just many client -> one server strategies, that fits perfectly over the web.
Related
I've been working through a bunch of WebRTC examples, and they all require a custom Websocket server for exchanging the signalling data. OTOH, every WebRTC doc states that you can use anything for signalling, including carrier pidgeons.
So I've been wondering, just out of curiosity: why isn't signalling usually done using a boring old REST API (or similar)? It's not as if the setup process has realtime requirements, for which using Websockets would make sense...
Because you want the setup process to be as quick as possible—usually—and there can be quite a few messages to exchange, especially if you use ICE trickling. Using AJAX you'd have to use repeated polling, which is certainly slower. If that's good enough for you and you see some advantage in doing it that way vs. web sockets, more power to you. But typically you'd want to forward messages to the other peer as soon as you get them, not whenever the other peer happens to poll the server next. And the only practical option to push data from the server to the client are web sockets.
You could use server-sent events for the server-to-client push and AJAX for the client-to-server sending… but why, when web sockets already provide duplex communication?
I'm trying to enhance a server-app-website architecture in reliability, another programmer has developed.
At the moment, android smartphones start a tcp connection to a server component to exchange data. The server takes the data, writes them into a DB and another user can have a look on the data through a website. The problem is that the smartphones very regularly are in locations where connectivity is really bad. The consequence is that the smartphones lose the tcp connection and it's hard to reconnect. Now my question is, if there are any protocols that are so lightweight or accomodating concerning bad connectivity that the data exchange could work better or more reliable.
For example, I was thinking about replacing the raw TCP interface with a RESTful API, but I don't really know how well REST works in this scenario, as I don't have any experience in this area.
Maybe useful to know for answering this question: The server component is programmed in c#. The connecting components are android smartphones.
Please understand that I dont add some code to this question, because in my opinion its just a theoretically question.
Thank you in advance !
REST runs over HTTP which runs over TCP so it would have the same issues with connectivity.
Moving up the stack to the application you could perhaps think in terms of 'interference'. I quite often have to use technical stuff in remote areas with limited reception and it reminds of trying to communicate in a storm. If you think about it, if you're trying to get someone to do something in a storm where they can hardly hear you and the words get blown away (dropped signal), you don't read them the manual on how to fix something, you shout key words such as 'handle', 'pull', 'pull', 'PULL', 'ok'. So the information reaches them in small bursts you can repeat (pull, what? pull, eh? PULL! oh righto!)
Can you redesign the communications between the android app and the server so the server can recognise key 'words' with corresponding data and build up the request over a period of time? If you consider idempotency, each burst of data would not alter the request if it has already been received (pull, PULL!) and over time the android app could send/receive smaller chunks of the request. If the signal stays up, just keep sending. If it goes down, note which parts of the request haven't been sent and retry them when the signal comes back.
So you're sending the request jigsaw-style but the server knows how to reassemble the pieces in the right order. A STOP word at the end tells the server ok this request is complete, go work on it. Until that word arrives the server can store the incomplete request or discard it if no more data comes in.
If the server respond to the first request chunk with an id, the app can use the id to get the response and keep trying until the full response comes back, at which point the server can remove the response from its jigsaw cache. A fair amount of work though.
Background
We are writing a Messenger-like app. We have setup Websockets to Inbox and Chat.
Question
My question is simple. What are the advantages and disadvantages when sending data from Client to Server using REST instead of Websockets? (I am not interested in updates now.)
We know that REST has higher overhead in terms of message sizes and that WS is duplex (thus open all time). What about the other things we didn't keep in mind?
Here's a summary of the tradeoffs I'm aware of.
Reasons to use webSocket:
You need/want server-push of data.
You are sending lots of small pieces of data from client to server and doing it very regularly. Using webSocket has significantly less overhead per transmission.
Reasons to use REST:
You want to use server-side frameworks or modules that are built for REST, not for webSocket (such as auth, rate limiting, security, streaming, etc...).
You aren't sending data very often from client to server and thus the server-side burden of keeping a webSocket connection open all the time may lessen your server scalability.
You want your client to run in places where a long-connected webSocket during inactive periods of time may not be practical (perhaps mobile).
You want your client to run in old browsers that don't support webSocket.
You want the browser to enforce same-origin restrictions (those are enforced for REST Ajax calls, but not for webSocket connections).
You don't want to have to write code that detects when the webSocket connection has died and then auto-reconnects and handles back-offs and handles mobile issues with battery usage issues, etc...
You need to run in situations where there are proxies or other network infrastructure that may not support long running webSocket connections.
If you want request/response built in. REST is request/response. WebSocket is not - it's message based. Responses from a webSocket are done by sending a messge back. That message back is not, by itself, a response to any specific request, it's just data being sent back. If you want request/response with webSocket, then you have to build some infrastructure yourself where you tag an id into a request and the response for that particular request contains that specific id. Otherwise, if there are every multiple requests in flight at the same time, then you don't know which response belongs with which request because all the data is being sent over the same connection and you would have no way of matching response with request.
If you want other clients to be able to carry out this operation via an Ajax call.
So, if you already have a webSocket implementation, don't have any problem with it that are lessened with REST and aren't interested in any of the reasons that REST might be better, then stick with your webSocket implementation.
Related references:
websocket vs rest API for real time data?
Ajax vs Socket.io
Adding comments per your request:
It sounds like you're expecting someone to tell you the "right" way to do it. There are reasons to pick one way over the other. If none of those reason compel you one way vs. the other, then it's just an architectural choice and you must take in the whole context of what you are doing and decide which architectural choice makes more sense to you. If you already have the reliably established webSocket connection and none of the advantages of REST apply to your situation then you can optimize for "efficiency" and send your data to the server over the webSocket connection.
On the other hand, if you wanted there to be a simple API on your server that could be reached with an Ajax call from other clients, then you'd want your server to support this operation via REST so it would simplest for these other clients to carry out this one operation. So, it all depends upon which direction your requirements drive you and, if there is no particular driving reason to go one way or the other, you just make an architectural choice yourself.
I got a project using GWTP (which involves MVP separation, Gin and Dispatch), now I'm on the situation where it is required that changes on the server are pushed to specific clients
I've reading the gwt-comet and gwteventservice documentation, It seems the first doesn't work with RPC and the second Ecnapsulates RPC, for which I don't know how to fit it in my current command pattern from GWTP. Ideas?
I have been using gwt-comet (http://code.google.com/p/gwt-comet/). It's a native comet implementation working pretty good like RPC, you can send Strings or your GWT-serialized objects as well. And the best thing you don't need to do many things to make it works.
i used "Server Push in GWT" described here http://code.google.com/p/google-web-toolkit-incubator/wiki/ServerPushFAQ - it seemed to work fairly well for a small project.
This is really a servlet problem, not a GWT or GWTP problem.
So there are a few approaches to doing this, the most stable (in my opinion) is to have a long or blocking poll servlet. This is basically a servlet that is polled by the client, and holds the connection open for some period of time if there is no message to 'push' to the client, and if too much time passes (this is to get around http timeouts) a heartbeat is returned of some kind. Either way, when the servlet request request returns, the client just makes another request. This is the most portable and stable way to my mind, since it uses only the core servlet api, doesn't suffer from network issues, and the blocking portion allows you to have the poll 'park' at the server for some period of time and reduces total request load, while allowing very quick return of new information to the client when there is some available.
The next way to achieve this is via WebSockets, this is great once you get it working and in my opinion is the way of the future without question. I think this is a good one to work with since this will be, in my opinion, a paradigm shift in web applications once it catches a head of steam, so we all need to be up to speed. Basically, you have a javascript 'socket' open via port 80 (this is one of the best features, since you don't have to open any firewall holes) and can communicate in two directions across that socket.
Comet can also work, but it will generally lock you down to one server type, which may be alright for your application. Caveat here!!!! I have only done very small tests with comet, it was flaky for me when I set it up, and was not as steady as the blocking poll solution as I had it set up.
Now the neatest one in my opinion, but this one is very limited due to network constraints probably to single domain intranet applications, is to use an applet based push. This setup (which could be done with udp or a straight socket, I did all web just to keep it all simpler conceptually) takes the applet, uses it to spin up a jetty server instance on the client, and the has the page publish the client's jetty 'endpoint' to the server. At this point, the client can contact the server using it's servlets, and the server can contact the client at the servlet(s) exposed on the jetty server. This is true push, it's neato, but there are network nightmares.
So of all the above, I use long polling, keep my eye on web sockets since they are the future in my mind, and really like the applet based version, although it's quite restricted in use due to the network resolution limitations.
Once you have this decided, from GWTP you would just have actions or JSNI bridge methods as needed to connect to your server and receive responses. I won't go into this, since this is really a core servlet/http/javascript question more than a GWT or GWTP centric question.
I hope that helps!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm looking to establish some kind of socket/COMET type functionality from my server(s) to my iPhone application. Essentially, anytime a user manages to set an arbitrary object 'dirty' on the server, by say, updating their Address.. the feedback should be pushed from the server to any clients keeping a live poll to the server. The buzzword for this is COMET I suppose. I know there is DWR out there for web browser applications, so I'm thinking, maybe it's best to set a hidden UIWebView in each of my controllers just so I can get out of the box COMET from their javascript framework? Is there a more elegant approach?
There are a couple of solutions available to use a STOMP client.
STOMP is incredibly simple and lightweight, perfect for the iPhone.
I used this one as my starting point, and found it very good. It has a few object allocation/memory leak problems, but once I got the hang of iPhone programming, these were easy to iron out.
Hope that helps!
Can you use ordinary TCP/IP socket in your application?
A) If yes then definitely a raw TCP/IP socket is more elegant solution. From your iPhone app you just wait for notification events. The socket is open as long as your application is open. If you want you can even use HTTP protocol / headers.
On the server side you can use some framework to write servers which efficiently handle thousands of open TCP/IP connections. e.g Twisted, EventMachine or libevent. Then just bind the server main socket to http port (80).
The idea is to use a server which keeps just a single data structure per client. Receives update event from some DB application and then pushes it to right client.
B) No, you have to use Apache and http client on iPhone side. Then you should know that whole COMET solution is in fact work around for limitations of HTTP protocol and Apache / PHP.
Apache was designed to handle many short time connections. As far I know only newest versions Apache (mpm worker) can handle efficiently big number of opened connection. Previously Apache was keeping one process per connection.
Web browsers have a limit of concurrent open connections to one web server (URL address in fact, eg. www.foo.com, not IP address of www.foo.com). And the limit is 2 connections. Additionally, a browser will allow only for AJAX connections to the same server from which the main HTML page was downloaded.
I wrote a web server for doing exactly this kind of thing. I'm pushing realtime updates through the server with long polling and, as an example, I had safari on the iPhone displaying that data.
A given instance of the server should be able to handle a few thousand concurrent clients without trying too hard. I've got a plan to put them in a hierarchy to allow for more horizontal scaling (should be quite trivial, but doesn't affect my current application).
WebSync has a javascript client that works on the iPhone, if that's what you're after
Would long-polling work for what you want to achieve? You can implement the client-side in a few lines of regular Javascript, which will be lighter than any framework could possibly be.
It would also be trivial to implement it in ObjC (connect, wait for a response or timeout, repeat)
The answers to my question Simple "Long Polling" example code? hopefully explain how extremely simple Long Polling is..
Basically you would just request a URL as usual - the web-server would accept the connection, but not send any data until it's available. When you receive data, or the connection times-out, you reconnect (and repeat)
The most complicated bit would be server server-side, as you cannot use a regular threaded web-server like Apache, although this is also the case with Comet..
StreamHub Comet Server works with the iPhone out of the box, no plugins or anything required. Just browsed to their website on my iPhone and all the examples worked, didn't need to install Flash or anything.
Do you want/have do the communication for your app over http? If not, you can use CFNetwork framework to use sockets (TCP/UDP) to allow your app and server to communicate. From what I have seen of the CFNetwork stack, it is pretty cool, and makes it fairly straitforward to read and write to streams, and allows for synchronous and asynchronous communication. It also allows for you to define callbacks on your socket allowing you to get notified of events like data received, connection made, etc. So, in your example you could send the information over the socket to your server, and then you could define a callback that would listen for incoming data on the stream and then update your app accordingly.
EDIT: Did a little more research, and if you go the socket approach, you may want to also look at the NSStream classes. They are Cocoa abstractions build on top of the CFSocket stuff.
you didn't mention what serverside tech you're using. But in case it's microsoft .net (or for any other googlers who come across this), there is a simple option for comet: http://www.codeplex.com/ncomet.
COMET, LightStreamer, AJAX all that junk is broken. It is basics of TCP that no 'keep-alives' are ever guaranteed without pinging traffic.. So you can forget that long-polling if any decent reliability or timely delivery is to be guaranteed..
It's just hype everyone saw through back in 2003 when the lame-mania kicked off..