I am developing a socket based software. The client asks a question and the server answers the question and I have some problems.
The client asks a "question" to the server, the server may receive the "question" after 2 minutes. Then the client may ask the "question" once more 1 minute after it asked for the first time. So how to avoid the server answering twice?
If the server answers the client, the client may receive the answer immediately because of the network delay. So the client may ask the server once again. Server receives the question, then answers again.
Is there any good communication protocol/rules for this application?
To avoid sending duplicate answers you can use a nonce. Select an integer and send it to the server, together with the question data. When answering, the server will attach the same integer to it's answer. The client should drop all received answers with duplicated nonce values.
If you somehow store questions server side, it should be easy to just check the last questions asked, so that you avoid sending or receiving questions twice. Unless you have very limited server resources, or you have a lot of traffic, this solution should work for you.
You might also give each visitor to the site an ID, and store the questions from each ID in a format that is easy to search though and find possible duplicates.
As for the protocol, basic HTTP or Websocket is a good protocol for sending requests back and forth.
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
Related
I'm trying to enhance a server-app-website architecture in reliability, another programmer has developed.
At the moment, android smartphones start a tcp connection to a server component to exchange data. The server takes the data, writes them into a DB and another user can have a look on the data through a website. The problem is that the smartphones very regularly are in locations where connectivity is really bad. The consequence is that the smartphones lose the tcp connection and it's hard to reconnect. Now my question is, if there are any protocols that are so lightweight or accomodating concerning bad connectivity that the data exchange could work better or more reliable.
For example, I was thinking about replacing the raw TCP interface with a RESTful API, but I don't really know how well REST works in this scenario, as I don't have any experience in this area.
Maybe useful to know for answering this question: The server component is programmed in c#. The connecting components are android smartphones.
Please understand that I dont add some code to this question, because in my opinion its just a theoretically question.
Thank you in advance !
REST runs over HTTP which runs over TCP so it would have the same issues with connectivity.
Moving up the stack to the application you could perhaps think in terms of 'interference'. I quite often have to use technical stuff in remote areas with limited reception and it reminds of trying to communicate in a storm. If you think about it, if you're trying to get someone to do something in a storm where they can hardly hear you and the words get blown away (dropped signal), you don't read them the manual on how to fix something, you shout key words such as 'handle', 'pull', 'pull', 'PULL', 'ok'. So the information reaches them in small bursts you can repeat (pull, what? pull, eh? PULL! oh righto!)
Can you redesign the communications between the android app and the server so the server can recognise key 'words' with corresponding data and build up the request over a period of time? If you consider idempotency, each burst of data would not alter the request if it has already been received (pull, PULL!) and over time the android app could send/receive smaller chunks of the request. If the signal stays up, just keep sending. If it goes down, note which parts of the request haven't been sent and retry them when the signal comes back.
So you're sending the request jigsaw-style but the server knows how to reassemble the pieces in the right order. A STOP word at the end tells the server ok this request is complete, go work on it. Until that word arrives the server can store the incomplete request or discard it if no more data comes in.
If the server respond to the first request chunk with an id, the app can use the id to get the response and keep trying until the full response comes back, at which point the server can remove the response from its jigsaw cache. A fair amount of work though.
I am trying to create a network architecture which has a single server and multiple clients. The clients can connect and disconnect at any time so they need to announce their existence or shut-down to the server. The server must be able to send data to any particular client.
What is the best scalability protocols/architecture to use for this?
Currently I use a REQ/REP so that clients can 'login' and 'logout', and a SURVEY socket so that the server can send data to all clients. The message sent has an ID for the particular client it wants to process the message.
Is this good, or is there something better?
Sounds more like you need publisher subscriber. With both 0MQ and nanomsg you don't need to do anything in particular to manage connection / disconnection, the library does that for you.
However if you want more sophisticated message management (such as caching outgoing messages just in case another client chooses to connect) then you will have to manage that yourself. You might use a single push pull from the clients for them to announce their presence (they'd send a message saying who they were), followed by more push pulls from the server to each of the clients to send the messages from the cache that you have. Fiddly, but still a whole lot easier than programming up with raw sockets.
Using req rep can be problematic - if either end crashes or unexpectedly disconnects the other can be left in a stalled, unrecoverable state.
Sorry, in real world, there is no "One Size Fits All"
There are many further aspects, that influence architecture - The Bigger Picture.
While both Martin SUSTRIK's cool kids -- ZeroMQ & nanomsg -- have made a gread help in providing excellend bases + LEGO-type building blocks of Scaleable Formal Communication Patterns, they are only the beginning and saying that REQ/REP or SURVEY Behavioural Primitives ( great innovation, nevertheless still a building block ) are The Architecture would get upset almost all architects and evangelists.
The original question is important, however you already have gotten a first proposal to get it administratively closed as some people with "wider wingspan" feel your question is "too wide" or "opinion"-oriented instead an MCVE code-example driven ( ... yes, StackOverflow life is sometimes fast and cruel ).
So, without any further details available, my recommendation would be to check recent versions of PUB/SUB ( which can and do filter on PUB-side ( not on the SUB as was the design in the early ZeroMQ versions, once already xmited / delivered zillions of bytes round the world to just realise on the globally distributed peers level that no-one has yet SUB-ed to receive anything of that ) instead of the mentioned SURVEY.
Without any context it is nonsense to seriously judge, the less to improve what you try to design and implement.
I would do a poor service I were trying to do so.
The best next step?
What I can do for you right now is to direct you to see a bigger picture on this subject >>> with more arguments, a simple signalling-plane / messaging-plane illustration and a direct link to a must-read book from Pieter HINTJENS.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 9 years ago.
Improve this question
I have my own OPC Server written using the SLIK-DA4 ActiveX control in VB6. It hosts quite a large collection of tags (probably 2,000).
A customer uses a Siemens OPC client to connect (no private security). Everything goes fine and subscription reads appear just fine on the client.
Some time later, the IP link is lost between client and server for a while. However, the customer is telling me that when the link recovers, they then have to "do something" on the OPC client to get it to start subscribing again, after which things return to normal.
... Yes, I know, I'm trying to find out what they mean by "do something" !!
However, in the meantime, I'm trying to think of what I might not be doing correctly in my server code to handle this situation. My tag values don't update too often in the attached field equipment, so is it possible that, on reconnection, the client isn't receiving any callbacks simply because there are no tag changes taking place ?
On recovery of the link, how can I get the server to push an up-to-date status for all tags to the client rather than rely on someone "doing something" on the client end ? Do I need to use the OnConnect event and then SetVQT(,,sdaSGood) for all tags, or will this not have any effect ?
Thanks
When the OPC server receives a new connection (which seems to be the case), or more precisely, when the OPC client it creates an active group and puts items in it, the server is supposed to send an initial notification about each item (value/timestamp/quality, or error) - even if it has not changed recently.
If you are developing the server using a reasonable OPC toolkit, however, this should be taken care of automatically by the toolkit code. It certainly makes no sense to try to change the quality of the tags just because the OPC client had connected. The quality in the VQT should reflect whatever comes from your underlying system, or the communication problems communicating to THAT system, but not anything between the OPC server and the client.
It may also very well be a problem on the client side - simply not resilient enough to handle certain situations. The authoritative way to tell what is happening (and put the "blame" into the direction of the server or the client) would be place an OPC Analyzer (available from OPC Foundation to OPC Members) in between, and log the OPC calls and check which side is not behaving right.
what they mean by "do something"
I think it means that customer needs to restart the client, create a group and add tags. But this is how things should be, because OPC Specification doesn't say anything about handling of connection breaks. It has only description of interfaces which can be used to check connection (server) status (e.g. IOPCServer::GetStatus). Typically, clients reconnect (create a new connection with new group and add tags) automatically, but only if they have been noticed that connection was lost.
a "re-open" request from their OPC Client
You can ask someone from Siemens to provide you quotation from OPC Specification where "re-open" mechanism and/or interfaces are described. Because I don't remember such notations in specification.
But if "re-open" means reusing of old connection (i.e. DCOM objects left from old connection) then I may assume the following situation:
client and server interacts properly
connection breaks
server sends OnDataChange callback to client and receives an error (something like "RPC service is unavailable")
server just stops sending of OnDataChange callbacks and do nothing with DCOM objects related to current connection
connection restores
client still can call server using existing DCOM objects but doesn't receive OnDataChange callbacks
customer need to restart client manually to repair connection
In such case you (or toolkit) should remove all objects related to broken connection or don't stop sending of callbacks.
When IP changes ,try to reconnect the OPC Server or Make your OPC Server IP address as static
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I am trying to make a realtime notification just like facebook.After learning and searching alot i m very confuse please explain me what is right and what is wrong..
Please make sure that the site may would have same number of users as Facebook
We can make Realtime notification with long polling or not? IF yes what is the advantages, disadvantages and limitations.
we can make Realtime notifiaction with websockets or not?Please mind the number of users can be same as facebook .If yes what is the advantages, disadvantages and limitations.
If there is any other method please explain.
Confusion
How Far I learn in web and found that Websocket is good but there is a limitation(Max 5K) in number of open connection which means that at a time the max number of user is just 5K,this is very less than facebook's number of users.. if I m wrong please explain.
You're wrong, a websocket based solution is not limited to 5K concurrent connections.
According to the Facebook Newsroom they have about 727 million daily active users on average in September 2013 or about 504k unique users that hit the Facebook page every minute. Given an average visit time of 18 minutes (researched by statisticbrain.com) their notification infrastructure must be able to serve about 9 million (18*504k) concurrent TCP connections 24/7. Although this number is a very far approximation it gives a far idea of what they are dealing with and what you have to deal with if you are going to build such a system.
You can use long polling as well as websockets to build your realtime notification system. In both cases you face similar problems which are related to your OS (Explanations are for a Unix based system):
limitation of ports, one tcp listener socket can only accept 2^16 connections on the same IP/Port it is listening, so you'll need to listen on multiple ports and/or multiple IP adresses.
memory, every open connection uses at least one file descriptor
Read more about the limitations in What is the theoretical maximum number of open TCP connections that a modern Linux box can have
Long-polling vs. Websockets:
Every poll in your long-poll solution requires a new HTTP request, which requires more bandwidth than what is needed to keep a websocket connection alive. Moreover the notification is returned as a HTTP response resulting in a new poll request. Although the websocket solution can be more efficient in terms of bandwidth and consumption of system resources, it has a major drawback: lack of browser support.
Depending on the stats at hand, a websocket-only solution ignores about 20-40% of your visitors (stats from statscounter.com). For this reason different server libraries were developed that abstract the concept of a connection away from the 'physical' underlying transport model. As a result more modern browsers create the connection using websockets and older browsers fall back to an alternative transport such as e.g. HTTP long polling, jsonp polling, or flash. Prominent examples of such libraries are Sock.js and Socket.io.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm looking to establish some kind of socket/COMET type functionality from my server(s) to my iPhone application. Essentially, anytime a user manages to set an arbitrary object 'dirty' on the server, by say, updating their Address.. the feedback should be pushed from the server to any clients keeping a live poll to the server. The buzzword for this is COMET I suppose. I know there is DWR out there for web browser applications, so I'm thinking, maybe it's best to set a hidden UIWebView in each of my controllers just so I can get out of the box COMET from their javascript framework? Is there a more elegant approach?
There are a couple of solutions available to use a STOMP client.
STOMP is incredibly simple and lightweight, perfect for the iPhone.
I used this one as my starting point, and found it very good. It has a few object allocation/memory leak problems, but once I got the hang of iPhone programming, these were easy to iron out.
Hope that helps!
Can you use ordinary TCP/IP socket in your application?
A) If yes then definitely a raw TCP/IP socket is more elegant solution. From your iPhone app you just wait for notification events. The socket is open as long as your application is open. If you want you can even use HTTP protocol / headers.
On the server side you can use some framework to write servers which efficiently handle thousands of open TCP/IP connections. e.g Twisted, EventMachine or libevent. Then just bind the server main socket to http port (80).
The idea is to use a server which keeps just a single data structure per client. Receives update event from some DB application and then pushes it to right client.
B) No, you have to use Apache and http client on iPhone side. Then you should know that whole COMET solution is in fact work around for limitations of HTTP protocol and Apache / PHP.
Apache was designed to handle many short time connections. As far I know only newest versions Apache (mpm worker) can handle efficiently big number of opened connection. Previously Apache was keeping one process per connection.
Web browsers have a limit of concurrent open connections to one web server (URL address in fact, eg. www.foo.com, not IP address of www.foo.com). And the limit is 2 connections. Additionally, a browser will allow only for AJAX connections to the same server from which the main HTML page was downloaded.
I wrote a web server for doing exactly this kind of thing. I'm pushing realtime updates through the server with long polling and, as an example, I had safari on the iPhone displaying that data.
A given instance of the server should be able to handle a few thousand concurrent clients without trying too hard. I've got a plan to put them in a hierarchy to allow for more horizontal scaling (should be quite trivial, but doesn't affect my current application).
WebSync has a javascript client that works on the iPhone, if that's what you're after
Would long-polling work for what you want to achieve? You can implement the client-side in a few lines of regular Javascript, which will be lighter than any framework could possibly be.
It would also be trivial to implement it in ObjC (connect, wait for a response or timeout, repeat)
The answers to my question Simple "Long Polling" example code? hopefully explain how extremely simple Long Polling is..
Basically you would just request a URL as usual - the web-server would accept the connection, but not send any data until it's available. When you receive data, or the connection times-out, you reconnect (and repeat)
The most complicated bit would be server server-side, as you cannot use a regular threaded web-server like Apache, although this is also the case with Comet..
StreamHub Comet Server works with the iPhone out of the box, no plugins or anything required. Just browsed to their website on my iPhone and all the examples worked, didn't need to install Flash or anything.
Do you want/have do the communication for your app over http? If not, you can use CFNetwork framework to use sockets (TCP/UDP) to allow your app and server to communicate. From what I have seen of the CFNetwork stack, it is pretty cool, and makes it fairly straitforward to read and write to streams, and allows for synchronous and asynchronous communication. It also allows for you to define callbacks on your socket allowing you to get notified of events like data received, connection made, etc. So, in your example you could send the information over the socket to your server, and then you could define a callback that would listen for incoming data on the stream and then update your app accordingly.
EDIT: Did a little more research, and if you go the socket approach, you may want to also look at the NSStream classes. They are Cocoa abstractions build on top of the CFSocket stuff.
you didn't mention what serverside tech you're using. But in case it's microsoft .net (or for any other googlers who come across this), there is a simple option for comet: http://www.codeplex.com/ncomet.
COMET, LightStreamer, AJAX all that junk is broken. It is basics of TCP that no 'keep-alives' are ever guaranteed without pinging traffic.. So you can forget that long-polling if any decent reliability or timely delivery is to be guaranteed..
It's just hype everyone saw through back in 2003 when the lame-mania kicked off..