Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I have some problem.
I have client and server. Client connect with server over TCP.
Then, client send some data (separated by chunks), I don't know what is the length of data (TLS handshake). But I know that client send some data with fixed length, and then stop, until not received some response, then he send data with fixed length again.
I need read all chunks, until client stopped send (because so many chunks). How to do that ?
I have only one idea, it's timeout. Read data in loop and set timeout between iterate. If timeout is ended, then data complete collected.
Perhaps there is a more elegant solution?
Based on the information in your comments, you're doing this wrong. The correct way to write an HTTPS proxy is to read the CONNECT line, make the upstream connection, send the appropriate response back o the client, and then if successful start copying bytes in both directions simultaneously. You're not in the least concerned with packets or read sizes, and you should certainly not make any attempt to 'collect' packets before retransmission, as that will just add latency to the system.
You can accomplish this either by starting two threads per connection, one in each direction, or via non-blocking sockets and select()/poll()/epoll(), or whatever that looks like in Go.
BUT I have no idea why you're doing this at all. There are plenty of open-source HTTP proxies already in existence, and as you're dealing with HTTPS there is no value you can possibly add to them. Your claim about 'business logic' is meaningless, or at least unimplementable.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 1 year ago.
Improve this question
As TCP contains a checksum, and the TCP/IP stack will detect broken packets, is it redundant to add an extra checksum or CRC in a TCP payload (I mean the user data which could be read out by the socket api on the remote end), to make it possible for the receiver verify whether the data is same with the sent one?
Some thought about this question:
It's very common seen that there is a SHA256 value to verify the consistency when downloading files from internet.
The checksum contained in TCP packets already could detect broken packets in most cases.
The Modbus protocol for TCP dropped the CRC, which is used by the Modbus protocol for serial because there is already a checksum in TCP packets.
So, I am really confused now. Could somebody shed some light on this question?
I have googled, it's really not a new question, but the answer is still not clear, there are two opposite voices about this question.
For details, see these:
CRC checking done automatically on Tcp/Ip?
Is it possible for packets sent using TCP to ever arrive with different data?
ADDED:
The two questions aforementioned have been there for more than ten years! But there are still two opposite voices.
TCP has a checksum, which provides only some protection, simply because the checksum is only 16bit. For how much robustness the TCP checksum actually provides, or how much it is lacking, see Can a TCP checksum fail to detect an error? If yes, how is this dealt with?.
If you need more protection, you need to have additional and longer protection, because the more bits are used in protection, the better protection can be provided. If you need protection against active tampering with the traffic (ie not just accidental errors), you also need a cryptographic protection, such as offered by TLS.
I am developing a socket based software. The client asks a question and the server answers the question and I have some problems.
The client asks a "question" to the server, the server may receive the "question" after 2 minutes. Then the client may ask the "question" once more 1 minute after it asked for the first time. So how to avoid the server answering twice?
If the server answers the client, the client may receive the answer immediately because of the network delay. So the client may ask the server once again. Server receives the question, then answers again.
Is there any good communication protocol/rules for this application?
To avoid sending duplicate answers you can use a nonce. Select an integer and send it to the server, together with the question data. When answering, the server will attach the same integer to it's answer. The client should drop all received answers with duplicated nonce values.
If you somehow store questions server side, it should be easy to just check the last questions asked, so that you avoid sending or receiving questions twice. Unless you have very limited server resources, or you have a lot of traffic, this solution should work for you.
You might also give each visitor to the site an ID, and store the questions from each ID in a format that is easy to search though and find possible duplicates.
As for the protocol, basic HTTP or Websocket is a good protocol for sending requests back and forth.
https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I am trying to make a realtime notification just like facebook.After learning and searching alot i m very confuse please explain me what is right and what is wrong..
Please make sure that the site may would have same number of users as Facebook
We can make Realtime notification with long polling or not? IF yes what is the advantages, disadvantages and limitations.
we can make Realtime notifiaction with websockets or not?Please mind the number of users can be same as facebook .If yes what is the advantages, disadvantages and limitations.
If there is any other method please explain.
Confusion
How Far I learn in web and found that Websocket is good but there is a limitation(Max 5K) in number of open connection which means that at a time the max number of user is just 5K,this is very less than facebook's number of users.. if I m wrong please explain.
You're wrong, a websocket based solution is not limited to 5K concurrent connections.
According to the Facebook Newsroom they have about 727 million daily active users on average in September 2013 or about 504k unique users that hit the Facebook page every minute. Given an average visit time of 18 minutes (researched by statisticbrain.com) their notification infrastructure must be able to serve about 9 million (18*504k) concurrent TCP connections 24/7. Although this number is a very far approximation it gives a far idea of what they are dealing with and what you have to deal with if you are going to build such a system.
You can use long polling as well as websockets to build your realtime notification system. In both cases you face similar problems which are related to your OS (Explanations are for a Unix based system):
limitation of ports, one tcp listener socket can only accept 2^16 connections on the same IP/Port it is listening, so you'll need to listen on multiple ports and/or multiple IP adresses.
memory, every open connection uses at least one file descriptor
Read more about the limitations in What is the theoretical maximum number of open TCP connections that a modern Linux box can have
Long-polling vs. Websockets:
Every poll in your long-poll solution requires a new HTTP request, which requires more bandwidth than what is needed to keep a websocket connection alive. Moreover the notification is returned as a HTTP response resulting in a new poll request. Although the websocket solution can be more efficient in terms of bandwidth and consumption of system resources, it has a major drawback: lack of browser support.
Depending on the stats at hand, a websocket-only solution ignores about 20-40% of your visitors (stats from statscounter.com). For this reason different server libraries were developed that abstract the concept of a connection away from the 'physical' underlying transport model. As a result more modern browsers create the connection using websockets and older browsers fall back to an alternative transport such as e.g. HTTP long polling, jsonp polling, or flash. Prominent examples of such libraries are Sock.js and Socket.io.
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
Ok, I'm a bit confused with these mail protocols so I just have to ask.
I'm making an app for a very special use case we have at work. We have two e-mail servers sending mail to eachother (two seperate domains). The problem is that one of the servers is frequently moved (the server is in the field, and has to be mobile). When that server is moved, it can't be connected to our network and therefore mail cannot be sendt to this server.
What I'm making is an app that will be between our two e-mail servers, intercepting the e-mail trafic that goes between them and, if one of the servers cannot be reached, my app should store the e-mails in a buffer, so that when it detects that the e-mail server is back online, it can safely send the e-mails the other server did not recieve due to it being offline. The e-mails in the buffer should also be able to be sendt to an alternative e-mail so they can be acted upon immidietly should the recieving e-mail server be down for longer than normal...
So, I've figured out that I need to use the SMTP-protocoll to send e-mail, but what do I use to recieve them? All examples I've read so far in C# and Python are about connecting to an allready established POP3/Imap4 server and recieve mail from there, and I think setting up my own POP3/IMAP4 server for intercepting mail before sending it on is doing it the really hard way...
So how do i recieve/intercept e-mail without the use of POP3/IMAP4?
If I understand the question correctly you simply want to buffer your emails. You could be much better off either using a third party to act as a backup mail server or set up additional mail servers as lower priority servers to collect the mail should one of the servers not be reached. You can ask on ServerFault about setting something like this up. I can't see the real benefit of writing an app yourself.
If you do wish to write something yourself then you will most likely wish to write both an SMTP server and and SMTP client one to accept the emails and one to deliver the message.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm looking to establish some kind of socket/COMET type functionality from my server(s) to my iPhone application. Essentially, anytime a user manages to set an arbitrary object 'dirty' on the server, by say, updating their Address.. the feedback should be pushed from the server to any clients keeping a live poll to the server. The buzzword for this is COMET I suppose. I know there is DWR out there for web browser applications, so I'm thinking, maybe it's best to set a hidden UIWebView in each of my controllers just so I can get out of the box COMET from their javascript framework? Is there a more elegant approach?
There are a couple of solutions available to use a STOMP client.
STOMP is incredibly simple and lightweight, perfect for the iPhone.
I used this one as my starting point, and found it very good. It has a few object allocation/memory leak problems, but once I got the hang of iPhone programming, these were easy to iron out.
Hope that helps!
Can you use ordinary TCP/IP socket in your application?
A) If yes then definitely a raw TCP/IP socket is more elegant solution. From your iPhone app you just wait for notification events. The socket is open as long as your application is open. If you want you can even use HTTP protocol / headers.
On the server side you can use some framework to write servers which efficiently handle thousands of open TCP/IP connections. e.g Twisted, EventMachine or libevent. Then just bind the server main socket to http port (80).
The idea is to use a server which keeps just a single data structure per client. Receives update event from some DB application and then pushes it to right client.
B) No, you have to use Apache and http client on iPhone side. Then you should know that whole COMET solution is in fact work around for limitations of HTTP protocol and Apache / PHP.
Apache was designed to handle many short time connections. As far I know only newest versions Apache (mpm worker) can handle efficiently big number of opened connection. Previously Apache was keeping one process per connection.
Web browsers have a limit of concurrent open connections to one web server (URL address in fact, eg. www.foo.com, not IP address of www.foo.com). And the limit is 2 connections. Additionally, a browser will allow only for AJAX connections to the same server from which the main HTML page was downloaded.
I wrote a web server for doing exactly this kind of thing. I'm pushing realtime updates through the server with long polling and, as an example, I had safari on the iPhone displaying that data.
A given instance of the server should be able to handle a few thousand concurrent clients without trying too hard. I've got a plan to put them in a hierarchy to allow for more horizontal scaling (should be quite trivial, but doesn't affect my current application).
WebSync has a javascript client that works on the iPhone, if that's what you're after
Would long-polling work for what you want to achieve? You can implement the client-side in a few lines of regular Javascript, which will be lighter than any framework could possibly be.
It would also be trivial to implement it in ObjC (connect, wait for a response or timeout, repeat)
The answers to my question Simple "Long Polling" example code? hopefully explain how extremely simple Long Polling is..
Basically you would just request a URL as usual - the web-server would accept the connection, but not send any data until it's available. When you receive data, or the connection times-out, you reconnect (and repeat)
The most complicated bit would be server server-side, as you cannot use a regular threaded web-server like Apache, although this is also the case with Comet..
StreamHub Comet Server works with the iPhone out of the box, no plugins or anything required. Just browsed to their website on my iPhone and all the examples worked, didn't need to install Flash or anything.
Do you want/have do the communication for your app over http? If not, you can use CFNetwork framework to use sockets (TCP/UDP) to allow your app and server to communicate. From what I have seen of the CFNetwork stack, it is pretty cool, and makes it fairly straitforward to read and write to streams, and allows for synchronous and asynchronous communication. It also allows for you to define callbacks on your socket allowing you to get notified of events like data received, connection made, etc. So, in your example you could send the information over the socket to your server, and then you could define a callback that would listen for incoming data on the stream and then update your app accordingly.
EDIT: Did a little more research, and if you go the socket approach, you may want to also look at the NSStream classes. They are Cocoa abstractions build on top of the CFSocket stuff.
you didn't mention what serverside tech you're using. But in case it's microsoft .net (or for any other googlers who come across this), there is a simple option for comet: http://www.codeplex.com/ncomet.
COMET, LightStreamer, AJAX all that junk is broken. It is basics of TCP that no 'keep-alives' are ever guaranteed without pinging traffic.. So you can forget that long-polling if any decent reliability or timely delivery is to be guaranteed..
It's just hype everyone saw through back in 2003 when the lame-mania kicked off..