What is the difference between a TCP server/Net server in vertex and HTTP server?
What are the use cases for each?
I tried googling and went through the official website, none of them have a clear explanation.
First off, in General Networking, there are 2 common types of handling connections. This can be done either over TCP (Transmission Control Protocol) or UDP (User Datagram Protocol). The most import difference between these two is that UDP will continuously send out streams/buffers of bytes without checking to see if the network packets made it to the other side of the line. This is useful in situations where security isn't much of an issue and where speed is important. Most VoIP services (Skype, Hangouts), XMPP (chat) and even YouTube (I think) use UDP for their streaming, as it has huge gains on performances and it doesn't matter all that much if a frame made it to the other side of the line, as the person could simply repeat themselves.
TCP on the other hand, is "secure" by default. It performs several handshakes on a regular basis with the endpoint so maintain connectivity and make sure that all packets are received on the other side of the line.
Now, there are a LOT of protocols out there in the Wild Wild West called Internet.
List of TCP and UDP port numbers
As you can see, a lot of protocols support either TCP or UDP. HTTP on it's own is a TCP protocol with port 80 (as you might know). Therefore, HTTPServer is pretty much just an extension of a TCPServer, but with some add-ons such as REST. These add-ons are much welcome as HTTP processing is a pretty common use case. Without HTTPServer, you would need to declare loads of functions on your own.
There are many articles online explaining the difference between HTTP and TCP, so here is: http://www.differencebetween.net/technology/internet/difference-between-tcp-and-http/
Vert.x naturally offers the capacity to do "raw" networking at TCP level or at HTTP-level, the latter offering facilities to deal with the protocol, including decoding TCP packets into HTTTP requests, supporting the creation of HTTP responses, ...
Related
There is the IP - Internet Protocol. It allows us to address some computer in global network - Internet.
There is also a port - endpoint destination on that addressed computer. Someone can establish such an endpoint through socket - a so-named handful of operating system calls (API) and combination of IP address and port. It can be done using one of two "transport" protocols - either TCP or UDP. TCP provides a guarantee of delivery, UDP doesn't. In both cases someone given with a stream of bytes which he/she can read and write back. Such a stream has no meaning as it's own - someone can and should interpret it as he/she likes.
Socket API is well established and straightforward.
There is major wave nowadays: many global companies tend to implement it's own, "brand new" protocols. In fact, seems like many of them just discard using TCP reliable delivery and implement it's own using UDP (what is the reason to fight TCP reliability, though?).
Is there a way to implement a brand new "transport" protocol which will coexists with TCP and UDP? Will it be accessible through socket APIs? If not, then how?
Are there any practical reasons to do this?
What is performance (I mean latency while sending all messages, maximum fan-out rate for many messages to many receivers) of ZMQ in comparison to "simple" UDP and its multicast implementation?
Assume, I have one static 'sender', which have to send messages to many,many 'receivers'. PUB/SUB pattern with simple TCP transport seems very comfortable to handle such task - ZMQ does many things without our effort, one ZMQ-socket is enough to handle even numerous connections.
But, what I am afraid is: ZMQ could create many TCP sockets in background, even if we don't "see" that. That could create latency. However, if I create "common" UDP socket and will transmit all my messages with multicast - there would be only one socket (multicast), so I think latency problem would be solved. To be honest, I would like to stay with ZMQ and PUB/SUB on TCP. Are my concerns valid?
I don't think you can really compare them in that way. It depends on what is important to you.
TCP provides reliability and you as the sender can choose if loss is more important than latency by setting your block/retry options on send.
mcast provides network bandwidth saving especially if you network has multiple segments/routers.
Other options in zeromq
Use zmq_proxy's to split/share the load of tcp connections
Use pub/sub with pgm/epgm which is just a layer over multicast (I use this)
Use the new radio dish pattern (with this you have limited subscription options)
http://api.zeromq.org/4-2:zmq-udp
Behind the scenes, a TCP "socket" is identified (simplified) by both the "source" and "destination" - so there will be a socket for each direction you're communicating with each peer (for a more full description of how a socket is set up/identified, see here and here). This has nothing to do with ZMQ, ZMQ will set up exactly as many sockets as is required by TCP. You can optimize this if you choose to use multicast, but ZMQ does not optimize this for you by using multicast behind the scenes, except for PUB/SUB see here.
Per James Harvey's answer, ZMQ has added support for UDP, so you can use that if you do not need or want the overhead of TCP, but based on what you've said, you'd like to stay with TCP, which is likely the better choice for most applications (i.e. we often unconsciously expect the reliability of TCP when we're designing our applications, and should only choose UDP if we know what we're doing).
Given that, you should assume that ZMQ is as efficient as TCP enables it to be with regards to low-level socket management. In your case, with PUB/SUB, you're already using multicast. I think you're good.
I am new to networking.So I need to know the relations between sockets,IP, protocols(TCP/UDP), what is a socket for,and few other related words. I'm just trying to figure out how it works and want to learn. Can anyone help. at least put some links so that i can follow them. need it soon if possible
More elaborate information here : https://en.wikipedia.org/wiki/Internet_protocol_suite
Concise:
Your computer is on a LAN, that LAN is very likely to run on Ethernet.
On top of this Ethernet runs another protocol, IP.
On top of IP run several other protocols 2 of them are TCP and UDP.
UDP and TCP are multiplexing multiple communication channels that are each distinct on top of the same wire. It is doing that by using port numbers which are part of the protocol and which you can find in their respective header.
TCP and UDP are very much different:
TCP is Connection oriented whereas UDP is not.
UDP uses packets whereas TCP is a byte streaming protocol
TCP is reliable whereas UDP is not
other differences, this list is not exhaustive
To make TCP and UDP accessible to programs there is an application interface based on sockets. So you need a socket if you want to send or receive something.
https://en.wikipedia.org/wiki/Berkeley_sockets
But this is very broad topic and if it is your intention to start using this technology then you have to do a lot of reading.
I want an http message to be sent and processed quickly by a remote server, via an already established persistent TCP connection
How can I optimize the communication?
I have a few ideas, but I am not knowledgeable enough about networking to know if they make sense:
HTTP sits on top of TCP. But how does it work exactly? Specifically, if I send 1 http message, does it translate into only 1 tcp message? (I know the initial handshake takes 3 round trip time, but I do not care about this, as the connection is already established). I guess it depends on the Maximum Segment Size that the server can accept?
Can I ask the server for a bigger maximum segment size if needed? How can I do it (I use python, httplib and socket modules, it would be ideal in this language).
The remote server works with TCP, but could I try sending it UDP messages? I know UDP is faster, but could this idea work?
I'll allow myself to comment/answer in-text:
I have a few ideas, but I am not knowledgeable enough about networking
to know if they make sense:
Exactly! Read up on TCP and HTTP, on wikipedia, for a starter. It will make things much easier to discuss. Probably also faster than asking on stackoverflow ;)
HTTP sits on top of TCP. But how does it work exactly?
Well, exactly like protocols work over each other in a layered protocol stack. Read Wikipedia's TCP article, and about the OSI/ISO layer model.
Specifically, if I send 1 http message, does it translate into only 1
tcp message?
No. HTTP itself doesn't care (and it doesn't have to) into how many lower level packets communication gets split.
(I know the initial handshake takes 3 round trip time,
but I do not care about this, as the connection is already
established).
3 time? That's not something that makes sense. Read about the TCP handshake and how HTTP asks for a document.
I guess it depends on the Maximum Segment Size that the
server can accept?
Among a lot of different other factors; really: HTTP doesn't care the least!
Can I ask the server for a bigger maximum segment size if needed?
No. Your network stack will most probably automatically use the biggest MTU that works.
How can I do it (I use python, httplib and socket modules, it would be
ideal in this language).
The remote server works with TCP, but could I try sending it UDP messages?
There are some specialized HTTP-over-UDP protocols, but they are not HTTP. Generally, HTTP is spoken over TCP, but again, the internet works on a layered protocol stack, and higher level protocols usually don't care what transports their data -- you could perfectly well have an HTTP session over carrier pidgeons!
I know UDP is faster, but could this idea work?
It's not. That's a misconception. UDP doesn't have automatic re-requesting for packets that got lost along the way, which, for things like multimedia or games might make sense, but using TCP gives you an ordered session, which is necessary for HTTP.
I need to implement a client server architecture where the server sends
the same message to many clients over the internet.
I need to send a single message every 5 minutes about.
The message won't excede 5KB.
I need the solution to scale to a big number of clients connected (50.000-100.000)
I considered a bunch of solutions:
TCP Sockets
UDP Multicast
WCF http duplex service (comet)
I think I have to discard UDP solution because it is a good solution only for clients on the same network and it won't work over the internet.
I read somewhere that WCF multicast will cause a bottleneck if I have many clients connected but I can't find anywhere documentation showing performance statistics.
Tcp sockets seems to me the solution to chose.
What do you think about? Am I correct?
I'm certainly wrong when I say UDP doesn't work on internet... I thought
this because I read some articles pointing out that you need properly
configured routers in the network to support multicasting... I read of the
udp ports multicast range and thought it was meant to be locally.
Instead, the range 224.0.0.1 - 239.255.255.255 (Class D address group), can be reached over the internet
Considering that in my case reliability is not a crucial point, the udp multicast is a good choice.
The .net framework offers really helpful classes to accomplish this.
I can easily start an UdpClient and begin send data on a multicast address with two lines of code.
At client side it is really easy to.
There is the UdpSingleSourceMulticastClient class that does exactly what I need.
For what concernes reliability and security the .net framework has a smart and simple way of handle DoS attacks, DNS Rebinding attacks and Revers tunnel attacks that is described here: http://msdn.microsoft.com/en-us/library/ee707325(v=vs.95).aspx
The main question is: Do you care if the updates get to the clients?
If you DO then you will need to build something on top of UDP to add reliability. UDP datagrams are NOT reliable and so you should expect that some wont get to the destination. This is more likely if you are pushing UDP datagrams out quickly. Note that your clients might also get multiple copies of the same datagram in some situations with UDP.
50-100k connections with this level of traffic shouldn't be that difficult to achieve with TCP if you have a decent architecture.
See here for some blog posts that I've done on the subject.
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-concurrent-tcp-connections.html
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-or-more-concurrent-tcp-connections---part-2---perf-tests-from-day-0.html
http://www.serverframework.com/asynchronousevents/2010/12/one-million-tcp-connections.html
And here's some example code that deals with sending data to many clients.
http://www.serverframework.com/ServerFramework/latest/Docs/examples-datadistributionservers.html
Unicast (tcp sockets) will work fine for a relatively small amount of traffic such as this, but keep on top of multicasting technology, the situation is changing every year.