I'm writing a server that, among other things, needs to be constantly sending data in different multicast addresses. The packages being sent might be received by a client side (an app) which will be switching between the mentioned addresses.
I'm using Perfect (https://github.com/PerfectlySoft/Perfect) for writing the server side, however had no luck using the Perfect-Net module nor using CocoaAsyncSocket. How could i implement both the sender and the receiver using swift? Any could snippet would be really useful.
I've been reading about multicasting and when it comes to the receiver, i've notice that in most languages (i.e. java or c#) the receiver often indicates a port number and a multicast ip-address, but when is the connection with the server being made? When does the socket bind to the real server ip-address?
Thanks in advance
If we talk about the TCP/IP stack, only IP and UDP support broadcasts and multicasts. They're both connectionless, and this is why you see only sending and receiving to special multicast addresses, but no binds and connects. You see it in different languages because (a) protocols are language-agnostic and (b) most implementations put reasonable efforts in trying to be compatible with BSD sockets interface.
If you want that true multicast, you'll need to find a swift implementation of sockets that allow setting options. Usual names for this operation is setsockopt. Multicast sender side doesn't need anything beyond a basic UDP socket (I suggest using UDP, not IP), while sender needs to be added to a multicast group. This Python example pretty much describes it.
However, it's worth noting that routers don't route broadcasts and multicasts. Hence you cannot use it over internet. If you need to use internet in your project, I'd advise you to use TCP - or websockets if your clients are browsers - and send messages to "groups" of them manually.
I guess you actually want Perfect-Kafka or Perfect-Mosquitto - Message Queue which allows a server to publish live streams to the client side subscribers. Low-level sockets will not easily fulfill your requirement.
Related
I'm looking for the name of a protocol and example code that permits handing off IP/port connections to establish unmediated P2P after introduction through a server.
Simple example:
You and I both start chat programs that connect to chatintroduce.com (fictional server). I send you a "Hi! Wanna chat?" message. It doesn't get sent. Instead my chat program tells chatintroduce to send your chat program a request for connection. You respond to a prompt and your chat program tells chatintroduce to broker the connection. Chatintroduce establishes an initial two-way connection between us. Now, this final step is important, chatintroduce releases control and our two chat programs now talk directly to each other without any traffic through chatintroduce.
In other words, I construct packets which have your IP address and you receive them without interference from firewalls, NATs or any other technologies. In other words, true peer-to-peer connection independent of intermediate server.
I need to know what search terms to use to find appropriate technology. An RFC name would suffice. I've been searching for days without success.
I think what you are looking for is TCP/UDP hole punching which typically coordinates the P2P connection using a STUN server to determine the "capabilities" of the firewalls (e.g. is it a full cone nat? symmetric?).
https://en.wikipedia.org/wiki/Hole_punching_(networking)
We employed this at a company I worked for to create a kind of BitTorrent that could circumvent firewalls for streaming video between two peers.
Note that sometimes it is NOT possible to establish a connection without the intermediary.
What you are looking for is ICE protocol. RFC 5245. This protocol is used for connecting two peers through NAT traversal. There are some open source libraries and also some proprietary libraries for this. You can search google with ICE implementation.
You will also need to read about some additional protocols. These are used with ICE protocol. They are STUN and TURN.
For some cases you can't make P2P call 100% time. You will have to use a relay server. Like if the NAT combination of two peers are Symmetric vs Symmetric/PRC. That relay server is called TURN server.
Some technique like Port forwarding and TCP/UDP hole punching will help you to increase P2P rates.
See this answer for more information about which combination of NAT will require a relay server and which don't.
Thank you. I will be looking further into ICE, STUN, TURN, and hole-punching.
I also found n2n which looks like almost exactly what I wanted.
https://github.com/meyerd/n2n
http://xmodulo.com/configure-peer-to-peer-vpn-linux.html
With n2n, one makes a VPN with a super node that all other edge nodes know.
But once the introductions are made, the super node can be absent.
This was exactly what I wanted. I hope it works across platforms (linux, MacOS, Windows).
Again, I am still researching before implementation, so your advice was very important to me.
Thank you.
Use PJNATH. Its open source.
http://www.pjsip.org/pjnath/docs/html/
There is not much open source on NAT Traversal. As far as I know PJNATH is good.
For server you can use Google's Open source STUN and TURN server.
What is the purpose for using sip with VOIP ? is it just to know the remote ip address ?
If i know (by server) the remote IP address ,then established a direct TCP socket connection for call negotiation, and send the media over RTP protocol ,so am i still need sip protocol ? or how can sip help me here?
The Session Initiation Protocol does rather a lot more than find out a remote IP address/port/transport triple.
It lets two parties
negotiate the media streams (including codecs and transports) and
establish commonly understood extensions to the protocol.
It also describes how to build scalable infrastructure (proxies, using SRV and NAPTR records, back to back user agents), location services and a host of other details that go into making a voice (or any other kind of) call to arbitrary third parties.
then you have implemented a sip alternative. sip (session instanciation protocol) does just control the "phone call". if you want to do that on your own, why not? the only problem would be that there are many sip clients and just one (or few) clients using your protocol.
SIP allocates a IP:port to a voip call. The RTP flows (one for each direction) will then use this IP:port as a destination address. If you have only one static RTP flow to send to your server, it may be useful and ok to do what you said.
Otherwise, if there are many clients, or if your system has to change a lot, it's better to use a polished protocol which will dynamically allocate ports and establish your sessions.
nobody forces you to implement a standard.
e.g. why do you implement the media stream in RTP? most likely because you already have code that "talks" RTP (e.g. a library, or a raedymade application).
the nice thing about standards is, that it will work "out-of-the-box" with all other applications implementing the same standard.
if it is an open standard, there's another nice thing: other people have already spent a lot of brain power into getting the implementation right. you don't need to fall into the same problems.
I have no experience with sockets nor multiplayer programming.
I need to code a multiplayer mode for a game I made in c++. It's a puzzle game but the game mode will not be turn-based, it's more like cooperative.
I decided to use UDP, so I've read some tutorials, and all the samples I find decribes how to create a client that sends data and a server that receives it.
My game will be played by two players, and both will send and receive data to/from the other.
Do I need to code a client and a server?
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
Thanks, I'm kind of lost.
Read how the masters did it:
http://www.bluesnews.com/abrash/chap70.shtml
Read the code:
git clone git://quake.git.sourceforge.net/gitroot/quake/quake
Open one UDP socket and use sendto and recvfrom. The following file contains the functions for the network client.
quake/libs/net/nc/net_udp.c
UDP_OpenSocket calls socket (PF_INET, SOCK_DGRAM, IPPROTO_UDP)
NET_SendPacket calls sendto
NET_GetPacket calls recvfrom
Do I need to code a client and a server?
It depends. For a two player game, with both computers on the same LAN, or both on the open Internet, you could simply have the two computers send packets to each other directly.
On the other hand, if you want your game to work across the Internet, when one or both players are behind a NAT and/or firewall, then you have the problem that the NAT and/or firewall will probably filter out the other player's incoming UDP packets, unless the local player goes to the trouble of setting up port-forwarding in their firewall... something that many users are not willing (or able) to do. In that case, you might be better off running a public server that both clients can connect to, which forwards data from one client to another. (You might also consider using TCP instead of UDP in that case, at least as a fallback, since TCP streams are in general likely to have fewer issues with firewalls than UDP packets)
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
You don't have to, but you might as well -- there's no downside to using just a single socket and a single port, and it will simplify your code a bit.
Note that this answer is all about using UDP sockets. If you change your mind to use TCP sockets, it will almost all be irrelevant.
Do I need to code a client and a server?
Since you've chosen to to use UDP (a fair choice if your data isn't really important and benefits more from lower latency than reliable communication), you don't have much of a choice here: a "server" is a piece of code for receiving packets from the network, and your "client" is for sending packets into the network. UDP doesn't provide any mechanism for the server to communicate to the client (unlike TCP which establishes a 2 way socket). In this case, if you want to have two way communication between your two hosts, they'll each need server and client code.
Now, you could choose to use UDP broadcasts, where both clients listen and send on the broadcast address (usually 192.168.1.255 for home networks, but it can be anything and is configurable). This is slightly more complex to code for, but it would eliminate the need for client/server configuration and may be seen as more plug 'n play for your users. However, note that this will not work over the Internet.
Alternatively, you can create a hybrid method where hosts are discovered by broadcasting and listening for broadcasts, but then once the hosts are chosen you use host to host unicast sockets. You could provide fallback to manually specify network settings (remote host/port for each) so that it can work over the Internet.
Finally, you could provide a true "server" role that all clients connect to. The server would then know which clients connected to it and would in turn try to connect back to them. This is a server at a higher level, not at the socket level. Both hosts still need to have packet sending (client) and receiving (server) code.
Should I use the same socket to send and receive?
Well, since you're using UDP, you don't really have a choice. UDP doesn't establish any kind of persistent connection that they can communicate back and forth over. See the above point for more details.
Should I send and receive data in the same port?
In light of the above question, your question may be better phrased "should each host listen on the same port?". I think that would certainly make your coding easier, but it doesn't have to. If you don't and you opt for the 3rd option of the first point, you'll need a "connect back to me on this port" datafield in the "client's" first message to the server.
I need to implement a client server architecture where the server sends
the same message to many clients over the internet.
I need to send a single message every 5 minutes about.
The message won't excede 5KB.
I need the solution to scale to a big number of clients connected (50.000-100.000)
I considered a bunch of solutions:
TCP Sockets
UDP Multicast
WCF http duplex service (comet)
I think I have to discard UDP solution because it is a good solution only for clients on the same network and it won't work over the internet.
I read somewhere that WCF multicast will cause a bottleneck if I have many clients connected but I can't find anywhere documentation showing performance statistics.
Tcp sockets seems to me the solution to chose.
What do you think about? Am I correct?
I'm certainly wrong when I say UDP doesn't work on internet... I thought
this because I read some articles pointing out that you need properly
configured routers in the network to support multicasting... I read of the
udp ports multicast range and thought it was meant to be locally.
Instead, the range 224.0.0.1 - 239.255.255.255 (Class D address group), can be reached over the internet
Considering that in my case reliability is not a crucial point, the udp multicast is a good choice.
The .net framework offers really helpful classes to accomplish this.
I can easily start an UdpClient and begin send data on a multicast address with two lines of code.
At client side it is really easy to.
There is the UdpSingleSourceMulticastClient class that does exactly what I need.
For what concernes reliability and security the .net framework has a smart and simple way of handle DoS attacks, DNS Rebinding attacks and Revers tunnel attacks that is described here: http://msdn.microsoft.com/en-us/library/ee707325(v=vs.95).aspx
The main question is: Do you care if the updates get to the clients?
If you DO then you will need to build something on top of UDP to add reliability. UDP datagrams are NOT reliable and so you should expect that some wont get to the destination. This is more likely if you are pushing UDP datagrams out quickly. Note that your clients might also get multiple copies of the same datagram in some situations with UDP.
50-100k connections with this level of traffic shouldn't be that difficult to achieve with TCP if you have a decent architecture.
See here for some blog posts that I've done on the subject.
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-concurrent-tcp-connections.html
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-or-more-concurrent-tcp-connections---part-2---perf-tests-from-day-0.html
http://www.serverframework.com/asynchronousevents/2010/12/one-million-tcp-connections.html
And here's some example code that deals with sending data to many clients.
http://www.serverframework.com/ServerFramework/latest/Docs/examples-datadistributionservers.html
Unicast (tcp sockets) will work fine for a relatively small amount of traffic such as this, but keep on top of multicasting technology, the situation is changing every year.
I have to implement IPC mechanism (Sending short messages) between java/c++/python process running on the same system. One way to implement is using socket using TCP protocol. This requires maintain connection and other associated activities.
Instead I am thinking of using UDP protocol which does not requires connection and I can send messages.
My question is , does UDP on same machine ( for IPC ) still has same disadvantage has it is applicable when communicating across machines ( like un reliable packet delivery, out of order packet.
Yes, is still unrealiable. For local communication try to use named pipes or shared memory
Edit:
Don't know the requirements of your applications, did you considered something like MPI (altough Java is not well supported...) or, Thrift? ( http://thrift.apache.org/ )
Local UDP is still unreliable, but the major advantage is UDP multicast. You can have one data publisher and many data subscribers. The kernel does the job of delivering a copy of the datagram to each subscriber for you.
Unix local datagram sockets, on the other hand, are required to be reliable but they do not support multicast.
Local UDP is more unreliable than on a network, like 50%+ packet drop unreliable. It is a terrible choice, kernel developers have attributed the quality down to lack of demand.
I would recommend investigating message based middleware preferably with a BSD socket compatible interface for easy learning curve. A suggestion would be ZeroMQ which includes C++, Java and Python bindings.
Local UDP is both still unreliable and sometimes blocked by firewalls. We faced this in our MsgConnect product which uses local UDP for interthread communication. BTW MsgConnect can be an option for your task so that you don't need to deal with sockets. Unfortunately there's no Python binding, but "native" C++ and Java implementations exist.