I have found many code examples for sniffing packets through the network, but I am wondering if there is a way to sniff a specific local connection without having to filter by ip or port number. Assuming it will be lighter with less computation.
I thought of a hypothetical solution, forward the the packet to a socket that read the packet then transfer it to it destination by an other socket, like an internal middel man, witch is a heavy and less dynamic solution it will need an interaction with the kernel, :/
Related
I want to create a P2P network with the following characteristics:
low latency is not really important
loosing packages is okay
the nodes would only send tiny amounts of data around
there will be no NAT/firewall issues, every node has an open port on its public ip
every node is connected to every other node
Usually I would use TCP for anything not time-critical but the last requirements causes the nodes to have lots of open connections for a long time. If I remember correctly, using TCP to connect to 1000 servers would mean I had to use 1000 ports to handle these connections. UDP on the other hand, would only require a single port for each node.
So my question is: Is TCP able to handle the above requirements in a network with e.g. 1000 nodes without tweaking the system? Would UDP be better suited in this case? Is there anything else that would be a deal-breaker for either protocol?
With UDP you control the "connection state" and it is pretty much the best way to do anything peer to peer related IF you have a high number of nodes or care about bandwidth, memory and CPU overhead. By moving all the control to your application in regards to the "connection state" of each node you minimize the amount of wasted resources by making it fit your needs exactly.
You will bypass a lot of operating system specific weirdness that limits the effectiveness of TCP with high numbers of connections. There is TIME_WAIT bloat and tens to hundreds of OS specific settings which will need tweaking for every user of your P2P app if it needs those high numbers. A test app I made which allowed you to use UDP with ack or TCP showed only a 10% difference in performance regardless of operating system using UDP. TCP performance was always lower than the best UDP and its performance varied wildly by over 600% depending upon the OS. With tweaks you can make most OS perform roughly the same using TCP but by default most are not properly tweaked.
So in my opinion it is harder to make a reliable UDP P2P network compared to TCP but it is often needed. However I would only advise that route it if you were quite experienced with networking as there are a lot of "gotchas" to deal with. There are libraries which help with this like Raknet or Enet. They provide ways to do reliable UDP but it still takes a higher amount of networking knowledge to know how this all ties in together, whereas with TCP it is mostly hidden from you.
In a peer to peer network you often have messages like NODE PINGs that you may not care if each one is always received, you just care if you have received one recently. ie You may send a ping every 10 seconds, and disconnect the node after 60 seconds of no ping. This would mean you would need 6 ping packets in a row to fail, which is highly unlikely unless the node is really down. If you received even one ping in that 60 second period then the node is still active. A TCP implementation of this would have involved more latency and bandwidth as it makes sure EACH ping message gets through and will block any other data going out until it does. And since you cannot rely on TCP to reliably tell you if a connection is dead, you are forced to add similar PING features for TCP, on top of all the other things TCP is already doing extra with your packets.
Games also often have data that if its not received by a client it is no big deal because there are more packets coming in a few milliseconds which will invalidate any missed packets. ie Player is moving from A to Z over a time span of 1 second, his client sends out each packet, roughly 40 milliseconds apart ABCDEFG__I__KLMNOPQRSTUVWXYZ Do we really care if we miss "H and J" since every 40ms we are receiving updates? Not really, this is where prediction can come into it, but this is usually not relevant to most P2P projects. If that was TCP instead of UDP then it would have increased bandwidth requirements and added latency to the rest of the packets being received as the data will be resent until it arrives, on top of the extra latency it is already adding by acking everything.
Essentially you can lower latency and network overhead for many messages in a peer to peer network using UDP. However there will always be some messages which NEED to be sent reliably and that requires you to basically implement some reliable way to get packets to that node, similar to that of TCP. And this is where you need some level of expertise if you want a reliable peer to peer network. Some things to look into include sequencing packets with a number, message ACKs, etc.
If you care a lot about efficiency or really need tens of thousands of connections then implementing your specific protocol in UDP will always be better than TCP. But there are cases to be made for TCP, like if the time to make the project matters or if you are a new to network programming.
If I remember correctly, using TCP to connect to 1000 servers would mean I had to use 1000 ports to handle these connections.
You remember wrong.
Take a web server which is listening on port 80 and can handle 1000s of connections at the same time on this single port. This is because a connection is defined by the tuple of {client-ip,client-port,server-ip,server-port}. And while server-ip and server-port are the same for all connections to this server the client-ip and client-port are not. Even if the client-ip is the same (i.e. same client) the client would pick a different source port.
... with e.g. 1000 nodes without tweaking the system?
This depends on the system since each of the open connections needs to preserve the state and thus needs memory. This might be a problem for embedded systems with only little memory.
In any case: if your protocol is just sending small messages and if packet loss, reordering or duplication are acceptable than UDP might be the better choice because the overhead (connection setup, ACK..) is smaller and it takes less memory. You could also use a single socket to exchange data with all 1000 nodes whereas with TCP you would need a separate socket for each connection (socket is not the same as port!). Using only a single socket might allow for a simpler application design.
I want to amend the answer by Steffen with a few points:
1000 connections are nothing for any normal computer and OS.
UDP fits your requirements. It might be easier to program because it is message oriented. TCP provides a stream of bytes. You need to layer a messaging protocol on top of that which is not that easy. Also, you need to handle broken TCP connections by reconnecting.
Ports are not scarce. No problem with consuming 1000 ports.
I have a machine, with no external IP address, it will need to send UDP packets to the outside world. Only NAT access.
Will this work?
It is really hard to prototype this in our environment.
It is still really under construction.
Any thoughts on how I can prototype this?
Most of the home network configurations in the world are made of a PC with an internal IP and a router with a public IP that NAT the internal one. (Independently of UDP/TCP or whatever protocol that needs to go out)
I see no troubles with it
It should work.
Ensure that for the socket created, set the TTL (time-to-live) to a value that is sufficiently large to cover the possible number of router hops to reach the destination. Running traceroute to the destination IP will give you a rough idea on the number of hops. Note that this value can change depending on network conditions. So it's best to set this to a larger value. Refer to sockets IOCtl API documentation for the syntax for setting TTL.
Finally, remember that UDP is not a reliable protocol. So even after taking the necessary steps above, the packet may not reach its destination. However, if the entire network, including the intermediary routers, is within a controlled environment, such as a corporate intranet, chances of packet drop are minimal.
If you want to add reliability on top of UDP, you can adopt a NAK based algorithm where packets are stamped with a sequence number. Various resources might advise you that if you need to add reliability over UDP you should consider TCP, but my experience has been that if your app runs in a controlled environment with very minimal chance of packet drops and you need fast connection setup and tear down, adding a lightweight reliability over UDP has its merits. Also TCP connections take up valuable space in the OS kernel whereas UDP don't. This could also be a consideration if you want to support very large number of 'connections' in a constrained environment.
At the end of the day you need to experiment a little to figure out what works best for you.
To prototype, I would set up a NAT server using something like Linux and then start working from there. Real world traffic scenarios that you want to simulate will determine where the client and server are to be located on either side of the NAT. That is, if the traffic should go through an ISP or all within a controlled environment.
HTH
I have no experience with sockets nor multiplayer programming.
I need to code a multiplayer mode for a game I made in c++. It's a puzzle game but the game mode will not be turn-based, it's more like cooperative.
I decided to use UDP, so I've read some tutorials, and all the samples I find decribes how to create a client that sends data and a server that receives it.
My game will be played by two players, and both will send and receive data to/from the other.
Do I need to code a client and a server?
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
Thanks, I'm kind of lost.
Read how the masters did it:
http://www.bluesnews.com/abrash/chap70.shtml
Read the code:
git clone git://quake.git.sourceforge.net/gitroot/quake/quake
Open one UDP socket and use sendto and recvfrom. The following file contains the functions for the network client.
quake/libs/net/nc/net_udp.c
UDP_OpenSocket calls socket (PF_INET, SOCK_DGRAM, IPPROTO_UDP)
NET_SendPacket calls sendto
NET_GetPacket calls recvfrom
Do I need to code a client and a server?
It depends. For a two player game, with both computers on the same LAN, or both on the open Internet, you could simply have the two computers send packets to each other directly.
On the other hand, if you want your game to work across the Internet, when one or both players are behind a NAT and/or firewall, then you have the problem that the NAT and/or firewall will probably filter out the other player's incoming UDP packets, unless the local player goes to the trouble of setting up port-forwarding in their firewall... something that many users are not willing (or able) to do. In that case, you might be better off running a public server that both clients can connect to, which forwards data from one client to another. (You might also consider using TCP instead of UDP in that case, at least as a fallback, since TCP streams are in general likely to have fewer issues with firewalls than UDP packets)
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
You don't have to, but you might as well -- there's no downside to using just a single socket and a single port, and it will simplify your code a bit.
Note that this answer is all about using UDP sockets. If you change your mind to use TCP sockets, it will almost all be irrelevant.
Do I need to code a client and a server?
Since you've chosen to to use UDP (a fair choice if your data isn't really important and benefits more from lower latency than reliable communication), you don't have much of a choice here: a "server" is a piece of code for receiving packets from the network, and your "client" is for sending packets into the network. UDP doesn't provide any mechanism for the server to communicate to the client (unlike TCP which establishes a 2 way socket). In this case, if you want to have two way communication between your two hosts, they'll each need server and client code.
Now, you could choose to use UDP broadcasts, where both clients listen and send on the broadcast address (usually 192.168.1.255 for home networks, but it can be anything and is configurable). This is slightly more complex to code for, but it would eliminate the need for client/server configuration and may be seen as more plug 'n play for your users. However, note that this will not work over the Internet.
Alternatively, you can create a hybrid method where hosts are discovered by broadcasting and listening for broadcasts, but then once the hosts are chosen you use host to host unicast sockets. You could provide fallback to manually specify network settings (remote host/port for each) so that it can work over the Internet.
Finally, you could provide a true "server" role that all clients connect to. The server would then know which clients connected to it and would in turn try to connect back to them. This is a server at a higher level, not at the socket level. Both hosts still need to have packet sending (client) and receiving (server) code.
Should I use the same socket to send and receive?
Well, since you're using UDP, you don't really have a choice. UDP doesn't establish any kind of persistent connection that they can communicate back and forth over. See the above point for more details.
Should I send and receive data in the same port?
In light of the above question, your question may be better phrased "should each host listen on the same port?". I think that would certainly make your coding easier, but it doesn't have to. If you don't and you opt for the 3rd option of the first point, you'll need a "connect back to me on this port" datafield in the "client's" first message to the server.
I need to implement a client server architecture where the server sends
the same message to many clients over the internet.
I need to send a single message every 5 minutes about.
The message won't excede 5KB.
I need the solution to scale to a big number of clients connected (50.000-100.000)
I considered a bunch of solutions:
TCP Sockets
UDP Multicast
WCF http duplex service (comet)
I think I have to discard UDP solution because it is a good solution only for clients on the same network and it won't work over the internet.
I read somewhere that WCF multicast will cause a bottleneck if I have many clients connected but I can't find anywhere documentation showing performance statistics.
Tcp sockets seems to me the solution to chose.
What do you think about? Am I correct?
I'm certainly wrong when I say UDP doesn't work on internet... I thought
this because I read some articles pointing out that you need properly
configured routers in the network to support multicasting... I read of the
udp ports multicast range and thought it was meant to be locally.
Instead, the range 224.0.0.1 - 239.255.255.255 (Class D address group), can be reached over the internet
Considering that in my case reliability is not a crucial point, the udp multicast is a good choice.
The .net framework offers really helpful classes to accomplish this.
I can easily start an UdpClient and begin send data on a multicast address with two lines of code.
At client side it is really easy to.
There is the UdpSingleSourceMulticastClient class that does exactly what I need.
For what concernes reliability and security the .net framework has a smart and simple way of handle DoS attacks, DNS Rebinding attacks and Revers tunnel attacks that is described here: http://msdn.microsoft.com/en-us/library/ee707325(v=vs.95).aspx
The main question is: Do you care if the updates get to the clients?
If you DO then you will need to build something on top of UDP to add reliability. UDP datagrams are NOT reliable and so you should expect that some wont get to the destination. This is more likely if you are pushing UDP datagrams out quickly. Note that your clients might also get multiple copies of the same datagram in some situations with UDP.
50-100k connections with this level of traffic shouldn't be that difficult to achieve with TCP if you have a decent architecture.
See here for some blog posts that I've done on the subject.
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-concurrent-tcp-connections.html
http://www.serverframework.com/asynchronousevents/2010/10/how-to-support-10000-or-more-concurrent-tcp-connections---part-2---perf-tests-from-day-0.html
http://www.serverframework.com/asynchronousevents/2010/12/one-million-tcp-connections.html
And here's some example code that deals with sending data to many clients.
http://www.serverframework.com/ServerFramework/latest/Docs/examples-datadistributionservers.html
Unicast (tcp sockets) will work fine for a relatively small amount of traffic such as this, but keep on top of multicasting technology, the situation is changing every year.
I've been reading up on basic network programming, but am having a difficult time finding a straight-forward explanation for what exactly and socket is, and how it relates to either the OSI or TCP/IP stack.
Can someone explain to me what a socket is? Is it a programmer- or API-defined data structure, or is it a hardware device on a network card?
What layers of the mentioned network models deal with "raw" sockets? Transport layer? Network layer?
In terms of the data they pass between them, are socket text-based or binary?
Is there an alternative to sockets-based network programming? Or do all networked applications use some form of socket?
If I can get this much I should have a pretty clear understanding of everything else I'm reading. Thanks for any help!
Short answers:
Socket is an abstraction of an IP connection endpoint - so if you think of it as an API structure, you are not very far off. Please do read http://en.wikipedia.org/wiki/Internet_socket
Internet layer i.e. IP Protocol. In practice you usually use explicitly sockets that bind to a certain transport layer parameters (datagram/UDP or stream/TCP)
Sockets send data, in network byte order - whether it is text or binary, depends on the upper layer protocol.
Theoretically, probably yes - but in practice all IP traffic is done using 'sockets'
Socket is a software mechanism provided by the operating system. Like its name implies, you can think of it like an "electrical outlet" or some electrical connector, even though socket is not a physical device, but a software mechanism. In real world when you have two electrical connectors, you can connect them with a wire. In the same way in network programming you can create one socket on one computer and another socket on another computer and then connect those sockets. And when you write data to one of them, you receive it on the other one. There are also a few different kinds of sockets. For example if you are programming a server software, you want to have a listening socket which never sends or receives actual data but only listens for and accepts incoming connections and creates a new socket for each new connection.
A socket, in C parlance, is a data structure in kernel space, corresponding to one end-point of a UDP or TCP session (I am using session very loosely when talking about UDP). It's normally associated with one single port number on the local side and seldom more than one "well-known" number on either side of the session.
A "raw socket" is an end-point on, more or less, the physical transport. They're seldom used in applications programming, but sometimes used for various diagnostic things (traceroute, ping, possibly others) and may required elevated privileges to open.
Sockets are, in their nature, a binary octet-transport. It is not uncommon to treat sockets (TCP sockets, at least) as being text-based streams.
I have not yet seen a programming model that doesn't involve something like sockets, if you dig deep enough, but there have certainly been other models of doing networking. The "/net/" pseudo-filesystem, where opening "/net/127.0.0.0.1/tcp/80" (or "tcp/www") would give you a file handle where writes end up on a web server on localhost is but one.
Suppose your PC at home, and you have two browser windows open.
One looking at the facebook website, and the other at the Yahoo website.
The connection to facebook would be:
Your PC – IP1+port 30200 ——– facebook IP2 +port 80 (standard port)
The combination IP1+30200 = the socket on the client computer and IP2 + port 80 = destination socket on the facebook server.
The connection to Yahoo would be:
your PC – IP1+port 60401 ——–Yahoo IP3 +port 80 (standard port)
The combination IP1+60401 = the socket on the client computer andIP3 + port 80 = destination socket on the Yahoo server.