I need to write a ftp program linking two, only two, computers, but this must be done using raw sockets on promiscuous mode. Since the socket will receive any data that come across the network device, my problem is how to address the correct program. For example: Imagine I have a server on a machine and two clients on the other. The server sends a message, how can the two clients know which of them will receive the message? Another problematic situation would be if I open a client and a server in the same machine, both may start to commune with each other.
Related
I have a device (made in china) that works great but with a very annoying flaw. It's a massage device that will only work if it is connected to the internet at startup. Once it starts, it would work without the internet until it is turned off. So in short, if the internet is down, it will never work or it is impossible to bring this device on an area with no connection.
What I'm planning to do is emulate its server/client communication at start-up by using an old-PC that would run the server emulator software and some DNS re-routing and a WiFi router (using the PC as WAN).
I'll connect the device on the router, and its communication with the Chinese Server is emulated by the PC.
Can anyone give me an idea where to start, tools needed etc? I'm a programmer but new to packet emulation/replay. I've sniffed packets that the device does and its always consistent on every start/restart of the device. Here's the wireshark capture (IMG):
WireShark Packet Capture
You have two tasks, IMHO neither of them requires special tools:
Redirect requests to local address
You need to setup local DNS server and configure it to provide local IP address on hengdetech lookup. I didn't dive into this part, but google has plenty of forums discussing necessary config for different DNS servers.
Device most likely gets DNS server address from DHCP server. So you need to point device to local DNS instead of default one by configure DHCP server (which is run on your router, I suppose).
Once your DHCP configures local DNS and local DNS redirects hedgdete to local IP address, you are free to go to the second part which is
Emulate server behavior
Server must wait for some initial data from client before sending some data back. Such logic doesn't fit well into common "replay packets" tool capabilities, but it's quite easy to implement dumb TCP server with desired functionality on whatever language you like. Open socket, wait for data, send data, not a big deal, isn't it? So I wouldn't bother searching for the right tool, but would creat new instead.
If you need quick proof of concept, that you understand data flow right, before start coding, you may try netcat utility on Unix system for quick server emulation. I did the following once: extracted exchanged payload data from traces and saved as binaries (e.g. file1.bin is sent by server after the first request, file2.bin is sent after the second etc), then used something like this: cat - file1.bin - file2.bin | nc -l -p <local_port> -n. Using cat without dashes results into nc sending all files content to client, once it connects. Dash means "use data from console input rather than file" and they are used only as flow control. cat hangs on the first dash waiting for user input, so pressing ctrl+d proceeds to next file content and sends file1.bin, then hangs on the second dash. So you wait for the first request, press ctrl+d to send file1.bin content, then wait for the second request, press ctrl+d to send file2.bin content and so on.
EDIT
You generally got the idea of extracting data from wireshark traces right, but I want to clarify some subtlenesses.
When you said "saved binaries", did you mean to save individually captured packets as binaries so that netcat could send them to clients as a reply one by one (with the help of dash) ? If so, how do I save captured packets as binaries? Can I just extract them from the wireshark capture files, paste it on a new file and save as binary?
Only payload part of the packet needs to be saved, not the whole packet. TCP header, IP header and so on shouldn't be extracted, only tcp data section should be saved. Check second part of this answer for howto. I suppose, TCP data in your case consists of binary data rather than plain text, so you need to copy it as "raw binary" and use some binary editor allowing to paste "raw binary" into file such as frhed. So yes, you create new file and paste data copied from wireshark into it, so file consists of exactly the same bytes as packet payload.
Another thing to mention, TCP is a stream protocol, one packet doesn't always mean one response message. If you see several TCP packets of maximum size (aka MTU, which is usually 1.5 Kbytes) without PSH flag followed by packet of lesser size with PSH flag, they all contain parts of the single response message, so their payloads should be combined into one "fileX.bin".
I am presently working on a client-server solution to transfer files to another machine via a socket network connection. I am fairly new to the whole client-server thing and therefore have the following - admittedly very basic - question:
For the file transfer, does it make any difference if I am sending the file from a client to a server or from a server to a client?
Any qualified insight into this will be much appreciated!
For the file transfer, does it make any difference if I am sending the file from a client to a server or from a server to a client?
Basically, no it does not matter. Once you have the connection made you are free to send data in both directions. Although you have to consider that a server won't accept data that is sent to it unless it explicitly reads from the socket.
To be a bit more general, server and client are completely arbitrary for a home brewed implementation of data transfer. If you boil this down to the simplest concept then you are just opening a socket and writing data to it on one side, and on the other side you are reading from a different socket.
You might choose to implement a single client program capable of connecting other clients (P2P) and sending files back and forth. In that case you could call the "server" the program that is currently sending the file, and the "client" is the program currently receiving.
Alternatively, you could implement two programs, one for client and one for server. Your server will listen for connections and the client will decide when it wants to connect to the server.
Remember that there are network limitations for connecting. If the program that is listening for connections is behind a firewall then you have to be sure you are forwarding the correct ports. If you are connecting machines within a LAN then you probably have nothing to worry about.
We have an application that broadcasts data using UDP from a server system to client applications running on multiple Windows XP PC's. This is on a LAN, typically Gigabit. This has been running fine for some years.
We now have a requirement to have two (or more) of the client applications running on each quad core PC, with each instance of the application receiving the broadcast data. The method I have used to implement this is to give each client PC multiple IP addresses. Each client app then connects to the server using the same port number but on a different IP. This works functionally but the performance for some reason is very poor. My data transfer rate is cut by around a factor of 10!
To get multiple IP addresses I have tried both using two NIC adapters and assigning multiple IP addresses to a single NIC in the advanced TCP/IP network properties. Both methods seem to give similarly poor performance. I also tried using several different manufacturers NICs but that didn't help either.
One thing I did notice is that the data seems to come over more fragmented. With just a single client on a PC if I send 20kBytes of data to the client it almost always receives it all in one chunk. But with two clients running the data seems to mostly come over in blocks the size of a frame (1500 bytes) so my code has to iterate around more times. But I wouldn't expect this on its own to cause such a dramatic performance hit.
So I guess my question is does any one know why the performance is so much slower and if anything can be done to speed it up?
I know I could re-design things so that the server only sends data to one client per PC, and that client could then mirror the data on to the other clients on the same PC. But that is a major redesign and re-coding effort so I'd like to keep that as a last resort.
Instead of creating one IP address for each client, try using setsockopt() to enable the SO_REUSEADDR option for each of your sockets. This will allow all of your clients to bind to the same port on the same host address and receive the broadcast data. Should be easier to manage than the multiple NIC/IP address approach.
SO_REUSEADDR will allow broadcast and multicast sockets to share the same port and address. For more info see:
SO_REUSEADDR and UDP behavior in Windows
and
Uses of SO_REUSEADDR?
I have no experience with sockets nor multiplayer programming.
I need to code a multiplayer mode for a game I made in c++. It's a puzzle game but the game mode will not be turn-based, it's more like cooperative.
I decided to use UDP, so I've read some tutorials, and all the samples I find decribes how to create a client that sends data and a server that receives it.
My game will be played by two players, and both will send and receive data to/from the other.
Do I need to code a client and a server?
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
Thanks, I'm kind of lost.
Read how the masters did it:
http://www.bluesnews.com/abrash/chap70.shtml
Read the code:
git clone git://quake.git.sourceforge.net/gitroot/quake/quake
Open one UDP socket and use sendto and recvfrom. The following file contains the functions for the network client.
quake/libs/net/nc/net_udp.c
UDP_OpenSocket calls socket (PF_INET, SOCK_DGRAM, IPPROTO_UDP)
NET_SendPacket calls sendto
NET_GetPacket calls recvfrom
Do I need to code a client and a server?
It depends. For a two player game, with both computers on the same LAN, or both on the open Internet, you could simply have the two computers send packets to each other directly.
On the other hand, if you want your game to work across the Internet, when one or both players are behind a NAT and/or firewall, then you have the problem that the NAT and/or firewall will probably filter out the other player's incoming UDP packets, unless the local player goes to the trouble of setting up port-forwarding in their firewall... something that many users are not willing (or able) to do. In that case, you might be better off running a public server that both clients can connect to, which forwards data from one client to another. (You might also consider using TCP instead of UDP in that case, at least as a fallback, since TCP streams are in general likely to have fewer issues with firewalls than UDP packets)
Should I use the same socket to send and receive?
Should I send and receive data in the same port?
You don't have to, but you might as well -- there's no downside to using just a single socket and a single port, and it will simplify your code a bit.
Note that this answer is all about using UDP sockets. If you change your mind to use TCP sockets, it will almost all be irrelevant.
Do I need to code a client and a server?
Since you've chosen to to use UDP (a fair choice if your data isn't really important and benefits more from lower latency than reliable communication), you don't have much of a choice here: a "server" is a piece of code for receiving packets from the network, and your "client" is for sending packets into the network. UDP doesn't provide any mechanism for the server to communicate to the client (unlike TCP which establishes a 2 way socket). In this case, if you want to have two way communication between your two hosts, they'll each need server and client code.
Now, you could choose to use UDP broadcasts, where both clients listen and send on the broadcast address (usually 192.168.1.255 for home networks, but it can be anything and is configurable). This is slightly more complex to code for, but it would eliminate the need for client/server configuration and may be seen as more plug 'n play for your users. However, note that this will not work over the Internet.
Alternatively, you can create a hybrid method where hosts are discovered by broadcasting and listening for broadcasts, but then once the hosts are chosen you use host to host unicast sockets. You could provide fallback to manually specify network settings (remote host/port for each) so that it can work over the Internet.
Finally, you could provide a true "server" role that all clients connect to. The server would then know which clients connected to it and would in turn try to connect back to them. This is a server at a higher level, not at the socket level. Both hosts still need to have packet sending (client) and receiving (server) code.
Should I use the same socket to send and receive?
Well, since you're using UDP, you don't really have a choice. UDP doesn't establish any kind of persistent connection that they can communicate back and forth over. See the above point for more details.
Should I send and receive data in the same port?
In light of the above question, your question may be better phrased "should each host listen on the same port?". I think that would certainly make your coding easier, but it doesn't have to. If you don't and you opt for the 3rd option of the first point, you'll need a "connect back to me on this port" datafield in the "client's" first message to the server.
I have clients that need to all connect to a single server process. I am using UDP discovery for the clients to find the server. I have the client and server exchange IP address and port number, so that a TCP/IP connection can be established after completion of the discovery. This way the packet size is kept small. I see that this could be done in one of two ways using UDP:
Each client sends out its own multicast message in search of the server, which the server then responds to. The client can repeat sending this multicast message in regular intervals (in the case that the server is down) until the server responds.
The server sends out a multicast message beacon at regular intervals. The clients subscribe to the multicast group and in this way receives the server's multicast message and complete the discovery.
In 1. if there are many clients then initially there would be many multicast messages transmitted (one from each client). Only the server would subscribe and receive the multicast messages from the clients. Once the server has responded to the client, the client ceases to send out the multicast message. Once all clients have completed their discovery of the server no further multicast messages are transmitted on the network. If however, the server is down, then each client would be sending out a multicast message beacon in intervals until the server is back up and can respond.
In 2. only the server would submit a multicast message beacon in regular intervals. This message would end up getting routed to all clients that are subscribed to the multicast group. Once the clients receive the packet the client's UDP listening socket gets closed and they are no longer subscribed to the multicast group. However, the server must continue to send the multicast beacon, so that new clients can discover it. It would continue sending out the beacon at regular intervals regardless of whether any clients are out their requiring discovery or not.
So, I see pros and cons either way. It seems to me that #1 would result in heavier load initially, but this load eventually reduces down to zero. In #2 the server would continue sending out a beacon forever.
UDP and multicast is a fairly new topic to me, so I am interested in finding out which would be the preferred approach and which would result in less network load.
I've used option #2 in the past several times. It works well for simple network topologies. We did see some throughput problems when UDP datagrams exceeded the Ethernet MTU resulting in a large amount of fragmentation. The largest problem that we have seen is that multicast discovery breaks down in larger topologies since many routers are configured to block multicast traffic.
The issue that Greg alluded to is rather important to consider when you are designing your protocol suite. As soon as you move beyond simple network topologies, you will have to find solutions for address translation, IP spoofing, and a whole host of other issues related to the handoff from your discovery layer to your communications layer. Most of them have to do specifically with how your server identifies itself and ensuring that the identification is something that a client can make use of.
If I could do it over again (how many times have we uttered this phrase), I would look for standards-based discovery mechanisms that fit the bill and start solving the other protocol suite problems. The last thing that you really want to do is come up with a really good discovery scheme that breaks the week after you deploy it because of some unforeseen network topology. Google service discovery for a starting list. I personally tend towards DNS-SD but there are a lot of other options available.
I would recommend method #2, as it is likely (depending on the application) that you will have far more clients than you will servers. By having the server send out a beacon, you only send one packet every so often, rather than one packet for each client.
The other benefit of this method, is that it makes it easier for the clients to determine when a new server becomes available, or when an existing server leaves the network, as they don't have to maintain a connection to each server, or keep polling each server, to find out.
Both are equally viable methods.
The argument for method #1 would be that in normal principle, clients initiate requests, and servers listen and respond to them.
The argument for method #2 would be that the point of multicast is so that one host can send a packet and it can be received by many clients (one-to-many), so it's meant to be the reverse of #1.
OK, as I think about this I'm actually drawn to #2, server-initiated beacon. The problem with #1 is that let's say clients broadcast beacons, and they hook up with the server, but the server either goes offline or changes its IP address.
When the server is back up and sends its first beacon, all the clients will be notified at the same time to reconnect, and your entire system is back up immediately. With #1, all of the clients would have to individually realize that the server is gone, and they would all start multicasting at the same time, until connected back with the server. If you had 1000 clients and 1 server your network load would literally be 1000x greater than method #2.
I know these messages are most likely small, and 1000 packets at a time is nothing to a UDP network, but just from a design standpoint #2 feels better.
Edit: I feel like I'm developing a split-personality disorder here, but just thought of a powerful point of why #1 would be an advantage... If you ever wanted to implement some sort of natural load balancing or scaling with multiple servers, design #1 works well for this. That way the first "available" server can respond to the client's beacon and connect to it, as opposed to #2 where all the clients jump to the beaconing server.
Your option #2 has a big limitation in that it assumes that the server can communicate more or less directly with every possible client. Depending on the exact network architecture of your operational system, this may not be the case. For example, you may be depending that all routers and VPN software and WANs and NATs and whatever other things people connect networks together with, can actually handle the multicast beacon packets.
With #1, you are assuming that the clients can send a UDP packet to the server. This is an entirely reasonable expectation, especially considering the very next thing the client will do is make a TCP connection to the same server.
If the server goes down and the client wants to find out when it's back up, be sure to use exponential backoff otherwise you will take the network down with a packet storm someday!