So I am looking to get into network programming.
Specifically for a real time strategy game (in Unity 3D). I've done some basic GET and REQUEST stuff before, but I'm curious as to what I will need for doing real time game programming over a network.
My questions about the programming nomenclature are:
-What is the name of programming ("network programming" is too broad) involved in coding real time game networks?
-Can I do real time network programming with JSON or is there another technology I need to use?
-What search terms should I use to research network programming further (as what I've been pulling up has been to broad)?
Thank you!
Update:
Adding to the helpful question below, here is a good link explaining the different types of Authoritative Servers.
http://www.gabrielgambetta.com/fast_paced_multiplayer.html
Unity 4 uses a built-in version of RakNet to achieve network programming, typically referred to as Unity Networking.
Rather than using GET requests or sending JSON messages, you generally use the various Network APIs that Unity offers:
The host starts up with Network.InitializeServer.
Clients connect to the host with Network.Connect.
GameObjects that need to be synchronised across all players should have a NetworkView component attached to them; objects are then synched automatically without needing much effort on your part (for a basic object anyway).
Remote Procedure Calls let you invoke functions on clients.
Data is transferred with UDP packets since things need to happen quickly without ACK packets being sent back and forth, though there are different options to ensure the packets are ordered reliably within Unity if needed.
The Unity Network Reference Guide is a good read.
Related
I am writing my thesis about different game architectures and I have not found an answer to this one question I have. I have googled for hours but have not found an answer.
I was reading this article: https://gafferongames.com/post/what_every_programmer_needs_to_know_about_game_networking/
And there is this paragraph talking about p2p limitations for games :
The final limitation occurs because of the way the game synchronizes by sending just the command messages which change the state. In order for this to work it is necessary for all players to start from the same initial state. Typically this means that each player must join up in a lobby before commencing play, although it is technically possible to support late join, this is not common due to the difficulty of capturing and transmitting a completely deterministic starting point in the middle of a live game.
As stated by the author that when having a p2p connection, then players should join a lobby before starting a game, not joining an ongoing game.
He says that it is because it is hard transmitting a completely deterministic starting point in the middle of a live game.
Can someone please tell me why is that? Is it because there might be too much data to transfer and it might create some latency problems or is there another problem?
The amount of data to be transferred is not really the biggest problem, but can definitely have an impact.
To make things a little simpler, I will give you an example on what should be done in a client/server model, i.e. players connect to a central server, which runs the game engine and sends the state of the game to the players' computers. The server's task, is to keep the clients view of the game as consistent as possible, i.e. player health/position, position of objects in the map, etc.
Let's assume that you have 7 players in the match, with an 8th looking to join the in-progress game. The server needs to ensure that the new player receive the current state of the game in a timely manner, which requires processing/network resources on the server. Not only this is challenging from a programming perspective, but it results in a spike in resource usage on the server, which can theoretically impact other processes as well as the current match itself.
In the case of a P2P game model, where the server is one of the players' machines, the issue regarding the network amplifies, as usually servers have better Internet connections compared to your average household. The above still stands.
In the case of a pure P2P model, where each machine communicates with each other directly (I don't think there are modern games that actually do this), this becomes a very challenging problem of maintaining consistency in a fully meshed distributed system.
I am asking this question as a small part of my question series regarding game programming .
Refer this question as the main one .
Now suppose I want to develop a game on iphone - a simple online multiplayer board game.
Suppose its a casino table.
In the main question ChrisF has tell me of sort of Client - Server architecture for iphone online multiplayer game.
I want to know what sort network programming I have to do for this type of application .
What will be the responsibilities and activities carried out by client and server .
You can provide me link, tutorials or to the point answers , anything will be great help for me and will be really appreciable .
thanks
You'll want to write a socket application running on a server. When you have access to a wifi access point or edge/3g you can send data to it from your iphone application. This server can then handle the incoming data and send an appropriate reply to the people connected.
For server socket programming, take a look at this guide - http://beej.us/guide/bgnet/.
For iphone specific socket programming, take a look at the samples supplied with the Iphone SDK. This link also has some basic information.
A simple online multiplayer board game
Given that the iPhone isn't always connected to the internet, you might need a server to store state. Alternatively you could always stipulate that if one person loses their connection the game finishes.
Client to client would be the obvious choice for the latter. Both clients have a port they listen to, and send the other commands based on the board state. Like almost all online games the obvious choice would be to use UDP as it's fast and compact.
For the server architecture you will of course need some kind of server listening for commands and a game number. It would store your state in a datastore on the server, a mySQL database for example. UDP or even SOAP or JSON over HTTP would be the two obvious choices for this.
This second approach using JSON/SOAP route would be a lot easier for you to get started with, assuming the iPhone has a decent JSON or SOAP library which is not likely. I have no idea about UDP in Objective C, however in C it requires a certain level of knowledge which won't get you something to play with quickly.
As you already said, you will need a server, but you can have two kinds of design:
The server can serve only as a gateway between the players to connect one to each other: it's two uses are, first, to list the running games, and second, to list the IP addresses of the players so that each client will read the IP addresses and connect to them. This will require less bandwidth and processing power for the web server and more for the client which will host the game. In this configuration, one of the clients will act as a server and will send data to the others. If your game has only one of the players playing at a time, and not a huge lot a players, this is what you should use as what you pay is the server's power.
The server can also store all games' states: it might require far more processing power and/or bandwidth depending on your game.
Anyway, most of the time you will want only one machine (which can change during the game as in the first case) to do the processing and the others will only receive the computed data.
To program a networked game, you will need knowledge of sockets (deep knowledge in the first case because you will have to deal with NAT issues, routers blocking the way between clients). The guide to that is Beej's Guide to Network Programming, the number one ressource on this topic, although it doesn't focus on games.
If not too much processing is needed on the WWW server, you could deal with it with server scripting languages like PHP along with MySQL, but you're most likely to use your own server programmed in C++ (or in C). If you use C++, you might want to use an already existing library such as RakNet.
The client will obviously be programmed in Objective-C as it's on the iPhone. I believe there is a good networking framework looking at the number of online games, so you might as well not want to use an external server networking library.
It might be a bit too much for what you want to do, but you could also use the well known Torque Engine.
I'm looking for advice on the best way to implement some kind of bi-directional communication between a "server-side" application, written in Objective-C and running on a mac, and a client application running on an iPhone.
To cut a long story short, I'm adapting an existing library for use in a client-server environment. The library (which runs on the server) is basically a search engine which provides periodic results, and additionally can provide updates for any of those results at a later date. In an ideal world therefore I would be able to achieve the following with my hypothetical networking solution:
Start queries on the server.
Have the server "push" results to the client as they arrive.
Have the server "push" updates to individual results to the client as they arrive.
If I was writing this client to run on another Mac, I might well look at using Distributed Objects to mask the fact that the server was actually running remotely, but DO is not available on an iPhone.
If I was writing a more generic client-server application I would probably look at using HTTP to provide some kind of RESTful interface to searches, but this solution does not lend itself well to asynchronous updates and additionally what I am proposing does not fit well with the "stateless" tennet of REST: I would have to model my protocol so I "created" a search resource that I could subsequently query the state of and I would have to poll for updates to it.
One suggestion someone made was to make use of something like BLIP to provide me with a two-way pipe between the client and the server and implement my own "proxy" type objects for the server-side resources that knew how to fetch data from the server and additionally were addressable so that the server could push updates to them. Whilst BLIP provides the low-level messaging framework needed to communicate bi-directionally it still leaves me with a few questions:
How will I manage the lifetime of the objects on the server? I can have a message type that "creates" a search object, but when should that object be destroyed?
How well with this perform on an iPhone: if I have a persistent connection to the server will this drain the batteries too fast? This question is also pertinent in the HTTP world: most async updates are done using a COMET type hack which again requires a persistent connection.
So right now I'm still completely unsure what the best way to go is: I've done a lot of searching and reading but have not settled on any solution. I'm asking here on SO because I'm sure that there are many of you out there who have already solved this problem.
How have you gone about achieving real-time bidirectional networking between the iPhone and an Objective-C server-side app?
I plan on writing an automated bot for a game.
The tricky part is figuring out how they encoded their protocol... To make the bot run around is easy, simply make the character run and record what it does in wireshark. However, interpreting the environment is more difficult... It recieves about 5 packets each second if you are idle, hence lots of garbarge.
My plan: Because the game runs under TCP, I will use freecap (http://www.freecap.ru/eng) to force the game to connect to a proxy running on my machine. I will need this proxy to be capable of packet injection, or perhaps a server that is capable of resending captured packets. This way I can recreate and tinker around with what the server sends, and understand their protocol encoding.
Does anyone know where I can get a proxy that allows packet injection or where I can perform packet injection (not via hardware, as is the case with wireless or anything!)
Where of if I can find a server/proxy that resends captured packets (ie: replays a connection).
Any better tools or methodologies for pattern matching? Something which can highlight patterns from mutliple messages would be GREAT.
OR, is there a better way to decipher this here? Possibly a dissasembly strategy (via hooking a winsock function and starting the dissassembly from there) ? I have not done this before so I am not sure. OR , any other ideas?
Network traffic interception and protocol analysis is generally a less favored method to accomplish your goal here. For most modern games, encryption is a serious factor, and there are serious headaches associated with the protocol analysis for any but trivial factors of the most common gameplay scenarios.
Most modern implementations* of what you are trying to do rely on reading and manipulating the memory space and process of a running client. The client will have already done all the hard parts for you, including decrypting the traffic and sorting it into far more easy to read data structures. For interacting with the server you can call functions built into the client instead of crafting entire series of packets from scratch. The plus to this approach is that you have to do far less work to interpret the data and produce activity. The minus is that there is often some data in the network traffic that would be useful to a bot but is discarded by the client, or that you may want to send traffic to the server that the client cannot produce (which, in my own well-developed hierarchy for such, is a few steps farther down the "cheating" slope).
*...I say this having seen the evolution of the majority of MMORPG botting/hacking communities from network protocol analyzers like ShowEQ and Odin's Eye / Excalibur to memory-based applications like MacroQuest and InnerSpace. On that note, InnerSpace provides an excellent extensible framework for the memory/process-based variant of what you are attempting, and you should look into it as a basis for your project if you abandon the network analysis approach.
As I've done a few game bots in the past (for fun, not profit or griefing of course - writing game bots is a lot of fun), I recommend the following:
If you can code and there isn't cheat protection preventing you from doing it, I highly recommend writing an injected DLL for the following reasons:
Your DLL will be able to access the game's memory space directly, and once you reverse-engineer the data structures (either by poking around memory or by code disassembly), you'll have access to lots of data. This will also allow you to bypass any network encryption the game may have. The downside of accessing process memory directly is that offsets and data structures change between versions - however, data structures don't change very often with a stable game, and you can compensate offset changes by searching for code patterns instead of using fixed offsets.
Either way, you'll still be able to hook WinSock functions using API hooks (check out Microsoft Detours and the excellent but now-commercial madCodeHook).
otherwise, I can only advise that you give live/interactive packet editors like WPE Pro a try.
In most scenarios, the coolest methods (code reverse-engineering and direct memory access) tend to be the least productive. They require a lot of skill (to understand the code) and time, both initially (to go through all the code and develop code to interact with the data structure) and for maintainance (in case the game is being updated). (Of course, they sometimes do allow doing cool stuff which is impossible to do with the official client, but most of the time this is obvious as blatant cheating, and likely to attract the GMs quickly). Most of the time bots are made by replacing game graphics/textures with solid colours, and creating simple "pixel" bots which search for certain colours on the screen and react accordingly (e.g. click them).
Hope this helps, and remember - cheating is only fun when it doesn't make the game less fun for everyone else ;)
There are probably a few reasonable assumptions you can make that should simplify your task enormously. However, to make the best use of them you will probably need greater comfort with sleeves-rolled-up programming than it sounds like you have.
First, it's a safe bet that the encryption they are using falls into one of three categories:
None
Cheesy
Far better than you are likely to crack
With the odds of the middle case being very low.
Next, it's a safe bet that the packets are encrypted / decrypted close to the edge of the program (right as they come in, right before they go out) and that the body of the game deals with them in decrypted form.
Finally, the protocol they are using most likely consists of either
ascii with data blocks
binary goo
So do a little packet sniffing with a card set in promiscuous mode for unencrypted ascii. If you see some, great, you're ahead of the game. But if you don't give up the whole tapping-the-line idea and instead start following the code as it returns from the sending data out by breakpointing and stepping with a debugger. Figure the outermost layer or three will be standard network stuff, then will come the encryption layer, and beyond that the huge mass of stuff that deals with the protocol unencrypted.
You should be able to get this far in an hour if you're hot, a weekend if you're reasonably skilled, motivated, and diligent, and never if you are hopeless. But it is possible in principle (and doubtlessly far easier in practice) to do it this way.
Once you get to where something that looks like unencrypted goo comes in, gets mungled, and the mungled form goes out, then start worrying about what it means.
-- MarkusQ
A) I play a MMO and do not support bots, voting down...
B) Download Backtrack v.3, run an arpspoof on your default gateway and your host. There is an application that will spoof the remote host's SSL cert sslmitm (I believe is the name) which will then allow you to create a full connection through your host. Then fireup tcpdump/ethereal/wireshark (choose your pcap poison) and move around do random stuff to find out what packet is doing what. That will be your biggest challenge; but proxying with a Man in the Middle attack on yourself is the way to go.
C) I do not condone this activity, this information is only being provided as free information.
Sounds like there is not encryption going on, so you could do a network approach.
A great place to start would be to find the packet ID's - most of the time, something near the front of the packet is going to be an ID of the type of the packet. For example move could be 1, shoot fired could be "2", chat could be "4".
You can write your own proxy that listens on one port for your game to connect, and then connects to the server. You can make keypresses to your proxy fire off commands, or you can make your proxy write out debugging info to help you go further.
(I've written a bot for an online in game in PHP - of all things.)
I have some experience making multiplayer turn-based games using sockets, but I've never attempted a realtime action game. What kind of extra issues would I have to deal with? Do I need to keep a history of player actions in case lagged players do something in the past? Do I really need to use UDP packets or will TCP suffice? What else?
I haven't really decided what to make, but for the purpose of this question you can consider a 10-player 2D game with X Y movement.
'client server' or 'peer to peer' or something in between: which computer has authority over which game actions.
With turn based games, normally it's very easy to just say 'the server has ultimate authority and we're done'. With real time games, often that design is a great place to start, but as soon as you add latency the client movement/actions feels unresponsive. So you add some sort of 'latency hiding' allowing the clients input to affect their character or units immediately to solve that problem, and now you have to deal with reconciling issues when the client and servers gamestate starts to diverge. 9 times outta 10 that just fine, you pop or lerp the objects the client has affected over to the authoritative position, but that 1 out of 10 times is when the object is the player avatar or something, that solution is unacceptable, so you start give the client authority over some actions. Now you have to reconcile the multiple gamestates on the server, and open yourself up to a potentially 'cheating' via a malicious client, if you care about that sort of thing. This is basically where every teleport/dupe/whatever bug/cheat comes up.
Of course you could start with a model where 'every client has authority over 'their' objects' and ignore the cheating problem (fine in quite a few cases). But now you're vulnerable to a massive affect on the game simulation if that client drops out, or even 'just falls a little behind in keeping up with the simulation' - effectively every players game will end up being/feeling the effects of a lagging or otherwise underperforming client, in the form of either waiting for lagging client to catch up, or having the gamestate they control out of sync.
'synchronized' or 'asynchronus'
A common strategy to ensure all players are operating on the same gamestate is to simply agree on the list of player inputs (via one of the models described above) and then have the gameplay simulation play out synchronously on all machines. This means the simulation logic has to match exactly, or the games will go out of sync. This is actually both easier and harder than it sounds. It's easier because a game is just code, and code pretty much executes exactly the same when it's give the same input (even random number generators). It's harder because there are two cases where that's not the case: (1) when you accidently use random outside of your game simulation and (2) when you use floats. The former is rectified by having strict rules/assertions over what RNGs are use by what game systems. The latter is solved by not using floats. (floats actually have 2 problems, one they work very differently based on optimization configuration of your project, but even if that was worked out, they work inconsistently across different processor architectures atm, lol). Starcraft/Warcraft and any game that offers a 'replay' most likely use this model. In fact, having a replay system is a great way to test that your RNGs are staying in sync.
With an asynchronus solution the game state authorities simply broadcast that entire state to all the other clients at some frequency. The clients take that data and slam that into their gamestate (and normaly do some simplistic extrapolation until they get the next update). Here's where 'udp' becomes a viable option, because you are spamming the entire gamestate every ~1sec or so, dropping some fraction of those updates is irrelevant. For games that have relatively little game state (quake, world of warcraft) this is often the simplest solution.
There are a few factors involved in setting up multiplayer
The protocol, it's important that you decide whether you want TCP or UDP. UDP has less overhead but isn't guaranteed delivery. Conversely TCP is more trustworthy. Each game will have their preferred protocol. UDP for instance will work for a first person shooter but may not be suited for an RTS where information needs to be consistent
Firewall/Connection. Making sure your multiplayer game doesn't have to make 2000 outbound connections and uses a standard port so portforwarding is easy. Interfacing it with windows firewall will probably be an added bonus.
Bandwidth. This is important, how much data are you intending to push through a network connection? I guess this will come down to play testing and recording throughput. If you're requiring upwards of 200kb/s for each client you may want to rethink a few things.
Server Load. This is also important, how much processing is required by a server for a normal game? Do you need some super 8 core server with 16gb of RAM to run it? Are there ways of reducing it?
I guess there are heaps more, but really you want a game that is comfortable to play over the network and over a variety of connections.
Planning is your best friend. Figure out what your needs truly are.
Loading Data: Is every computer going to have the same models and graphics, and just names and locations are moved over the net. If every player can customize their character or other items, you will have to move this data around.
Cheating: do you have to worry about it? Can you trust what each client is saying. If not then you server side logic will look different than you client side logic. Imagine this simple case, each of your 10 players may have a different movement speed because of power ups. To minimize cheating you should to calculate how far each player can move between communication updates from the server, otherwise a player could hack there speed up and nothing would stop them. If a player is consistently a little faster than expected or has a one time jump, the server would just reposition them in the closest location that was possible, because it is likely clock skew or a one time interruption in communications. However if a player is constantly moving twice as far as possible then it may be prudent to kick them out of the game. The more math, the more parts of the game state you can double check on the server, the more consistent the game will be, incidentally this will make cheating harder.
How peer to peer is it: Even if the game is going to be peer to peer you will probably want to have one player start a game and use them as a server, this is much easier than trying to manage some of the more cloud based approaches. If there is no server then you need to work a protocol for solving disputes between 2 machines with inconsistent game states.
Again planning is your best friend Plan, Plan, Plan. If you think about a problem enough you can think your way through most of the problems. Then you can start thinking about the ones you haven't solved yet.
How important is avoiding cheating ?
[Can you trust any information coming from a client or can they be trusted and authenticated ?]
Object model
How are objects communicated from one machine to another ? How are actions carried out on an object ?
Are you doing client/server or peer to peer ?
Random Numbers
If you do a peer to peer then you need to keep them lock-stepped and the random numbers synchronized.
If you are doing client/server how do you deal with lag ? [dead reckoning ?]
There are a lot of non-trivial problems involved in network coding.
Check out RakNet both it's free to download code and it's discussion groups.
TCP is fine if your run on a LAN. But if you want to play online, you must use UDP and implement your own TCP-like layer: it's necessary to pass throw NAT routers.
You need to choose between Peer-to-peer or Client-Server communication. In Client-Server model, synchronisation and state of the world are easier to implement, but you might have a lack of reactivity online. In Pee-to-peer it's more complicated, but faster for the player.
Don't keep history of player action for game purpose (do it, but only for replay functionality). If you reach a point where it is necessary, prefer make every player wait.