Finding out where the bandwidth is being used - bandwidth

Recently on some of our sites the bandwidth being used has risen tenfold and we want to know what is using it. All we have so far is the MRTG graphs that just say 'you used X much at this time'.
I would like to break down this information by service (ftp, http etc), direction (inbound or outbound), source and destination address and when.
To this end I have started logging all the packets through iptables. This seems to give me what I want at a very detailed level. Is there some tool that will take that log file and condense the information in some way.
Ideally it would provide information like:
1.1.1.1 send 123 bytes to 2.2.2.2 via http at 2011-02-24T10:03:17
Any ideas as to how I might achieve this or at least some pointers to making sense of the iptables log format.

A good tool for network analysis is Ethereal
Not sure if it will give you what you want exactly, but it will tell you what's going on with your network.
http://www.ethereal.com/

Related

Prioritizing packets streaming in OpenFlow, mininet

I am doing work on security and priority in SDN.
I have a typology of this form
h1---s1--s2---h2
I have to pass packets of h1 to h2, prioritizing packets streaming. And check somehow they are correct. i don't know how to do that with OpenFlow and MiniNet. Anyone have any idea how to do it?
I believe from security and priority, you mean to say FireWall and Traffic shaping services respectively. As far as I know you cannot achieve traffic shaping just be OpenFlow(Though firewall service could be achieved to an extent by OpenFlow). In short OpenFlow is meant for setting forwarding rules for an in coming packet. Though there are plugins such as NICIRA which can add some functionality such as handling TCP, UDP, ARP packets, but the primary job of openflow packet is to forward packet to next port, drop or punt to controller based on flow rules.
However in your case, You need to implement SFC. You need to deploy firewall or traffic shaping applications(an opensource traffic shaping application here) in a separate server. you need to configure VXLAN on S1 and S2. Then add flows on S1 and S2 such that packets are forwarded to the server. and other flow to forward the flows from the server to next destination. In the Server you apply firewall and traffic shaping rules on packet by packet basis.
Hope this basic information atleast helps you to where to look for information / get going.
EDIT
Please look at really good tutorial from David Mahler, specifically the introduction to OpenFlow, this would clear your doubts on Packet traversal in OpenvSwitch.
Coming to Controller, there are number of controller implementations with languages from C++, Python, Java to Ruby.
Assuming you are a python developer, you can check with POX, here are good tutorial links(link1 and link2) however I'm not sure how well its maintained, I see the commits for code was last done 3 years ago, not sure it is actively maintained. There is also example to parse the packet here, where you can apply appropriate algorithm. If you are looking for Java there are controllers such as Floodlight and Opendaylight, which has pretty much better community support.

How to detect Facebook friend IP via chat using Fiddler

So, I have read that it is possible to trace the IP of a Facebook friend while talking to him on chat by using Fiddler and Firebug. Now, as far as I am know, Facebook uses HTTPS and all in all, I cannot seem to get anything precise from Fiddler.
May anyone be kind enough to explain if this is really possible and if so, how the process goes?
I dont have a direct answer to your question, but i can give you some guidelines:
If the chat is working as a peer-to-peer network (which i highly doubt) you can trace the incoming tcp/udp connection and search it for the message using a sniffing program (like wireshark) and from there fetch their IP.
If the chat is based on that every message goes through a server (which is probably based on SOMM. not that it matters. (Server Oriented Messaging Model)). now if it is that way there is virtually no way to figure out the IP because the servers are acting as a proxy and masking the original IP, however. if FB includes some sort of meta data (which they do for location for phones etc.) that might contain the sender IP and some other stuff like the mac address etc etc. now i'm not sure of that but its a good place to look.
If you want help with firebug and all those other html/http/browser development tools there are plenty of tutorials out there. If you already know how to use it you might want to check the resources that's loaded when a message is sent. From experience i can tell that when a comment is added to something a whole bunch of crap happens that append stuff to the current html document. but i have never ever seen some meta data through that. anyway it's a good place to start.

Where would I learn more about interpreting network packets?

I'm working on a personal project. It's to recreate server software for the game "Chu Chu Rocket" for the Sega Dreamcast. Its' servers went down in 2004 I believe. My approach is to use dnsmasq to change the originl hostname that the game originally connected to, to my own system. With a DC-PC server set up, I have done just that, now instead of it looking up a non-existent dns record, it connects to my computer which will eventually run the server software. I've used tshark (cli wireshark) to capture what's going on between the client (dreamcast) and the server (my computer). The problem is, I'm getting data, but I'm not sure how to interpret it, I don't know what it's saying, but I'm sure it can be done because private PSO servers were created, those are far more complex.
Very simply, where would I go about learning how to interpret data packets, and possibly creating packets that will respond to such queries from the client?
Thanks,
Dragos240
If you can get the source code for the server software on your PC, then that is the best place to look.
Otherwise, all you can do is look at the protocol, compare runs, and make notes of similarities and differences. With any luck, the protocol won't be encrypted.

Creating RAW Sockets tutorial/explanation?

So Im trying to further my understanding of sockets, but I want to start out at the lowest level first (well in C, not assembly lol)
However most sites that I deal with use the SOCK_STREAM or SOCK_DGRAM. However I have read around on Beejs guide....but I don't know if that actually deals with RAW sockets or not.
I'll obviously need to call SOCK_RAW in my call to sockets, but theirs not really a WHOLE lot of information about it. And this is just for learning purposes, i always try to understand the root of whats going on in abstraction.
Thanks
Assuming you're using Linux, I would look at the man pages for socket and packet as a start. From here and after reading a few other man pages to figure out how to actually send packets on the wire, I'd install Wireshark and experiment with sending hand-crafted packets and using Wireshark to capture and analyze them. You can learn a lot about the different network layers this way and how you can tweak different values in various headers.

Deciphering MMORPG Protocol Encoding

I plan on writing an automated bot for a game.
The tricky part is figuring out how they encoded their protocol... To make the bot run around is easy, simply make the character run and record what it does in wireshark. However, interpreting the environment is more difficult... It recieves about 5 packets each second if you are idle, hence lots of garbarge.
My plan: Because the game runs under TCP, I will use freecap (http://www.freecap.ru/eng) to force the game to connect to a proxy running on my machine. I will need this proxy to be capable of packet injection, or perhaps a server that is capable of resending captured packets. This way I can recreate and tinker around with what the server sends, and understand their protocol encoding.
Does anyone know where I can get a proxy that allows packet injection or where I can perform packet injection (not via hardware, as is the case with wireless or anything!)
Where of if I can find a server/proxy that resends captured packets (ie: replays a connection).
Any better tools or methodologies for pattern matching? Something which can highlight patterns from mutliple messages would be GREAT.
OR, is there a better way to decipher this here? Possibly a dissasembly strategy (via hooking a winsock function and starting the dissassembly from there) ? I have not done this before so I am not sure. OR , any other ideas?
Network traffic interception and protocol analysis is generally a less favored method to accomplish your goal here. For most modern games, encryption is a serious factor, and there are serious headaches associated with the protocol analysis for any but trivial factors of the most common gameplay scenarios.
Most modern implementations* of what you are trying to do rely on reading and manipulating the memory space and process of a running client. The client will have already done all the hard parts for you, including decrypting the traffic and sorting it into far more easy to read data structures. For interacting with the server you can call functions built into the client instead of crafting entire series of packets from scratch. The plus to this approach is that you have to do far less work to interpret the data and produce activity. The minus is that there is often some data in the network traffic that would be useful to a bot but is discarded by the client, or that you may want to send traffic to the server that the client cannot produce (which, in my own well-developed hierarchy for such, is a few steps farther down the "cheating" slope).
*...I say this having seen the evolution of the majority of MMORPG botting/hacking communities from network protocol analyzers like ShowEQ and Odin's Eye / Excalibur to memory-based applications like MacroQuest and InnerSpace. On that note, InnerSpace provides an excellent extensible framework for the memory/process-based variant of what you are attempting, and you should look into it as a basis for your project if you abandon the network analysis approach.
As I've done a few game bots in the past (for fun, not profit or griefing of course - writing game bots is a lot of fun), I recommend the following:
If you can code and there isn't cheat protection preventing you from doing it, I highly recommend writing an injected DLL for the following reasons:
Your DLL will be able to access the game's memory space directly, and once you reverse-engineer the data structures (either by poking around memory or by code disassembly), you'll have access to lots of data. This will also allow you to bypass any network encryption the game may have. The downside of accessing process memory directly is that offsets and data structures change between versions - however, data structures don't change very often with a stable game, and you can compensate offset changes by searching for code patterns instead of using fixed offsets.
Either way, you'll still be able to hook WinSock functions using API hooks (check out Microsoft Detours and the excellent but now-commercial madCodeHook).
otherwise, I can only advise that you give live/interactive packet editors like WPE Pro a try.
In most scenarios, the coolest methods (code reverse-engineering and direct memory access) tend to be the least productive. They require a lot of skill (to understand the code) and time, both initially (to go through all the code and develop code to interact with the data structure) and for maintainance (in case the game is being updated). (Of course, they sometimes do allow doing cool stuff which is impossible to do with the official client, but most of the time this is obvious as blatant cheating, and likely to attract the GMs quickly). Most of the time bots are made by replacing game graphics/textures with solid colours, and creating simple "pixel" bots which search for certain colours on the screen and react accordingly (e.g. click them).
Hope this helps, and remember - cheating is only fun when it doesn't make the game less fun for everyone else ;)
There are probably a few reasonable assumptions you can make that should simplify your task enormously. However, to make the best use of them you will probably need greater comfort with sleeves-rolled-up programming than it sounds like you have.
First, it's a safe bet that the encryption they are using falls into one of three categories:
None
Cheesy
Far better than you are likely to crack
With the odds of the middle case being very low.
Next, it's a safe bet that the packets are encrypted / decrypted close to the edge of the program (right as they come in, right before they go out) and that the body of the game deals with them in decrypted form.
Finally, the protocol they are using most likely consists of either
ascii with data blocks
binary goo
So do a little packet sniffing with a card set in promiscuous mode for unencrypted ascii. If you see some, great, you're ahead of the game. But if you don't give up the whole tapping-the-line idea and instead start following the code as it returns from the sending data out by breakpointing and stepping with a debugger. Figure the outermost layer or three will be standard network stuff, then will come the encryption layer, and beyond that the huge mass of stuff that deals with the protocol unencrypted.
You should be able to get this far in an hour if you're hot, a weekend if you're reasonably skilled, motivated, and diligent, and never if you are hopeless. But it is possible in principle (and doubtlessly far easier in practice) to do it this way.
Once you get to where something that looks like unencrypted goo comes in, gets mungled, and the mungled form goes out, then start worrying about what it means.
-- MarkusQ
A) I play a MMO and do not support bots, voting down...
B) Download Backtrack v.3, run an arpspoof on your default gateway and your host. There is an application that will spoof the remote host's SSL cert sslmitm (I believe is the name) which will then allow you to create a full connection through your host. Then fireup tcpdump/ethereal/wireshark (choose your pcap poison) and move around do random stuff to find out what packet is doing what. That will be your biggest challenge; but proxying with a Man in the Middle attack on yourself is the way to go.
C) I do not condone this activity, this information is only being provided as free information.
Sounds like there is not encryption going on, so you could do a network approach.
A great place to start would be to find the packet ID's - most of the time, something near the front of the packet is going to be an ID of the type of the packet. For example move could be 1, shoot fired could be "2", chat could be "4".
You can write your own proxy that listens on one port for your game to connect, and then connects to the server. You can make keypresses to your proxy fire off commands, or you can make your proxy write out debugging info to help you go further.
(I've written a bot for an online in game in PHP - of all things.)