Is it possible to dump contents of tcp recv/send to a pcap file and then have wireshark analyse this. I want to do this because I want to log an applications conversations in order to debug issues but I don't want to have to write my own parsers (wireshark already supports it) and don't necessarily trust the application to parse frames correctly.
What follows is based on work that I've done exclusively on Windows.
I can't speak to how well any of this might apply to other systems.
FWIW, I would suggest editing your question (or at least tagging it)
to be more clear about what OS you're targeting.
If you are talking about writing what's received directly from a TCP client to a pcap file, the answer is no. The pcap file format requires at least some networking headers, e.g., the IP and TCP headers, in order for the file to be valid.
There are four approaches I see.
You could fake these headers for each read from the socket. This wouldn't necessarily require a lot of work, but it would require some knowledge of the relationships between the network header, the transport header, and the payload (i.e., your data) in order for everything to work right. If you're only interested in analysis of the payload in Wireshark, this might be a valid approach. However, if the analysis you're wanting to do in Wireshark is network analysis (i.e., analysis that includes the network headers), then you'll want to avoid this option and figure out how to capture the network headers directly.
You could use a raw socket to capture network traffic at the network layer. This will give you the IP packet - IP header, TCP header, and payload. You could dump this to a pcap file using LINKTYPE_RAW as the link-layer header type, and Wireshark will read the file fine. I've used this approach before.
The third approach would be to use WinPcap to collect your data. This is the same software that Wireshark uses behind the scenes to collect the data, and it's the approach I'm using now. If you have Wireshark installed, the WinPcap is already installed, too. WinPcap has a simple API with numerous examples for opening an interface, collecting the data, and dumping it to a pcap or pcap-ng file.
The fourth option is the simplest...just use Wireshark to collect the data simultaneously with your application.
HTH.
Related
I have a device (made in china) that works great but with a very annoying flaw. It's a massage device that will only work if it is connected to the internet at startup. Once it starts, it would work without the internet until it is turned off. So in short, if the internet is down, it will never work or it is impossible to bring this device on an area with no connection.
What I'm planning to do is emulate its server/client communication at start-up by using an old-PC that would run the server emulator software and some DNS re-routing and a WiFi router (using the PC as WAN).
I'll connect the device on the router, and its communication with the Chinese Server is emulated by the PC.
Can anyone give me an idea where to start, tools needed etc? I'm a programmer but new to packet emulation/replay. I've sniffed packets that the device does and its always consistent on every start/restart of the device. Here's the wireshark capture (IMG):
WireShark Packet Capture
You have two tasks, IMHO neither of them requires special tools:
Redirect requests to local address
You need to setup local DNS server and configure it to provide local IP address on hengdetech lookup. I didn't dive into this part, but google has plenty of forums discussing necessary config for different DNS servers.
Device most likely gets DNS server address from DHCP server. So you need to point device to local DNS instead of default one by configure DHCP server (which is run on your router, I suppose).
Once your DHCP configures local DNS and local DNS redirects hedgdete to local IP address, you are free to go to the second part which is
Emulate server behavior
Server must wait for some initial data from client before sending some data back. Such logic doesn't fit well into common "replay packets" tool capabilities, but it's quite easy to implement dumb TCP server with desired functionality on whatever language you like. Open socket, wait for data, send data, not a big deal, isn't it? So I wouldn't bother searching for the right tool, but would creat new instead.
If you need quick proof of concept, that you understand data flow right, before start coding, you may try netcat utility on Unix system for quick server emulation. I did the following once: extracted exchanged payload data from traces and saved as binaries (e.g. file1.bin is sent by server after the first request, file2.bin is sent after the second etc), then used something like this: cat - file1.bin - file2.bin | nc -l -p <local_port> -n. Using cat without dashes results into nc sending all files content to client, once it connects. Dash means "use data from console input rather than file" and they are used only as flow control. cat hangs on the first dash waiting for user input, so pressing ctrl+d proceeds to next file content and sends file1.bin, then hangs on the second dash. So you wait for the first request, press ctrl+d to send file1.bin content, then wait for the second request, press ctrl+d to send file2.bin content and so on.
EDIT
You generally got the idea of extracting data from wireshark traces right, but I want to clarify some subtlenesses.
When you said "saved binaries", did you mean to save individually captured packets as binaries so that netcat could send them to clients as a reply one by one (with the help of dash) ? If so, how do I save captured packets as binaries? Can I just extract them from the wireshark capture files, paste it on a new file and save as binary?
Only payload part of the packet needs to be saved, not the whole packet. TCP header, IP header and so on shouldn't be extracted, only tcp data section should be saved. Check second part of this answer for howto. I suppose, TCP data in your case consists of binary data rather than plain text, so you need to copy it as "raw binary" and use some binary editor allowing to paste "raw binary" into file such as frhed. So yes, you create new file and paste data copied from wireshark into it, so file consists of exactly the same bytes as packet payload.
Another thing to mention, TCP is a stream protocol, one packet doesn't always mean one response message. If you see several TCP packets of maximum size (aka MTU, which is usually 1.5 Kbytes) without PSH flag followed by packet of lesser size with PSH flag, they all contain parts of the single response message, so their payloads should be combined into one "fileX.bin".
Whenever I open a website in my browser, I want to see the packet that is going out of my computer. I want to do this to check how the proxy extension on my browser is encrypting the information.
Is there a way to check the packet that is going out of my NIC?
Some sniffer tool like tcpdump or wireshark may help you.
Wireshark is easier, with a good GUI and easy to learn and advanced filtering rules, analyse each packet on all levels of the packet, save packets into external files, load from external files, filter prototyped protocols and more advanced usage.
tcpdump is fast and useful but it will require a little bit more learning than wireshark GUI. But, is a really good solution for command line, ready to use.
A simple step-by-step for wirshark:
Install wireshark
Open it with root/admin permissions
Choose the target NIC
Click on 'Start' for start packet sniffer
On this new window, you will see a list of continous packets passing--through chosen NIC
On the top of this window, you have a filtering field. By example, you can write down on it:
tcp.port == 443 and ip.addr = 10.0.0.106
Then, only packets matching this rule will be shown
On this example, we are filtering all traffic passing by TCP port 443 (SSL) and have even target (request) and source (response) host under address 10.0.0.106.
It is possible filter by mac-addresses, and a lot of parameters under each packet, protocol specific parameters, and a lot of things.
I was reading through the libpcap tutorial and was using some captures that I want to analyze (I didn't create them myself), so I am using the offline mode.
When applying the tutorial, nothing seemed to match. I didn't get valid MAC addresses from the ethernet frame and not even the preamble seemed to be correct. After some expirementing I fired up wireshark and noticed something odd. Each packet start with a 6: These are not packets on the ethernet (data link) layer, they are already one layer up on the IP (network) layer (They are IPv6 packets, which is expected).
Of course, I can now easily continue, but I am still wondering: How do I know at which layer the captured packages are? It seems there are at least two options: Saving at the data link layer and saving at the network layer. Could it also be that I receive packets on the transport layer? How do I differentiate? Must the user tell me? I'd really hate to have to guess, but seeing as wireshark gets it right, there must be a simple way to determine it.
The libpcap file format has a field in its global header called network that specifies the data link type. This corresponds to documentation over at the tcpdump project about various Link-Layer Header Types.
From there, you would be able to determine how many bytes the link layer consists (if any) and will be a sure fire way for you to parse pcap files created with the libpcap library.
I am not aware of a program options in the "libpcap family" that would allow you to just capture starting at the transport layer.
I'd like to be able to intercept/ modify data in tcp flow, on the side of tcp client. Examples for pcap show how to parse tcp packet header/ payload.
But suppose, i want to modify packet payload before tcp client reads it, or drop the packet entirely. How can i do that with pcap capure?
As above, you can't do interception/modification with pcap. For this you need one of the following OS-dependent techniques:
Linux: libnetfilter_queue + iptables
MacOS, FreeBSD: divert sockets + ipfw
Windows: WinPkFilter (commercial), WinDivert (LGPL), or write your own NDIS IM or WFP call-out driver.
(usual disclosure: WinDivert is my project).
Scapy used in conjunction with python is a very good tool cum library.
You can do all sorts of packet monitoring and editing via command line
Also you can build applications on top of scapy to do a specific task.
Scapy
You can't do that with libpcap or WinPcap; libpcap is built atop OS mechanisms that do not support that (those mechanisms exist to support passive packet capture and low-level packet capture and injection, not to support packet modification in the packet input and output path), and WinPcap's driver is built atop an OS mechanism of that sort.
You would have to find some mechanism, in whatever OS you're using, that supports tapping into the networking stack in a way that allows the tapping program to modify packets as they pass through the networking stack. Such a mechanism might not exist on some OSes; on OSes where it does exist, if there are any, it's probably very OS-dependent. (The mechanisms libpcap uses are also OS-dependent; libpcap exists, in part, to hide those differences from applications, to the maximum extend possible.)
(Apologies: I uninstalled and reinstalled WinPcap and now I can see the extra interface! Suggestion found in Wireshark FAQ. I leave the original question below.)
I use WireShark to examine ethernet packet contents at the byte level (in/out of custom FPGA-based hardware). I have a USB-Ethernet adapter to add a second Ethernet port to my laptop. It was a cheap Chinese device bought on Ebay but now that I've found an appropriate driver, it works OK. However, I see that, on Windows, WinPcap/WireShark doesn't support Ethernet capture over USB.
While it would be nice if WireShark could be made to work on USB capture, I'm really looking for an alternative way to grab the raw ethernet bytes. I have some perl scripts set up that operate on the raw frames output from tshark, (Wireshark command line) and I could easily feed it from any stream of frames/bytes.
Is anyone doing something similar or is there a tidy way to output the raw bytes?
Sniffed raw USB bytes would be OK, but it would be nicer if someone has already programmed/scripted extracting the Ethernet frames. I'm using perl but any compiled app or python or C# or C++ or .. would be fine.
You mentioned python, scapy can do a LOT of raw packet things, might want to look at that. From their git:
Scapy is a powerful Python-based interactive packet manipulation
program and library.
It is able to forge or decode packets of a wide number of protocols,
send them on the wire, capture them, store or read them using pcap
files, match requests and replies, and much more. It is designed to
allow fast packet prototyping by using default values that work.
It can easily handle most classical tasks like scanning, tracerouting,
probing, unit tests, attacks or network discovery (it can replace
hping, 85% of nmap, arpspoof, arp-sk, arping, tcpdump, wireshark, p0f,
etc.). It also performs very well at a lot of other specific tasks
that most other tools can't handle, like sending invalid frames,
injecting your own 802.11 frames, combining techniques (VLAN
hopping+ARP cache poisoning, VoIP decoding on WEP protected channel,
...), etc.
Scapy supports Python 2.7 and Python 3 (3.3 to 3.6). It's intended to
be cross platform, and runs on many different platforms (Linux, OSX,
*BSD, and Windows).
Check them out at https://github.com/secdev/scapy
I don't have a Windows PC readily at hand to test, but as far as I can tell, there is no problem capturing Ethernet frames in Wireshark on Windows, from a USB-Ethernet adapter.
What you can't do, is capturing USB bus traffic, but that is not what you wanted, right?
To clarify, just select the USB-Ethernet device as you would any other, and you are set.