I'm trying to emulate the UDP output of a particular piece of software (a very simple remote control for an audio player). Capturing its output with tcpdump -X was no problem, and resulted in packets like the ones below:
Turn the device off...
18:30:03.623499 IP x.x.x.1.27490 > x.x.x.2.23500: UDP, length 32
0x0000: 4500 003c 42e6 0000 8011 92ee 0a02 2810 E..<B.........(.
0x0010: 0a02 28c9 6b62 5bcc 0028 b213 0500 007f ..(.kb[..(......
0x0020: 0100 0000 1800 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0100 0000 0200 0000 ............
Turn the device on...
18:30:06.222808 IP x.x.x.1.27490 > x.x.x.2.23500: UDP, length 32
0x0000: 4500 003c 42e7 0000 8011 92ed 0a02 2810 E..<B.........(.
0x0010: 0a02 28c9 6b62 5bcc 0028 b213 0500 007f ..(.kb[..(......
0x0020: 0100 0000 1800 0000 0000 0000 0000 0000 ................
0x0030: 0000 0000 0200 0000 0100 0000 ............
Now I'm trying to reproduce these packets in Perl, so I can send them via IO::Socket::INET and have them arrive looking the same as the above.
I've been fooling around with the Perl pack function, but I'm afraid my understanding of what I'm looking at is insufficient to craft a working template.
Am I even on the right track here?
To turn the device off, send the following string to 10.2.40.201:23500 using UDP:
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x01\x00\x00\x00"."\x02\x00\x00\x00"
To turn it on,
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x02\x00\x00\x00"."\x01\x00\x00\x00"
IP packet structure
UDP packet structure
The first byte tells gives us important information.
45
Version: 4 (IPv4)
IHL: 5 (IP header is 20 bytes long, including this byte.)
Now we know how long the header is.
00 00 3c 42 e6 00 00 80 11 92 ee 0a 02 28 10
0a 02 28 c9
DSCP: 0
ECN: 0
Total length: 0x003c (60)
Identification: 0x42e6
Flags: 0
Fragment offset: 0
TTL: 0x80 (128)
Protocol: 0x11 (UDP)
Header checksum: 0x92ee
Source IP: 0x0a022810 (10.2.40.16)
Destination IP: 0x0a0228c9 (10.2.40.201)
It's an unfragmented packet, so it's the full packet.
The next 60 - 20 = 40 bytes is the IP packet payload.
It's a UDP packet, so the next 8 bytes form the UDP packet header.
6b 62 5b cc 00 28 b2 13
Source port: 0x6b62 (27490)
Destination port: 0x5bcc (23500)
Length (IP header excluded): 0x0028 (40)
Checksum: 0xb213
The next 40 - 8 = 32 bytes is the UDP packet payload.
05 00 00 7f 01 00 00 00 18 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 01 00 00 00 02 00 00 00
To send this message, you'd use UDP send the following string to 10.2.40.201:23500:
"\x05\x00\x00\x7f"."\x01\x00\x00\x00"."\x18\x00\x00\x00"."\x00\x00\x00\x00".
"\x00\x00\x00\x00"."\x00\x00\x00\x00"."\x01\x00\x00\x00"."\x02\x00\x00\x00"
Here, I divided into four-byte strings for readability. How you build the string is irrelevant. You could use pack in a million different ways to generate this string, but there's no way to know which one is significant without knowing the protocol.
Now, you asked to replicate the packets, but that's surely unnecessary. You only need to send the same message. There are going to be differences in the headers. You might be able to control some of them (e.g. by binding the socket to 10.2.40.16:27490). But others might be more difficult. The TTL field decreases as the the packet moves through the network. The packet can become fragmented (divided into smaller packets called fragments). etc. But none of that should be relevant.
Related
I'm currently working with NFC/NDEF and I'm running into an issue where I'm unable to understand the data coming in. I have a general understanding of the NDEF standard and have looked over the MIFARE datasheet, so I'm able to pick out a few things, but there are a few bytes that are seemingly out of place and are puzzling me.
Here is the hexdump of the data on the tag, collected via nfc-mfultralight r:
00000000 04 02 2f a1 d2 11 5f 81 1d 48 00 00 e1 10 12 00 |../..._..H......|
00000010 01 03 a0 0c 34 03 1b 91 01 05 54 02 65 6e 68 69 |....4.....T.enhi|
00000020 11 01 05 54 02 65 6e 68 69 51 01 05 54 02 65 6e |...T.enhiQ..T.en|
00000030 68 69 fe 00 00 00 00 00 00 00 00 00 00 00 00 00 |hi..............|
I know the first 16 bytes (04 02 2f a1 d2 11 5f 81 1d 48 00 00 e1 10 12 00) are the NFC/MIFARE header (first 9 being the serial number/check bytes, 1 byte for internal, 2 for lock, and then final 4 are OTP bytes.)
Starting at byte 21 I can see the start of a TLV record with the Terminator TLV flag at the end (03 1b ... fe), indicating a record of NDEF type with length 27. This matches the length of the expected NDEF record.
However, I'm confused by bytes 16..20 (01 03 a0 0c 34). What are these?
It appears these are a part of the Lock Control TLV, a part of the NFC Type 2 Tag standard (pages 10-11).
The bytes are laid out as such:
0x01 - Lock Control TLV block name
0x03 - Length is 3 bytes
0xa0 - Encodes the position within the tag the lock area is at, composed of two nibbles:
0b0000 - Higher 4 bits represent the number of pages, while the lower 4 bits are the number of bytes
0b1100 - The number of bits used in the lock area.
0x0c - Indicates size in bits of the lock area
0x34 - Provides number of bytes in a page and the number of bytes each dynamic lock bit is able to lock.
I have a database filled with a lot of logged IPV4 messages. It is used to get queries like: "give me all messages from MacAddress ... that were logged in the period ... to ... that have ..."
Some queries will result in a huge amount of logged messages. Therefore we decided to make a PCAP file if such a request was made.
"Please create a PCAP file containing all logged messages from your
database that ..."
So upon request, my service should fetch the requested data from the database (in pages) and create a PCAP file filled with the data fetched from the database. Later callers can ask for a read-only OWIN stream to this file
The service can create such a file. The problem is that it is not recognized as a proper WireShark file.
I've read Libcap File Format. Whenever I have to create a file filled with LoggedMessages I fill a binary file as follows.
Global Header
Per logged message:
A packet header
Packet data with:
Ethernet Frame: Destination Mac, Source Mac, EtherType (0x800)
IPV4 header
Logged Data
Wireshark starts complaining about the file when it attempts to read the Ethertype. It says this is a Length. Definition of Ethernet Frame with EtherType
So below I show the start of my file. Hexadecimal format per byte + my interpretation of it. After that the comments from wireshark
The created stream starts with the Global Header: a 32 bytes structure. First the hexadecimal values then the interpretation:
=== Global Header ====
D4 C3 B2 A1 02 00 04 00
00 00 00 00 00 00 00 00
FF FF 00 00 01 00 00 00
Magic number A1B2C3D4 (Original Time Precision)
Version: 2 - 4
ThisZone 0
sigFigs 0
snapLen 0000FFFF
datalinkType 1
Note that the magic number has the LSB first, indicating that every multi-byte number will have the least significant byte first. So a 2 byte value of 0x1234 will have in memory first 34 then 12.
After that the Packets should come. Every time one Packet Header, followed by one Packet Data
=== Packet header ===
09 89 58 5A C8 85 0B 00
6B 00 00 00 6B 00 00 00
Timestamp: 1515751689.7551446 (usec precision)
Number of saved bytes (incl_len) 107 bytes (0x006b)
Actual packet length (orig_len) 107 bytes (0x006b)
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 00 08 45 00
5D 00 00 00 00 00 FF 00 - E0 0D 8A 84 77 44 E0 2B
9C FB 4D 43 D5 8A 00 00 - 00 00 41 41 41 41 41 41
41 41 41 41 41 41 41 41 - 41 41 41 41 41 41 41 41
// etc, until total 107 bytes
The packet data consists of a Mac Header, IPV4 header and a couple of 0x41 as data
=== Mac Header ===
Destination Mac: CF:31:59:D3:E7:98
Source Mac: 53:39:17:F0:A9:9C
Ether type: 0800
Note that the magic number showed that every multi-byte number has the LSB first, so the two bytes 00 08 will have a 16-bit meaning of 0x0800
If you look at the PCAP file interpretation I show below, then the problem starts here: the Ether Type is not interpreted as Ether Type, but as length.
After remark in one of the answers, I tried to reverse the two byte ether type from 00 08 into 08 00 (MSB first), but that made the problems worse.
=== IPV4 header ===
- 45 00 5D 00
- 00 00 00 00
- FF 00 E0 0D
- 8A 84 77 44
- E0 2B 9C FB
Specification of the IPV4 header structure
DWORD 0
- bits 00..04: version; bits 04..07 IP Header Length: 04 05
- bits 08..13 DSCP; bits 14..15 ECN: 00
- bits 16..31 Total Length (header + Payload): 93 (005D)
DWORD 1
- bits 00..15 Identification: 0000
- bits 16..18 Flags; bits 19..31 offset: 0000
DWORD 2
- bits 00..07 Time to Live FF
- bits 08..15 Protocol; used protocol 00
- bits 16..31 Header Checksum 3552 (0DE0)
DWORD 3 and 4
Source IP: 138.132.119.68
Destination IP: 224.43.156.251
Bacause wireshark complains about checksum, I verify as follows:
Verify checksum:
Header: 0045 005D 0000 0000 00FF 0DE0 848A 4477 2BE0 FB9C
69 + 93 + 0 + 0 + 255 + 3552 + 33930 + 17527 + 11232 + 64412 = 131070 (01FFFE)
0001 + FFFE = FFFF
1's complement: 0000 (checksum ok)
This is what WireShark (version 2.4.4) makes of it:
The following seems normal:
Frame 1: 107 bytes on wire (856 bits), 107 bytes captured (856 bits)
Encapsulation type: Ethernet (1)
Arrival Time: Jan 12, 2018 11:08:09.755144000 W. Europe Standard Time
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1515751689.755144000 seconds
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 107 bytes (856 bits)
Capture Length: 107 bytes (856 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:llc:data]
[Coloring Rule Name: Checksum Errors]
[Coloring Rule String [truncated]: eth.fcs.status=="Bad" ||
ip.checksum.status=="Bad" || tcp.checksum.status=="Bad" ||
udp.checksum.status=="Bad" || sctp.checksum.status=="Bad" ||
mstp.checksum.status=="Bad" || cdp.checksum.status=="Bad" ||]
Here comes the first problem: EtherType is interpreted as Length
IEEE 802.3 Ethernet
Destination: cf:31:59:d3:e7:98 (cf:31:59:d3:e7:98)
Source: 53:39:17:f0:a9:9c (53:39:17:f0:a9:9c)
Length: 8
Padding: ff00e00d8a847744e02b9cfb4d43d58a0000000041414141...
Trailer: 414141414141414141414141414141414141414141414141...
Frame check sequence: 0x41414141 incorrect, should be 0xe19cae36
[FCS Status: Bad]
After the length, which I meant as an EtherType, comes a lot of padding, instead of interpretation of my 5 DWORDs.
The link to the Ethernet Frame in wikipedia I showed says:
The EtherType field is two octets long and it can be used for two
different purposes. Values of 1500 and below mean that it is used to
indicate the size of the payload in octets, while values of 1536 and
above indicate that it is used as an EtherType, to indicate which
protocol is encapsulated in the payload of the frame.
My value if 0x0800 = 2048. This certainly is above 1536
For example, an EtherType value of 0x0800 signals that the frame
contains an IPv4 datagram.
If value 0x0800 the incorrect value? Or is my error somewhere else?
Looks like your ethertype has the wrong byte order. It should be:
=== Packet Data ===
CF 31 59 D3 E7 98 53 39 - 17 F0 A9 9C 08 00 XX XX
I can read UDP packet using
void my_callback(u_char *useless, const struct pcap_pkthdr* pkthdr, const u_char* packet)
I have hexa output of my packet:
08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00 45 00 00 4C 7C E7 00 00 40 11 3C 28 5B BD 59 C6 0A 00 02 0F 00 7B 00 7B 00 38 B7 9D 24 02 03 E8 00 00 04 A8 00 00 07 51 83 BC 03 DC DC C5 CC 47 F1 F1 69 C3 DC C5 CF 37 D2 5F A7 F5 DC C5 CF 38 3C 2D C2 CF DC C5 CF 38 3C 32 0B 9A
I know, that it is NTP packet.
How can I extrath data? Cut ethernet frames, etc..
Thank you for your help.
I am using pcap c++.
If you read pcap you get raw packet from the network device. Several options may be there:
Packet is read from ethernet device
Packet is read from vlan device
Packet is read from some other device
What kind of device is used during pcap defines what protocol header is first in your packet. To know it you can look at link layer type field of global pcap header.
Once you defined first protocol header you need to open protocol specification and find:
Size of header (in your case it looks like regular ethenet header - 14 bytes 08 00 27 E5 B5 3B 52 54 00 12 35 02 08 00)
How to find encapsulated packet type (in your case last 08 00 means IP)
Once you found IP header (45 00 00 4C 7C E7 00 00 40 11 ...) you can determine IP header length:
0 1 2 3
0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|Version| IHL |Type of Service| Total Length |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Identification |Flags| Fragment Offset |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Time to Live | Protocol | Header Checksum |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Source Address |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
| Destination Address |
Here you need:
IHL defines sizeof IP header. This is lower 4 bits of first byte of IP header. In your case it is 0x5. This means 5 words or 20 bytes.
Protocol defines what data is encapsulated in IP header. In your case 0x11 (IPPROTO_UDP)
After that you can get UDP header (8 bytes) check ports if you need it and parse NTP header in according to NTP specification.
In your example total shift of the NTP header will be 14+20+8 bytes.
Out of curiosity, how much data is actually sent when establishing a connection to a port (using Java sockets). Is it the size of a Socket object? SocketConnection object?
Your understanding of TCP network connections seems to conflate them with electrical circuits. (Understandable, given your background.)
From a physical standpoint, there's no such thing as a connection, only data packets. Through the TCP protocol, two devices agree to establish a logical (that is, software) connection. A connection is established by a client first sending data to the remote host (SYN), the server sending data back to the client (SYN-ACK), and the client sending a final acknowledgement (ACK). All of this negotation necessarily consumes bandwidth, and when you terminate a connection, you must negotiate a completely new connection to begin sending data again.
For example, I'll connect from my machine to a local web server, 192.168.1.2:80.
First, my machine sends a TCP SYN. This sends 66 bytes over the wire: (headers deliniated with |)
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00|45 00 .$..L... hf y..E.
0010 00 34 53 98 40 00 80 06 00 00 c0 a8 01 0b c0 a8 .4S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a7 00 00 00 00 80 02 ..6..P.. :.......
0030 20 00 50 c8 00 00 02 04 05 b4 01 03 03 02 01 01 .P..... ........
0040 04 02 ..
The first 14 bytes are the Ethernet frame, specifying that this packet's destination MAC address. This will typically be an upstream router, but in this case, the server happens to be on the same switch, so it's the machine's MAC address, 00:24:8c:a9:4c:b4. The source (my) MAC follows, along with the payload type (IP, 0x0800). The next 20 bytes are the IPv4 headers, followed by 32 bytes of TCP headers.
The server responds with a 62-byte SYN-ACK:
0000 00 1e 68 66 20 79 00 24 8c a9 4c b4 08 00|45 00 ..hf y.$ ..L...E.
0010 00 30 69 b9 40 00 80 06 0d b1 c0 a8 01 02 c0 a8 .0i.#... ........
0020 01 0b|00 50 36 0a d3 ae 9a 73 09 ef 3a a8 70 12 ...P6... .s..:.p.
0030 20 00 f6 9d 00 00 02 04 05 b4 01 01 04 02 ....... ......
Again, 14 bytes of Ethernet headers, 20 bytes of IP headers, and 28 bytes of TCP headers. I send an ACK:
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00|45 00 .$..L... hf y..E.
0010 00 28 53 9a 40 00 80 06 00 00 c0 a8 01 0b c0 a8 .(S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a8 d3 ae 9a 74 50 10 ..6..P.. :....tP.
0030 fa f0 83 78 00 00 ...x..
14 + 20 + 20 = 54 bytes over the wire (this is the smallest possible TCP packet size, by the way – the SYN and SYN-ACK packets were larger because they included options).
This adds up to 182 bytes over the wire to establish a connection; now I can begin sending actual data to the server:
0000 00 24 8c a9 4c b4 00 1e 68 66 20 79 08 00 45|00 .$..L... hf y..E.
0010 01 9d 53 9d 40 00 80 06 00 00 c0 a8 01 0b c0 a8 ..S.#... ........
0020 01 02|36 0a 00 50 09 ef 3a a8 d3 ae 9a 74 50 18 ..6..P.. :....tP.
0030 fa f0 84 ed 00 00|47 45 54 20 2f 20 48 54 54 50 ......GE T / HTTP
0040 2f 31 2e 31 0d 0a 48 6f 73 74 3a 20 66 73 0d 0a /1.1..Ho st: fs..
...
14 Ethernet + 20 IP + 20 TCP + data, in this case HTTP.
So we can see that it costs ~182 bytes to establish a TCP connection, and an additional 162-216 bytes to terminate a TCP connection (depending on whether a 4-way FIN ACK FIN ACK or more common 3-way FIN FIN-ACK ACK termination handshake is used), adding up to nearly 400 bytes to "pulse" a connection by disconnecting and reconnecting.
Compared to the 55 bytes you'd use to send one byte of data over an already established connection, this is obviously wasteful.
What you want to do is establish one connection and then send data as-needed. If you're really bandwidth constrained, you could use UDP (which requires no handshaking at all and has an overhead of only 14 Ethernet + 20 IP + 8 UDP bytes per packet), but then you face the problem of using an unreliable transport, and having to handle lost packets on your own.
The minimum size of a TCP packet is 40 bytes. It takes three exchanged packets to create a connection, two from the client and one from the server, and four more to close it, two in each direction. The last packet in a connect exchange can also contain data, as can the first in the close exchange in each direction, which can amortize it a bit, as can combining the outgoing FIN and ACK as per #josh3736's comment below.
I'm working on an embedded device that connects on local network with RJ45 and when the system sends an ARP request to know the mac address of the gateway, no answer at all.
If I clear the arp table on my Windows, the Windows asks exactly the same ARP request and got an answer!
I sniffed the packet and the only difference inside the request packet is a 0 trailer on the embedded device at the end of the packet and that the target mac address is ff:ff:ff:ff:ff:ff where the windows one is 00:00:00:00:00:00 (wikipedia seems to say that it should be ffffffffff)
I tried to changed the mac address in case my gateway banned the mac due to arp spam but it doesn't change anything. I also try with DHCP IP and static IP, same problem...
Windows packet:
Frame 1 (42 bytes on wire, 42 bytes captured)
Frame is marked: False
Arrival Time: Jan 29, 2010 12:05:49.775534000
Time delta from previous packet: -77.580549000 seconds
Time since reference or first frame: 6354.738379000 seconds
Frame Number: 1
Packet Length: 42 bytes
Capture Length: 42 bytes
Protocols in frame: eth:arp
Ethernet II, Src: 00:1e:8c:b5:d0:00, Dst: ff:ff:ff:ff:ff:ff
Type: ARP (0x0806)
Address Resolution Protocol (request)
Hardware type: Ethernet (0x0001)
Protocol type: IP (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (0x0001)
Sender MAC address: 00:1e:8c:b5:d0:00 (00:1e:8c:b5:d0:00)
Sender IP address: 192.168.0.14 (192.168.0.14)
Target MAC address: 00:00:00:00:00:00 (00:00:00:00:00:00)
Target IP address: 192.168.0.1 (192.168.0.1)
0000: FF FF FF FF FF FF 00 1E 8C B5 D0 00 08 06 00 01 ................
0010: 08 00 06 04 00 01 00 1E 8C B5 D0 00 C0 A8 00 0E ................
0020: 00 00 00 00 00 00 C0 A8 00 01 ..........
Embedded device packet:
Frame 1 (60 bytes on wire, 60 bytes captured)
Frame is marked: False
Arrival Time: Jan 29, 2010 12:07:04.257748000
Time delta from previous packet: -3.098335000 seconds
Time since reference or first frame: 6429.220593000 seconds
Frame Number: 1
Packet Length: 60 bytes
Capture Length: 60 bytes
Protocols in frame: eth:arp
Ethernet II, Src: 00:04:a3:12:34:05, Dst: ff:ff:ff:ff:ff:ff
Type: ARP (0x0806)
Trailer: 000000000000000000000000000000000000
Address Resolution Protocol (request)
Hardware type: Ethernet (0x0001)
Protocol type: IP (0x0800)
Hardware size: 6
Protocol size: 4
Opcode: request (0x0001)
Sender MAC address: 00:04:a3:12:34:05 (00:04:a3:12:34:05)
Sender IP address: 192.168.0.129 (192.168.0.129)
Target MAC address: ff:ff:ff:ff:ff:ff (ff:ff:ff:ff:ff:ff)
Target IP address: 192.168.0.1 (192.168.0.1)
0000: FF FF FF FF FF FF 00 04 A3 12 34 05 08 06 00 01 ..........4.....
0010: 08 00 06 04 00 01 00 04 A3 12 34 05 C0 A8 00 81 ..........4.....
0020: FF FF FF FF FF FF C0 A8 00 01 00 00 00 00 00 00 ................
0030: 00 00 00 00 00 00 00 00 00 00 00 00 ............
In fact, It was a problem with the TX where the polarity was inverted and cause these problems.
I inverted the polarity and now it works perfectly.