I've faced with an interesting behavior of UDP socket with SO_REUSEADDR option set...
If I send 1472 bytes over IP/UDP I get it all in one frame - that' expected...
BUT for 1473 fragmentation differs with/without that option. Does anyone has a clue why that occurs (I'm using Debian 2.6.32-5-amd64)?
SO_REUSEADDR Enabled:
FRAME#1
Internet Protocol Version 4, Src: 173.59.3.22 (173.59.3.22), Dst: 173.70.1.5 (173.70.1.5)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: **1476**
Identification: 0x214d (8525)
Flags: 0x01 (More Fragments)
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..1. .... = More fragments: Set
Fragment offset: 0
Time to live: 64
Protocol: UDP (17)
Header checksum: 0xd53f [correct]
[Good: True]
[Bad: False]
Source: 173.59.3.22 (173.59.3.22)
Destination: 173.70.1.5 (173.70.1.5)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
User Datagram Protocol (**8** bytes)
Source port: icl-twobase1 (25000)
Destination port: 5000 (5000)
Length: **1481**
Checksum: 0x3cac [unchecked, not all data available]
[Good Checksum: False]
[Bad Checksum: False]
Data (**1448** bytes)
FRAME#2
Internet Protocol Version 4, Src: 173.59.3.22 (173.59.3.22), Dst: 173.70.1.5 (173.70.1.5)br/>
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: **45**
Identification: 0x214d (8525)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 1456
Time to live: 64
Protocol: UDP (17)
Header checksum: 0xfa20 [correct]
[Good: True]
[Bad: False]
Source: 173.59.3.22 (173.59.3.22)
Destination: 173.70.1.5 (173.70.1.5)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Data (**25** bytes)
SO_REUSEADDR Disabled:
FRAME#1
Internet Protocol Version 4, Src: 173.59.3.22 (173.59.3.22), Dst: 173.70.1.5 (173.70.1.5)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: **1500**
Identification: 0x214a (8522)
Flags: 0x01 (More Fragments)
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..1. .... = More fragments: Set
Fragment offset: 0
Time to live: 64
Protocol: UDP (17)
Header checksum: 0xd52a [correct]
[Good: True]
[Bad: False]
Source: 173.59.3.22 (173.59.3.22)
Destination: 173.70.1.5 (173.70.1.5)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
User Datagram Protocol (**8** bytes)
Source port: icl-twobase1 (25000)
Destination port: 5000 (5000)
Length: **1481**
Checksum: 0x3cac [unchecked, not all data available]
[Good Checksum: False]
[Bad Checksum: False]
Data (**1472** bytes)
FRAME#2
Internet Protocol Version 4, Src: 173.59.3.22 (173.59.3.22), Dst: 173.70.1.5 (173.70.1.5)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x00 (DSCP 0x00: Default; ECN: 0x00: Not-ECT (Not ECN-Capable Transport))
0000 00.. = Differentiated Services Codepoint: Default (0x00)
.... ..00 = Explicit Congestion Notification: Not-ECT (Not ECN-Capable Transport) (0x00)
Total Length: **21**
Identification: 0x214a (8522)
Flags: 0x00
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
Fragment offset: 1480
Time to live: 64
Protocol: UDP (17)
Header checksum: 0xfa38 [correct]
[Good: True]
[Bad: False]
Source: 173.59.3.22 (173.59.3.22)
Destination: 173.70.1.5 (173.70.1.5)
[Source GeoIP: Unknown]
[Destination GeoIP: Unknown]
Data (**1** byte)
The answer is NO - SO_REUSEADDR doesn't affect fragmentation...
The clue is MTU Discovery and Don't Fragment Flag in IP Header.
In case of sending of 1472 bytes with MTU Discovery = ON:
1500 bytes sent with Don't Frag Flag ON => I get ICMP:
Internet Control Message Protocol
Type: 3 (Destination unreachable)
Code: 4 (Fragmentation needed)
Checksum: 0x90db [correct]
MTU of next hop: 1480
After that the sender does fragmentation for all packets to fit 1480 MTU while route\arp cache info is kept for that receiver (5 min for route and 30 sec for arp - defaults configured on my system).
That's why I saw unexpected for me fragmentation of 1473 bytes (1456 + 25 bytes in two frames) (SO_REUSEADDR was ON in that experiment)...
Next time I tried sending the same 1473 bytes (SO_REUSEADDR was OFF) route\arp cache freed the info about the receiver and I saw that different confusing fragmentation (1480 + 1 bytes in two frames).
Sorry for everyone for the confusion...
However that was pretty cognitive for me to dig into such behavior and clarify things I observed. No miracles here.
Related
In our K8 cluster, we use haproxy app for connecting to Galera cluster.
Our haproxy.cnf file looks like
global
maxconn 2048
external-check
stats socket /var/run/haproxy.sock mode 600 expose-fd listeners level user
user haproxy
group haproxy
defaults
log global
mode tcp
retries 10
timeout client 30000
timeout connect 100500
timeout server 30000
frontend mysql-router-service
bind *:6446
mode tcp
option tcplog
default_backend galera_cluster_backend
# MySQL Cluster BE configuration
backend galera_cluster_backend
mode tcp
option tcpka
option mysql-check user haproxy
balance source
server pitipana-opsdb1 192.168.144.82:3306 check weight 1
server pitipana-opsdb2 192.168.144.83:3306 check weight 1
server pitipana-opsdb3 192.168.144.84:3306 check weight 1
Dockerfile for creating haproxy image
FROM haproxy:2.3
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg
In my Galera nodes, I get constant warning in /var/log/mysql/error.log
2021-12-20 21:16:47 5942 [Warning] Aborted connection 5942 to db: 'ourdb' user: 'ouruser' host: '192.168.1.2' (Got an error reading communication packets)
2021-12-20 21:16:47 5943 [Warning] Aborted connection 5943 to db: 'ourdb' user: 'ouruser' host: '192.168.1.2' (Got an error reading communication packets)
2021-12-20 21:16:47 5944 [Warning] Aborted connection 5944 to db: 'ourdb' user: 'ouruser' host: '192.168.1.2' (Got an error reading communication packets)
I had increased max_packet_size to 64MB and max_connections to 1000.
When I take a tcpdump from galera node :
Frame 16: 106 bytes on wire (848 bits), 106 bytes captured (848 bits)
Linux cooked capture
Internet Protocol Version 4, Src: 192.168.1.2, Dst: 192.168.10.3
Transmission Control Protocol, Src Port: 62495, Dst Port: 3306, Seq: 1, Ack: 1, Len: 50
Source Port: 62495
Destination Port: 3306
[Stream index: 2]
[TCP Segment Len: 50]
Sequence number: 1 (relative sequence number)
[Next sequence number: 51 (relative sequence number)]
Acknowledgment number: 1 (relative ack number)
0101 .... = Header Length: 20 bytes (5)
Flags: 0x018 (PSH, ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 1... = Push: Set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······AP···]
Window size value: 507
[Calculated window size: 64896]
[Window size scaling factor: 128]
Checksum: 0x3cec [unverified]
[Checksum Status: Unverified]
Urgent pointer: 0
[SEQ/ACK analysis]
[Timestamps]
TCP payload (50 bytes)
[PDU Size: 45]
[PDU Size: 5]
MySQL Protocol
Packet Length: 41
Packet Number: 1
Request Command SLEEP
Command: SLEEP (0)
Payload: 820000008000012100000000000000000000000000000000...
[Expert Info (Warning/Protocol): Unknown/invalid command code]
[Unknown/invalid command code]
[Severity level: Warning]
[Group: Protocol]
MySQL Protocol
Packet Length: 1
Packet Number: 0
Request Command Quit
Command: Quit (1)
Here 192.168.1.2 is a K8 worker node and 192.168.10.3 is the galera node.
When I connect our applications in K8, we can access to applications, but when we try to edit, we get stuck.
Any suggestion to fix this?
I ran tshark -V > file.log in my terminal and then in a separate terminal I ran curl 'www.google.com'. I then returned to the first terminal, shut off tshark and then looked at file.log. In it there are a number of 'frames'. For example, here is one of them:
Frame 42: 66 bytes on wire (528 bits), 66 bytes captured (528 bits) on interface en0, id 0
Interface id: 0 (en0)
Interface name: en0
Interface description: Wi-Fi
Encapsulation type: Ethernet (1)
Arrival Time: Nov 3, 2020 17:28:15.022217000 PST
[Time shift for this packet: 0.000000000 seconds]
Epoch Time: 1604453295.022217000 seconds
[Time delta from previous captured frame: 0.000244000 seconds]
[Time delta from previous displayed frame: 0.000244000 seconds]
[Time since reference or first frame: 12.164253000 seconds]
Frame Number: 42
Frame Length: 66 bytes (528 bits)
Capture Length: 66 bytes (528 bits)
[Frame is marked: False]
[Frame is ignored: False]
[Protocols in frame: eth:ethertype:ip:tcp]
Ethernet II, Src: *****, Dst: *****
Destination: *****
Address: *****
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Source: Apple_81:82:2f (ac:bc:32:81:82:2f)
Address: Apple_81:82:2f (ac:bc:32:81:82:2f)
.... ..0. .... .... .... .... = LG bit: Globally unique address (factory default)
.... ...0 .... .... .... .... = IG bit: Individual address (unicast)
Type: IPv4 (0x0800)
Internet Protocol Version 4, Src: *****, Dst: *****
0100 .... = Version: 4
.... 0101 = Header Length: 20 bytes (5)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 52
Identification: 0x0000 (0)
Flags: 0x40, Don't fragment
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
Fragment Offset: 0
Time to Live: 64
Protocol: TCP (6)
Header Checksum: 0xabe8 [validation disabled]
[Header checksum status: Unverified]
Source Address: 134.87.182.156
Destination Address: 172.217.165.14
Transmission Control Protocol, Src Port: 55888, Dst Port: 80, Seq: 76, Ack: 530, Len: 0
Source Port: 55888
Destination Port: 80
[Stream index: 3]
[TCP Segment Len: 0]
Sequence Number: 76 (relative sequence number)
Sequence Number (raw): 2379446728
[Next Sequence Number: 76 (relative sequence number)]
Acknowledgment Number: 530 (relative ack number)
Acknowledgment number (raw): 278173922
1000 .... = Header Length: 32 bytes (8)
Flags: 0x010 (ACK)
000. .... .... = Reserved: Not set
...0 .... .... = Nonce: Not set
.... 0... .... = Congestion Window Reduced (CWR): Not set
.... .0.. .... = ECN-Echo: Not set
.... ..0. .... = Urgent: Not set
.... ...1 .... = Acknowledgment: Set
.... .... 0... = Push: Not set
.... .... .0.. = Reset: Not set
.... .... ..0. = Syn: Not set
.... .... ...0 = Fin: Not set
[TCP Flags: ·······A····]
Window: 2048
[Calculated window size: 131072]
[Window size scaling factor: 64]
Checksum: 0xc5b6 [unverified]
[Checksum Status: Unverified]
Urgent Pointer: 0
Options: (12 bytes), No-Operation (NOP), No-Operation (NOP), Timestamps
TCP Option - No-Operation (NOP)
Kind: No-Operation (1)
TCP Option - No-Operation (NOP)
Kind: No-Operation (1)
TCP Option - Timestamps: TSval 190086814, TSecr 4203481592
Kind: Time Stamp Option (8)
Length: 10
Timestamp value: 190086814
Timestamp echo reply: 4203481592
[SEQ/ACK analysis]
[This is an ACK to the segment in frame: 41]
[The RTT to ACK the segment was: 0.000244000 seconds]
[iRTT: 0.064637000 seconds]
[Timestamps]
[Time since first frame in this TCP stream: 0.219685000 seconds]
[Time since previous frame in this TCP stream: 0.000244000 seconds]
I think each frame corresponds to a single request sent or received by my computer. I want to know how to reconstruct the exact request sent by my computer to googles server. Additionally, I want to know how to capture whatever was returned by the server.
During the packet capture you can use the -f (capture filter) option to extract only the packets linked to www.google.com
tshark -a duration:10 -T text -V -f "host www.google.com" > capture.txt
I set the -a (autostop) capture option to 10 seconds, because I was only doing the curl 'www.google.com' once.
The command above will capture all TCP and UDP related to the curl request.
If you want to reassemble the connection after a capture you need to create a pcap file during the capture:
# this is capturing all traffic
tshark -a duration:10 -w capture.pcap
You can query this pcap file in multiple ways:
tshark -r capture.pcap -Y http.request -T fields -e http.host -e http.user_agent
# output
www.google.com curl/7.64.1'
tshark -r capture.pcap -Y "(http.host == www.google.com)"
44 1.631029 192.168.86.35 → 64.233.177.105 HTTP 144 GET / HTTP/1.1
tshark -r capture.pcap -Y "dns.qry.name == www.google.com"
#output
36 1.546800 192.168.86.35 → 192.168.86.1 DNS 74 Standard query 0xe159 A www.google.com
39 1.587633 192.168.86.1 → 192.168.86.35 DNS 170 Standard query response 0xe159 A www.google.com A 64.233.177.105 A 64.233.177.104 A 64.233.177.103 A 64.233.177.99 A 64.233.177.147 A 64.233.177.106
# get the frame details for TCP packets associated with google
tshark -r capture.pcap -T fields -e tcp.stream -Y "tcp contains google"
#output frame number(s)
# get the frame details for UDP packets associated with google
tshark -r capture.pcap -T fields -e udp.stream -Y "udp contains google"
# follow the TCP frame
# this should return the curl request for www.google.com
tshark -r capture.pcap -q -z follow,tcp,ascii,frame_number
# follow the UDP frame
tshark -r capture.pcap -q -z follow,udp,ascii,frame_number
Hopefully, this answer helps solve your question.
I have mrd6 installed on my raspberry pi. It registers with a local interface (tun0) and periodically transmits MLDv2 queries over it.
According to [RFC3810], MLDv2 message types are a subset of ICMPv6 messages, and are identified in IPv6 packets by a preceding Next Header value of 58 (0x3a). They are sent with a link-local IPv6 Source Address, an IPv6 Hop Limit of 1, and an IPv6 Router Alert option [RFC2711] in a Hop-by-Hop Options header.
I can confirm that I'm seeing these packets periodically over tun0:
pi#machine:~ $ sudo tcpdump -i tun0 ip6 -vv -XX
01:22:52.125915 IP6 (flowlabel 0x71df6, hlim 1, next-header Options (0)
payload length: 36)
fe80::69bf:be2d:e087:9921 > ip6-allnodes: HBH (rtalert: 0x0000) (padn)
[icmp6 sum ok] ICMP6, multicast listener query v2 [max resp delay=10000]
[gaddr :: robustness=2 qqi=125]
0x0000: 6007 1df6 0024 0001 fe80 0000 0000 0000 `....$..........
0x0010: 69bf be2d e087 9921 ff02 0000 0000 0000 i..-...!........
0x0020: 0000 0000 0000 0001 3a00 0502 0000 0100 ........:.......
0x0030: 8200 b500 2710 0000 0000 0000 0000 0000 ....'...........
0x0040: 0000 0000 0000 0000 027d 0000 .........}..
I have a socket set up in my application on tun0 as follows, since I expect these to be ICMP packets:
int fd = socket(AF_INET6, SOCK_RAW, IPPROTO_ICMPV6); // ICMP
// ... bind this socket to tun0
int interfaceIndex = // tun0 interface Index
int mcastTTL = 10;
int loopBack = 1;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_IF,
&interfaceIndex,
sizeof(interfaceIndex))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_IF:: ");
}
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_LOOP,
&loopBack,
sizeof(loopBack))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_LOOP:: ");
}
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_MULTICAST_HOPS,
&mcastTTL,
sizeof(mcastTTL))
< 0) {
perror("setsockopt:: IPV6_MULTICAST_HOPS:: ");
}
struct ipv6_mreq mreq6 = {{{{0}}}};
MEMCOPY(&mreq6.ipv6mr_multiaddr.s6_addr, sourceAddress, 16);
mreq6.ipv6mr_interface = interfaceIndex;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_JOIN_GROUP,
&mreq6,
sizeof(mreq6))
< 0) {
perror("setsockopt:: IPV6_JOIN_GROUP:: ");
}
Setting up the socket this way, I can receive ICMP echo requests, replies to my own address, and multicasts sent using the link-local multicast address. However, I don't see any MLDv2 queries.
Here's my receive loop:
uint8_t received[1000] = { 0 };
struct sockaddr_storage peerAddress = { 0 };
socklen_t addressLength = sizeof(peerAddress);
socklen_t addressLength = sizeof(peerAddress);
int receivedLength = recvfrom(sockfd,
received,
sizeof(received),
0,
(struct sockaddr *)&peerAddress,
&addressLength);
if (receivedLength > 0) {
// Never get here for MLDv2 queries.
}
Researching this a bit further, I discovered the IPV6_ROUTER_ALERT socket option, which the man page describes as follows:
IPV6_ROUTER_ALERT
Pass forwarded packets containing a router alert hop-by-hop option to this socket.
Only allowed for SOCK_RAW sockets. The tapped packets are not forwarded by the
kernel, it is the user's responsibility to send them out again. Argument is a
pointer to an integer. A positive integer indicates a router alert option value
to intercept. Packets carrying a router alert option with a value field
containing this integer will be delivered to the socket. A negative integer
disables delivery of packets with router alert options to this socket.
So I figured I was missing this option, and tried setting it as follows. [RFC2710] 0 means Multicast Listener Discovery message.
int routerAlertOption = 0;
if (setsockopt(listener->socket,
IPPROTO_IPV6,
IPV6_ROUTER_ALERT,
&routerAlertOption,
sizeof(routerAlertOption))
< 0) {
perror("setsockopt:: IPV6_ROUTER_ALERT:: ");
}
However, this gives me the ENOPROTOOPT error (errno 92). Some more Googling (http://www.atm.tut.fi/list-archive/usagi-users-2005/msg00317.html) led me to the fact that you can't set the IPV6_ROUTER_ALERT option with the IPPROTO_ICMPV6 protocol. It needs a socket defined using the IPPROTO_RAW protocol.
However, defining my socket as:
int fd = socket(AF_INET6, SOCK_RAW, IPPROTO_RAW);
means I'm not able to receive any ICMP packets in my recvfrom anymore.
TL;DR: How do I read MLDv2 queries using an IPv6 socket?
edit (answer):
It appears conventional implementations of Linux will drop MLDv2 packets when passing them to an ICMPV6 socket. Why this is, I'm not sure. (Could be because of the next-header option.)
I followed the accepted answer below and went with an approach of reading raw packets on the tun0 interface. I followed the ping6_ll.c example here: http://www.pdbuchan.com/rawsock/rawsock.html.
It uses a socket with (SOCK_RAW, ETH_P_ALL). You can also set some SOL_PACKET options to filter on specific multicast rules on your interface.
From a quick look at RFCs things aren't looking good. Per RFC4443 (ICMPv6) 2.4:
2.4. Message Processing Rules
Implementations MUST observe the following rules when processing
ICMPv6 messages (from [RFC-1122]):
(b) If an ICMPv6 informational message of unknown type is received,
it MUST be silently discarded.
According to MLDv2 spec it makes use of types 130, 143, perhaps something else (not seeing more diagrams in the RFC), while valid ICMPv6 types are 1, 2, 3, 4, 101, 107, 127, 128, 129, 200, 201, 255.
It looks like the implementation (kernel) must drop MLDv2 packets if they are to be passed to an ICMPv6 socket. Personally I don't see much sense in making MLDv2 look like ICMPv6 if conventional implementations will drop the packet anyways, but I didn't see anything that contradicts this claim.
You can surely go deeper and use a raw socket, especially given that your stack doesn't recognize MLDv2 (perhaps there's a kernel patch to fix that?). But you'll have to parse IP and ICMP headers on your own then.
Are there a constants in scapy for TCP and UDP?
I mean
TCP=6, UDP=17
etc...
a)
looking up the implementation for IP we see that IP.proto is a ByteEnumField("proto", 0, IP_PROTOS),. This means, it takes values from the IP_PROTOS list which just loads your os /etc/protocols/. So you could either parse /etc/protocols yourself or, of scapy is already loaded, access the IP_PROTOS object directly:
>>> IP_PROTOS
</etc/protocols/ pim ip ax_25 esp tcp ah mpls_in_ip rohc ipv6_opts xtp st mobility_header dccp igmp ipv6_route igp ddp etherip wesp xns_idp ipv6_frag vrrp gre ipcomp encap ipv6 iso_tp4 sctp ipencap rsvp hip udp ggp hmp idpr_cmtp hopopt fc skip icmp pup manet isis rdp l2tp ipv6_icmp udplite egp ipip ipv6_nonxt eigrp idrp shim6 rspf ospf vmtp>
>>> IP_PROTOS.tcp
6
>>> IP_PROTOS.udp
17
>>> IP_PROTOS.ip
0
b) An alternative approach would be to read scapys layer binding information directly. This is the information that is added to a layer when you (or scapy core itself) calls bind_layers(lower,upper[,overload_fields]). You can easily read that information as follows:
>>> TCP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 6}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 6}}
Means, in case TCP is a payload to IPv4 (scapy.layers.inet.IP) it will override IP.proto=6.
Here's that same information for UDP
>>> UDP.overload_fields
{<class 'scapy.layers.inet6.IPv6'>: {'nh': 17}, <class 'scapy.layers.inet.IP'>: {'frag': 0, 'proto': 17}}
For reference, here is the bind_layers call for TCP/UDP
TCP and UDP are the initiators of TCP/UDP packets.
For example:
pack = IP(dst="www.google.com") / UDP(dport=80)
pack.show()
Result:
>>> pack = IP(dst="www.google.com") / UDP(dport=80)
>>> pack.show()
###[ IP ]###
version= 4
ihl= None
tos= 0x0
len= None
id= 1
flags=
frag= 0
ttl= 64
proto= udp
chksum= None
src= 'Your local address'
dst= Net('www.google.com')
\options\
###[ UDP ]###
sport= domain
dport= http
len= None
chksum= None
>>>
i'm a perl rookie :) and I need your help
i want to parsing my log.txt but i'm confused and i'm not sure to
this coding.
i have tried log.pcap but it's not like i need. so i want to parsing text not parsing pcap with recursive. Is there anyone edit my coding??
http://pastebin.com/mwZ1y2kM
this is my log.txt: (input)
http://pastebin.com/P0g0D6Pi
Frame 1 (640 bytes on wire, 640 bytes captured)
Arrival Time: Jan 31, 2012 19:41:17.121115000
[Time delta from previous captured frame: 0.000000000 seconds]
[Time delta from previous displayed frame: 0.000000000 seconds]
[Time since reference or first frame: 0.000000000 seconds]
Frame Number: 1
Frame Length: 640 bytes
Capture Length: 640 bytes
[Frame is marked: False]
[Protocols in frame: eth:ip:tcp:http]
Ethernet II, Src: SunMicro_45:39:78 (01:24:4c:50:79:95), Dst:
Cisco_03:3c:dc (03:49:12:65:3f:dc)
Destination: Cisco_03:3c:dc (03:49:12:65:3f:dc)
Address: Cisco_03:3c:dc (03:49:12:65:3f:dc)
.... ...0 .... .... .... .... = IG bit: Individual address
(unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique
address (factory default)
Source: SunMicro_45:39:78 (01:24:4c:50:79:95)
Address: SunMicro_45:39:78 (01:24:4c:50:79:95)
.... ...0 .... .... .... .... = IG bit: Individual address
(unicast)
.... ..0. .... .... .... .... = LG bit: Globally unique
address (factory default)
Type: IP (0x0800)
Internet Protocol, Src: 221.255.225.143 (221.255.225.143), Dst:
10.12.264.43 (10.12.264.43)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x01 (DSCP 0x00: Default; ECN:
0x01)
0000 00.. = Differentiated Services Codepoint: Default (0x01)
.... ..0. = ECN-Capable Transport (ECT): 0
.... ...0 = ECN-CE: 0
Total Length: 626
Identification: 0x3b68 (15208)
Flags: 0x02 (Don't Fragment)
0.. = Reserved bit: Not Set
.1. = Don't fragment: Set
..0 = More fragments: Not Set
Fragment offset: 0
Time to live: 118
Protocol: TCP (0x06)
Header checksum: 0xfc4b [correct]
[Good: True]
[Bad : False]
Source:221.255.225.143 (221.255.225.143)
Destination: 10.12.264.43 (10.12.264.43)
Transmission Control Protocol, Src Port: 45267 (45267), Dst Port: http
(80), Seq: 1, Ack: 1, Len: 566
Source port: 45267 (45267)
Destination port: http (80)
[Stream index: 0]
Sequence number: 1 (relative sequence number)
[Next sequence number: 587 (relative sequence number)]
Acknowledgement number: 1 (relative ack number)
Header length: 20 bytes
Flags: 0x18 (PSH, ACK)
0... .... = Congestion Window Reduced (CWR): Not set
.0.. .... = ECN-Echo: Not set
..0. .... = Urgent: Not set
...1 .... = Acknowledgement: Set
.... 1... = Push: Set
.... .0.. = Reset: Not set
.... ..0. = Syn: Not set
.... ...0 = Fin: Not set
Window size: 17520
Checksum: 0xc19e [validation disabled]
[Good Checksum: False]
[Bad Checksum: False]
[SEQ/ACK analysis]
[Number of bytes in flight: 586]
Hypertext Transfer Protocol
[truncated] GET /index.php?page=rilis&artikel=999999.9%27+union+all
+select+0x31303235343830303536%2C
[[truncated] Expert Info (Chat/Sequence): GET /index.php?
page=rilis&artikel=999999.9%27+union+all+select
+0x31303235343830303536%2C
[Message [truncated]: GET /index.php?
page=rilis&artikel=999999.9%27+union+all+select
+0x31303235343830303536%2C
[Severity level: Chat]
[Group: Sequence]
Request Method: GET
Request URI [truncated]: /index.php?
page=rilis&artikel=999999.9%27+union+all+select
+0x31303235343830303536%2C
Request Version: HTTP/1.1
Host: example.com\r\n
Accept: */*\r\n
User-Agent: Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1;
SV1; .NET CLR 2.0.50727) Havij\r\n
Connection: Close\r\n
\r\n
yeah, and I want to parsing like this: (output)
Time: Jan 31, 2012 19:41:17
IP Address Source: 221.255.225.143
Mac Address Source: 010912063cfc
Port Numbers Source: 44535
IP Address Destination: 10.12.264.43
Mac Address Destination: 00f04c080788
Port Numbers Destination: 3306
HTTP Host: example.com
Request Method: GET
Request URI: /index.php?page=rilis artikel=999999.9%27+union
+all+select+0x31303235343830303536%2C
Tool: Havij
thank you...
can you help me???
what i'm doing right now...
Your log lines are a bit strange. Some of the fields (Request URI and Tool) contain a space before the colon, which is unexpected. I would check that your log lines really contain those unexpected spaces. Also, it's a bit surprising that Port Numbers appears twice without the Source or Destination qualifier. Again, check your real log file. The following code snippet works for the log line posted in the question. Since the field names are not unique, it assumes they are always printed in the same order.
my #fields = ('Time', 'IP Address Source', 'Mac Address Source', 'Port Numbers',
'IP Address Destination', 'Mac Address Destination', 'Port Numbers', 'HTTP Host',
'Request Method', 'Request URI ', 'Tool ');
my $re = join(':\s+(.*?)\s+', #fields);
$re .= ':\s+(.*)';
warn $re;
$line = 'Time: Jan 31, 2012 19:41:17 IP Address Source: 221.255.225.143 Mac Address Source: 010912063cfc Port Numbers: 44535 IP Address Destination: 10.12.264.43 Mac Address Destination: 00f04c080788 Port Numbers: 3306 HTTP Host: example.com Request Method: GET Request URI : /index.php?page=rilis artikel=999999.9%27+union +all+select+0x31303235343830303536%2C Tool : Havij';
my #values = $line =~ /$re/;
print "values = #values\n"
Have you tried Net::Pcap or the approach described here?