I have python, scapy peace of code that store my data into database (IP src and dst, ports, ..)
which i use for some statistics.
On some packets i am doing some manipulation (changing dst port) and then send them back out on interface.
Problem is that this packet i was manipulating with have different pkt.time value than original one
and if I store those packets into database they have different packet time then they have originally.
Is there and option within creating UDP packet to put original pkt.time value?
With this option packet manipulation delay would not cause disorder with my packets.
Any help is welcome
Below is my manipulation script
#!/usr/bin/env python
from scapy.all import *
# VARIABLES
interface = 'eth1'
filter_bpf = "port 8000"
def pkt_change(pkt):
if pkt.haslayer(UDP):
# --> pkt.time is packet time
ts = pkt.time
src = pkt[IP].src
dst = pkt[IP].dst
sport = pkt[IP].sport
dport = pkt[IP].dport
msg = pkt[IP].load
#### Spoof Response
changed_pkt = Ether()/IP(dst=dst, src=src)/UDP(dport=8000, sport=sport)/msg
sendp(changed_pkt, iface="eth1")
print 'Sent:', changed_pkt.summary()
# ------------------------------------------------
# start sniffing
print "Start Sniffing"
sniff(iface=interface, filter=filter_bpf, store=0, prn=pkt_change)
After creating changed_pkt, you can simply set its time attribute as follows:
changed_pkt.time = ts
Note that even after changing the packet's timestamp and sending it, the updated timestamp won't be reflected in the received packet on the other end since the timestamp is set in the receiving machine as the packet is received, as described here.
If you're interested in transmitting the packets to a remote machine, while keeping their timestamp, consider storing the manipulated packets in a pcap file and sending that file over to the other machine.
Related
What will happen if I will establish a connection between a client and a server, and configure a different buffer size for each of them.
This is my client's code:
import socket,sys
TCP_IP = sys.argv[1]
TCP_PORT = int(sys.argv[2])
BUFFER_SIZE = 1024
MESSAGE = "World! Hello, World!"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
s.close()
print "received data:", data
Server's code:
import socket,sys
TCP_IP = '0.0.0.0'
TCP_PORT = int(sys.argv[1])
BUFFER_SIZE = 5
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
while True:
conn, addr = s.accept()
print 'New connection from:', addr
while True:
data = conn.recv(BUFFER_SIZE)
if not data: break
print "received:", data
conn.send(data.upper())
conn.close()
That means I will be limited to only 5 bytes? Which means I won't be able to receive the full packet and will lose 1024-5 packets?
I or does it mean I am able to get only packets of 5 bytes, which means that instead of receiving one packets of 1024 bytes as the client sent it, I'll have to divide 1024 by 5 and get 204.8 packets (?) which sounds not possible.
What in general is happing in that code?
Thanks.
Your arguments are based on the assumption that a single send should match a single recv. But this is not the case. TCP is a byte stream and not a message based protocol. This means all what matters are the transferred bytes. And for this is does not matter if it does not matter if one or 10 recv are needed to read 50 bytes.
Apart from that send is not guaranteed to send the full buffer either. It might only send parts of the buffer, i.e. the sender need actually check the return code to find out how much of the given buffer was actually send now and how much need to be retried for sending later.
And note that the underlying "packet" is again a different thing. If there is a send for 2000 bytes it will usually need multiple packets to be send (depending on the maximum transfer unit of the underlying data link layer). But this does not mean that one also need multiple recv. If all the 2000 bytes are already transferred to the OS level receive buffer at the recipient then they can be also be read at once, even if they traveled in multiple packets.
Your socket won't lose the remaining 1024 - 5 (1019) bytes.it just stored on the socket and ready to read again! so , all you need to do is to read from the socket again. the size of buffer you want to read to is decided by yourself. and you are not limited to 5 bytes, you are just limiting the read buffer for each single read to 5 bytes. so for 1024 bytes to read you have to read for 204 times plus another time read which would be the last one. but remember that the last time read fills your last buffer index with null. and that means there is no more bytes available for now.
I want to read IP packets from a non-blocking tun/tap file descriptor tunfd
I set the tunfd as non-blocking and register a READ_EV event for it in libevent.
when the event is triggered, I read the first 20 bytes first to get the IP header, and then
read the rest.
nr_bytes = read(tunfd, buf, 20);
...
ip_len = .... // here I get the IP length
....
nr_bytes = read(tunfd, buf+20, ip_len-20);
but for the read(tunfd, buf+20, ip_len-20)
I got EAGAIN error, actually there should be a full packet,
so there should be some bytes,
why I get such an error?
tunfd is not compatible with non-blocking mode or libevent?
thanks!
Reads and writes with TUN/TAP, much like reads and writes on datagram sockets, must be for complete packets. If you read into a buffer that is too small to fit a full packet, the buffer will be filled up and the rest of the packet will be discarded. For writes, if you write a partial packet, the driver will think it's a full packet and deliver the truncated packet through the tunnel device.
Therefore, when you read a TUN/TAP device, you must supply a buffer that is at least as large as the configured MTU on the tun or tap interface.
I am writing the callout driver for Hyper-V 2012 where I need to filter the packets sent from virtual machines.
I added filter at FWPM_LAYER_EGRESS_VSWITCH_TRANSPORT_V4 layer in WFP. Callout function receive packet buffer which I am typecasting it to NET_BUFFER_LIST. I am doing following to get the data pointer
pNetBuffer = NET_BUFFER_LIST_FIRST_NB((NET_BUFFER_LIST*)pClassifyData->pPacket);
pContiguousData = NdisGetDataBuffer(pNetBuffer, NET_BUFFER_DATA_LENGTH(pNetBuffer), 0, 1, 0);
I have simple client-server application to test the packet data. Client is on VM and server is another machine. As I observed, data sent from client to server is truncated and some garbage value is added at the end. There is no issue for sending message from server to client. If I dont add this layer filter client-server works without any issue.
Callback function receives the metadata which incldues ipHeaderSize and transportHeaderSize. Both these values are zero. Are these correct values or should those be non-zero??
Can somebody help me to extract the data from packet in callout function and forward it safely to further layers?
Thank You.
These are the TCP packets. I looked into size and offset information. It seems the problem is consistent across packets.
I checked below values in (NET_BUFFER_LIST*)pClassifyData->pPacket.
NET_BUFFER_LIST->NetBUfferListHeader->NetBUfferListData->FirstNetBuffer->NetBuffe rHeader->NetBufferData->CurrentMdl->MappedSystemVa
First 24 bytes are only sent correctly and remaining are garbage.
For example total size of the packet is 0x36 + 0x18 = 0x4E I don't know what is there in first 0x36 bytes which is constant for all the packets. Is it a TCP/IP header? Second part 0x18 is the actual data which i sent.
I even tried with API NdisQueryMdl() to retrieve from MDL list.
So on the receiver side I get only 24 bytes correct and remaining is the garbage. How to read the full buffer from NET_BUFFER_LIST?
I have a question concerning udp packets in MATLAB. I've been using the oscsend.m script to send udp packets to other applications.
http://www.mathworks.fr/matlabcentral/fileexchange/31400-send-open-sound-control-osc-messages/content/oscsend.m
It works normaly. however, now I need to make use of the pnet function from the TCP/UDP/IP Toolbox 2.0.6 http://www.mathworks.de/matlabcentral/fileexchange/345-tcpudpip-toolbox-2-0-6
because I no longer have access to the instrument control toolbox in Matlab, which oscsend makes use of.
I looked up online for answers and even contacted the authors with no luck so far. Even though the problem seems trivial I couldn't make it work.
normally the usage with the Instrument Control toolbox would be:
u = udp('127.0.0.1', 12345) %12345 being the port
fopen(u)
oscsend(u, /test, 'f', 1.05) %"f" indicating a floating number
but now without udp or fopen I tried to use pnet like this
%write data to UDP
data = [oscstr(path) types data];
sock=pnet('udpsocket',12345) %it returns 0, a sign that the socket is working
%however when I try to send the oscsend signal through this socket nothing happens
pnet(sock, 'writepacket' data) %data being the output of oscsend
I also tried with no luck integrating the pnet function to oscsend but I couldn't recieve a signal, (I am working with another instance of MATLAB which is recieving the data sent from the previoulsy mentioned port).
This is what I did inside oscsend and none of them worked
%write data to UDP
data = [oscstr(path) types data];
%pnet(u, 'writepacket', data, '127.0.0.1', 12345 );%
%pnet u 'write' data %
%pnet(u,'write',data, '127.0.0.1', 12345 )
%pnet(u,'write',data, '127.0.0.1', 12345 ) %returned value?
%pnet(0,'write',data, '127.0.0.1', 12345 ) %
%sock = pnet('udpsocket',12345 ); %
%pnet(sock,'write', data, '127.0.0.1', 12345 )
Thanks to all of you in advance who took the time to read this.
Best.
Mario.
It's not clear to me what UDP port you want to send your data to, and what sort of process you're using to monitor and test your code. There are two ports involved... One is the local port that you're binding to (that's the argument to 'udpsocket'), and the other port is the destination port of the packet, given along with the destination host.
It appears to me that the Instrument Control Toolbox syntax requires the destination hostname and port during creation. It allows you to specify the local port as an optional argument if you want to. In constrast, pnet('udpsocket') takes the local port. You should notice in your first set of code that nothing specifies the destination host, which should make you suspicious... If you read on in the UDP docs of pnet.m, you'll see that there's another function: pnet(sock, 'udpconnect', 'hostname', port), which "connects" the UDP socket to the host/port pair, so that you can leave out the hostname/port when doing writepacket.
So here is what I think the equivalent is of your original Instrument Control Toolbox code:
sock=pnet('udpsocket',1237); % Does local port matter? You haven't said...
pnet(sock, 'udpconnect', '127.0.0.1', 12345); % Destination port
pnet(sock, 'writepacket', data);
You should ALWAYS use 'writepacket' for UDP, never 'write'. And note that you can either do 'udpconnect' as I've listed above, OR you can supply hostname/port with every writepacket, as you were trying to do.
What I can't figure out from your question is how 6351 enters in, as that never showed up in your reference code.
btw, there is also oscmex, a library based on liblo that allows you to send/receive OSC messages directly from/within matlab.
Thanks to my supervisor here is the code if you ever need to send udps via oscsend.m using pnet
http://www.mathworks.fr/matlabcentral/fileexchange/31400-send-open-sound-control-osc-messages/content/oscsend.m
just add the following code at the end of oscsend.m
%write data to UDP
data = [oscstr(path) types data];
pnet(u, 'write', data)
pnet(u, 'writepacket', '127.0.0.1', 12345); %127.0.0.1 being the IP and 12345 the port
I starts learning TCP protocol from internet and having some experiments. After I read an article from http://www.diffen.com/difference/TCP_vs_UDP
"TCP is more reliable since it manages message acknowledgment and retransmissions in case of lost parts. Thus there is absolutely no missing data."
Then I do my experiment, I write a block of code with TCP socket:
while( ! EOF (file))
{
data = read_from(file, 5KB); //read 5KB from file
write(data, socket); //write data to socket to send
}
I think it's good because "TCP is reliable" and it "retransmissions lost parts"... But it's not good at all. A small file is OK but when it comes to about 2MB, sometimes it's OK but not always...
Now, I try another one:
while( ! EOF (file))
{
wait_for_ACK();//or sleep 5 seconds
data = read_from(file, 5KB); //read 5KB from file
write(data, socket); //write data to socket to send
}
It's good now...
All I can think of is that the 1st one fails because of:
1. buffer overflow on sender because the sending rate is slower than the writing rate of the program (the sending rate is controlled by TCP)
2. Maybe the sending rate is greater than writing rate but some packets are lost (after some retransmission, still fails and then TCP gives up...)
Any ideas?
Thanks.
TCP will ensure that you don't lose data but you should check how many bytes actually got accepted for transmission... the typical loop is
while (size > 0)
{
int sz = send(socket, bufptr, size, 0);
if (sz == -1) ... whoops, error ...
size -= sz; bufptr += sz;
}
when the send call accepts some data from your program then it's a job of the OS to get that to destination (including retransmission), but the buffer for sending may be smaller than the size you need to send, and that's why the resulting sz (number of bytes accepted for transmission) may be less than size.
It's also important to consider that sending is asynchronous, i.e. after the send function returns the data is not already at the destination, it's has been only assigned to the TCP transport system to be delivered. If you want to know when it will be received then you'll have to use other systems (e.g. a reply message from your counterpart).
You have to check write(socket) to make sure it writes what you ask.
Loop until you've sent everything or you've calculated a time out.
Do not use indefinite timeouts on socket read/write. You're asking for trouble if you do, especially on Windows.