I was looking through a slide by IOvisor project, https://events.static.linuxfound.org/sites/events/files/slides/iovisor-lc-bof-2016.pdf
#include <bcc/proto.h>
struct IPKey { u32 dip; u32 sip; };
BPF_TABLE("hash", struct IPKey, int, mytable, 1024);
int recv_packet(struct __sk_buff *skb) {
struct IPKey key;
u8 *cursor = 0;
struct ethernet_t *ethernet = cursor_advance(cursor, sizeof(*ethernet));
struct ip_t *ip = cursor_advance(cursor, sizeof(*ip));
key.dip = ip->dst;
key.sip = ip->src;
int *leaf = mytable.lookup(&key);
if (leaf)
*(leaf)++;
return 0;
}
This code is amongst the examples.
I've been using cursor_advance() quite often and now I'm trying to figure out what exactly it does.
I suspect that cursor is a pointer where we save the address of the packet we are parsing.
Then, with cursor_advance() we move the cursor by the size of the ethernet header, since ethernet_t contains all the ethernet header information.
Then, the cursor now at the address at the end of the ethernet header of the packet and if we use variables declared in the ethernet_t header, such as type, like : ethernet->type, we can access the information saved at type since the struct ethernet would read the values saved in that address?
I'm sorry my explanation is not really good.
I'm just looking for a general explanation or if my theory is correct.
Thanks!
Your understanding sounds correct to me. Just think of it as a “cursor” used to successively parse the different headers of your packet. The cursor_advance() macro is defined as:
#define cursor_advance(_cursor, _len) \
({ void *_tmp = _cursor; _cursor += _len; _tmp; })
It adds _len to the _cursor, and returns the value _cursor had before we added _len.
So the first call to cursor_advance() returns the initial value: ethernet points to the beginning of the packet, and we can use its attributes to access the different fields of the Ethernet header. But this same call also moves the cursor forwards by the length of the Ethernet header, so now it points to the beginning of the next header (L3, e.g. IP). The second call to cursor_advance() returns the pointer to the L3 layer, which we store in ip. The cursor is also moved forward and, assuming the packet is IPv4, would now point at the L4 header.
Note: I do not believe this mechanism is widely used in BPF programs aside from the few networking examples available in BCC. Instead, programs often navigate through packet headers with skb->data and skb->data_end.
Related
I've been creating a virtual ethernet interface. I've opened asynchronous communication with a controlling application and every time there are new packets, the controlling app is notified and then asks for the packet data. The packet data is stored in a simple struct, with uint8_t[1600] for the bytes, and uint32_t for the length. The dext is able to populate this struct with dummy data every time a packet is available, with the dummy data visible on the controlling application. However, I'm struggling to fill it with the real packet data.
The IOUserNetworkPacket provides metadata about a packet. It contains a packets timestamp, size, etc, but it doesn't seem to contain the packet's data. There are the GetDataOffset() and GetMemorySegmentOffset() methods which seem to return byte offsets for where the packet data is located in their memory buffer. My instinct tells me to add this offset to the pointer of wherever the packet data is stored. The problem is I have no idea where the packets are actually stored.
I know they are managed by the IOUserNetworkPacketBufferPool, but I don't think that's where their memory is. There is the CopyMemoryDescriptor() method which gives an IOMemoryDescriptor of its contents. I tried using the descriptor to create an IOMemoryMap, using it to call GetAddress(). The pointers to all the mentioned objects lead to junk data.
I must be approaching this entirely wrong. If anyone knows how to access the packet data, or has any ideas, I would appreciate any help. Thanks.
Code snippet within IOUserClient::ExternalMethod:
case GetPacket:
{
IOUserNetworkPacket *packet =
ivars->m_provider->getPacket();
GetPacket_Output output;
output.packet_size = packet->getDataLength();
IOUserNetworkPacketBufferPool *pool;
packet->GetPacketBufferPool(&pool);
IOMemoryDescriptor *memory = nullptr;
pool->CopyMemoryDescriptor(&memory);
IOMemoryMap *map = nullptr;
memory->CreateMapping(0, 0, 0, 0, 0, &map);
uint64_t address = map->GetAddress()
+ packet->getMemorySegmentOffset();
memcpy(output.packet_data,
(void*)address, packet->getDataLength());
in_arguments->structureOutput = OSData::withBytes(
&output, sizeof(GetPacket_Output));
// free stuff
} break;
The problem was caused by an IOUserNetworkPacketBufferPool bug. My bufferSize was set to 1600 except this value was ignored and replaced with 2048. The IOUserNetworkPackets acted as though the bufferSize was 1600 and so they gave an invalid offset.
Creating the buffer pool and mapping it:
kern_return_t
IMPL(FooDriver, Start)
{
// ...
IOUserNetworkPacketBufferPool::Create(this, "FooBuffer",
32, 32, 2048, &ivars->packet_buffer));
packet_buffer->CopyMemoryDescriptor(ivars->packet_buffer_md);
ivars->packet_md->Map(0, 0, 0, IOVMPageSize,
&ivars->packet_buffer_addr, &ivars->packet_buffer_length));
// ...
}
Getting the packet data:
void FooDriver::getPacketData(
IOUserNetworkPacket *packet,
uint8_t *packet_data,
uint32_t *packet_size
) {
uint8_t packet_head;
uint64_t packet_offset;
packet->GetHeadroom(&packet_head);
packet->GetMemorySegmentOffset(&packet_offset);
uint8_t *buffer = (uint8_t*)(ivars->packet_buffer_addr
+ packet_offset + packet_head);
*packet_size = packet->getDataLength();
memcpy(packet_data, buffer, *packet_size);
}
I have tried to contact the original designer for the ADS1115->EmonLib adaptation, but unfortunately his Github page has no contact information. I have also tried asking on the OpenEnergyMonitor forum and I get "Contact the person who made the changes".
So, I've come here in the hopes that the knowledgeable folk here can help me out.
Situation:
I have an ESP32 that I'm going to be using to monitor my energy consumption of my home. Because I will be monitoring several circuits I need lots of sensors. The ESP32 only has a 12-bit ADC and I'm limited to only a few sensors. Also limits future expansion if needed.
So here comes the ADS1115, which is a 16-bit ADC, but it uses I2C. Unfortunately, the EmonLib was designed to use onboard ADC, and not through an I2C device. So after doing some digging around, I found this modification to allow the ADS1115 to be used:
https://github.com/PaulWieland/EmonLib/
However, Paul Wieland is not able to be contacted through github.
Ok, so if I wanted to use only 1 ADS1115, I could just use his code as stock, which is this:
// In this example we will use an ADS1115 breakout board instead of the Arduino's local analog inputs
// This is especially useful for nodemcu/esp8266 users who only have a single analog input
#include <Wire.h>
#include <Adafruit_ADS1015.h>
// EmonLibrary examples openenergymonitor.org, Licence GNU GPL V3
#include <EmonLib_I2C.h> // Include Emon Library
EnergyMonitor emon1; // Create an instance
Adafruit_ADS1115 ads; // Create an instance of the ADS1115 object
// Make a callback method for reading the pin value from the ADS instance
int ads1115PinReader(int _pin){
return ads.readADC_SingleEnded(_pin);
}
void setup()
{
emon1.inputPinReader = ads1115PinReader; // Replace the default pin reader with the customized ads pin reader
emon1.current(1, 111.1); // Current: input pin, calibration.
}
void loop()
{
double Irms = emon1.calcIrms(1480); // Calculate Irms only
Serial.print(Irms*230.0); // Apparent power
Serial.print(" ");
Serial.println(Irms); // Irms
}
This compiles just fine. Though I am kinda confused as to how to select the "pin" of the ADS.
In the emonlib.cpp he added this code:
//--------------------------------------------------------------------------------------
// Constructor. Set the pinReader to the default pin reader method
//--------------------------------------------------------------------------------------
EnergyMonitor::EnergyMonitor()
{
this->inputPinReader = defaultInputPinReader;
}
//--------------------------------------------------------------------------------------
// By default we just call Arduino's analogRead
//--------------------------------------------------------------------------------------
int EnergyMonitor::defaultInputPinReader(int _pin)
{
return analogRead(_pin);
}
And changed this:
startV = analogRead(inPinV);
to:
startV = (this->inputPinReader)(inPinV);
and a few other spots where he made the adjustment using "this->".
And in the emonlib.h he added this to the Public section:
EnergyMonitor();
typedef int (*inputPinReaderMethod) (int _pin);
inputPinReaderMethod inputPinReader;
static int defaultInputPinReader(int _pin);
Now, for a single ADS1115, I think this works. At least the code compiles. But I need to use 4 ADS1115's as I'll need to be able to monitor up to 16 circuits. I can't for the life of me figure out how to add an array of ADS's to an array of emonlib's.
I tried doing this:
EnergyMonitor emon1[4]; // Create an instance
Adafruit_ADS1115 ads[4]; // Create an instance of the ADS1115 object
// Make a callback method for reading the pin value from the ADS instance
int ads1115PinReader(int _channel, int _pin){
return ads[1].readADC_SingleEnded(_pin);
}
void setup()
{
emon1[1].inputPinReader = ads1115PinReader; // Replace the default pin reader with the customized ads pin reader
emon1[1].current(1, 111.1); // Current: input pin, calibration.
}
void loop()
{
double Irms = emon1[1].calcIrms(1480); // Calculate Irms only
Serial.print(Irms*230.0); // Apparent power
Serial.print(" ");
Serial.println(Irms); // Irms
}
but then I get this error:
Emon_ADS_Test:18:27: error: invalid conversion from 'int (*)(int, int)' to 'EnergyMonitor::inputPinReaderMethod {aka int (*)(int)}' [-fpermissive]
emon1[1].inputPinReader = ads1115PinReader; // Replace the default pin reader with the customized ads pin reader
I'm not very advanced in coding, and function pointers just confuse the heck outta me.
So my dilemma is I have several ADS1115's I need to match to the several instances of the EmonLib.
There has to be a separate instance for each wire monitored. On my working model that uses an Arduino Nano I have 2 emonlib variables (emon1, emon2). This works fine as it uses the internal ADC, tho I've found out over time that it is not very accurate and not fast enough or high enough resolution for my needs. (ie. when 0-current is flowing, it shows 50W usage).
What do I need to make a change to make this thing work?
Kori
[UPDATE 1]
Ok, after some testing, I do still get this:
emon[i * j].inputPinReader = ads1115PinReader;
emon[i * j].current(j++, 111.1);
Unfortunately this throws an error still:
Emon_ADS_Test:21:33: error: invalid conversion from 'int (*)(int, int)' to 'EnergyMonitor::inputPinReaderMethod {aka int (*)(int)}' [-fpermissive]
emon1[i * j].inputPinReader = ads1115PinReader;
However, it comes to the function pointer as being a problem.
I use this code:
int ads1115PinReader(int _pin){
return ads.readADC_SingleEnded(_pin);
}
It works just fine. However if I do this:
int ads1115PinReader(int _channel, int _pin){
return ads[_channel].readADC_SingleEnded(_pin);
}
I get the error above. In the EmonLib_I2C library, the only thing I can think of is in this part:
typedef int (*inputPinReaderMethod) (int _pin);
inputPinReaderMethod inputPinReader;
static int defaultInputPinReader(int _pin);
not matching up. But even that doesn't make sense. In this line:
emon.inputPinReader = ads1115PinReader; // Replace the default pin reader with the customized ads pin reader
does that mean that when I call inputPinReader, it "actually" grabs the value from ads1115PinReader instead of the one in the library?
I need to parse the pcap files and count the packets separately (TCP,UDP,IP). I found a lot of libraries for this like pcap, jnetpcap but I want to do this without using any external libraries.I do not need a code but a just a conceptual explanation.
Question
While parsing pcap files how should I distinguish between the frames(be it TCP,UDP,IP). I tried reading about the format but what I do not understand is how would I come to know about how many bytes should I read for a particular frame and how would i know what type of a frame is it.Because only once I am able to extract the packets separately I will be able to filter out other information.
You'd have to parse each frame separately and have a counter for each value you are trying to count. Assuming the capture you are examining is in pcap/pcapng format you might find libpcap helpful.
To give a quick run of what you might have to do (assuming the lower level is Ethernet without VLAN tags)
uint64_t ip_count, tcp_count, udp_count;
void parse_pkt(uint8_t *data, uint32_t data_len) {
uint8_t *ether_hdr = data;
uint16_t ether_type = ntohs(*(uint16_t *) (data + 12))
if (ether_type != 0x800) {
return;
}
ip_count += 1;
uint8_t *ip_hdr = data + 14;
protocol = ntohs(*(uint16_t *) (ip_hdr + 9))
//protocol is either udp/tcp/sctp...etc
if (protocol == 0x11) {
udp_count++;
} else if (protocol == 0x06) {
tcp_count++;
}
}
// foreach pkt from libpcap_open call parse_pkt with the data and data_len
This code is fragile. Jumping to direct offsets without the proper length and type checks is not a good idea.
I am trying to broadcast data but the output is udp send failed. I chose a random port 33333. What's wrong with my code?
int main()
{
struct sockaddr_in udpaddr = { sin_family : AF_INET };
int xudpsock_fd,sock,len = 0,ret = 0,optVal = 0;
char buffer[255];
char szSocket[64];
memset(buffer,0x00,sizeof(buffer));
memset(&udpaddr,0,sizeof(udpaddr));
udpaddr.sin_addr.s_addr = INADDR_BROADCAST;
udpaddr.sin_port = htons(33333);
xudpsock_fd = socket(PF_INET,SOCK_DGRAM,IPPROTO_UDP);
optVal = 1;
ret = setsockopt(xudpsock_fd,SOL_SOCKET,SO_BROADCAST,(char*)&optVal,sizeof(optVal));
strcpy(buffer,"this is a test msg");
len = sizeof(buffer);
ret = sendto(xudpsock_fd,buffer,len,0,(struct sockaddr*)&udpaddr,sizeof(udpaddr));
if (ret == -1)
printf("udp send failed\n");
else
printf("udp send succeed\n");
return (0);
}
One problem is that the address family you are trying to send to is zero (AF_UNSPEC). Although you initialize the family to AF_INET at the top of the function, you later zero it out with memset.
On the system I tested with, the send actually works anyway for some strange reason despite the invalid address family, but you should definitely try fixing that first.
You probably had a problem with your default route (eg, you didn't have one). sendto needs to pick an interface to send the packet on, but the destination address was probably outside the Destination/Genmask for each defined interface (see the 'route' command-line tool).
The default route catches this type of packet and sends it through an interface despite this mismatch.
Setting the destination to 127.255.255.255 will usually cause the packet to be sent through the loopback interface (127.0.0.1), meaning it will be able to be read by applications that (in this case) are run on the local machine.
Is it possible to decipher the hostname given the sockaddr_in strucure?
struct sockaddr_in {
short sin_family; // e.g. AF_INET, AF_INET6
unsigned short sin_port; // e.g. htons(3490)
struct in_addr sin_addr; // see struct in_addr, below
char sin_zero[8]; // zero this if you want to
};
struct in_addr {
unsigned long s_addr; // load with inet_pton()
};
I'd tried printing out "udp_server.sin_addr.s_addr" (where udp_server is of type struct sockaddr_in) and it printed me a number.
I believe the s_addr variable is converted to network long? How do I convert it into a readable format so I can determine the host?
There's a number of functions for transforming the number you mentioned (a network address in long format) to human-readable addresses.
Take a look at inet_pton() and inet_ntop() (Presentation to Number, Number to Presentation). Those are AF_INET-specific, and work for both IPv4 and IPv6 if I'm not mistaken.
Suerte!
Yes, it's the IP address in network byte order. Use the inet_ntoa(3) or inet_ntop(3) to get back a string representation of the IP.