I need to parse the pcap files and count the packets separately (TCP,UDP,IP). I found a lot of libraries for this like pcap, jnetpcap but I want to do this without using any external libraries.I do not need a code but a just a conceptual explanation.
Question
While parsing pcap files how should I distinguish between the frames(be it TCP,UDP,IP). I tried reading about the format but what I do not understand is how would I come to know about how many bytes should I read for a particular frame and how would i know what type of a frame is it.Because only once I am able to extract the packets separately I will be able to filter out other information.
You'd have to parse each frame separately and have a counter for each value you are trying to count. Assuming the capture you are examining is in pcap/pcapng format you might find libpcap helpful.
To give a quick run of what you might have to do (assuming the lower level is Ethernet without VLAN tags)
uint64_t ip_count, tcp_count, udp_count;
void parse_pkt(uint8_t *data, uint32_t data_len) {
uint8_t *ether_hdr = data;
uint16_t ether_type = ntohs(*(uint16_t *) (data + 12))
if (ether_type != 0x800) {
return;
}
ip_count += 1;
uint8_t *ip_hdr = data + 14;
protocol = ntohs(*(uint16_t *) (ip_hdr + 9))
//protocol is either udp/tcp/sctp...etc
if (protocol == 0x11) {
udp_count++;
} else if (protocol == 0x06) {
tcp_count++;
}
}
// foreach pkt from libpcap_open call parse_pkt with the data and data_len
This code is fragile. Jumping to direct offsets without the proper length and type checks is not a good idea.
Related
I've been creating a virtual ethernet interface. I've opened asynchronous communication with a controlling application and every time there are new packets, the controlling app is notified and then asks for the packet data. The packet data is stored in a simple struct, with uint8_t[1600] for the bytes, and uint32_t for the length. The dext is able to populate this struct with dummy data every time a packet is available, with the dummy data visible on the controlling application. However, I'm struggling to fill it with the real packet data.
The IOUserNetworkPacket provides metadata about a packet. It contains a packets timestamp, size, etc, but it doesn't seem to contain the packet's data. There are the GetDataOffset() and GetMemorySegmentOffset() methods which seem to return byte offsets for where the packet data is located in their memory buffer. My instinct tells me to add this offset to the pointer of wherever the packet data is stored. The problem is I have no idea where the packets are actually stored.
I know they are managed by the IOUserNetworkPacketBufferPool, but I don't think that's where their memory is. There is the CopyMemoryDescriptor() method which gives an IOMemoryDescriptor of its contents. I tried using the descriptor to create an IOMemoryMap, using it to call GetAddress(). The pointers to all the mentioned objects lead to junk data.
I must be approaching this entirely wrong. If anyone knows how to access the packet data, or has any ideas, I would appreciate any help. Thanks.
Code snippet within IOUserClient::ExternalMethod:
case GetPacket:
{
IOUserNetworkPacket *packet =
ivars->m_provider->getPacket();
GetPacket_Output output;
output.packet_size = packet->getDataLength();
IOUserNetworkPacketBufferPool *pool;
packet->GetPacketBufferPool(&pool);
IOMemoryDescriptor *memory = nullptr;
pool->CopyMemoryDescriptor(&memory);
IOMemoryMap *map = nullptr;
memory->CreateMapping(0, 0, 0, 0, 0, &map);
uint64_t address = map->GetAddress()
+ packet->getMemorySegmentOffset();
memcpy(output.packet_data,
(void*)address, packet->getDataLength());
in_arguments->structureOutput = OSData::withBytes(
&output, sizeof(GetPacket_Output));
// free stuff
} break;
The problem was caused by an IOUserNetworkPacketBufferPool bug. My bufferSize was set to 1600 except this value was ignored and replaced with 2048. The IOUserNetworkPackets acted as though the bufferSize was 1600 and so they gave an invalid offset.
Creating the buffer pool and mapping it:
kern_return_t
IMPL(FooDriver, Start)
{
// ...
IOUserNetworkPacketBufferPool::Create(this, "FooBuffer",
32, 32, 2048, &ivars->packet_buffer));
packet_buffer->CopyMemoryDescriptor(ivars->packet_buffer_md);
ivars->packet_md->Map(0, 0, 0, IOVMPageSize,
&ivars->packet_buffer_addr, &ivars->packet_buffer_length));
// ...
}
Getting the packet data:
void FooDriver::getPacketData(
IOUserNetworkPacket *packet,
uint8_t *packet_data,
uint32_t *packet_size
) {
uint8_t packet_head;
uint64_t packet_offset;
packet->GetHeadroom(&packet_head);
packet->GetMemorySegmentOffset(&packet_offset);
uint8_t *buffer = (uint8_t*)(ivars->packet_buffer_addr
+ packet_offset + packet_head);
*packet_size = packet->getDataLength();
memcpy(packet_data, buffer, *packet_size);
}
I'm gonna to read/write under the modbus-tcp specification.
So, I'm trying to code the client and server in the linux environment.
(I would communicate with the windows program(as a client) using the modbus-tcp.)
but it doesn't work as I want, so I ask you here.
I'm testing the client code for linux as a client and the easymodbus as a server.
I used the libmodbus code.
I'd like to read coil(0x01) and write coil(0x05).
When the code is executed using the libmodbus, 'ff' is printed out from the Unit ID part.(according to the manual, 01 should be output for modbus-tcp.
I don't know why 'ff' is printed(photo attached).
Wrong result:
Expected result:
'[00] [00] .... [00]' ; Do you know where to control this part?
Do you have or do you know the sample code that implements the 'read/write' function using the libmodbus?
please let me know the information, if you know that.
ctx = modbus_new_tcp("192.168.0.99", 502);
modbus_set_debug(ctx, TRUE);
if (modbus_connect(ctx) == -1) {
fprintf(stderr, "Connection failed: %s\n",
modbus_strerror(errno));
modbus_free(ctx);
return -1;
}
tab_rq_bits = (uint8_t *) malloc(nb * sizeof(uint8_t));
memset(tab_rq_bits, 0, nb * sizeof(uint8_t));
tab_rp_bits = (uint8_t *) malloc(nb * sizeof(uint8_t));
memset(tab_rp_bits, 0, nb * sizeof(uint8_t));
nb_loop = nb_fail = 0;
/* WRITE BIT */
rc = modbus_write_bit(ctx, addr, tab_rq_bits[0]);
if (rc != 1) {
printf("ERROR modbus_write_bit (%d)\n", rc);
printf("Address = %d, value = %d\n", addr, tab_rq_bits[0]);
nb_fail++;
} else {
rc = modbus_read_bits(ctx, addr, 1, tab_rp_bits);
if (rc != 1 || tab_rq_bits[0] != tab_rp_bits[0]) {
printf("ERROR modbus_read_bits single (%d)\n", rc);
printf("address = %d\n", addr);
nb_fail++;
}
}
printf("Test: ");
if (nb_fail)
printf("%d FAILS\n", nb_fail);
else
printf("SUCCESS\n");
free(tab_rq_bits);
free(tab_rp_bits);
/* Close the connection */
modbus_close(ctx);
modbus_free(ctx);
return 0;
That FF you see right before the Modbus function is actually correct. Quoting the Modbus Implementation Guide, page 23:
On TCP/IP, the MODBUS server is addressed using its IP address; therefore, the
MODBUS Unit Identifier is useless. The value 0xFF has to be used.
So libmodbus is just sticking to the Modbus specification. I'm assuming, then, that the problem is in easymodbus, which is apparently expecting you to use 0x01as the unit id in your queries.
I imagine you don't want to mess with easymodbus, so you can fix this problem pretty easily from libmodbus: just change the default unit id:
modbus_set_slave(ctx, 1);
You could also go with:
rc = modbus_set_slave(ctx, MODBUS_BROADCAST_ADDRESS);
ASSERT_TRUE(rc != -1, "Invalid broadcast address");
to make your client address all slaves within the network, if you have more than one.
You have more info and a short explanation of where this problem is coming from in the libmodbus man page for modbus_set_slave function.
For a very comprehensive example, you can check libmodbus unit tests
And regarding your question number 5, I don't know how to answer it, the zeros you mean are supposed to be the states (true or false) you want to write (or read) to the coils. For writing you can change them with the value field of function modbus_write_bit(ctx, address, value).
I'm very grateful for your reply.
I tested the read/write function using the 'unit-test-server/client' code you recommended.
I've reviewed the code, but there are still many things I don't know.
However, there is an address value that acts after testing each other with unit-test-server/client code and there is an address value that does not work
(Do you know why?).
-Checked and found that the UT_BITS_ADDRESS (address value) value operates from 0x130 to 0x150
-'error Illegal data address' occurs at values below -0x130 and above 0x150
-The address I want to read/write is 0x0001 to 0x0004(Do you know how to do?).
I want to know how to process and transmit data like the TX part of the right picture.
enter image description here
I'm running both client and server in my Linux environment and I'm doing read/write testing.
Among the wrong pictures...[06][FF]... <-- I want to know how to modify FF part (to change the value to 01 as shown in the picture)
enter image description here
and "modbus_set_slave" is the function for modbus rtu?
I'd like to communicate PC Program and Linux device in the end.
so Which part do I use that function?
I thanks for your concern again.
I have stm32f4-discovery kit and I want to try i/o expander for hd44870 LCD . I have PCF8574AT link to io example like mine 8-bit expander where i2c address is 0x3f (checked by i2c scanner) on hi2c3 hardware. For c/c++ use HAL libraries on Eclipse environment. Ok take look at code.
First I initialize i2c3 - like Datasheet 100kHz on SCL:
static void MX_I2C3_Init(void)
{
hi2c3.Instance = I2C3;
hi2c3.Init.ClockSpeed = 100000;
hi2c3.Init.DutyCycle = I2C_DUTYCYCLE_2;
hi2c3.Init.OwnAddress1 = 0;
hi2c3.Init.AddressingMode = I2C_ADDRESSINGMODE_7BIT;
hi2c3.Init.DualAddressMode = I2C_DUALADDRESS_DISABLE;
hi2c3.Init.OwnAddress2 = 0;
hi2c3.Init.GeneralCallMode = I2C_GENERALCALL_DISABLE;
hi2c3.Init.NoStretchMode = I2C_NOSTRETCH_DISABLE;
if (HAL_I2C_Init(&hi2c3) != HAL_OK)
{
_Error_Handler(__FILE__, __LINE__);
}
}
Then try to send data to I/O expander. But before that I check that i/o is ready to use:
result = HAL_I2C_IsDeviceReady(&hi2c3,0x3f , 2, 2);
if (result == HAL_BUSY)
{
HD44780_Puts(6, 1, "busy");
}else{
HD44780_Puts(6, 1, "ready");
uint8_t data_io = 0xff;
HAL_I2C_Master_Transmit(&hi2c3, 0x3f, data_io, 1, 100);
}
On a same expander nothing changes. Any ideas what is wrong or maybe i/0 expander is broken ?
Im not sure about HAL driver, really never used HAL. But I have touched pcf8574 IO expander. As you said, you have checked it with scanner and if you get address, line and device is OK. As I am not expert on I2C and HAL libs,I'll show my I2C driver it relies on STM32 standard periphery drivers and it worked for PCF8574 and various I2C devices. There is an example,snippet(blocking mode, not irq based):
Checking if IO is not busy.
while(I2C_GetFlagStatus(&I2Cx, I2C_FLAG_BUSY) == SET){
if((timeout--) == 0) return -ETIMEDOUT;
}
Generate start condition and set write condition(with address for write mode).
I2C_TransferHandling(&I2Cx, dev_addr, 1, I2C_SoftEnd_Mode, I2C_Generate_Start_Write);
while(I2C_GetFlagStatus(&I2Cx, I2C_ISR_TXIS) == RESET){
if((timeout--) == 0) return -ENODEV;
}
Now you can send data byte( it is your IO states), This function writes directly to I2C TX(transceiver) register :
I2C_SendData(&I2Cx, reg_addr);
while(I2C_GetFlagStatus(&I2Cx, I2C_ISR_TC) == RESET){
if((timeout--) == 0) return -EIO;
}
Generate reading condition and than read from PCF8574, data should be same as it was just written(if nothing toggles IO expander). Basically you can read byte or more bytes (depends on device). In your case PCF8574(8bit) gives only 1 byte.
I2C_TransferHandling(dev->channel,dev_addr, len, I2C_AutoEnd_Mode,I2C_Generate_Start_Read);
size_t i;
for(i=0;i<len;i++){
timeout = I2C_TIMEOUT;
while(I2C_GetFlagStatus(dev->channel, I2C_ISR_RXNE) == RESET){
if((timeout--) == 0) return -EIO;
}
data[i] = I2C_ReceiveData(dev->channel);
}
You can continue RW operations, or just simply wait till device automatically stop transition on line:
while(I2C_GetFlagStatus(&I2Cx, I2C_FLAG_STOPF) == RESET){
if((timeout--) == 0) return -EIO;
}
I2C_ClearFlag(&I2Cx, I2C_ICR_STOPCF);
This steps will write and read data. Anyway this chip has some tricky logic there, it more simplistic than it looks like.Actually it works just as simple OUTPUT. Extern input just triggers up PCF8574 pin and nothing more, no special configuration for input mode. For input monitor for sure use PCF8574 INT pin, PCF8574 will trigger INT pin.
For example:
If you want input pins, than just simply set pins to logic zero. And monitor INT pin,if change happens on input, INT pin will be triggered and you should read data via I2C .
For OUTPUT low just write zero's.
And for OUTPUT high set logic true.
You are using HAL so you have to read what happens inside HAL_I2C_Master_Transmit function. Do not forget that address is 7bit and first byte with address also includes R/W condition.First byte bit0 is R/W bit. So you should handle it.
for example with defines:
#define PCF8574_WRITE_ADRESS (0x40) /*for writing to chip*/
#define PCF8574_READ_ADRESS ((0x40)|0x01) /*for reading chip*/
Here is some links:
i2c explanations
this may help
Really nice guide!
Hope this will help to understand your problem and solve it.:)
thanks , Bulkin
I found obvious mistake . HAL libs do not i2c_address << 1 . I/YOU must put that in code not same result !
HAL_I2C_Master_Transmit(&hi2c3, (0x3f<<1), data_io, 1, 100);
or
$define i2c_address_write (0x3f <<1 )
HAL_I2C_Master_Transmit(&hi2c3, i2c_address_write , data_io, 1, 100);
to read :
$define i2c_address_read ((0x3f <<1) | 0x01)
HAL_I2C_Master_Transmit(&hi2c3, i2c_address_read , data_io, 1, 100);
I have several Axis IP-Cameras and I want to stream their H264 output over RTP to my application.
So far everything is working most of the time, usually with one camera. As soon as I attach more than one cam, I get lots of missing packets on every jrtplib instance I am using, resulting in bad video (artifacts, broken images, etc.).
So, I created a small test setup connecting just one camera and using just one jrtplib instance, with code more or less directly taken from the samples.
using namespace jrtplib;
RTPUDPv4TransmissionParams transparams;
RTPSessionParams sessparams;
RTPSession sess;
sessparams.SetOwnTimestampUnit(1.0 / 90000.0);
sessparams.SetAcceptOwnPackets(true);
transparams.SetPortbase(rtp_port);
auto status = sess.Create(sessparams, &transparams);
checkerror(status);
uint16_t last_sn = 0;
while (1)
{
sess.BeginDataAccess();
// check incoming packets
if (sess.GotoFirstSourceWithData())
{
do
{
RTPPacket *pack;
while ((pack = sess.GetNextPacket()) != NULL)
{
// You can examine the data here
auto sn = pack->GetSequenceNumber();
if (0!=last_sn && sn - last_sn != 1)
{
std::cout << "\tmissing packets: " << (sn - last_sn) << std::endl;
}
std::cout << sn << std::endl;
last_sn = sn;
// we don't longer need the packet, so
// we'll delete it
sess.DeletePacket(pack);
}
} while (sess.GotoNextSourceWithData());
}
sess.EndDataAccess();
status = sess.Poll();
checkerror(status);
Sleep(1);
}
sess.BYEDestroy(RTPTime(10, 0), 0, 0);
Even with this simple test, I get missing packets (missing sequence numbers), I also checked wether the missing sequence number are just delayed, but no.
But when I add transparams.SetRTPReceiveBuffer to a rather high value, like 1048576 bytes, it stops missing packets, at least for this sample.
In my real world code, increasing the receive buffer does not help. I also tried moving the session.Poll() to a separate thread.
Capturing UDP packets using Wireshark shows no dropped packets, so it´s something with libjrtp?
Does anyone have experience with this, or maybe even a suggestion for a another library to use? I am quite stuck at this point...
Thanks for any hints, maybe it is just a small issue and I just don´t see it
Regards
Is there a known way to list existing MIDI clients using the ALSA API only, without reading the special file /proc/asound/seq/clients?
I searched the ALSA MIDI API reference, and could not find any match. I believe there must be a way to achieve this using the API, otherwise that's a lot surprising.
As shown in the source code of aplaymidi and similar tools, ALSA sequencer clients are enumerated with snd_seq_query_next_client():
snd_seq_client_info_alloca(&cinfo);
snd_seq_client_info_set_client(cinfo, -1);
while (snd_seq_query_next_client(seq, cinfo) >= 0) {
int client = snd_seq_client_info_get_client(cinfo);
...
}
I could figure it, finally: snd_seq_get_any_client_info to get informations about the first client (there should be at least one, the system one) and snd_seq_query_next_client. to get the next one.
Here is a snippet to list MIDI clients:
static void list_clients(void) {
int count = 0;
int status;
snd_seq_client_info_t* info;
snd_seq_client_info_alloca(&info);
status = snd_seq_get_any_client_info(seq_handle, 0, info);
while (status >= 0) {
count += 1;
int id = snd_seq_client_info_get_client(info);
char const* name = snd_seq_client_info_get_name(info);
int num_ports = snd_seq_client_info_get_num_ports(info);
printf("Client “%s” #%i, with %i ports\n", name, id, num_ports);
status = snd_seq_query_next_client(seq_handle, info);
}
printf("Found %i clients\n", count);
}
The snippet assumes seq_handle is declared and initialized elsewhere (initialized with snd_seq_open).
The use of 0 as the client ID in the invocation of snd_seq_get_any_client_info, is a guess: ALSA uses negative numbers for errors, so I guess the first valid client ID is 0.