Losing values with iterative realloc in C - realloc

I am working in C with Netbeans8.0
I have to read files in an iterative approach to get list of words. That is, in single iteration a file is read into an array of strings and then merge this array into a single array.
void merge_array(char** a,int* M, char** b,int N)
{
//............. Add extra memory to a ..............*/
void *tmp = realloc(a, (*M+N) * sizeof(*a));
if (tmp == NULL)
{
perror("Merging -> Could not reallocate");
exit(EXIT_FAILURE);
}
a = tmp;
memset(a+(*M), 0, N*sizeof(*a));
//............. copy strings in b to a ..............*/
int i,j=0;
for(i=*M; i<((*M)+N); i++)
{
size_t wlen = strlen(b[j]);
a[i] = malloc((wlen+1) * sizeof(char));
if (a[i] == NULL)
{
perror("Failed to replicate string");
exit(EXIT_FAILURE);
}
memcpy(a[i], b[j], wlen+1);
j++;
}
(*M) = (*M)+N; // resetting the count
printf("Confirm - %s, %d\n",a[0],*M);
}
Above function reads the contents of a file. In main above function is called iteratively and merged into a single array named 'termlist'. Main code is given below
char** termlist;
int termCount=0;
while(files[i]){
char **word_array;
int wdCnt,a;
char* tmp = (char*) malloc(strlen(path)*sizeof(char));
strcpy(tmp,path); strcat(tmp,files[i]); strcpy(files[i],tmp);
printf("\n\n******* Reading file %s...\n",files[i]);
word_array = getTerms_fscanf(files[i],&a); //reading contents of file
wdCnt = a;
if(i==0) // before reading the first file initializing the termlist
{
termlist = (char**) malloc(wdCnt*sizeof(char*));
}
merge_array(termlist,&termCount,word_array,wdCnt);
printf("CHECK - %s, %d\n",termlist[0],termCount);
free(word_array);
++i;
}
Now the problem is that,
After 1st two iterations, Inside function everything works fine but in main values of termlist[0], termlist[1] turns out to be junk.. That is first 2 words read from first file is lost. The 3rd iteration returns with failure at merge_array function call.
Output is
******* Reading F:/Netbeans C/Test Docs/doc1.txt...
Confirm - tour, 52
CHECK - tour, 52
******* Reading F:/Netbeans C/Test Docs/doc2.txt...
Confirm - tour, 71
CHECK - Ôk'aÔk'a`œ€`œ€äk'aäk'aìk'aìk'aôk'aôk'aük'aük'ah“€, 71
I am not able to identify problem with this.. Please help with this..

Related

Bad address error when comparing Strings within BPF

I have an example program I am running here to see if the substring matches the string and then print them out. So far, I am having trouble running the program due to a bad address. I am wondering if there is a way to fix this problem? I have attached the entire code but my problem is mostly related to isSubstring.
#include <uapi/linux/bpf.h>
#define ARRAYSIZE 64
struct data_t {
char buf[ARRAYSIZE];
};
BPF_ARRAY(lookupTable, struct data_t, ARRAYSIZE);
//char name[20];
//find substring in a string
static bool isSubstring(struct data_t stringVal)
{
char substring[] = "New York";
int M = sizeof(substring);
int N = sizeof(stringVal.buf) - 1;
/* A loop to slide pat[] one by one */
for (int i = 0; i <= N - M; i++) {
int j;
/* For current index i, check for
pattern match */
for (j = 0; j < M; j++)
if (stringVal.buf[i + j] != substring[j])
break;
if (j == M)
return true;
}
return false;
}
int Test(void *ctx)
{
#pragma clang loop unroll(full)
for (int i = 0; i < ARRAYSIZE; i++) {
int k = i;
struct data_t *line = lookupTable.lookup(&k);
if (line) {
// bpf_trace_printk("%s\n", key->buf);
if (isSubstring(*line)) {
bpf_trace_printk("%s\n", line->buf);
}
}
}
return 0;
}
My python code here:
import ctypes
from bcc import BPF
b = BPF(src_file="hello.c")
lookupTable = b["lookupTable"]
#add hello.csv to the lookupTable array
f = open("hello.csv","r")
contents = f.readlines()
for i in range(0,len(contents)):
string = contents[i].encode('utf-8')
print(len(string))
lookupTable[ctypes.c_int(i)] = ctypes.create_string_buffer(string, len(string))
f.close()
b.attach_kprobe(event=b.get_syscall_fnname("clone"), fn_name="Test")
b.trace_print()
Edit: Forgot to add the error: It's really long and can be found here: https://pastebin.com/a7E9L230
I think the most interesting part of the error is near the bottom where it mentions:
The sequence of 8193 jumps is too complex.
And a little bit farther down mentions: Bad Address.
The verifier checks all branches in your program. Each time it sees a jump instruction, it pushes the new branch to its “stack of branches to check”. This stack has a limit (BPF_COMPLEXITY_LIMIT_JMP_SEQ, currently 8192) that you are hitting, as the verifier tells you. “Bad Address” is just the translation of kernel's errno value which is set to -EFAULT in that case.
Not sure how to fix it though, you could try:
With smaller strings, or
On a 5.3+ kernel (which supports bounded loops): without unrolling the loop with clang (I don't know if it would help).

Decoding delimited frames from byte arrays

I have frames that are delimited by bytes to start and stop the frame (they do not appear in the stream).
I read a chunk from disk or network socket, i then need to pass to a deserializer but only after I have de-framed the packet first.
Frames may span multiple chunks that have been read, note how frame 3 is split across array 1 and array 2.
Rather than reinvent the wheel for this common problem, do any github or similar projects exist?
I am investigating ReadOnlySequenceSegment<T> from https://www.codemag.com/article/1807051/Introducing-.NET-Core-2.1-Flagship-Types-Span-T-and-Memory-T and will post updates as I work out the requirements.
Update
Further to Stephen Cleary link (thank you!!) to https://github.com/davidfowl/TcpEcho/blob/master/src/Server/Program.cs I have the below.
My data is json, so unlike the original question the delimiter tokens will appear in the stream. Therefore I have to count the array delimitator and only declare a frame when i have found the outermost [ and ] characters.
The below code works, and less manual copies done (not sure if still done behind the scenes - code is quite neater using David Fowl approach).
However I am casting to array instead of using buffer.PositionOf((byte)'[') since I was unable to see how I could call the PositionOf with an offset applied (i.e. scan deeper into the frame past previously found delimiter tokens).
Am i using/butchering the library in a brute force way, or is the below good to go with the array cast?
class Program
{
static async Task Main(string[] args)
{
using var stream = File.Open(args[0], FileMode.Open);
var reader = PipeReader.Create(stream);
while (true)
{
ReadResult result = await reader.ReadAsync();
ReadOnlySequence<byte> buffer = result.Buffer;
while (TryDeframe(ref buffer, out ReadOnlySequence<byte> line))
{
// Process the line.
var str = System.Text.Encoding.UTF8.GetString(line.ToArray());
Console.WriteLine(str);
}
// Tell the PipeReader how much of the buffer has been consumed.
reader.AdvanceTo(buffer.Start, buffer.End);
// Stop reading if there's no more data coming.
if (result.IsCompleted)
{
break;
}
}
// Mark the PipeReader as complete.
await reader.CompleteAsync();
}
private static bool TryDeframe(ref ReadOnlySequence<byte> buffer, out ReadOnlySequence<byte> frame)
{
int frameCount = 0;
int start = -1;
int end = -1;
var bytes = buffer.ToArray();
for (var i = 0; i < bytes.Length; i++)
{
var b = bytes[i];
if (b == (byte)'[')
{
if (start == -1)
start = i;
frameCount++;
}
else if (b == (byte)']')
{
frameCount--;
if (frameCount == 0)
{
end = i;
break;
}
}
}
if (start == -1 || end == -1) // no frame found
{
frame = default;
return false;
}
frame = buffer.Slice(start, end+1);
buffer = buffer.Slice(frame.Length);
return true;
}
}
do any github or similar projects exist?
David Fowler has an echo server that uses Pipelines to implement delimited frames.

Xilinx Echo Server Data Variable

I want to have my Zedboard return a numeric value using the Xilinx lwIP example as a base but no matter what I do I can't figure out what stores the data received or transmitted.
I have found the void type payload but I don't know what to do with it.
Snapshot of one instance of payload and a list of lwIP files
Below is the closest function to my goal:
err_t recv_callback(void *arg, struct tcp_pcb *tpcb,
struct pbuf *p, err_t err){
/* do not read the packet if we are not in ESTABLISHED state */
if (!p) {
tcp_close(tpcb);
tcp_recv(tpcb, NULL);
return ERR_OK;
}
/* indicate that the packet has been received */
tcp_recved(tpcb, p->len);
/* echo back the payload */
/* in this case, we assume that the payload is < TCP_SND_BUF */
if (tcp_sndbuf(tpcb) > p->len) {
err = tcp_write(tpcb, p->payload, p->len, 1);
//I need to change p->paylod but IDK where it is given a value.
} else
xil_printf("no space in tcp_sndbuf\n\r");
/* free the received pbuf */
pbuf_free(p);
return ERR_OK;
}
Any guidance is appreciated.
Thanks,
Turtlemii
-I cheated and just made sure that the function has access to Global_tpcb from echo.c
-tcp_write() reads in an address and displays each char it seems.
void Print_Code()
{
/* Prepare for TRANSMISSION */
char header[] = "\rSwitch: 1 2 3 4 5 6 7 8\n\r"; //header text
char data_t[] = " \n\r\r"; //area for storing the
data
unsigned char mask = 10000000; //mask to decode switches
swc_value = XGpio_DiscreteRead(&SWCInst, 1); //Save switch values
/* Write switch values to the LEDs for visual. */
XGpio_DiscreteWrite(&LEDInst, LED_CHANNEL, swc_value);
for (int i =0; i<=7; i++) //load data_t with switch values (0/1)
{
data_t[8+2*i] = '0' + ((swc_value & mask)/mask); //convert one bit to 0/1
mask = mask >> 1;//move to next bit
}
int len_header = *(&header + 1) - header; //find the length of the
header string
int len_data = *(&data_t + 1) - data_t; //find the length of the data string
tcp_write(Global_tpcb, &header, len_header, 1); //print the header
tcp_write(Global_tpcb, &data_t, len_data, 1); //print the data
}

Querying Mongodb Key and Value using C driver

mongo_cursor *cursor=mongo_find(conn,TEST_NS,query,NULL,0,0,0);
count_matched=0;
bson *doc;
while(mongo_cursor_next(cursor)==MONGO_OK)
{
count_matched++;
doc=(bson *)mongo_cursor_bson(cursor);
bson_iterator_init(&it,doc);
while(bson_iterator_next(&it) != BSON_EOO)
{
fprintf(stderr,"%s : %s\n\n",bson_iterator_key(&it),bson_iterator_string(&it));
}
}
This code is working perfectly and i can see the matched documents (Key + Value) but now i want to save the matched document's key and value to a string. Can any tell me how i can save the return value of key and value in to a string?
One document includes (all strings)
Total Key=10
Total value=10
and i want to save 10 document's key and value at one time. I am using C driver of mongodb.
The following code shows how you would be doing copy of the key and values from the bson iterator into your key-value arrays temp_key and temp_value. The specific block of code is in between the comments marked START and END.
Additionally, you can find documentation for accessing BSON document contents at http://api.mongodb.org/c/current/bson.html .
mongo_cursor *cursor = mongo_find(&conn, TEST_NS, &query, NULL, 0, 0, 0);
int count_matched = 0;
bson *doc;
// Assuming you are just looking for 100 key / value pair of max length of 99 characters
const unsigned KV_ARRAY_LENGTH = 100;
const unsigned MAX_KV_LENGTH = 105;
char temp_key[KV_ARRAY_LENGTH][MAX_KV_LENGTH + 1], temp_value[KV_ARRAY_LENGTH][MAX_KV_LENGTH + 1];
int i = 0;
while (mongo_cursor_next(cursor) == MONGO_OK) {
count_matched++;
doc=(bson *)mongo_cursor_bson(cursor);
bson_iterator it;
bson_iterator_init(&it,doc);
while (bson_iterator_next(&it) != BSON_EOO) {
fprintf(stderr,"%s : %s\n", bson_iterator_key(&it), bson_iterator_string(&it));
/******* START - Code to capture key-value into appropriate array */
if (i < KV_ARRAY_LENGTH) {
/* - Collect key-value pairs only if there is space in the array
* - Key / Value woud be captured only till the max amount of space available for them i.e. MAX_KV_LENGTH in this case
* */
strncpy(temp_key[i], bson_iterator_key(&it), MAX_KV_LENGTH);
strncpy(temp_value[i], bson_iterator_string(&it), MAX_KV_LENGTH);
temp_key[i][MAX_KV_LENGTH] = temp_value[i][MAX_KV_LENGTH] = '\0';
++i;
} else {
/* whatever need to be done if there is no room in the array */
}
/******* END - Code to capture key-value into appropriate array */
}
}
/* Test iterating through the key-value pair constructed in query iteration */
fprintf(stdout, "--- Fields collected ---\n");
int keyIndex = 0;
for ( ; keyIndex < i; ++keyIndex) {
fprintf(stdout, "{key: %s, value: %s}\n", temp_key[keyIndex], temp_value[keyIndex]);
}
mongo_cursor *cursor=mongo_find(conn,TEST_NS,query,NULL,0,0,0);
count_matched=0;
bson *doc;
//Answer
const char* temp_key[100][100],temp_value[100][100];
int i=0;
while(mongo_cursor_next(cursor)==MONGO_OK)
{
count_matched++;
doc=(bson *)mongo_cursor_bson(cursor);
bson_iterator_init(&it,doc);
while(bson_iterator_next(&it) != BSON_EOO)
{
fprintf(stderr,"%s : %s\n\n",bson_iterator_key(&it),bson_iterator_string(&it));
temp[i][0]=bson_iterator_key[&it]; //Answer
temp_value[i][0]=bson_iterator_key[&it]; //Answer
i++; //Answer
}
}
Just for the record, this is the rough sketch and i know about corruption of the temp variables and their overflow but i will remove it according to my code.

AudioFileWriteBytes fails with error code -40

I'm trying to write raw audio bytes to a file using AudioFileWriteBytes(). Here's what I'm doing:
void writeSingleChannelRingBufferDataToFileAsSInt16(AudioFileID audioFileID, AudioConverterRef audioConverter, ringBuffer *rb, SInt16 *holdingBuffer) {
// First, figure out which bits of audio we'll be
// writing to file from the ring buffer
UInt32 lastFreshSample = rb->lastWrittenIndex;
OSStatus status;
int numSamplesToWrite;
UInt32 numBytesToWrite;
if (lastFreshSample < rb->lastReadIndex) {
numSamplesToWrite = kNumPointsInWave + lastFreshSample - rb->lastReadIndex - 1;
}
else {
numSamplesToWrite = lastFreshSample - rb->lastReadIndex;
}
numBytesToWrite = numSamplesToWrite*sizeof(SInt16);
Then we copy the audio data (stored as floats) to a holding buffer (SInt16) that will be written directly to the file. The copying looks funky because it's from a ring buffer.
UInt32 buffLen = rb->sizeOfBuffer - 1;
for (int i=0; i < numSamplesToWrite; ++i) {
holdingBuffer[i] = rb->data[(i + rb->lastReadIndex) & buffLen];
}
Okay, now we actually try to write the audio from the SInt16 buffer "holdingBuffer" to the audio file. The NSLog will spit out an error -40, but also claims that it's writing bytes. No data is written to file.
status = AudioFileWriteBytes(audioFileID, NO, 0, &numBytesToWrite, &holdingBuffer);
rb->lastReadIndex = lastFreshSample;
NSLog(#"Error = %d, wrote %d bytes", status, numBytesToWrite);
return;
What is this error -40? By the way, everything works fine if I write straight from the ringBuffer to the file. Of course it sounds like junk, because I'm writing floats, not SInt16s, but AudioFileWriteBytes doesn't complain.
The key is to explicitly change the endianness of the incoming data to big endian. All I had to do was wrap CFSwapInt16HostToBig around my data to get:
float audioVal = rb->data[(i + rb->lastReadIndex) & buffLen];
holdingBuffer[i] = CFSwapInt16HostToBig((SInt16) audioVal );