Perl XS Memory Handling of Strings - perl

I have an XSUB like this:
char *
string4()
CODE:
char *str = strdup("Hello World4");
int len = strlen(str) + 1;
New(0, RETVAL, len, char);
Copy(str, RETVAL, len, char);
free(str);
OUTPUT:
RETVAL
But this shows up as a memory leak, on the New(), in valgrind and if I run it in a loop the resident memory will continue to grow.
I get the same thing if I use this one too:
char *
string2()
CODE:
char *str = strdup("Hello World2");
RETVAL = str;
OUTPUT:
RETVAL
I'm able to prevent the leak and the increasing memory size by doing:
char *
string3()
PPCODE:
char *str = strdup("Hello World3");
XPUSHs(sv_2mortal(newSVpv(str, 0)));
free(str);
but the problem with this solution is that when I compile with -Werror I get the following warnings/errors.
test.c: In function ‘XS_test_string3’:
/usr/lib/x86_64-linux-gnu/perl/5.20/CORE/XSUB.h:175:28: error: unused variable ‘targ’ [-Werror=unused-variable]
#define dXSTARG SV * const targ = ((PL_op->op_private & OPpENTERSUB_HASTARG) \
^
test.c:270:2: note: in expansion of macro ‘dXSTARG’
dXSTARG;
^
test.c:269:9: error: unused variable ‘RETVAL’ [-Werror=unused-variable]
char * RETVAL;
the c file gets built with an unused RETVAL:
XS_EUPXS(XS_test_string3); /* prototype to pass -Wmissing-prototypes */
XS_EUPXS(XS_test_string3)
{
dVAR; dXSARGS;
if (items != 0)
croak_xs_usage(cv, "");
PERL_UNUSED_VAR(ax); /* -Wall */
SP -= items;
{
char * RETVAL;
dXSTARG;
#line 61 "test.xs"
char *str = strdup("Hello World3");
XPUSHs(sv_2mortal(newSVpv(str, 0)));
free(str);
#line 276 "test.c"
PUTBACK;
return;
}
}
So is there a better way to handle the returning of allocated strings in XS? Is there a way to return the string using RETVAL and free the memory? I appreciate any help.

Among other problems[1], your first snippet allocates memory using New, but never deallocates it.
Among other problems, your second snippet allocates memory using strdup, but never deallocates it.
The underlying problem with your third snippet is that you claim the XS function returns a value and it doesn't. That value would have been assigned to RETVAL, which is automatically created for that very purpose. The variable won't be created if you correctly specify that you don't return anything.
void
string3()
PREINIT:
char *str;
PPCODE:
str = strdup("Hello World3");
XPUSHs(sv_2mortal(newSVpv(str, 0)));
free(str);
or just
void
string3()
PPCODE:
XPUSHs(sv_2mortal(newSVpv("Hello World3", 0)));
Note that I moved your declarations out of PPCODE. In C, declarations can't appear after non-declarations, and the code in PPCODE can appear after non-declarations (depending on the options used to build Perl). Declarations belong in PREINIT. You could also use curlies around the code in PPCODE.
One of them is the use of New. You shoudln't be using New. New was deprecated in favour of Newx ages ago. New hasn't even been in the documentation for as long as I can remember.

Related

ebpf unknown opcode comparing strings

I currently try to filter calls to a function by command. I try to do so with the following code where ##REPLACE_comm## is replaced by python by the command name. The double backslash are cause I am using bcc. The following code throws an error when loading:
if(1){
char filter[TASK_COMM_LEN] = "##REPLACE_comm##";
char command[TASK_COMM_LEN];
bpf_get_current_comm(&command, sizeof(command));
for(u16 i = 0; i<=TASK_COMM_LEN; i++){
if(command[i] == '\\0' && filter[i] == '\\0'){
break;
}
if(command[i] == filter[i]){
continue;
}
return 0;
}
}
The error is:
unknown opcode 70
HINT: The 'unknown opcode' can happen if you reference a global or static variable, or data in read-only section. For example, 'char *p = "hello"' will result in p referencing a read-only section, and 'char p[] = "hello"' will have "hello" stored on the stack.
I feel like I already made sure the variables are on the stack by allocating space and not just having a pointer but it doesnt work. What am I missing?

Bad address error when comparing Strings within BPF

I have an example program I am running here to see if the substring matches the string and then print them out. So far, I am having trouble running the program due to a bad address. I am wondering if there is a way to fix this problem? I have attached the entire code but my problem is mostly related to isSubstring.
#include <uapi/linux/bpf.h>
#define ARRAYSIZE 64
struct data_t {
char buf[ARRAYSIZE];
};
BPF_ARRAY(lookupTable, struct data_t, ARRAYSIZE);
//char name[20];
//find substring in a string
static bool isSubstring(struct data_t stringVal)
{
char substring[] = "New York";
int M = sizeof(substring);
int N = sizeof(stringVal.buf) - 1;
/* A loop to slide pat[] one by one */
for (int i = 0; i <= N - M; i++) {
int j;
/* For current index i, check for
pattern match */
for (j = 0; j < M; j++)
if (stringVal.buf[i + j] != substring[j])
break;
if (j == M)
return true;
}
return false;
}
int Test(void *ctx)
{
#pragma clang loop unroll(full)
for (int i = 0; i < ARRAYSIZE; i++) {
int k = i;
struct data_t *line = lookupTable.lookup(&k);
if (line) {
// bpf_trace_printk("%s\n", key->buf);
if (isSubstring(*line)) {
bpf_trace_printk("%s\n", line->buf);
}
}
}
return 0;
}
My python code here:
import ctypes
from bcc import BPF
b = BPF(src_file="hello.c")
lookupTable = b["lookupTable"]
#add hello.csv to the lookupTable array
f = open("hello.csv","r")
contents = f.readlines()
for i in range(0,len(contents)):
string = contents[i].encode('utf-8')
print(len(string))
lookupTable[ctypes.c_int(i)] = ctypes.create_string_buffer(string, len(string))
f.close()
b.attach_kprobe(event=b.get_syscall_fnname("clone"), fn_name="Test")
b.trace_print()
Edit: Forgot to add the error: It's really long and can be found here: https://pastebin.com/a7E9L230
I think the most interesting part of the error is near the bottom where it mentions:
The sequence of 8193 jumps is too complex.
And a little bit farther down mentions: Bad Address.
The verifier checks all branches in your program. Each time it sees a jump instruction, it pushes the new branch to its “stack of branches to check”. This stack has a limit (BPF_COMPLEXITY_LIMIT_JMP_SEQ, currently 8192) that you are hitting, as the verifier tells you. “Bad Address” is just the translation of kernel's errno value which is set to -EFAULT in that case.
Not sure how to fix it though, you could try:
With smaller strings, or
On a 5.3+ kernel (which supports bounded loops): without unrolling the loop with clang (I don't know if it would help).

Can't write Double word on STM32F429 using HAL driver

I am trying to write uint64_t(double word) variable into the flash memory, without success though. Here is the code.
#define APPLICATION_START_ADDRESS 0x8008000
void flashErase(uint8_t startSector, uint8_t numberOfSectors)
{
HAL_FLASH_Unlock();
Flash_eraseInitStruct.TypeErase = FLASH_TYPEERASE_SECTORS;
Flash_eraseInitStruct.VoltageRange = FLASH_VOLTAGE_RANGE_3;
Flash_eraseInitStruct.Sector = startSector;
Flash_eraseInitStruct.NbSectors = numberOfSectors;
if(HAL_FLASHEx_Erase(&Flash_eraseInitStruct, &Flash_halOperationSectorError) != HAL_OK)
{
Flash_raiseError(errHAL_FLASHEx_Erase);
}
HAL_FLASH_Lock();
}
int main(void)
{
HAL_Init();
main_clockSystemInit();
__IO uint64_t word = 0x1234567890;
flashErase(2, 1);
// flashProgramWord(aTxBuffer, APPLICATION_START_ADDRESS, 2 );
HAL_FLASH_Unlock();
HAL_FLASH_Program(FLASH_TYPEPROGRAM_DOUBLEWORD, APPLICATION_START_ADDRESS, word);
}
I get error flag raised PGSERR and PGAERR. The erase operation goes without problems. But programming returns ERROR.
Some Ideas?
There is no STM32F249, did you mean STM32F429?
In order to use 64 bit programming, VPP (BOOT0) has to be powered by 8 - 9 Volts. Is it?
See the Reference Manual Section 3.6.2
By the way,
__IO uint64_t word = 0x1234567890;
would not work as (presumably) expected. It is a 32 bit architecture, integer constants will be truncated to 32 bits, unless there is an L suffix. U wouldn't hurt either, because the variable is unsigned. __IO is unnecessary.
uint64_t word = 0x1234567890UL;

mex file run out of memory

I wrote a mexfunction on c++ code, then compiled it to get mexa64 file. After I used mexa64 file for many times, my PC memory run out.
Here is my mexfunction:
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
// double *res ;// the final result index
int status1,status2,status3;
double *res ;
double *out_mat;
double *x_idx,*y_idx,*width_idx,*height_idx;
char* sequence_path;// the tracking image sequences
char* sequence_name;// the tracking image sequences
char* output_path;// save the output
int start_frame=(int)mxGetScalar(prhs[0]);
int end_frame=(int)mxGetScalar(prhs[1]);
int Max_track_number=(int)mxGetScalar(prhs[2]);
/* Extract the inputs */
int buflen1=(mxGetM(prhs[3])*mxGetN(prhs[3]))+1;
sequence_path= (char *)mxCalloc(buflen1,sizeof(char));
status1=mxGetString(prhs[3],sequence_path,buflen1);
int buflen2=(mxGetM(prhs[4])*mxGetN(prhs[4]))+1;
sequence_name= (char *)mxCalloc(buflen2,sizeof(char));
status2=mxGetString(prhs[4],sequence_name,buflen2);
int buflen3=(mxGetM(prhs[5])*mxGetN(prhs[5]))+1;
output_path= (char *)mxCalloc(buflen3,sizeof(char));
status3=mxGetString(prhs[5],output_path,buflen3);
if(status3!=0)
mexWarnMsgTxt("Not enough space. String is truncated.");
float x=(float)mxGetScalar(prhs[6]);
float y=(float)mxGetScalar(prhs[7]);
float width=(float)mxGetScalar(prhs[8]);
float height=(float)mxGetScalar(prhs[9]);
x_idx=mxGetPr(prhs[6]);
y_idx=mxGetPr(prhs[7]);
width_idx=mxGetPr(prhs[8]);
height_idx=mxGetPr(prhs[9]);
mexPrintf("the dimension is %f,%f,%f,%f\n",*x_idx,*y_idx,*width_idx,*height_idx);
int N1=mxGetM(prhs[6])*mxGetN(prhs[6]);
int N2=mxGetM(prhs[7])*mxGetN(prhs[7]);
int N3=mxGetM(prhs[8])*mxGetN(prhs[8]);
int N4=mxGetM(prhs[9])*mxGetN(prhs[9]);
if(N1!=N2|N2!=N3|N3!=N4)
mexErrMsgTxt("the dimension of the initial bounding box parameter is wrong");
int NN=N1;
plhs[0] = mxCreateDoubleMatrix(Max_track_number*5*NN,1,mxREAL);
out_mat = mxGetPr(plhs[0]);
plhs[1] = mxCreateDoubleMatrix(1,1,mxREAL);
res = mxGetPr(plhs[1]);
*res=struck_track(start_frame,end_frame,Max_track_number,sequence_path,sequence_name,output_path,out_mat,x_idx,y_idx,width_idx,height_idx,NN);
}
Is there some problems that may cause the exhaustion of memory?
you should use mxFree whenever using the following:
mxCalloc
mxMalloc
mxRealloc
mxArrayToString
I can see from your code that you are using mxCalloc but there is no corresponding mxFree for the same.
Besides using mxFree you can also increase the Java Heap Size. You can find it under settings, general, java heap memory.

Losing values with iterative realloc in C

I am working in C with Netbeans8.0
I have to read files in an iterative approach to get list of words. That is, in single iteration a file is read into an array of strings and then merge this array into a single array.
void merge_array(char** a,int* M, char** b,int N)
{
//............. Add extra memory to a ..............*/
void *tmp = realloc(a, (*M+N) * sizeof(*a));
if (tmp == NULL)
{
perror("Merging -> Could not reallocate");
exit(EXIT_FAILURE);
}
a = tmp;
memset(a+(*M), 0, N*sizeof(*a));
//............. copy strings in b to a ..............*/
int i,j=0;
for(i=*M; i<((*M)+N); i++)
{
size_t wlen = strlen(b[j]);
a[i] = malloc((wlen+1) * sizeof(char));
if (a[i] == NULL)
{
perror("Failed to replicate string");
exit(EXIT_FAILURE);
}
memcpy(a[i], b[j], wlen+1);
j++;
}
(*M) = (*M)+N; // resetting the count
printf("Confirm - %s, %d\n",a[0],*M);
}
Above function reads the contents of a file. In main above function is called iteratively and merged into a single array named 'termlist'. Main code is given below
char** termlist;
int termCount=0;
while(files[i]){
char **word_array;
int wdCnt,a;
char* tmp = (char*) malloc(strlen(path)*sizeof(char));
strcpy(tmp,path); strcat(tmp,files[i]); strcpy(files[i],tmp);
printf("\n\n******* Reading file %s...\n",files[i]);
word_array = getTerms_fscanf(files[i],&a); //reading contents of file
wdCnt = a;
if(i==0) // before reading the first file initializing the termlist
{
termlist = (char**) malloc(wdCnt*sizeof(char*));
}
merge_array(termlist,&termCount,word_array,wdCnt);
printf("CHECK - %s, %d\n",termlist[0],termCount);
free(word_array);
++i;
}
Now the problem is that,
After 1st two iterations, Inside function everything works fine but in main values of termlist[0], termlist[1] turns out to be junk.. That is first 2 words read from first file is lost. The 3rd iteration returns with failure at merge_array function call.
Output is
******* Reading F:/Netbeans C/Test Docs/doc1.txt...
Confirm - tour, 52
CHECK - tour, 52
******* Reading F:/Netbeans C/Test Docs/doc2.txt...
Confirm - tour, 71
CHECK - Ôk'aÔk'a`œ€`œ€äk'aäk'aìk'aìk'aôk'aôk'aük'aük'ah“€, 71
I am not able to identify problem with this.. Please help with this..