I'm currently wrapping the Armadillo library in Cython and have some trouble to convert scipy.sparse.csc_matrix in Python through Cython to the sparse matrix class from Armadillo.
The function to do so is actually like this :
cdef inline SpMat[double] * sp_to_spmatp(X):
cdef np.ndarray[int, ndim = 1, mode = 'c'] col_ptrs = X.indptr
cdef np.ndarray[int, ndim = 1, mode = 'c'] row_indices = X.indices
cdef np.ndarray[double, ndim = 1, mode = 'c'] values = X.data
cdef SpMat[double] *p_spRes = new SpMat[double](<int*> &row_indices[0], <int*> &col_ptrs[0], <double*> &values[0], X.shape[0], X.shape[1])
return p_spRes
where X is the sparse matrix from scipy and p_spRes the sparse matrix from Armadillo.
By building it I get this error:
arma_wrapper.cpp:3418:549: error: no matching function for call to
‘arma::SpMat::SpMat(int*, int*, double*, int&, int&)’
along with:
Armadillo/include/armadillo_bits/SpMat_bones.hpp:103:10: note: candidate: template<class T1, class T2, class T3> arma::SpMat<eT>::SpMat(const arma::Base<long long unsigned int, T1>&, const arma::Base<long long unsigned int, T2>&, const arma::Base<eT, T3>&, arma::uword, arma::uword)
inline SpMat(const Base<uword,T1>& rowind, const Base<uword,T2>& colptr, const Base<eT,T3>& values, const uword n_rows, const uword n_cols);
^~~~~
Armadillo/include/armadillo_bits/SpMat_bones.hpp:103:10: note: template argument deduction/substitution failed:
arma_wrapper.cpp:3418:549: note: mismatched types ‘const arma::Base<long long unsigned int, T1>’ and ‘int*’
].strides)))), ((double *)(&(*__Pyx_BufPtrCContig1d(__pyx_t_5numpy_double_t *, __pyx_pybuffernd_values.rcbuffer->pybuffer.buf, __pyx_t_8, __pyx_pybuffernd_values.diminfo[0].strides)))), __pyx_t_10, __pyx_t_11);
^
which makes me think that some type casts may be incorrect.
If anyone does know anything about it, it will be greatly appreciated.
Related
I have encountered an issue when using Eigen in MATLAB MEX files.
Consider this excerpt of code, in which I call a mex function to create an object of the class vars. The class has an integer N, an integer S, and two Eigen arrays.
//constructMat.cpp
class vars {
public:
int N
int S
Eigen::ArrayXd upperLims
Eigen::ArrayXd lowerLims
stateVars (double *, double *, double *)
};
stateVars::stateVars (double *upperInput, double *lowerInput, double *gridInput)
Eigen::ArrayXd upper; upper = Eigen::Map<Eigen::VectorXd>(upperInput, sizeof(*upperInput),1);
Eigen::ArrayXd lower; lower = Eigen::Map<Eigen::VectorXd>(lowerInput, sizeof(*lowerInput),1);
Eigen::ArrayXd gridSizes; gridSizes = Eigen::Map<Eigen::VectorXd>(gridInput, sizeof(*gridInput),1);
upperLims = upper;
lowerLims = lower;
N = upperLims.size();
S = gridSizes.prod();
}
//MEX CODE
void
mexFunction(int nlhs,mxArray *plhs[],int nrhs,const mxArray *prhs[])
{
//....checks to make sure the inputs are okay...
double* upper = mxGetPr(prhs[0]);
double* lower = mxGetPr(prhs[1]);
double* grids = mxGetPr(prhs[2]);
stateVars stateSpace(upper, lower, grids);
mexPrintf("N =%f \n\n", stateSpace.N );
mexPrintf("S =%f \n\n", stateSpace.S );
}
However, when I execute the function, I call constructMat([7.0, 8.0, 9.0], [4.0, 2.0, 3.0], [10, 10, 10]), and I expect mexPrintf("N =%f \n\n", stateSpace.N ) to yield 3 since the array upperLims only has three elements. However, it yields 7. Similarly, I expect mexPrintf("S =%f \n\n", stateSpace.S ) to yield 10^3 = 1000, but it yields 7 as well. I'm not sure what I did wrong. The mex file was compiled successfully.
Also, if I call mexPrintf("upperlims =%f \n\n",stateSpace.upperLims(0) ), i.e. printing out the first element of the Eigen array upperLims, it gives me the right number. Does it have something to do with the method .size() and .prod()?
[environment]
OS: OSX10.11 and maxOS Sierra(10.12)
MATLAB: matlab2013a and matlab2016a
Xcode: xcode7 and xcode8
In my work, I must use the following mex file in the old package.
//destructiveMatrixWriteAtIndices.c
//------------------------------------------------------
#include <matrix.h> /* Matlab matrices */
#include <mex.h>
#include <stddef.h> /* NULL */
#define notDblMtx(it) (!mxIsNumeric(it) || !mxIsDouble(it) || mxIsSparse(it) || mxIsComplex(it))
void mexFunction(int nlhs, /* Num return vals on lhs */
mxArray *plhs[], /* Matrices on lhs */
int nrhs, /* Num args on rhs */
const mxArray *prhs[] /* Matrices on rhs */
)
{
double *mtx;
double *newValues;
double *doubleStartIndex;
int i, startIndex, size;
mxArray *arg;
if (nrhs != 3) mexErrMsgTxt("requires 3 arguments.");
/* ARG 1: MATRIX */
arg = prhs[0];
if notDblMtx(arg) mexErrMsgTxt("MTX arg must be a real non-sparse matrix.");
mtx = mxGetPr(arg);
arg = prhs[1];
if notDblMtx(arg) mexErrMsgTxt("MTX arg must be a real non-sparse matrix.");
newValues = mxGetPr(arg);
size = (int) mxGetM(arg) * mxGetN(arg);
arg = prhs[2];
if notDblMtx(arg) mexErrMsgTxt("MTX arg must be a real non-sparse matrix.");
doubleStartIndex = mxGetPr(arg);
startIndex = (int) doubleStartIndex[0];
for (i=0; i<size; i++){
mtx[i+startIndex] = newValues[i];
}
return;
}
//------------------------------------------------------
This mex file is the function to overwrite the scalar and the part of matrix through the pointer.
e.g. in matlab2013a command window (scalar in matlab2013a)
a = 1;
destructiveMatrixWriteAtIndices(a, 3, 0);
and the variable "a" becomes "3".
e.g. in matlab2013a and matlab2016a command window (matrix in matlab2013a and matlab2016a)
a = [1, 2];
destructiveMatrixWriteAtIndices(a, 3, 0);
and the variable "a" becomes "[3, 2]".
e.g. in matlab2016a command window (scalar in matlab2016a)
a = 1;
destructiveMatrixWriteAtIndices(a, 3, 0);
and the variable "a" becomes "1"! Why?
I also used the lldb, and revealed the strange behavior of this code.
In matlab2013a and matlab2016a, when I run the following snippet.
a = 1;
destructiveMatrixWriteAtIndices(a, 3, 0);
The lldb revealed "*mtx = 3" at the end of the mex function in both matlab, but the mex function couldn't pass the result(*mtx = 3, or prhs[0] = 3) through the pointer in the only matlab2016a.
It's very strange behavior!
※I have understood that this mex function is very danger, but this mex function was used at some points in the package that I must use. Therefore, I must fix this mex file and make the package run in matlab2016a.
Please help me.
I'm pretty sure you're not meant to modify the input array in a mex function. More details here Does Matlab ever copy data passed to a mex function?. The "matlab" solution is probably to return the modified array as an output of the mex rather modifying in place.
I have this code and get the following error: Implicit Conversion Loses Integer Precision
size_t BitArray::wordsForBits(size_t bits) {
int arraySize = (bits + bitsPerWord_ - 1) >> logBits_;
return arraySize;
}
How can I resolve this?
I have the following kernel
__global__ void func( float * arr, int N ) {
int rtid = blockDim.x * blockIdx.x + threadIdx.x;
if( rtid < N )
{
float* row = (float*)((char*)arr + rtid*N*sizeof(float) );
for (int c = 1; c < N; ++c)
{
//Manipulation
}
}
}
When I call the kernel from MATLAB using
gtm= parallel.gpu.GPUArray(ones(a,b,'double'));
OR gtm= parallel.gpu.GPUArray(ones(1,b,'double'));
gtm=k.feval(gtm,b);
it is giving the following error:
Error using ==> feval
parallel.gpu.GPUArray must match the exact input type as specified on the kernel
prototype.
Error in ==> sameInit at 65 gtm=k.feval(gtm,b);
Can someone please tell me where am I going wrong.
Thanking You,
Viharri P L V.
The kernel object "k" that is being created in MATLAB have the following structure:
MaxNumLHSArguments: 1
NumRHSArguments: 2
ArgumentTypes: {'inout single' 'in int32 scalar'}
with the above mentioned CUDA kernel prototype i.e.,
__global__ void func( float * arr, int N )
So, there was an mismatch error. We need to either change the prototype of the CUDA kernel to
__global__ void func( double * arr, int N )
or create the MATLAB array with 'single' type.
gtm= parallel.gpu.GPUArray(ones(a,b,'single'));
Numpy has a basic pxd that declares its c interface to cython. Is there such a pxd for scipy components (especially scipy.integrate.quadpack)?
Alternatively, could someone give an example of how to link directly with c/fortran functions included in scipy from cython? So far I have always used pyximport ... will that work here, or will I have to link using distutils (or make..)?
Thanks!
---- UPDATE ----
The cython code below compiles; however, I get
ImportError: Building module failed: ['ImportError: dlopen(/Users/shauncutts/.pyxbld/lib.macosx-10.7-intel-2.7/factfiber/stat/pmodel/c/meer.so, 2): Symbol not found: _DQAGSE\n Referenced from: /Users/shauncutts/.pyxbld/lib.macosx-10.7-intel-2.7/factfiber/stat/pmodel/c/meer.so\n Expected in: flat namespace\n in /Users/shauncutts/.pyxbld/lib.macosx-10.7-intel-2.7/factfiber/stat/pmodel/c/meer.so\n']
So I think I've focused my problem on actually "pointing" my reference of fortran QDAGSE to the _quadpack.so in scipy.integrate. (N.B. tried lowercase version as well.)... without having to hack scipy by putting a pxd there. Is there some way I can do this? Perhaps dynamically? (Can I load the .so dynamically and bind the appropriate function pointer onto a global variable, perhaps?)
cdef extern from "stdlib.h":
void free(void* ptr)
void* malloc(size_t size)
void* realloc(void* ptr, size_t size)
ctypedef double (*qagfunc)( double* )
cdef extern:
# in scipy.integrate._quadpack
cdef void dqagse(
qagfunc f, double *a, double *b, double *epsabs, double *epsrel,
int *limit,
double *result, double *abserr, int *neval, int *ier,
double *alist, double *blist, double *rlist, double *elist,
int *iord, int *last
)
class QAGError( ValueError ):
code = None
cdef double qags(
qagfunc quad_function, double a, double b,
double epsabs=1.49e-8, double epsrel=1.49e-8,
int limit = 50
):
'''
wrapper for QUADPACK quags/quagse
'''
cdef double result, abserr
cdef int neval = 0, ier = 0, last = 0
cdef int *iord = <int *>malloc( limit * sizeof( double ) )
cdef double *alist = <double *>malloc( limit * sizeof( double ) )
cdef double *blist = <double *>malloc( limit * sizeof( double ) )
cdef double *rlist = <double *>malloc( limit * sizeof( double ) )
cdef double *elist = <double *>malloc( limit * sizeof( double ) )
try:
DQAGSE(
quad_function, &a, &b, &epsabs, &epsrel, &limit,
&result, &abserr, &neval, &ier,
alist, blist, rlist, elist, iord, &last);
finally:
free( iord )
free( alist )
free( blist )
free( rlist )
free( elist )
if ier > 0:
raise QAGError( QUAGS_IER_MAP[ ier ] )
return result
Thanks to: http://codespeak.net/pipermail/cython-dev/2009-October/007239.html
Adapting, the following seems to work.
from os import path
import ctypes
from scipy.integrate import quadpack
ctypedef double (*qagfunc)( double* )
ctypedef void (*quadpack_qagse_fp)(
qagfunc f, double *a, double *b, double *epsabs, double *epsrel,
int *limit,
double *result, double *abserr, int *neval, int *ier,
double *alist, double *blist, double *rlist, double *elist,
int *iord, int *last
)
quadpack_path = path.dirname( quadpack.__file__ )
quadpack_ext_path = path.join( quadpack_path, '_quadpack.so' )
quadpack_lib = ctypes.CDLL( quadpack_ext_path )
cdef quadpack_qagse_fp quadpack_qagse
quadpack_qagse = (
<quadpack_qagse_fp*><size_t> ctypes.addressof( quadpack_lib.dqagse_ ) )[ 0 ]
With this prepended, the code above seems to work... well except that, annoyingly enough, the answer is a bit different than the python quad, and also quadpack dblquad (I'm evaluating a double integral). I assume I either need more resolution, or need to specify a parameter with a non-default value. (Also it won't port to Windows perhaps? But that would just mean changing ".so" to ".dll")
It gets me just under 5x speed up from previous code that used python callback, but otherwise was as optimized as I could get it.
For extra credit... how do I smuggle in extra parameters to the c callback? I assume its impossible without global variables? or I could use weave?
In any case -- thanks for your attention -- this is an obscure example, but has more general applicability to calling undeclared c functions in extension modules from cython. If anyone has a more elegant way of doing it, I'd be eager to hear.