getting libstruct to work in matlab for dll pointer argument - matlab

I'm trying to call a dll function in matlab. I have a C++ struct as shown in sixense.h:
typedef struct _sixenseControllerData {
float pos[3];
float rot_mat[3][3];
float joystick_x;
float joystick_y;
float trigger;
...
} sixenseControllerData;
and functions I could call:
SIXENSE_EXPORT int sixenseInit( void );
SIXENSE_EXPORT int sixenseGetAllNewestData( sixenseAllControllerData * );
I can easily get this to work with calllib('sixense','sixenseInit') since there is no input, but for the function sixenseGetAllNewestData I need to have a struct pointer. I realize that libstruct is what I need to use. However, I don't seem to be doing it right.
So I tried libstruct like so:
libstruct('sixenseControllerData')
and I get the error:
??? Error using ==> feval
Undefined function or variable 'lib.sixenseControllerData'.
Error in ==> libstruct at 15
ptr=feval(['lib.' structtype]);
EDIT: here is my current unedited proto file:
http://pastebin.com/PemmmMqF
the full header file is available here:
https://github.com/rll/sixense/blob/master/include/sixense.h

For C structures, loadlibrary generates types named: s_{NAME} where {NAME} is the name of the structure. In your case we create a pointer as:
s = libstruct('s_sixenseControllerData');
We can see this fact by instructing MATLAB to generate a prototype file:
>> loadlibrary('sixense', 'sixense.h', 'proto','sixense_proto')
A prototype file is a file of MATLAB commands which we can modify and use in place of a header file. In this case, the file will contain something like:
sixense_proto.m
...
structs.s_sixenseControllerData.members = struct('pos', 'single#3', 'rot_mat', 'single#9', 'joystick_x', 'single', 'joystick_y', 'single', 'trigger', 'single', 'buttons', 'uint32', 'sequence_number', 'uint8', 'rot_quat', 'single#4', 'firmware_revision', 'uint16', 'hardware_revision', 'uint16', 'packet_type', 'uint16', 'magnetic_frequency', 'uint16', 'enabled', 'int32', 'controller_index', 'int32', 'is_docked', 'uint8', 'which_hand', 'uint8', 'hemi_tracking_enabled', 'uint8');
structs.s_sixenseAllControllerData.members = struct('controllers', 's_sixenseControllerData#4');
....
Unfortunately, a limitation of loadlibrary is that it does not support nested structure very well, especially if a structure contains a pointer to another structure (or an array in this case):
Nested structures or structures containing a pointer to a structure are
not supported. However, MATLAB can access an array of
structures created in an external library.
So you will not be able to directly create the sixenseAllControllerData structure on the MATLAB side, which is defined in the C header file as:
typedef struct _sixenseAllControllerData {
sixenseControllerData controllers[4];
} sixenseAllControllerData;
According to the following discussion, one workaround is to "unroll"/"flatten" the array into separate variables. You can either do this in a copy of the header file, or making the changes in the generated prototype file (which I think is the preferred way). You can do this without having to recompile the shared library.
In your case, change the nested structure in the generated sixense_proto.m file into:
structs.s_sixenseAllControllerData.members = struct(...
'controllers1', 's_sixenseControllerData', ...
'controllers2', 's_sixenseControllerData', ...
'controllers3', 's_sixenseControllerData', ...
'controllers4', 's_sixenseControllerData');
Now we can create a pointer to this structure, and call the C method:
s = libstruct('s_sixenseAllControllerData');
s.controllers1 = libstruct('s_sixenseControllerData');
s.controllers2 = libstruct('s_sixenseControllerData');
s.controllers3 = libstruct('s_sixenseControllerData');
s.controllers4 = libstruct('s_sixenseControllerData');
out = calllib('sixense', 'sixenseGetAllNewestData', s);
get(s)
A completely different solution is to write a MEX-function to interface with the library. It is just like any other C/C++ code, only using mxArray and the MX-API to interface with MATLAB...
Example:
To test the above, I created a simple DLL with similar structures, and implemented the above solution. Here is the code if someone wants to test it:
helper.h
#ifndef HELPER_H
#define HELPER_H
#ifdef _WIN32
#ifdef EXPORT_FCNS
#define EXPORTED_FUNCTION __declspec(dllexport)
#else
#define EXPORTED_FUNCTION __declspec(dllimport)
#endif
#else
#define EXPORTED_FUNCTION
#endif
#endif
mylib.h
#ifndef MYLIB_H
#define MYLIB_H
#include "helper.h"
typedef struct _mystruct {
int pos[3];
double value;
} mystruct;
typedef struct _mystruct2 {
mystruct arr[2];
int num;
} mystruct2;
EXPORTED_FUNCTION void myfunc(mystruct *);
EXPORTED_FUNCTION void myfunc2(mystruct2 *);
#endif
mylib.c
#define EXPORT_FCNS
#include "helper.h"
#include "mylib.h"
void myfunc(mystruct *s)
{
s->pos[0] = 10;
s->pos[1] = 20;
s->pos[2] = 30;
s->value = 4.0;
}
void myfunc2(mystruct2 *s)
{
int i;
for(i=0; i<2; i++) {
myfunc(&(s->arr[i]));
}
s->num = 99;
}
After compiling the above into a DLL, we generate the initial prototype file:
loadlibrary('./mylib.dll', './mylib.h', 'mfilename','mylib_proto')
unloadlibrary mylib
I edit the prototype file as described before:
function [methodinfo,structs,enuminfo,ThunkLibName] = mylib_proto()
MfilePath = fileparts(mfilename('fullpath'));
ThunkLibName = fullfile(MfilePath,'mylib_thunk_pcwin64');
enuminfo = [];
structs = [];
structs.s_mystruct.members = struct('pos','int32#3', 'value','double');
structs.s_mystruct2.members = struct('arr1','s_mystruct', ...
'arr2','s_mystruct', 'num','int32');
ival = {cell(1,0)};
methodinfo = struct('name',ival, 'calltype',ival, 'LHS',ival, ...
'RHS',ival, 'alias',ival, 'thunkname',ival);
methodinfo.thunkname{1} = 'voidvoidPtrThunk';
methodinfo.name{1} = 'myfunc';
methodinfo.calltype{1} = 'Thunk';
methodinfo.LHS{1} = [];
methodinfo.RHS{1} = {'s_mystructPtr'};
methodinfo.thunkname{2} = 'voidvoidPtrThunk';
methodinfo.name{2} = 'myfunc2';
methodinfo.calltype{2} = 'Thunk';
methodinfo.LHS{2} = [];
methodinfo.RHS{2} = {'s_mystruct2Ptr'};
end
Now we can finally invoke functions exposed by the DLL:
%// load library using proto file
loadlibrary('./mylib.dll', #mylib_proto)
%// call first function with pointer to struct
s = struct('pos',[0,0,0], 'value',0);
ss = libstruct('s_mystruct',s);
calllib('mylib', 'myfunc', ss)
get(ss)
%// call second function with pointer to struct containing array of struct
xx = libstruct('s_mystruct2');
xx.arr1 = libstruct('s_mystruct');
xx.arr2 = libstruct('s_mystruct');
calllib('mylib', 'myfunc2', xx)
get(xx)
%// clear references and unload library
clear ss xx
unloadlibrary mylib

Related

Pass String array as input into external C function

I would like to pass a String vector into an external C function.
In a minimal example I just want to pass the String vectors (or 1D array) through the C function.
My Modelica function looks like:
function testreadstri
input String instri[2];
output String outstri[2];
external "C" test_stri(instri,, size(instri, 1), outstri);
annotation (Include="#include <ebcmysql.cpp>", Library="libmysql");
end testreadstri;
My C fucntion looks like:
void test_stri(const char* thestring, size_t nLines, const char **testresult)
{
//bout = 12.3;
size_t iLines;
//size_t nLines;
iLines = 0;
//nLines = 1;
while ( iLines <= nLines ) {
<LINE_OF_INTEREST>
iLines++;
}
}
I tried for <LINE_OF_INTEREST> the following lines:
testresult[iLines] = thestring[iLines];
strcpy(testresult[iLines], thestring[iLines]);
What works, but of course does not pass the input through as an output, is:
testresult[iLines] = "aTestString";
Is there any possibility to handle Modelica input String vectors in the external C function?
Thanks in advance!
Here's a short, self-contained and compilable example demonstrating both input string and output string handling of a pure external function in Modelica
model Model
function testreadstri
input String instri[2];
output String outstri[2];
external "C" test_stri(instri, size(instri, 1), outstri, size(outstri, 1));
annotation(Include="
#include \"ModelicaUtilities.h\"
#include <stdlib.h>
#include <string.h>
void test_stri(const char** thestring, size_t nLinesIn, const char** testresult, size_t nLinesOut)
{
size_t iLines;
// example for input string handling
for (iLines = 0; iLines < nLinesIn; iLines++) {
ModelicaFormatMessage(\"%s\\n\", thestring[iLines]);
}
// example for output string handling
for (iLines = 0; iLines < nLinesOut; iLines++) {
char* line = ModelicaAllocateStringWithErrorReturn(6);
if (line != NULL) {
strcpy(line, \"result\");
testresult[iLines] = line;
}
}
}");
end testreadstri;
String s[:] = testreadstri({"first", "second"});
end Model;
Yes, this is supported by the Modelica specification, see https://specification.modelica.org/v3.4/Ch12.html#argument-type-mapping.

[AccelStepper][PlatformIO][mbed] Request for member '' in '', which is of non-class type

so I have been trying to create 3 'accelstepper' objects.
This is a screenshot of my code in case the code section doesn't appear. Also, this is a screenshot of the file "stepper_directory.h"
#include <mbed.h>
#include "stepper_directory.h"
//Include accelstepper library
#include <AccelStepper.h>
//Create accelstepper object for the Z-Axis actuator
AccelStepper zaxis(uint8_t interface = AccelStepper::DRIVER, uint8_t zstep = ZSTEP, uint8_t zdir = ZDIR);
//Create accelstepper object for the theta axis actuator
AccelStepper taxis(uint8_t interface = AccelStepper::DRIVER, uint8_t tstep = TSTEP, uint8_t tdir = TDIR);
//Create accelstepper object for the magnet actuator
AccelStepper maxis(uint8_t interface = AccelStepper::DRIVER, uint8_t mstep = MSTEP, uint8_t mdir = MDIR);
This is the header file i've used "stepper_directory.h"
#ifndef _STEPPER_DIRECTORY
#define _STEPPER_DIRECTORY
#include "PinNames.h"
//Pin Definitions
#define ZSTEP PA_7
#define ZDIR PA_0
#define TSTEP PA_8
#define TDIR PA_1
#define MSTEP PA_9
#define MDIR PA_2
I've tried to setup one stepper in my main code in main.cpp as follows:
int main() {
// put your setup code here, to run once:
zaxis.setMaxSpeed(188000);
while(1) {
// put your main code here, to run repeatedly:
}
}
But the platformIO compiler is throwing out this lines:
src\main.cpp: In function 'int main()':
src\main.cpp:17:7: error: request for member 'setMaxSpeed' in 'zaxis', which is of non-class type 'AccelStepper(uint8_t, uint8_t, uint8_t)
{aka AccelStepper(unsigned char, unsigned char, unsigned char)}'
zaxis.setMaxSpeed(188000);
^~~~~~~~~~~
*** [.pio\build\nucleo_f410rb\src\main.o] Error 1
I have been attempting to search for what is wrong with my object instantiations to no avail. I would really appreciate if there someone could explain what is wrong with this. This is a screenshot of the error in question
The problem is here.
//Create accelstepper object for the Z-Axis actuator
AccelStepper zaxis(uint8_t interface = AccelStepper::DRIVER, uint8_t zstep = ZSTEP, uint8_t zdir = ZDIR);
This is a function declaration. It takes three arguments and returns AccelStepper. You cannot initialize an instance of AccelStepper with this line of code.
I assume AccelStepper's constructor is something like this:
AccelStepper AccelStepper(uint8_t interface, uint8_t zstep, uint8_t zdir);
You can initialize your instance this way:
AccelStepper zaxis(AccelStepper::DRIVER, ZSTEP, ZDIR);

pybind with array as class attribute

I want to wrap the following C++ code into python using pybind
class Galaxy {
public:
double x[3];
double v[3];
};
class GalaxyCatalogue {
public:
long n_tracers;
Galaxy *object;
GalaxyCatalogue(long n_tracers);
~GalaxyCatalogue();
};
GalaxyCatalogue::GalaxyCatalogue(long n_tracers) : n_tracers(n_tracers) {
std::cout << "from galaxies " << n_tracers << std::endl;
object = new Galaxy[n_tracers];
std::cout << "has been allocated " << std::endl;
}
GalaxyCatalogue::~GalaxyCatalogue() {
delete[] object;
}
The first problem I have is that Galaxy doesn't have a constructor, so I'm not sure what to do with that. Even if I declare an empty constructor I don't know how to treat the array in a way that I don't get an error when compiling. This is what I've tried:
#include <pybind11/pybind11.h>
#include <iostream>
namespace py = pybind11;
class Galaxy {
public:
Galaxy();
double x[3];
};
PYBIND11_MODULE(example, m){
py::class_<Galaxy>(m, "Galaxy")
.def(py::init<>())
.def_readwrite("x", &Galaxy::x);
}
This is how I compile it:
c++ -O3 -Wall -shared -std=c++11 -fPIC `python3 -m pybind11 --includes` gal.cpp -o example`python3-config --extension-suffix`
and this is the error I get:
In file included from gal.cpp:1:
/home/florpi/.conda/envs/virtualito/include/python3.5m/pybind11/pybind11.h: In instantiation of ‘pybind11::class_<type_, options>& pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = Galaxy; D = double [3]; Extra = {}; type_ = Galaxy; options = {}]’:
gal.cpp:19:33: required from here
/home/florpi/.conda/envs/virtualito/include/python3.5m/pybind11/pybind11.h:1163:65: error: invalid array assignment
fset([pm](type &c, const D &value) { c.*pm = value; }, is_method(*this));
~~~~~~^~~~~~~
In file included from gal.cpp:1:
/home/florpi/.conda/envs/virtualito/include/python3.5m/pybind11/pybind11.h:64:5: error: ‘pybind11::cpp_function::cpp_function(Func&&, const Extra& ...) [with Func = pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = Galaxy; D = double [3]; Extra = {}; type_ = Galaxy; options = {}]::<lambda(pybind11::class_<Galaxy>::type&, const double (&)[3])>; Extra = {pybind11::is_method}; <template-parameter-1-3> = void]’, declared using local type ‘pybind11::class_<type_, options>::def_readwrite(const char*, D C::*, const Extra& ...) [with C = Galaxy; D = double [3]; Extra = {}; type_ = Galaxy; options = {}]::<lambda(pybind11::class_<Galaxy>::type&, const double (&)[3])>’, is used but never defined [-fpermissive]
cpp_function(Func &&f, const Extra&... extra) {
^~~~~~~~~~~~
Thank you in advance.
In C++, you can't assign directly to an array, which is what pybind11 is trying to do inside its wrapping magic. In general, C++ arrays are not great abstractions for numerical arrays. As you've noticed, you can't even say galaxy.x = other_galaxy.x.
Your best bet is to use a higher-level library for matrices and vectors, which will
a) give you a much better experience writing your C++
b) perform better
c) map more cleanly to Python
Eigen is a good choice. pybind11 automatically knows how to map Eigen matrices and vectors to numpy arrays. Your Galaxy would become:
class Galaxy {
public:
Eigen::Vector3d x;
Eigen::Vector3d v;
};
If you absolutely can't do this, you'll have to supply manual getter/setter functions to the property, where you do your own conversion to and from python types:
https://pybind11.readthedocs.io/en/master/classes.html?highlight=def_property#instance-and-static-fields

How to add a built-in module to a C-Python API after Py_Initialize?

I have a module defined in my C code like so:
static struct PyModuleDef module_def = {
PyModuleDef_HEAD_INIT,
"the_module_name", /* m_name */
module_documentation, /* m_doc */
//....
};
and a function to initialize it:
PyMODINIT_FUNC init_the_module(void)
{
PyObject *mod, *submodule;
PyObject *sys_modules = PyThreadState_GET()->interp->modules;
mod = PyModule_Create(&module_def);
PyModule_AddObject(mod, "some_submodule", (submodule = init_the_submodule()));
PyDict_SetItemString(sys_modules, PyModule_GetName(submodule), submodule);
Py_INCREF(submodule);
// more submodules..
return mod;
}
The application that I am embedding python into is quite big and I can not change the workflow much. At this point Py_Initialize has already been called, so I can not call PyImport_ExtendInittabor PyImport_AppendInittab .
How can I create and add the module to the system modules?
Maybe I can manipulate the modules dictionary directly? Like so:
PyObject *modules, *the_module;
modules = PyImport_GetModuleDict();
PyDict_SetItemString(modules, "the_module_name", init_the_module());
the_module = PyDict_GetItemString(modules, "the_module_name"); //this is getting something back
std::cout << PyDict_Size(the_module) << std::endl; // this prints -1
The easiest way to handle this is to statically initialize your statically-linked modules by directly calling initspam() after the call to Py_Initialize() or PyMac_Initialize():
int main(int argc, char **argv)
{
/* Pass argv[0] to the Python interpreter */
Py_SetProgramName(argv[0]);
/* Initialize the Python interpreter. Required. */
Py_Initialize();
/* Add a static module */
initspam();
An example may be found in the file Demo/embed/demo.c in the Python source distribution.

Matlab: How to survey compiled m-code progression from external API?

My question is extremely specific to the arcanes of the matlab compiler and runtime. As only people familiar with matlab runtime API may answer, I shortened much details. Please let me know if I should be more verbose.
Introduction
Using the matlab compiler & runtime I can call a function written in m-code from a C# program. Let's say calling:
function [result] = foo(n)
%[
result = 0;
for k = 1:n,
pause(1.0); % simulate long processing
result = result + 42;
end
%]
with (somewhere behind some dllimports in the C# code):
mclFeval(IntPtr inst, string name, IntPtr[] plhs, IntPtr[] prhs)
So far, so good, I have no issue with this (i.e intializing the runtime, loading the '.cft' file, marshalling back and forth MxArray with .Net types, etc...)
My Problem
I would like to survey the progression of my foo function using some cancel and progress callbacks:
function [result] = foo(n, cancelCB, progressCB)
%[
if (nargin < 3), progressCB = #(ratio, msg) disp(sprintf('Ratio = %f, Msg = %s', ratio, msg)); end
if (nargin < 2), cancelCB = #() disp('Checking cancel...'); end
result = 0;
for k = 1:n,
if (~isempty(cancelCB)),
cancelCB(); % Up to the callback to raise some error('cancel');
end;
if (~isempty(progressCB)),
progressCB(k/n, sprintf('Processing (%i/%i)', k, n));
end
pause(1.0); % simulate long processing
result = result + 42;
end
%]
But of course I would like these callbacks to be in the C# code, not within the m-one.
Investigations
Looking at 'mclmcr.h' header file, it looks like these functions may be of help:
extern mxArray* mclCreateSimpleFunctionHandle(mxFunctionPtr fcn);
extern bool mclRegisterExternalFunction(HMCRINSTANCE inst, const char* varname, mxFunctionPtr fcn);
Unfortunatly these are fully undocumented and I found no use case I could mimic to understand how they work.
I've also thought about creating a COM visible object in C# and pass it as a parameter to the matlab code:
// Somewhere within C# code:
var survey = new ComSurvey();
survey.SetCancelCallback = () => { if (/**/) throw new OperationCancelException(); };
survey.SetProgressCallback = (ratio, msg) => { /* do something */ };
function [result] = foo(n, survey)
%[
if (nargin < 2), survey = []; end
result = 0;
for k = 1:n,
if (~isempty(survey)),
survey.CheckCancel(); % up to the COM object to raise exception
survey.SetProgress(k/n, sprintf('Processing... %i/%i', k, n));
end
pause(1.0); % simulate long processing
result = result + 42;
end
%]
I'm very familiar with functions to create numeric and structure arrays and know how to use them:
extern mxArray *mxCreateNumericArray(...)
extern mxArray *mxCreateStructArray(...)
Anyhow, how COM objects are packaged to MxArrays, I don't know?
Further investigations
Day+1
Even if still unstable, I succeeded to have matlab to callback into my C# code and it seems that mclCreateSimpleFunctionHandle is the direction to go.
Note: Below code is for reference only. It may not be suitable in your own context as is. I'll provide simpler code later on (i.e. once I'll get stable solution).
Looking to the signature of the mxFunctionPtr, I created two delegates like this:
// Mimic low level signature for a Matlab function pointer
[UnmanagedFunctionPointer(CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
delegate void MCRInteropDelegate(int nlhs, IntPtr[] plhs, int nrhs, IntPtr[] prhs);
and
// Same signature (but far more elegant from .NET perspective)
delegate void MCRDelegate(MxArray[] varargouts, MxArray[] varargins);
I also linked to the runtime like this:
[DllImport("mclmcrrt74.dll", EntryPoint = "mclCreateSimpleFunctionHandle", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi, ExactSpelling = true)]
static extern IntPtr _mclCreateSimpleFunctionHandle(MCRInteropDelegate fctn);
Assuming MxArray is a .NET class of mine that simply encapsulate for mxArray* handles, I then marshaled my delegates like this:
// Create MxArray from corresponding .NET delegate
static MxArray CreateFromDelegate(MCRDelegate del)
{
// Package high level delegate signature to a 'dllimport' signature
MCRInteropDelegate interopDel = (nlhs, plhs, nrhs, prhs) =>
{
int k = 0;
var varargouts = new MxArray[nlhs];
var varargins = new MxArray[nrhs];
// (nrhs, prhs) => MxArray[] varargins
Array.ForEach(varargins, x => new MxArray(prhs[k++], false)); // false = is to indicate that MxArray must not be disposed on .NET side
// Call delegate
del(varargouts, varargins); // Todo: varargouts created by the delegate must be destroyed by matlab, not by .NET !!
// MxArray[] varargouts => (nlhs, plhs)
k = 0;
Array.ForEach(plhs, x => varargouts[k++].getPointer());
};
// Create the 1x1 array of 'function pointer' type
return new MxArray(MCRInterop.mclCreateSimpleFunctionHandle(interopDel));
}
Finally, assuming module is an instance of MCRModule (again, a class of mine to encapsulate hInst* in low level mclFeval API), I was able to call foo function and have it to enter my .NET cancel delegate like this:
// Create cancel callback in .NET
MCRDelegate cancel = (varargouts, varargins) =>
{
if ((varargouts != null) && (varargouts.Length != 0) { throw new ArgumentException("'cancel' callback called with too many output arguments"); }
if ((varargins != null) && (varargins.Length != 0) { throw new ArgumentException("'cancel' callback called with too many input arguments"); }
if (...mustCancel...) { throw new OperationCanceledException(); }
}
// Enter the m-code
// NB: Below function automatically converts its parameters to MxArray
// and then call low level mclFeval with correct 'mxArray*' handles
module.Evaluate("foo", (double)10, cancel);
This .NET code worked fine, and foo really made callback to the cancel delegate properly.
Only problem, is that it is quite unstable. My guess is that I used too many anonymous functions, and probably some of them are disposed too early ...
Will try to provide with stable solution within the next few days (hopefully with simpler code to read and copy-paste in your own context for immediate testing).
Please let me know if you think I'm going the wrong direction with mclCreateSimpleFunctionHandle.
Got it
mclCreateSimpleFunctionHandle was effectively the right API function to call at in order to create an array variable (on matlab's side) holding for a function pointer (on external's side). I'm now able to have compiled m-code to call back into my C# code for cancellation and progression purposes.
Correct marshalling for mclCreateSimpleFunctionHandle is described here