Linking and LOADING static .lib with mex - matlab

So, I have a MEX gateway script file that calls my C source code. I've used the -L and -I commands to link my 64-bit compiled GSL libraries (.libs) to my mex executable, which is then compiled under the extension of .mexw64.
I want for this executable to be transferred to another windows machine and run fine, without any GSL libraries installed. That is the the only solution, I don't care what he arguments are regarding the benefits of the dynamic linking/code generation upon compile-time are. I want an executable that has every function not only (of course) pre-declared, but also PRE-DEFINED.
I was lead to believe that this is what 'static' linking is vs. dynamic; but I've read some contradictory definitions all around the interwebs. I need a completely 100% standalone, singular file.
Supposedly you can link the actual .obj file in the mex function, which I can generate, but unfortunately I then get unresolved symbol errors.
Someone else mentioned that I can use the -l (lowercase L) to directly link the actual .lib(s) needed, statically, but that is NOT true.
So is there anyone that can lead me in the right direction, either how to have everything not only linked but to also have the DEFINITIONS linked and ready to load when executable is run--completely standalone, or why I am running into unresolved symbols/linker errors when I include my .obj file? Am I misunderstanding something elementary about the linking process?
Also: To elaborate a bit more, I have the GSL libraries built and linked via Visual Studio for the 64 bit architecture, and I can link it easily with MATLAB, so that is not my problem (any more).
EDIT: I've seen the post here:
Generating standalone MEX file with GNU compilers, including libraries
This doesn't solve my problem, however, although it is the same question. I don't have access to gcc; it's finally compiling on the MSVS12 compiler in MATLAB, I'm not going try to recompile using GCC via MinGW (already tried, couldn't figure it out), so -static and .a options are out.

In your previous post, you mentioned that you decided to compile GSL library with Visual C++, using the VS solution provided by Brian Gladman.
Here is a step-by-step illustration on how to build a MEX-function that links against GSL libraries statically:
Download GNU GSL sources (GSL v1.16)
Download the matching Visual Studio project files (VS2012 for GSL v1.16)
Extract the GSL tarball, say to C:\gsl-1.16
Extract the VS project files on top of the sources, this will overwrite three files as well as add a folder C:\gsl-1.16\build.vc11.
Open Visual Studio 2012, and load the solution: C:\gsl-1.16\build.vc11\gsl.lib.sln
Change the configuration to the desired output: for me I chose platform=x64 and mode=Release
First you must build the gslhdrs project first
Now build the whole solution. This will create two static libraries cblas.lib and gsl.lib stored in C:\gsl-1.16\lib\x64\Release (along with corresponding PDB debugging symbols). It will also create a directory containing the final header files: C:\gsl-1.16\gsl
Next we proceed to build a MEX-function. Take the following simple program (computes some value from a Bessel function, and return it as output):
gsl_test.c
#include "mex.h"
#include <gsl/gsl_sf_bessel.h>
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
if (nrhs != 0 || nlhs > 1) mexErrMsgTxt("Wrong number of args.");
plhs[0] = mxCreateDoubleScalar(gsl_sf_bessel_J0(5.0));
}
This is how to compile the above C code in MATLAB:
>> mex -largeArrayDims gsl_test.c -I"C:\gsl-1.16" -L"C:\gsl-1.16\lib\x64\Release" cblas.lib gsl.lib
Finally we test the MEX-file, and compare it against the value reported by MATLAB's own Bessel function:
>> x = gsl_test()
ans =
-0.1776
>> y = besselj(0,5)
y =
-0.1776
>> max(x-y) % this should be less than eps
ans =
8.3267e-17
Note that the built MEX-function has no external DLL dependencies (other than "Visual C Runtime" which is expected, and the usual MATLAB libraries). You can verify that by using Dependency Walker if you want. So you can simply deploy the gsl_test.mexw64 file alone (assuming the users already have the corresponding VC++ runtime installed on their machines).

Related

Link to gfortran for mex file

I am trying to create mex files for using some C++ code in Matlab, but running into an error with the compilation which, I guess, goes beyond Matlab. I am on Windows.
The example I'm struggling with is the Mex Interface example in the Armadillo library for linear algebra, available: here. The Matlab file is what I'm running and it looks like this:
% Compile the demo as a mex file
mex -lgfortran armaMex_demo.cpp -I/path/to/armadillo
The error I get is:
Error using mex
MEX cannot find library 'gfortran' specified with the -l option.
MEX looks for a file with one of the names:
libgfortran.lib
gfortran.lib
Please specify the path to this library with the -L option.
Trying some things I found online, I installed MinGW and cygwin64 and added the paths to their bin folders to my PATH variable. Before that I also installed MinGW via Matlab.
My question: How do I get it to link with gfortran?
If I search C:for libgfortran I find some files, but none of them have the file ending .lib (but e.g. .dll and .a) and if I search for gfortranI find some .exe files but again, no .lib file. If I run gfortran -dumpversion in the terminal, I get 4.9.3 back, so it is obviously there somewhere. I am obviously missing something, and this is really not my forte, to say the least.

Matlab missing dependency MEX-file

I have a script in matlab that calls other libraries. I use matlab version 2012a on linux . I get below error and I don't know how to fix it.
The error is :
Invalid MEX-file '/home/XXX/nearest_neighbors.mexa64':
libflann.so.1.8: cannot open shared object file: No such file or
directory
Error in flann_search (line 82)
[indices,dists] = nearest_neighbors('find_nn', data, testset, n, params);
Error in MyScript (line 73)
[nresult, ndists] = flann_search(Ntraindata', Ntraindata', resu.KNN, struct('algorithm','composite',...
That library you are referring to - https://github.com/mariusmuja/flann/ - has the nearest_neighbors function written in MEX code. MEX code is C code that is used to interface with MATLAB. People usually write heavily computationally burdening code over in MEX as it is known to handle loops and other things faster. The inputs come from MATLAB and are sent to this MEX function, and the outputs come from the MEX function and are piped back to MATLAB. It's basically a nice black box where you can use it just like any other MATLAB function. In fact, a lot of the functions that come shipped with MATLAB have MEX wrappers written to promote acceleration.
You are getting that error because you need to compile the nearest_neighbors function so that there is a MEX wrapper that can be called in MATLAB. That wrapper is missing because you haven't compiled the code.
First, you need to set up MEX. Make sure you have a compiler that is compatible with your version of MATLAB. You can do that by visiting this link:
http://www.mathworks.com/support/compilers/R20xxy/index.html
xx is the version number that belongs to your MATLAB and y is the character code that comes after it. For example, if you are using R2013a, you would visit:
http://www.mathworks.com/support/compilers/R2013a/index.html
Once you're there, go to your Operating System and ensure you have one of those supported compilers installed. Once you have that installed, go into MATLAB, and in the command prompt, type in:
mex -setup
This will allow you to set up MEX and choose the compiler you want. Given your error, you're using Linux 64-bit, so it should be very easy for you to get GCC. Just make sure that you choose a version of GCC that is compatible with your version of MATLAB. Once you choose the compiler, you can compile the code by doing this in the command prompt:
>> mex -v -O nearest_neighbors.cpp
This should generate the nearest_neighbors MEX file for you. Once that's done, you can now run the code.
For more detailed instructions, check out FLANN's user manual - http://people.cs.ubc.ca/~mariusm/uploads/FLANN/flann_manual-1.8.4.pdf - It tells you how to build and compile it for MATLAB use.
Good luck!

Problems compiling CUDA with MEX in Matlab2014a

I am trying to compile a CUDA program using MEX within Matlab2014a and can't get it to work.
I installed the newest CUDA toolbox (6.5) and driver (340.62) and the samples work correctly, so I guess it is correctly installed.
I also installed MS Visual Studio 2012 Professional and Matlab 2014a.
I followed the exact description in the Matlab Help in "GPU Computing -> Examples and How To -> Run MEX-Functions Containing CUDA Code".
My CUDA file ends with .cu (so I am trying to compile Matlab's standard example mexGPUExample.cu, but I get the same error with other .cu files), the mex_CUDA_win64.xml is in the same folder, the environment variable MW_NVCC_PATH is set correctly in the user variables and just to be sure I also called "setenv('MW_NVCC_PATH,...)" in Matlab.
If I try to compile a CUDA example with
mex -largeArrayDims mexGPUExample.cu
I get the following error:
cl : Command line warning D9024 : unrecognized source file type 'mexGPUExample.cu', object file assumed
cl : Command line warning D9027 : source file 'mexGPUExample.cu' ignored
cl : Command line warning D9021 : no action performed
D:\PROGRAMS\MATLAB~1\BIN\MEX.PL: Error: Compile of 'mexGPUExample.cu' failed.
Unable to complete successfully.
I also tried to set the compiler correctly using
mex -setup
and chose the MS Visual Studio Compiler.
The code itself works because I tested it on other systems.
I have no idea what I am doing wrong.
Any help is appreciated.
Matlab only supports CUDA 5.5 on R2014a, so your CUDA 6.5 by default cannot compile mex files under Matlab. Matlab R2014a supports VS 2012 now, but you need to verify you have VS 64-bit compiler if your Matlab is 64-bit. You can compile a normal .mex file with .cpp source file, (lots of these files under matlabroot/extern folder) to see whether Matlab works well with your VS.
Note that Matlab has a lag in supporting the latest compilers, both VS/gcc and Cuda. It is always good choice to check the compiler requirement by Matlab, before using the latest compilers.
Also check whether your Cuda kit and graphics drivers are correctly installed and work seamlessly with VS. You can compile and run some .cu files under the VS environment.
If you have finished all above and changed to the correct version of compilers, follow the instructions on matheworks website to compile the mexGPUExample file. If my memory is correct, you need to set environment variables and copy the XML file to the .mex file path. Then it should work.

mex linking of cuda code in separate compilation mode

I'm trying to use CUDA code inside MATLAB mex, under linux. With the "whole program compilation" mode, it works good for me. I take the following two steps inside Nsight:
(1) Add "-fPIC" as a compiler option to each .cpp or .cu file, then compile them separately, each producing a .o file.
(2) Set the linker command to be "mex" and add "-cxx" to indicate that the type of all the .o input files are cpp files, and add the library path for cuda. Also add a cpp file that contains the mexFunction entry as an additional input.
This works good and the resulted mex file runs well under MATLAB. After that when I need to use dynamical parallelism, I have to switch to the "separate compilation mode" in Nsight. I tried the same thing above but the linker produces a lot of errors of missing reference, which I wasn't able to resolve.
Then I checked the compilation and linking steps of the "separate compilation" mode. I got confused by what it is doing. It seems that Nsight does two compilation steps for each .cpp or .cu file and produces a .o file as well as a .d file. Like this:
/usr/local/cuda-5.5/bin/nvcc -O3 -gencode arch=compute_35,code=sm_35 -odir "src" -M -o "src/tn_matrix.d" "../src/tn_matrix.cu"
/usr/local/cuda-5.5/bin/nvcc --device-c -O3 -gencode arch=compute_35,code=compute_35 -gencode arch=compute_35,code=sm_35 -x cu -o "src/tn_matrix.o" "../src/tn_matrix.cu"
The linking command is like this:
/usr/local/cuda-5.5/bin/nvcc --cudart static --relocatable-device-code=true -gencode arch=compute_35,code=compute_35 -gencode arch=compute_35,code=sm_35 -link -o "test7" ./src/cu_base.o ./src/exp_bp_wsj_dev_mex.o ./src/tn_main.o ./src/tn_matlab_helper.o ./src/tn_matrix.o ./src/tn_matrix_lib_dev.o ./src/tn_matrix_lib_host.o ./src/tn_model_wsj_dev.o ./src/tn_model_wsj_host.o ./src/tn_utility.o -lcudadevrt -lmx -lcusparse -lcurand -lcublas
What's interesting is that the linker does not take the .d file as input. So I'm not sure how it dealt with these files and how I should process them with the "mex" command when linking?
Another problem is that the linking stage has a lot of options I don't understand (--cudart static --relocatable-device-code=true), which I guess is the reason why I cannot make it work like in the "whole program compilation" mode. So I tried the following:
(1) Compile in the same way as in the beginning of the post.
(2) Preserve the linking command as provided by Nsight but change to use "-shared" option, so that the linker produces a lib file.
(3) Invoke mex with input the lib file and another cpp file containing the mexFunction entry.
This way mex compilation works and it produces a mex executable as output. However, running the resulted mex executable under MATLAB produces a segmentation fault immediately and crashes MATLAB.
I'm not sure if this way of linking would cause any problem. More strangely, I found that the mex linking step seems to finish trivially without even checking the completeness of the executable, because even if I miss a .cpp file for some function that the mexFunction will use, it still compiles.
EDIT:
I figured out how to manually link into a mex executable which can run correctly under MATLAB, but I haven't figured out how to do that automatically under Nsight, which I can in the "whole program compilation" mode. Here is my approach:
(1) Exclude from build the cpp file which contains the mexFunction entry. Manually compile it with the command "mex -c".
(2) Add "-fPIC" as a compiler option to each of the rest .cpp or .cu file, then compile them separately, each producing a .o file.
(3) Linking will fail because it cannot find the main function. We don't have it since we use mexFunction and it is excluded. This doesn't matter and I just leave it there.
(4) Follow the method in the post below to manually dlink the .o files into a device object file
cuda shared library linking: undefined reference to cudaRegisterLinkedBinary
For example, if step (2) produces a.o and b.o, here we do
nvcc -gencode arch=compute_35,code=sm_35 -Xcompiler '-fPIC' -dlink a.o b.o -o mex_dev.o -lcudadevrt
Note that here the output file mex_dev.o should not exist, otherwise the above command will fail.
(5) Use mex command to link all the .o files produced in step (2) and step (4), with all necessary libraries supplied.
This works and produces runnable mex executable. The reason I cannot automate step (1) inside Nsight is because if I change the compilation command to "mex", Nsight will also use this command to generate a dependency file (the .d file mentioned in the question text). And the reason I cannot automate step (4) and step (5) in Nsight is because it involves two commands, which I don't know how to put them in. Please let me know if you knows how to do these. Thanks!
OK, I figured out the solution. Here are the complete steps for compiling mex programs with "separate compilation mode" in Nsight:
Create a cuda project.
In the project level, change build option for the following:
Switch on -fPIC in the compiler option of "NVCC compiler" at the project level.
Add -dlink -Xcompiler '-fPIC' to "Expert Settings" "Command Line Pattern" of the linker "NVCC Linker"
Add letter o to "Build Artifact" -> "Artifact Extension", since by -dlink in the last step we are making the output a .o file.
Add mex -cxx -o path_to_mex_bin/mex_bin_filename ./*.o ./src/*.o -lcudadevrt to "Post Build Steps", (add other necessary libs)
UPDATE: In my actual project I moved the last step to a .m file in MATLAB, because otherwise if I do it while my mex program is running, it could cause MATLAB crash.
For files needs to be compiled with mex, change these build option for each of them:
Change the compiler to GCC C++ Compiler in Tool Chain Editor.
Go back to compiler setting of GCC C++ Compiler and change Command to mex
Change command line pattern to ${COMMAND} -c -outdir "src" ${INPUTS}
Several additional notes:
(1) Cuda specific details (such as kernel functions and calls to kernel functions) must be hidden from the mex compiler. So they should be put in the .cu files rather than the header files. Here is a trick to put templates involving cuda details into .cu files.
In the header file (e.g., f.h), you put only the declaration of the function like this:
template<typename ValueType>
void func(ValueType x);
Add a new file named f.inc, which holds the definition
template<>
void func(ValueType x) {
// possible kernel launches which should be hidden from mex
}
In the source code file (e.g., f.cu), you put this
#define ValueType float
#include "f.inc"
#undef ValueType
#define ValueType double
#include "f.inc"
#undef ValueType
// Add other types you want.
This trick can be easily generalized for templated classes to hide details.
(2) mex specific details should also be hidden from cuda source files, since the mex.h will alter the definitions of some system functions, such as printf. So including of "mex.h" should not appear in header files that can potentially be included in the cuda source files.
(3) In the mex source code file containing the entry mexFunction, one can use the compiler macro MATLAB_MEX_FILE to selectively compile code sections. This way th source code file can be compiled into both mex executable or ordinarily executable, allowing debugging under Nsight without matlab. Here is a trick for building multiple targets under Nsight: Building multiple binaries within one Eclipse project
First of all, it should be possible to set up Night to use a custom Makefile rather than generate it automatically. See Setting Nsight to run with existing Makefile project.
Once we have a custom Makefile, it may be possible to automate (1), (4), and (5). The advantage of a custom Makefile is that you know exactly what compilation commands will take place.
A bare-bones example:
all: mx.mexa64
mx.mexa64: mx.o
mex -o mx.mexa64 mx.o -L/usr/local/cuda/lib64 -lcudart -lcudadevrt
mx.o: mxfunc.o helper.o
nvcc -arch=sm_35 -Xcompiler -fPIC -o mx.o -dlink helper.o mxfunc.o -lcudadevrt
mxfunc.o: mxfunc.c
mex -c -o mxfunc.o mxfunc.c
helper.o: helper.c
nvcc -arch=sm_35 -Xcompiler -fPIC -c -o helper.o helper.c
clean:
rm -fv mx.mexa64 *.o
... where mxfunc.c contains the mxFunction but helper.c does not.
EDIT: You may be able achieve the same effect in the automatic compilation system. Right click on each source file and select Properties, and you'll get a window where you can add some compilation options for that individual file. For linking options, open Properties of the project. Do some experiments and pay attention to the actual compilation commands that show up in the console. In my experience, custom options sometimes interact with the automatic system in a weird way. If this method proves too troublesome for you, I suggest that you make a custom Makefile; this way, at least we are not caught by unexpected side-effects.

FORTRAN-MATLAB MEX file error

This is Subhash. I am having problem in linking fortran-matlab mex files.
I am running MATLAB R2010a. I am trying to mex fortran subroutines in MATLAB. I have installed MSYS, MinGW and gfortran. I have also downloaded and extracted gnumex in the c:\documents\MATLAB folder. I entered gnumex to run the installation settings.
At this point, I see a window pop-up and I entered the following:
MinGW's root directory ---> Pointed towards c:\mingw b) Cygwin's
root directory -----> left blank c)
Path to g95.exe ------> c:\mingw\bin
path to gfortran.exe ------> c:\program files\gfortran\bin e)
path to gnumex utilities ------> c:\users.....\matlab\gnumex
environment linking type ------> MinGW
generate mex dll or engine-----> mex
language for compilation -------> fortran 77
add stub ------> no
optimization level -----> -O3
path for libraries and .def files
c:\users...\Roaming\MathWorks\MATLAB\R2010a\gnumex
Mex options file to create:
c\users...\Roaming\MathWorks\MATLAB\R2010a\mexopts.bat
The above is my installation setup for mex with gnumex. When I hit make options file, an warning message pops out which says
" cannot corret mex.pl .....
"cannot open c:...\R2010a\bin\mex.pl for writing"
and when I hit ok, again it pops out saying to confirm overwrite of mexopts.bat .
I hit confirm and the opt file is created.
At this point I should have my mex setup ready. But, when I run the fortran mex examples xtimesy.f, etc.. ( I use mex filename.f at the command prompt) and I get the following error
g77: getValidInputLinkLibraries: No such file or directory
c:\prog~1\MATLAB\R2010A\BIN\MEX.PL: Error: Link of 'xtimesy.mexw32' failed.
Error using ---> mex at 222
unable to complete successfully.
I tried this uninstalling several times, but ended up with the same error.
Could anyone please let me know where am I going wrong. Is this not the correct procedure to link the g77? Please throw some light on this.
I appreciate your time..
Thanks.
I recently tried to mex some fortran routines and failed due to the poor support for fortran compilers by matlab in windows. I got around the problem by using f2c to convert the fortran to C code, then creating a gateway routine in C which calls the automatically generated C code functions and mexing the resul using the visual studio 2008 compiler which integrates nicely with matlab (the free express edition version is sufficient, you will also be able to use it for debugging).
You can get f2c from http://www.netlib.org/f2c/. You must link the produced C code to the f2c library, which you may also have to build from the sources.
I know this all sounds very complicated, but it was easiest solution I found.