Training weka models in matlab: how to evaluate? Mssing some model type? - matlab

despite using weka I have models like libSVM and RseslibKnn, when I call them in Matlab it can't find them
enter image description here
and I noticed that actually they are not part of the models of the directories "classifiers".
enter image description here
I know how to crossvalidate a generic model but I can't use the simple validation with determinated training and test set. Here the code I'm using:
% memorize current folder and change folder
base_folder = cd;
cd 'C:\Program Files'
% adding weka in the java folder
javaaddpath("weka-3-8-4/weka.jar");
% loading arff training set
l = javaObject("weka.core.converters.ArffLoader");
l.setFile(javaObject("java.io.File","weka-3-8-4/data/weka_BEST_train_ESS.arff"));
tr = l.getDataSet;
tr.toString;
% building classfiers
c = javaObject("weka.classifiers.functions.MultilayerPerceptron");
tr.setClassIndex(tr.numAttributes - 1);
c.buildClassifier(tr);
c.toString;
% loading arff test set
l = javaObject("weka.core.converters.ArffLoader");
l.setFile(javaObject("java.io.File","weka-3-8-4/data/weka_BEST_test_ESS.arff"));
ts = l.getDataSet;
ts.toString;
% validation
e = javaObject("weka.classifiers.Evaluation", tr);
e.evaluateModel(c, ts); %?????????????????????????????????????????????????????????????????????
e.toSummaryString;
% going back to the current folder
cd (base_folder)

Answering to the first question:
libSVM and RseslibKnn are models available in separate Weka packages that need to be additionally installed in Weka. To install the packages run Weka package manager (menu Tools -> Package Manager in Weka GUI Chooser) and install LibSVM and Rseslib packages. The packages with their java jars are installed in the wekafiles directory. On Windows it is C:\Users\ <username>\wekafiles\packages by default.
I don't know Matlab but I guess that to use models from extra Weka packages you need to add also the package jar files to Matlab javapath after installing the packages in Weka:
javaaddpath("C:\Users\ <username>\wekafiles\packages\Rseslib\rseslib.jar");
javaaddpath("C:\Users\ <username>\wekafiles\packages\LibSVM\LibSVM.jar");
Alternatively, you can download the libraries directly from the web sites of the package providers, unpack them and add jars to Matlab javapath:
http://rseslib.mimuw.edu.pl/download.html
https://www.csie.ntu.edu.tw/~cjlin/libsvm/

Related

Unable to find scripts in darknet package where I can change file paths

I cloned this darknet package. When training:
./darknet detector train /home/philip/Downloads/YOLO/output.json /home/philip/darknet/cfg/yolov3_test.cfg /home/philip/obj_ws/src/darknet_ros/darknet_ros/yolo_network_config/weights/yolov3.weights -dont_show
I get:
train: Using default 'data/train.txt'
valid: Using default 'data/train.txt'
backup: Using default '/backup/'
at the onset of training.
At the end of training I get this error:
Couldn't open file: /backup//yolov3_test_final.weights
I have a backup folder with the weights. It seems to be a path directory probelm. For anyone who has used this package, I would appreciate help with:
[i]The name of the script I should be looking for to modify my path to fix my error.
[ii] Where to modify darknet so it has access to my OpenCV 3.20
[iii] where to change the default path of the train and valid parameters.
I've checked the folder and having a hard time finding the scripts I need to change.

Yocto custom application through custom layer does not copy source files to destination rootfs

I am trying to install a custom application on my Yocto build.
I currently use a Raspberry Pi 4 64 bit setup, for which I want PyQt5 to display an application directly on the frame buffer (so no windowing manager or desktop envoirement).
My current build with Yocto completes and boots on the Raspberry Pi. All the Qt5 libraries are also present in the root fs after the bitbake build.
Although, I'm having problems getting a custom layer, that adds a custom recipe with a custom application to also copy over to the destination root fs.
My custom layer is called 'meta-rpikms' with recipe 'recipes-kms-qt-app' which contains the application bb files. This files is called 'basicquick_0.1.bb' and has the following contents (this test application tries to add a EGLFS friendly Qt5 applicaiton, i'll try PyQt5 later):
SUMMARY = "Simple Qt5 Quick application"
#SECTION = "examples"
LICENSE = "MIT"
#PACKAGE_ARCH = "all"
LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda2f7b4f302"
DEPENDS += "qtbase qtdeclarative qtquickcontrols2"
SRCREV = "${AUTOREV}"
SRC_URI = "git://github.com/shigmas/BasicQuick.git"
S = "${WORKDIR}/git"
require recipes-qt/qt5/qt5.inc
do_install() {
install -d ${D}${bindir_native}
install -m 0755 BasicQuick ${D}${bindir_native}
}
FILES_${PN} += "${bindir_native}"
When I bitbake the custom layer of my meta-rpikms (called 'qt5-kms-rpi-image'), it does compute. When I take a look at the 'image_initial_manifest' file, my custom application does show up, suggesting that it does compile and try to install the application:
# This file was generated automatically and contains the packages
# passed on to the package manager in order to create the rootfs.
# Format:
# <package_type>,<package_name>
# where:
# <package_type> can be:
# 'mip' = must install package
# 'aop' = attempt only package
# 'mlp' = multilib package
# 'lgp' = language package
mip,basicquick
mip,bluez5
mip,bridge-utils
mip,hostapd
mip, bla bla bla etc etc etc.
And if I take a look in: '~/builds/pyqt5_try1/poky/rpi64-build/tmp/work/cortexa72-poky-linux/basicquick/0.1-r0', the expected files do show up. Also suggesting that it does atleast build the application. This makes the think that the 'install' arguments in my .bb file are pointing to the wrong folder.
In my basicquick_*.bb file this is to make the directory and install (copy) the built files:
install -d ${D}${bindir_native}
install -m 0755 BasicQuick ${D}${bindir_native}
I used the 'bitbake -e' command to trace the variable D and bindir_native:
D="/home/mats/builds/pyqt5_try1/poky/rpi64-build/tmp/work/raspberrypi4_64-poky-linux/qt5-kms-rpi-image/1.0-r0/image"
bindir_native="/usr/bin"
This seems okay at first glance, but when I manually follow the destination of the variable 'D', there is no 'images' folder created. I also wonder why everybody installs their custom applications on ${D}/usr/bin? Should this not be written to the build/tmp/deploy directory? Or am I missing a step here.
So, in this 'qt5-kms-rpi-image/1.0-r0' folder, there is a folder called 'deploy-qt5-kms-rpi-image-image-complete', which contains a rootfs. But there also is a rootfs folder in the 'qt5-kms-rpi-image/1.0-r0' folder. Both of these rootfs's do not contain any mention of my basicquick application, or a BasicQuick folder being created in /usr/bin.
Also, the rootfs found in the "build/tmp/deploy/images/raspberrypi4-64/qt5-kms-rpi-image-raspberrypi4-64.tar.bz2" does not contain any mention of the basicquick application being present in the filesystem.
Does anybody have any clues on what I am missing? Am I just not copying my files to the correct location? Or does the final deploy image end up somewhere else from where I am expecting it?
Thanks in advance.
With kind regards,
Mats de Waard

Linking and LOADING static .lib with mex

So, I have a MEX gateway script file that calls my C source code. I've used the -L and -I commands to link my 64-bit compiled GSL libraries (.libs) to my mex executable, which is then compiled under the extension of .mexw64.
I want for this executable to be transferred to another windows machine and run fine, without any GSL libraries installed. That is the the only solution, I don't care what he arguments are regarding the benefits of the dynamic linking/code generation upon compile-time are. I want an executable that has every function not only (of course) pre-declared, but also PRE-DEFINED.
I was lead to believe that this is what 'static' linking is vs. dynamic; but I've read some contradictory definitions all around the interwebs. I need a completely 100% standalone, singular file.
Supposedly you can link the actual .obj file in the mex function, which I can generate, but unfortunately I then get unresolved symbol errors.
Someone else mentioned that I can use the -l (lowercase L) to directly link the actual .lib(s) needed, statically, but that is NOT true.
So is there anyone that can lead me in the right direction, either how to have everything not only linked but to also have the DEFINITIONS linked and ready to load when executable is run--completely standalone, or why I am running into unresolved symbols/linker errors when I include my .obj file? Am I misunderstanding something elementary about the linking process?
Also: To elaborate a bit more, I have the GSL libraries built and linked via Visual Studio for the 64 bit architecture, and I can link it easily with MATLAB, so that is not my problem (any more).
EDIT: I've seen the post here:
Generating standalone MEX file with GNU compilers, including libraries
This doesn't solve my problem, however, although it is the same question. I don't have access to gcc; it's finally compiling on the MSVS12 compiler in MATLAB, I'm not going try to recompile using GCC via MinGW (already tried, couldn't figure it out), so -static and .a options are out.
In your previous post, you mentioned that you decided to compile GSL library with Visual C++, using the VS solution provided by Brian Gladman.
Here is a step-by-step illustration on how to build a MEX-function that links against GSL libraries statically:
Download GNU GSL sources (GSL v1.16)
Download the matching Visual Studio project files (VS2012 for GSL v1.16)
Extract the GSL tarball, say to C:\gsl-1.16
Extract the VS project files on top of the sources, this will overwrite three files as well as add a folder C:\gsl-1.16\build.vc11.
Open Visual Studio 2012, and load the solution: C:\gsl-1.16\build.vc11\gsl.lib.sln
Change the configuration to the desired output: for me I chose platform=x64 and mode=Release
First you must build the gslhdrs project first
Now build the whole solution. This will create two static libraries cblas.lib and gsl.lib stored in C:\gsl-1.16\lib\x64\Release (along with corresponding PDB debugging symbols). It will also create a directory containing the final header files: C:\gsl-1.16\gsl
Next we proceed to build a MEX-function. Take the following simple program (computes some value from a Bessel function, and return it as output):
gsl_test.c
#include "mex.h"
#include <gsl/gsl_sf_bessel.h>
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
if (nrhs != 0 || nlhs > 1) mexErrMsgTxt("Wrong number of args.");
plhs[0] = mxCreateDoubleScalar(gsl_sf_bessel_J0(5.0));
}
This is how to compile the above C code in MATLAB:
>> mex -largeArrayDims gsl_test.c -I"C:\gsl-1.16" -L"C:\gsl-1.16\lib\x64\Release" cblas.lib gsl.lib
Finally we test the MEX-file, and compare it against the value reported by MATLAB's own Bessel function:
>> x = gsl_test()
ans =
-0.1776
>> y = besselj(0,5)
y =
-0.1776
>> max(x-y) % this should be less than eps
ans =
8.3267e-17
Note that the built MEX-function has no external DLL dependencies (other than "Visual C Runtime" which is expected, and the usual MATLAB libraries). You can verify that by using Dependency Walker if you want. So you can simply deploy the gsl_test.mexw64 file alone (assuming the users already have the corresponding VC++ runtime installed on their machines).

How to run clojure from matlab

How can I run a clojure script from matlab?
I tried following:
run matlab with jdk 1.7 and then call java
MATLAB_JAVA=/usr/lib/jvm/java-7-oracle/jre matlab
in the matlab, set classpath and use clojure compiler
javaaddpath([pwd '/lib/clojure-1.5.1.jar'])
import clojure.lang.RT
Here I got error:
Error using import
Import argument 'clojure.lang.RT' cannot be found or cannot be imported.
When I writing java class that runs clojure, everything working from console, but whould not run from matlab.
please advice.
It looks like this is a problem with Clojure not being happy running from Matlab's "dynamic classpath". I got the same error with Matlab R2014a on OS X 10.9, using either the bundled JVM or Java 1.7.0u51. But if I add clojure-1.5.1.jar to the static classpath by putting it in a custom javaclasspath.txt in the Matlab startup directory, then the Clojure classes become visible.
>> version -java
ans =
Java 1.7.0_51-b13 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
>> cloj = clojure.lang.RT
cloj =
clojure.lang.RT#77de6590
Hacking the Java classpath
You use the "class path hacking" approach in this answer to add entries to the static classpath from the Matlab command line and not have to muck around with a custom Matlab setup. The answer there involves writing a new Java class, but you can do the equivalent in pure M-code.
function javaaddpathstatic(file)
%JAVAADDPATHSTATIC Add an entry to the static classpath at run time
%
% javaaddpathstatic(file)
%
% Adds the given file to the STATIC classpath. This is in contrast to the
% regular javaaddpath, which adds a file to the dynamic classpath.
%
% Files added to the path will not show up in the output of
% javaclasspath(), but they will still actually be on there, and classes
% from it will be picked up.
%
% Caveats:
% * This is a HACK and bound to be unsupported.
% * You need to call this before attempting to reference any class in it,
% or Matlab may "remember" that the symbols could not be resolved.
% * There is no way to remove the new path entry once it is added.
parms = javaArray('java.lang.Class', 1);
parms(1) = java.lang.Class.forName('java.net.URL');
loaderClass = java.lang.Class.forName('java.net.URLClassLoader');
addUrlMeth = loaderClass.getDeclaredMethod('addURL', parms);
addUrlMeth.setAccessible(1);
sysClassLoader = java.lang.ClassLoader.getSystemClassLoader();
argArray = javaArray('java.lang.Object', 1);
jFile = java.io.File(file);
argArray(1) = jFile.toURI().toURL();
addUrlMeth.invoke(sysClassLoader, argArray);
So, use this javaaddpathstatic() instead of javaaddpath() and your code might work.

SUN benchmark code Invalid .mex file error

I've downloadad the SUN benchmark code from MIT:
version 2:
http://people.csail.mit.edu/jxiao/SUN/source_code/
Once I download the 15 image dataset and run the function:
compute_features.m
I get an Invalid MEX file error:
Invalid MEX-file
'/home/arturo/Documents/Virality/reddit_data/SUN_source_code_v2/code/OpenCVmexWrapper/cvlib_mex.mexa64':
libcv.so.1: cannot open shared object file: No such file or directory
The file /home/arturo/Documents/Virality/reddit_data/SUN_source_code_v2/code/OpenCVmexWrapper/cvlib_mex.mexa64 actually does exist, so I'm thinking the problem hast to do with libcv.so.1, or the .mexa64 file is broken (which is very unlikely)
I read something about adding a path via LD_LIBRARY_PATH when I googled the error, but I'm not sure what path I should exactly add, at least temporarily. I have opencv installed along with its library, so I'm not sure what the problem is. My current system is Ubuntu 12.04 64bit.
There's a similar question here, but the answer says that for their specific case they have the source code of the MEX files posted, which is not the same fortune I have: running old mex file on new matlab releases
Found the answer. The key is to link explicitly to the folder the file 'libcv.so.1' is in just before running matlab.
so in the matlab folder:
LD_LIBRARY_PATH=/home/arturDocuments/Virality/reddit_data/SUN_source_code_v2/code/scene_sun ./matlab
And in matlab:
matlab>> compute_feature
Compling the source code in the other folders is also necessary.(vlfeat,VOC,libsvm). As well as editing the path in the function im2superpixels # GeometricContext_dhoiem from :
segcmd = '../segment_pedro/segment 0.8 100 100';
to:
segcmd = '/YOUR_PATH/SUN_source_code_v2/code/segment_pedro/segment/ 0.8 100 100';
*(The folder is one level above.)