I am getting an error when i test CBLAS in CLAPACK. When i run the test code
./xblat2d < dblat2.in
I get the error output:
"TransA must be 111, 112 or 113, but is set to -1Parameter 2 to routine cblas_dgemv was incorrect"
Does anyone know what kind of problem this indicates?
Details
I am trying not to use reference CBLAS, and use ATLAS CBLAS instead. So, i compiled wrapper library libcblaswr.a and changed the line in make.inc to
BLASLIB = ../../libcblaswr.a -L/usr/local/atlas/lib -lcblas -L/usr/local/atlas/lib -latlas
CLAPACK installation suggest doing
BLASLIB = ../../libcblaswr.a -lcblas -latlas
But linker doesn't find cblas and atlas without me using -L option, so i included it.
Details of my setup:
Ubunty Lycid Lynx 10.04
CLAPACK-3.2.1
ATLAS.3.9.51
This appears to be a problem with the CLAPACK test in that it passes the wrong parameters to the BLAS routines. In calling dgemv there is an option for the matrix to operate as itself or as the transpose or as the conjugate transpose, corresponding to 111, 112 or 113 (see line 6 of cblas.h). The code in CLAPACK doesn't set the parameter correctly. I haven't looked deep enough to know if this is an issue with just the test or if the issue runs deeper; but I suspect it's just with the test as I haven't run in to this issue using any CLAPACK code.
I have ignored it because 1) I'm not depending on CLAPACK to test the BLAS routines installed by ATLAS and 2) if there's a deeper issue it will cause an error like this rather than produce invalid results, and I'll track it down then.
And yes, the linker won't find -lcblas and -latlas until you install them somewhere normally searched, this is normal.
Related
Currently I am working a project that has have been using the pathlib library so I can work on my Windows desktop when I need too and on my MacBook Pro. Essentially be able to work between both operating systems. I have not have any issues at all until right now. Here is the set up:
I have a pipeline set up to automatically save a .joblib and a whole lot of .png files that will go to a directory called
output_dir = Path('../Trained_Models/Differential_gene_analysis/A Kidney Cancer Transcriptome Molecular Signature Identifies Tumors with Tumor Thrombus/Models train on TCGA data and test on Rodriguez data/Oct-XX-20XX')
For example, if I want to save a .joblib file under the name RandomForest_TumorThrombus_104.joblib,I would use the command
joblib.dump(model ,output_dir / 'RandomForest_TumorThrombus_104.joblib')
On my MacBook Pro, I have no issues when this is ran, but on Windows it gives me the following error
FileNotFoundError: [Errno 2] No such file or directory: '..\\Trained_Models\\Differential_gene_analysis\\A Kidney Cancer Transcriptome Molecular Signature Identifies Tumors with Tumor Thrombus\\Models train on TCGA data and test on Rodriguez data\\Oct-17-2022\\RandomForest_TumorThrombus_104.joblib'
I have tried to use the .resolve() method to get the absolute path but still gives me the same error. I have tried to experiment to try to see what is goin on such as using os.path.exists(). When using the os.path.exists() method I get True for the follwoing command:
os.path.exists(output_dir)
So it does indeed recognize that the directory exists. The next thing I tried was to rename the file to something like dddddd.joblib and that worked. But I find that only a few names for the file would allow me to save the files. During debug I found that the most recent Traceback occurs here:
with open(filename, 'wb') as f:```
I was wondering if anyone here had any idea what was going on here and how I can fix this issue? Please and Thank you.
The solution was to enable long paths on Windows.
everyone!
I'm trying to parallelize an algorithm that uses mex files from mexopencv (KNearest.m, KNearest_.mexw32).
The program is based vlfeat (vlsift.mex32) + mexopencv (KNearest.m and KNearest_.mexw32).I classify descriptors obtained from images.
All the code is located on the fileshare
\\ LAB-07 \ untitled \ DISTRIB \ (this is the program code)
\\ LAB-07 \ untitled \ + cv (mexopencv)
When I run the program with matlabpool close everything works well.
Then I make matlabpool open (2 computers on 2 cores each. ultimately 4 worker, but now I use for testing only 2 workers on the computer and run the program which)
PathDependencises from fileshare -> \LAB-07\untitled\DISTRIB\ , \LAB-07\untitled+cv
Before parfor loop I train classifier on the local machine
classifiers = cv.KNearest
classifiers.train(Descriptors',Labels','MaxK',1)
Then run parfor
descr=vlsift(img);
PredictClasses = classifiers.predict(descr');
Error
Error in ==> KNearest>KNearest.find_nearest at 173
Invalid MEX-file '\\LAB-07\untitled\+cv\private\KNearest_.mexw32':
The specified module could not be found.
That is KNearest.m finds, but no KNearest_.mexw32. Because KNearest_.mexw32 located in private folder, I changed the code KNearest.m (everywhere where it appeal KNearest_ () changed to cv.KNearest_ (). Example: this.id = сv.KNearest_ ()) and placed in a folder with KNearest_.mexw32 KNearest.m. As a result, get the same error
Immediately after matlabpool open file search on workers
pctRunOnAll which ('KNearest.m')
'KNearest.m' not found.
'KNearest.m' not found.
'KNearest.m' not found.
pctRunOnAll which ('KNearest_.mexw32')
'KNearest_.mexw32' not found.
'KNearest_.mexw32' not found.
'KNearest_.mexw32' not found.
after cd \LAB-07\untitled+cv
pctRunOnAll which ('KNearest.m')
\\LAB-07\untitled\+cv\KNearest.m
\\LAB-07\untitled\+cv\KNearest.m % cv.KNearest constructor
\\LAB-07\untitled\+cv\KNearest.m
>> pctRunOnAll which ('KNearest_.mexw32')
\\LAB-07\untitled\+cv\KNearest_.mexw32
\\LAB-07\untitled\+cv\KNearest_.mexw32
\\LAB-07\untitled\+cv\KNearest_.mexw32
I ran and FileDependecies, but the same result.
I do not know this is related or not, I display during the execution of the program classifiers
after training and before parfor
classifiers =
cv.KNearest handle
Package: cv
Properties:
id: 5
MaxK: 1
VarCount: 128
SampleCount: 9162
IsRegression: 0
Methods, Events, Superclasses
Within parfor before classifiers.predict
classifiers =
cv.KNearest handle
Package: cv
Properties:
id: 5
I tested the file cvtColor.mexw32. I left in a folder only 2 files cvtColor.mexw32 and vl_sift
parfor i=1:2
im1=imread('Copy_of_start40.png');
im_vl = im2single(rgb2gray(im1));
desc=vl_sift(im_vl);
im1 = cvtColor(im1,'RGB2GRAY');
end
The same error, and vl_sift work, cvtColor no...
If the worker machines can see the code in your shared filesystem, you should not need FileDependencies or PathDependencies at all. It looks like you're using Windows. It seems to me that the most likely problem is file permissions. MDCS workers running under a jobmanager on Windows by default run not using your own account (they run using the "LocalSystem" account I think), and so may well simply not have access to files on a shared filesystem. You could try making sure your code is world-readable.
Otherwise, you can add the files to the pool by using something like
matlabpool('addfiledependencies', {'\\LAB-07\untitled\+cv'})
Note that MATLAB interprets directories with a + in as defining "packages", not sure if this is intentional in your case.
EDIT
Ah, re-reading your original post, and your comments below - I suspect the problem is that the workers cannot see the libraries on which your MEX file depends. (That's what the "Invalid MEX-file" message is indicating). You could use http://www.dependencywalker.com/ to work out what are the dependencies of your MEX file, and make sure they're available on the workers (I think they need to be on %PATH%, or in the current directory).
Edric thanks. There was a problem in the PATH for parfor. With http://www.dependencywalker.com/ looked missing files and put them in a folder +cv. Only this method works in parfor.
But predict in parfor gives an error
PredictClasses = classifiers.predict(descr');
??? Error using ==> parallel_function at 598
Error in ==> KNearest>KNearest.find_nearest at 173
Unexpected Standard exception from MEX file.
What() is:..\..\..\src\opencv\modules\ml\src\knearest.cpp:365: error: (-2) The search
tree must be constructed first using train method
I solved this problem by calling each time within parfor train
classifiers = cv.KNearest
classifiers.train(Descriptors',Labels','MaxK',1)
But it's an ugly solution :)
This is very simple to recreate.
If my script foo.py is:
import scipy
Then run:
python pyinstaller.py --onefile foo.py
When I launch foo.exe I get:
WARNING: file already exists but should not: C:\Users\username\AppData\Local\Temp\_MEI86402\Include\pyconfig.h
I've tested a few versions but the latest I've confirmed is 2.1dev-e958e02 running on Win7, Python 2.7.5 (32 bit), Scipy version 0.12.0
I've submitted a ticket with the Pyinstaller folks but haven't heard anything yet. Any clues how to debug this further?
You can hack the spec file to remove the second instance by adding these lines after a=Analysis:
for d in a.datas:
if 'pyconfig' in d[0]:
a.datas.remove(d)
break
The answer by wtobia# worked for me. See https://github.com/pyinstaller/pyinstaller/issues/783
Go to C:\Python27\Lib\site-packages\PyInstaller\build.py
Find the def append(self, tpl): function.
Change if tpl[2] == "BINARY": to if tpl[2] in ["BINARY", "DATA"]:
Expanding upon Ilya's solution, I think this is a little bit more robust solution to modifying the spec file (again place after the a=Analysis... statement).
a.datas = list({tuple(map(str.upper, t)) for t in a.datas})
I only tested this on a small test program (one with a single import and print statement), but it seems to work. a.datas is a list of tuples of strings which contain the pyconfig.h paths. I convert them all to lowercase and then dedup. I actually found that converting all of them all to lowercase was sufficient to get it to work, which suggests to me that pyinstaller does case-sensitive deduping when it should be case-insensitive on Windows. However, I did the deduping myself for good measure.
I realized that the problem is that Windows is case-insensitive and these 2 statements are source directories are "duplicates:
include\pyconfig.h
Include\pyconfig.h
My solution is to manually tweak the .spec file with after the a=Analysis() call:
import platform
if platform.system().find("Windows")>= 0:
a.datas = [i for i in a.datas if i[0].find('Include') < 0]
This worked in my 2 tests.
A more flexible solution would be to check ALL items for case-insensitive collisions.
I ran the archive_viewer.py utility (from PyInstaller) on one of my own --onefile executables that has the same error and found that pyconfig.h is included twice:
(31374007, 6521, 21529, 1, 'x', 'include\\pyconfig.h'),
(31380528, 6521, 21529, 1, 'x', 'Include\\pyconfig.h'),
(31387049, 984, 2102, 1, 'x', 'pytz\\zoneinfo\\CET'),
Sadly though, I don't know how to fix it.
PyInstaller Manual link:
http://www.pyinstaller.org/export/d3398dd79b68901ae1edd761f3fe0f4ff19cfb1a/project/doc/Manual.html#archiveviewer
I am new to S-Functions and Real TIme WorkShop. The S-Function has been created using an Embedded Matlab Function. The S-Function Works fine, but when i try to build the same, i get the following error report:
It appears that the build process was unable to locate some utility (e.g. make, compiler, linker, etc.). Please verify your path and tool environment variables are correct. You should be able to execute the make command: .\Radius_S_func2.bat at an MS DOS Command Prompt in the directory: C:\Users\skaushik\Desktop\Matlab Models\WIP\TESTs\Sfunc_2\Radius_S_func2_Source Currently, this generates the following error:
C:\Users\skaushik\Desktop\Matlab Models\WIP\TESTs\Sfunc_2\Radius_S_func2_Source>set MATLAB=C:\MATLAB\R2010b
C:\Users\skaushik\Desktop\Matlab Models\WIP\TESTs\Sfunc_2\Radius_S_func2_Source>.......\u_Utils\Build\make -f Radius_S_func2.mk GENERATE_REPORT=0 INCLUDE_MDL_TERMINATE_FCN=0 COMBINE_OUTPUT_UPDATE_FCNS=1 MAT_FILE=0 MULTI_INSTANCE_CODE=0 INTEGER_CODE=0 PORTABLE_WORDSIZES=0 GENERATE_ERT_S_FUNCTION=0 GENERATE_ASAP2=0 EXT_MODE=0 EXTMODE_STATIC_ALLOC=0 EXTMODE_STATIC_ALLOC_SIZE=1000000 EXTMODE_TRANSPORT=0 TMW_EXTMODE_TESTING=0 MODELLIB=Radius_S_func2lib.lib RELATIVE_PATH_TO_ANCHOR=.. MODELREF_TARGET_TYPE=NONE OPTS="-DRT -DUSE_RTMODEL -DERT" The system cannot find the path specified.
PLease guide me how to understand the error, so i can take care of it myself.
Thank you!!
Solution: The build target was not specified therefore matlab gave the above mentioned error. To resolve this one should specify the build environment for a specific target.
I installed moodle 1.9.12 and now I want to use Algebra notation in content. I enable "TeX Notation" and "Algebra Notation" in administrator panel and also install mimetext and dvips and Imagemagic on the server. fortunately Tex Notation works fine but I got the following error for Algebra:
sh: /var/www/html/moodle/filter/tex/mimetex.linux: not found
The shell command
"/var/www/html/moodle/filter/tex/mimetex.linux" -e "/var/www/moodledata/filter/algebra/de06d6c44d98ba4e42dffca988bf530b.gif" -- '\Large \frac{\sin\left(z\right)}{x^{2}+y^{2}}'
returned status = 127
File size of mimetex executable /var/www/html/moodle/filter/tex/mimetex.linux is 830675
The file permissions are: 100775
The md5 checksum of the file is 56bcc40de905ce92ebd7b083c76e019e
Image not found!
Note: /var/www/html/moodle/filter/tex/mimetex.linux exists on the server and is executable!!!
What is the problem?? Any idea?????
From what you have described, calling the general tex filter debug page works and does not show up the same error.
/filter/tex/texdebug.php works, but /filter/algebra/algebradebug.php does not.
If this is the case, perhaps you could check for an open_basedir, or safe_mode_exec_dir being set to include the current working directory, or otherwise restricting the execution of /var/www/html/moodle/filter/tex/mimetex.linux, while the current working directory is /var/www/html/moodle/filter/algebra.
You could look at this by visiting /admin/phpinfo.php at your site, and look carefully at the effective values of open_basedir, safe_mode and safe_mode_exec_dir.
You could also check the apache error log or add the following lines to the top of the algebra debug php file, and you might see some extra error messages:
$CFG->debug = 6143 ;
$CFG->debugdisplay= 1 ;
Hope that helps