I am using Matlab to do some image processing on a cluster that uses Sun Grid Engine. On my personal laptop the code runs fine but when I run it on the cluster I get several errors of files that cannot be found. For example a .nii (nifti) file that exists (I can read it when I run matlab interactively in the shell) is not found. An excerpt from the output log:
{^HError using load_nii_ext (line 97)
Cannot find file
"/path/imageFile.nii".
And I also get errors from an xml structured file (that needs to have a .mps extension to be readable by a postprocessing toolbox, which all worked fine on my own laptop). Another excerpt from the output log:
/path/pointSetFile.mps exists
{^HError using readpointsmps (line 24)
Failed to read XML file
/path/pointSetFile.mps.
In this second error message the first line is the output I get from including in the script,
if exist(strcat(folder, fileName), 'file') == 2
disp([strcat(folder, fileName) ' exists'])
end
So it's weird that 1) I can see the files, 2) I can open them manually with Matlab, 3) according to the matlab function exist() they do indeed exist, but when the functions xmlread() and read_niigz() want to open them they suddenly can't be found.
As extra information: I run the scripts with the flags -nodisplay -nodesktop -nosplash, and I currently run the scripts as 2 tasks with the SGE. Memory should be good, I give it 5GB and all my images combined are about 1.5GB.
I'm using absolute paths starting at the root /, have been reading the paths letter by letter about 200 times now and have no clue anymore what's going on.
I have solved the problems now.
#Xiangrui Li pointed out in the comments that the missing .nii files were due to interference with the unzipping, reading and deletion of the .nii and .nii.gz files. That was indeed the problem. Thanks!
I found that the second problem was due to umlauts in the filenames. Apparently there was a difference between how the system and matlab and even other processes involved encode the filenames. Removing the characters with umlauts solved the problem.
Related
I have several Windows sourced p-files internally coded with the '\' file separator that I want to run on Matlab under macOS.
I get errors caused by the '\' because macOS uses '/'.
eg The pfile tries to call a file named "model\xyz' which causes a warning:
"Name is nonexistent or not a directory: model\ "
1) Is there code that I might insert somewhere to recognise the 'model\' call from the pcode file and change it to 'model/' before it is used by MATLAB addpath?
2) Is there a generic fix I could apply to the addpath code?
3) Or better still is there a way to modify the Windows p-file without access to its source code so that it will run under macOS?
There are several things you can do (none of which are particularly easy; listed here in increasing order of how nasty I think they are):
Contact the author of the code and ask them to fix it.
Install an older MATLAB version (R2007b <= ver < R2015b, I think) which allowed debugging (stepping into) p-files within MATLAB, then, assuming that there is some line in the original source code that does
filepath = ['model' '\' 'xyz.m'];
step until you see filepath appear in the workspace (having the wrong path in it), then simply edit the value to the correct path, and hope for the best.
(Essentially the same idea as before, but on newer MATLAB versions, VERY DIFFICULT to pull off) Obtain an external debugger, attach to the MATLAB process, run the p-file and scan the memory for the contents of filepath. When you discover it, change the value to the correct path and detach/disable the debugger.
If this code relies on some external (and open source) function to get (or use) the path, you can modify its behavior to output the string you want. This requires some knowledge about the source code.
Find/make a tool for decoding p-files and fix the resulting source code yourself.
For completeness I describe how my problem was solved:
As suggested by Dev-iL I was eventually able to locate the author and he modified his code. This is the best solution but it took some time and is not always possible.
Based on: https://au.mathworks.com/matlabcentral/answers/117110-dealing-with-and-windows-vs-unix-for-path-definition I located a module (I've forgotten its name) within the Matlab package which handles file calls and intercepted all incoming file path names containing the '\' Windows separator, and replaced them with the always acceptable '/'. This quick and dirty fix worked until solution 1. was obtained.
Thanks to all who responded so quickly to this question.
For this project we have been given code, and will be changing some inputs and assumptions. Thus, I already possess the original codes, but just changing all the creator's file paths to match my own computer is yielding me a lot of trouble. The following, and many variations of, continually yield errors.
load \Users\myname\Library\Documents\...
The error is
Error using load
'Unable to read file
\Users\myname\Library\Documents...'.
No such file or directory.
My files are stored in my Documents. Another person in my group on windows has used
load C:\Users\hisname\Desktop\...
Is there something I'm missing in my line, similar to the C drive but on Mac? Is my code just completely wrong, I'm able to load files in R quite easily, but Matlab is posing a huge hurdle. I have no experience with Matlab and have been asked simply to run this code.
On the Mac, path components are separated by /, not \. Thus, you should type
load /Users/myname/Documents/filename.mat
You can use the location bar at the top of the command window to change to the directory where your file is located, and then you can type
load filename
to load filename.mat.
Also, are you sure you have a Documents directory under Library? Why?
To run code from a file called "my_file.m", than just open your Matlab and type run my_file.m. This will run your script in the Command Window.
The load function is used, if you want to load a .mat file. These are normally files, where variables from your workspace are stored.
I have a script in MATLAB that outputs various files, including NIFTI (MRI images) files and text log files. Sometimes the output files that are created have their file permissions set to no read or write for anyone, including the file owner, and this appears to occur randomly.
This normally isn't a problem unless it occurs with the logs, as it leads MATLAB to endlessly recurse as it tries to write the error to the logs. Unfortunately, I haven't been able to find anyone who has experienced similar behavior from any program, including MATLAB.
The script, input files, and output files are all located on a NAS drive connected to a server running Ubuntu 14.04, so I'm wondering if this is a problem with the script (probably not because it happens intermittently regardless of input), the matlab environment, or the NAS drive.
I'm not sure why yoru fileattributes/permissions are changing, but I know the solution. You want file attributes if you know chmod from unix this should be familiar, if not you will still be fine
something like this will make your files writeable for all user groups on a unix (ubuntu) system
fileattrib('/home/work/results/my_file.log','+w','a')
edit
since these files are non-existant this may work better. Simply use fopen with the 'w+' flag. Which means write mode for appending. The + will also create the file if it doesn't exist(and you have permission to create files in the specified directory)
fid = fopen('my_new_log.txt','w+');
fprintf(fid,'some strings for my file');
fclose(fid);
it is also important you make sure to close the file after you are done.
Assuming a text file with 7718 characters (e.g. the contents cut and pasted from here)
The command fileread('myfile.txt') fails when I launch matlab with matlab -nodesktop but works in desktop mode.
Does anyone have any idea why?
If the number of characters in the text file is reduced to 1981 it works in -nodesktop mode.
Note:I do not know the exact number of characters at which it stops working.
Finally, I am using Matlab 7.8.0 (R2009a) on Windows7.
I can't reproduce this odd behaviour (Matlab 2010b & 2012b, both on Linux).
You could at least try to "debug" this type of behaviour by restricting the problem. The "fileread" routine does some checks on the file name (ischar, isempty), then opens the file (fopen), reads it (fread) and closes it (fclose) - rather simple basic stuff. For reference, type "edit fileread" into the command line of the matlab desktop and try running down the routine step by step.
If "fread" causes the whole thing to crash and if you're bound to this version of Matlab, try implementing a "fix". "fread" accepts a few more parameters than what is used in "fileread" (the file ID and the precision "char" only): FREAD(FID,SIZE,PRECISION,SKIP,MACHINEFORMAT). Specifying more parameters may help (wild guess but worth a try).
I have a fairly simple question that has me stymied. I am trying to run an executable built from a simple C program using MATLAB as a shell, i.e. using the following MATLAB code:
FileName = ['D:\Users\person\Desktop\MATLAB\GUI','\Program.exe &'];
dos(FileName);
The executable correctly begins running, but crashes giving the error:
Debug Assertion Failed!
Program: D:\Users\person\Desktop\MATLAB\GUI\Program.exe
File: f:\\dd\vctools\crt_bld\self_x86\crt\src\fscanf.c
Expression: (stream != NULL)
The programs opens a text file, reads the input, performs math functions, and writes outputs back to another text file. I assume that this error means there is a problem reading from the text file, BUT-- when I run the executable by itself (i.e. Windows Explorer doubleclick), it executes flawlessly, as I would expect.
So, it's only MATLAB pointing to the file location that is causing the crash. Any ideas? Thank you.
Sounds like relative paths are the culprit. The Matlab command is running from whatever directory you've specified within Matlab; cmd runs from root or something like that (don't know much about Dos). That's why it works when you specify absolute paths. Change your Matlab directory to DOS root, and see if it works as originally coded.
Edit: Note that it's the path to whatever file Program.exe is trying to read that's the problem, not the path to Program.exe itself.