I'm batch processing statistical analyses and need to save down graphs as part of the output. I want to include some parameters settings as part of the graphs' filenames, to associate them easier at a later stage.
However, if I the FileName is 'too long' the print command throws errors. MATLAB Help is silent on this issue.
h = figure;
plot(somedata, moredata);
FileName = strcat(Name(1),'_',Name(2),'_',num2str(a),'_',num2str(b),'_','.jpeg');
print(h,'-djpeg',FileName);
Any idea how many chars 'too long' is?
Related
I am very new to MATLAB and i am currently trying to learn how to import files in matlab and work on it. I am importing a "*.dat" file which contains a single column of floating point numbers[they are just filter coefficients I got from a c++ code] into an array in MATLAB. When I am displaying the output in command window the first line is always " 1.0e-03 * " followed by the contents of my file. I want to know what it means? When I check my workspace the array connects the correct number of inputs. My sample code and first few lines of output are below:
Code:-
clear; clc;
coeff = fopen('filterCoeff.dat');
A = fscanf(coeff, '%f');
A
fclose(coeff);
Output:-
A =
**1.0e-03 *** <===== What does this mean?
-0.170194000000000
0
0.404879000000000
0
-0.410347000000000
P.S: I found many options to read file eg. textscan, fscanf etc. Which one is the best to use?
It is a multiplier that applies to all the numbers displayed after that. It means that, for example, the last entry of A is not -0.410347 but -0.410347e-3, that is, -0.000410347.
I think it is is just Matlab's display number type. It means each of your results are scaled by that amount.
format longg
A
And see what it displays. Look at the docs for format for other options.
I'm a Java programmer and have no background of matlab hence I'm really clueless with these lines of code from MATLAB. When I run the code I got an error :
??? Undefined function or variable 'nfile'.
Error in ==> texture_id at 29
fprintf(' \nneural network processing \n',nfile);
I understand that 'path' is a variable that stores string, 'demo' is boolean, but for the other lines, I don't want to assume what it does...Can you please help me and explain each lines?
Here's the code:
path = 'C:\Users\Dais\Documents\MATLAB\Data Sets\';
demo = true;
elfile = dir('*.jpg');
[lu ri] = size(elfile); feat=zeros(lu,29); nomf=cell(lu,1);
for nfi = 1:lu
nfile = elfile(nfi).name;
fprintf(' feature extraction file: %s \n',nfile);
nomf{nfi} = upper(nfile);
feat(nfi,:) = feature_ex([path nfile],demo);
end
fprintf(' \nneural network processing \n',nfile);
I would guess that whats happening here is that elfile = dir('*.jpg'); does not find any jpegs in the local directory and hence lu is empty and nfile is never populated. Place a breakpoint there in the code and check this. The way I would set up the loop would be something like this:
for nfi=1:numel(elfile)
As #Rody Oldenhuis said, use doc and help to elarn more about each function (or press F1 when the cursor is in the function name) but this should get you started..
%Looks for all files with extention .jpg in current directory
elfile = dir('*.jpg');
%lu and ri hold the rows, column lengths of elfile respectively
[lu ri] = size(elfile);
%creates an array of zeros of dimensions lu rows by 29 columns
feat=zeros(lu,29);
%creates an empty cell array (doc cell) dimensions lu rows by 1
nomf=cell(lu,1); columns
for nfi = 1:lu %look through all files
nfile = elfile(nfi).name; %get index nfi file
fprintf(' feature extraction file: %s \n',nfile); %print string
nomf{nfi} = upper(nfile); %upper case
feat(nfi,:) = feature_ex([path nfile],demo); %some external function
end
fprintf(' \nneural network processing \n',nfile); %print string
Rather than explain all and everything about MATLAB, I'll say this: MATLAB is interactive! And, one of the things why you pay good money for MATLAB, is that the documentation is awesome, and getting help is super easy.
For instance, you can type help <command> on the MATLAB command line, and get a short help on that command, or doc <command> to get the complete documentation, often with examples and demonstrations. The whole documentation is also online, should you prefer Google and being in a browser.
Should you have a script or function or class that has problems, you can issue dbstop if error, so that you drop into the debugger when an error occurs, and then you can view the contents of all variables just prior to the error, type new commands to investigate the error, etc. You can set breakpoints by clicking on the line number next to where you want to break, dbstep then makes a single step, dbup moves you up a level, etc. Have a look at doc dbstop.
You can select portions of code and press F9, which will execute those lines of code. Note that that is equivalent to copy-pasting the code to the command window and running it, so you will often have problems with undefined variables (and similar problems) that way (this or something similar is what I suspect happened in your particular case, as the code you posted should not give that error).
I have decided to use memmapfile because my data (typically 30Gb to 60Gb) is too big to fit in a computer's memory.
My data files consist two columns of data that correspond to the outputs of two sensors and I have them in both .bin and .txt formats.
m=memmapfile('G:\E-Stress Research\Data\2013-12-18\LD101_3\EPS/LD101_3.bin','format','int32')
m.data(1)
I used the above code to memory map my data to a variable "m" but I have no idea what data format to use (int8', 'int16', 'int32', 'int64','uint8', 'uint16', 'uint32', 'uint64', 'single', and 'double').
In fact I tried all of the data formats listed that MATLAB supports, but when I used the m.data(index number) I never get a pair of numbers (2 columns of data) which is what I expected, also the number will be different depending on the format I used.
If anyone has experience with memmapfile please help me.
Here are some smaller versions of my data files so people can understand how my data is structured:
cheers
James
memmapfile is designed for reading binary files, that's why you are having trouble with your text file. The data in there is characters, so you'll have to read them as characters and then parse them into numbers. More on that below.
The binary file appears to contain more than just a stream of floating point values written in binary format. I see identifiers (strings) and other things in the file as well. Your only hope of reading that is to contact the manufacturer of the device that created the binary file and ask them about how to read in such files. There'll probably be an SDK, or at least a description of the format. You might want to look into this as the floating point numbers in your text file might be truncated, i.e., you have lost precision compared to directly reading the binary representation of the floats.
Ok, so how to read your file with memmapfile? This post provides some hints.
So first we open your file as 'uint8' (note there is no 'char' option, so as a workaround we read the content of the file into a datatype of the same size):
m = memmapfile('RTL5_57.txt','Format','uint8'); % uint8 is default, you could leave that off
We can render the data read in as uint8 as characters by casting it to char:
c = char(m.Data(1:19)).' % read the first three lines. NB: transpose just for getting nice output, don't use it in your code
c =
0.398516 0.063440
0.399611 0.063284
0.398985 0.061253
As each line in your file has the same length (2*8 chars for the numbers, 1 tab and 2 chars for newline = 19 chars), we can read N lines from the file by reading N*19 values. So m.Data(1:19) gets you the first line, m.Data(20:38), the second line, and m.Data(20:57) the second and third lines. Read as much as you want at once.
Then we'll have to parse the read-in data into floating point numbers:
f = sscanf(c,'%f')
f =
0.3985
0.0634
0.3996
0.0633
0.3990
0.0613
All that's left now is to reshape them into your two column format
d = reshape(f,2,[]).'
d =
0.3985 0.0634
0.3996 0.0633
0.3990 0.0613
Easier ways than using memmapfile:
You don't need to use memmapfile to solve your problem, and I think it makes things more complicated. You can simply use fopen followed by fread:
fid = fopen('RTL5_57.txt');
c = fread(fid,Nlines*19,'*char');
% now sscanf and reshape as above
% NB: one can read the values the text file directly with f = fscanf(fid,'%f',Nlines*19).
% However, in testing, I have found calling fread followed by sscanf to be faster
% which will make a significant difference when reading such large files.
Using this you can read Nlines pairs of values at a time, process them and simply call fread again to read the next Nlines. fread remembers where it is in the file (as does fscanf), so simply use same call to get next lines. Its thus easy to write a loop to process the whole file, testing with feof(fid) if you are at the end of the file.
An even easier way is suggested here: use textscan. To slightly adapt their example code:
Nlines = 10000;
% describe the format of the data
% for more information, see the textscan reference page
format = '%f\t%f';
fid = fopen('RTL5_57.txt');
while ~feof(fid)
C = textscan(fid, format, Nlines, 'CollectOutput', true);
d = C{1}; % immediately clear C at this point if you need the memory!
% process d
end
fclose(fid);
Note again however that the fread followed by sscanf will be fastest. Note however that the fread method would die as soon as there is one line in the text file that doesn't exactly match your format. textscan is forgiving of whitespace changes on the other hand and thus more robust.
i have some data files that i would like to load into matlab. unfortunatly, they have a quite complex structure - at least compared to what i am used to. you should be able to download an old example of this here, https://www.dropbox.com/s/vbh6kl334c5zg1s/fn1_2.out (it opens fine in notepad or wordpad)
it is data files based on synchrotron data where both the raw data, regularized "raw" data and the (indirect) fourier transformed data+fit to data is listed. there are furthermore some statistics from the fourier transformation.
I just need to quote the results from the statistics in my paper, so while it would be nice to plot some of the results, it is not strictly necessary. I need, however, the raw and regularized data together with the fit, and the fourier transformed data.
My problem
in the data file, the results from the statistical analysis is shown before the data i need. but the size of the columns from the statistical analysis varies from data file to data file. this means that i cannot just include the statistics in the header unless i manually change the number of header lines for each file i import. i need to analysis groups of 5 data files together and i would at least need to analyze around 30 files this time so i would like to avoid it if possible. in the future i would again need to load this kind of data files - so even if changing the number of headerlines 30 times does not sound bad it would be nice to be able do it automatically
Possible solution
both the he raw and regularized data together with the fit as well as the fourier transformed data are preceded by a specific line that tells me that after this and a blank/empty line, the data begins
so i though that maybe i could use regular expressions to tell matlab to ignore everything until you see this specific line, ignore this line and one more, and then import data
i googled and found this topic where regular expressions are used: Trying to parse a fairly complex text file
but i am new to regular expressions and the code suggested is a bit complex for me. i can gather that he uses named capture but i am not quite sure i understand how he uses it and if i can adopt it to me need. i have checked the official matlab documentation but their examples are somewhat simpler :) (http://www.mathworks.se/help/matlab/matlab_prog/regular-expressions.html#bqm94nz-1)
Sorry for writing such a long post. any suggestions on how to proceed with this problem will be greatly appreciated
/Martin
EDIT
the code i have used based on the link in the comment:
fileName = 'data.dat';
inputfile = fopen(fileName);
% Ignore all until we see one that just consists of this:
startString = ' R P(R) ERROR';
mydata = [];
while 1
tline = fgetl(inputfile);
% Break if we hit end of file, or the start marker
if ~ischar(tline) || strcmp(tline, startString)
break
end
data = sscanf(tline, '%f', 3 );
mydata(end+1,:) = data;
end
fclose(inputfile);
When i run the code i get the error:
Subscripted assignment dimension mismatch.
mydata(end+1,:) = data;
any suggestions will be greatly appriciated and my apologize for the strange layout/leaving the link in the comment. i am not allowed to include more than two links in a post and i cannot add a new answer yet - both due to me having to low rep :)
Since the blocks are separated by at least two new lines you can use that to separate the text into blocks and analyse them individually. Try this code
fileH = fopen('fn1_2.out');
content = fscanf(fileH, '%c', inf);
fclose(fileH);
splitstring = regexp(content, '\r\n\r\n', 'split');
blocks = regexp(splitstring, '\d\.\d{4}.*\r\n.*\d\.\d{4}','match');
numericBlocksIdx = find(cellfun(#(x) ~isempty(x), blocks));
numericBlocks = splitstring(numericBlocksIdx);
Now the numericBlocks{1}, numericBlocks{2}, ... contain the tables that you are interested in. Note that for some tables the headers are also included because they are not separated by two new lines. From here you can use functions like textscan to read the data into matrices.
Matlab is failing to read in the specified number of elements from a file. I have a simple program that needs to read in two files, perform a linear operation on the data and write a combined result to a third file.
My questions are: 1) Why does Matlab fail to read the specified number of elements and 2) is there a workaround for this? Any of your thoughts will be helpful.
Some details on the input files:
they are large (~18GB)
they are both the same size (exactly)
Details on the procedure (2-4 are conditioned on an feof check of both files:
Open the input and output files for reading and writing (resp.)
Read in N floats (N*4 bytes) from each of the input files
Perform an operation on the data (say 0.5*(datin1+datin2))
Write the result to the output file.
Granted, this is all very simple and in most cases in the past this has worked well. Unfortunately, at some point in the cycle, MATLAB doesn't get all N floats from one of the files and gives a matrix dimension error on step 3.
CODE SNIP:
N = 2048;
fidin1 = fopen('file1.dat','r','l');
fidin2 = fopen('file2.dat','r','l');
fidout = fopen('outfile.dat','w','l');
%# I could do some assertions on the file sizes,
%# but I know they are the same size (w/o question).
while(~feof(fidin1) && ~feof(fidin2))
datin1 = fread(fidin1,N,'float=>single',0,'l');
datin2 = fread(fidin2,N,'float=>single',0,'l');
%# the following line produces an error after 100
%# or more iterations in to the procedure
datout = 0.5*(datin1+datin2);
fwrite(fidout,datout,'float',0,'l');
end
UPDATE 1
The error message I'm receiving is:
???Error using ==> plus
Matrix dimension must agree.
UPDATE 2
I followed a suggestion and included ferrorchecks after each read and magically the problem went away. So now a modification to my questions: What could be the root of the problem here? Is this simply a timing issue or bug?
Here is a snip of the updated code (showing only a portion of the code). I'm sure there are better ways to do this. Regardless, the addition of these checks allowed Matlab to complete all the reads from each of the files successfully.
[datin1 count1]= fread(fidin1,N,'float=>single',0,'l');
[msg errn1]=ferror(fidin1);
if errn1
pos1 = ftell(fidin1);
error('Error at Position %d in file. %d bytes were read.',...
pos1,count1);
end
[datin2 count2]= fread(fidin2,N,'float=>single',0,'l');
[msg errn2]=ferror(fidin2);
if errn2
pos2 = ftell(fidin2);
error('Error at Position %d in file. %d bytes were read.',...
pos2,count2);
end
%# the following line produces an error after 100
%# or more iterations in to the procedure
datout = 0.5*(datin1+datin2);
fwrite(fidout,datout,'float',0,'l');
Have you specifically looked at the datin1 and datin2 variables at the time the error occurs? Try going to 'Debug-->Stop if Errors/Warnings...' then select 'Always stop if error (dstop if error)'. Run your program and then once it crashes, look at datin1 and datin2. Hopefully that will explain why adding them together is now working.