I'm currently attempting to load a number of MATLAB files which all contain the same variable in order to make a matrix of all the values.
These files all start with a number (i.e. 40_analysed.mat), which was previously extracted from different raw data files using regular expressions, meaning I have a vector composed of all the individual numbers (id).
When I try to load the values and display the data for all individuals in a single matrix using the code below, the files aren't loaded alphanumerically (i.e. according to id), instead appearing to be loaded randomly.
file = dir('*_analysed.mat');
for i=1:length(id);
load(file(i).name,'means');
overallThresholds{i} = means;
end
overallMeans = cell2mat(overallThresholds)
How could I do this so the resulting matrix would be in the correct order? Apologies if this question doesn't make much sense, the problem is a little hard to articulate!
If your filenames don't have a fixed-precision number (as #FakeDIY points out, that would mean they would already be sorted), you could do something like this:
file = dir('*_analysed.mat');
overalThresholds = cell(1, length(id));
IDs = zeros(1, length(id));
for i = 1:length(id)
fileName = file(i).name;
IDs(i) = str2double( strrep( fileName, '_analysed.mat', '' ) );
data = load(fileName, 'means');
overallThresholds{i} = data.means;
end
[~, reordering] = sort(IDs);
overallThresholds = overallThresholds(reordering);
In other words, store the file ID in a separate array as you're going along, and then reorder overallThresholds to be in sorted order of IDs using the second output of SORT.
(I also pre-allocated the arrays, and use the functional form of LOAD, but you don't really need to do that).
When one uses dir command, it is not promised that the results will be in alphabetical order. In fact, the manual says explicitly that:
dir lists the files and folders in the MATLAB current folder. Results
appear in the order returned by the operating system.
Even if you did get this in alphabetical order, nothing assures you that you will get it next time. Thus, you must order the results from dir using sort command.
[~,order] = sort( {file.name} );
file = file(order);
Related
I am reading in a binary EDF file and I have to split it into multiple smaller EDF files at specific points and then adjust some of the values inside. Overall it works quite well but when I read in the file it combines 2 character arrays with each other. Obviously everything afterwords gets corrupted as well. I am at a dead end and have no idea what I'm doing wrong.
The part of the code (writing) that has to contain the problem:
byt=fread(fid,8,'*char');
fwrite(tfid,byt,'*char');
fwrite(tfid,fread(fid,44));
%new number of records
s = records;
fwrite(tfid,s,'*char');
fseek(fid,8,0);
%test
fwrite(tfid,fread(fid,8,'*char'),'*char');
When I use the reader it combines the records (fwrite(tfid,s,'*char'))
with the value of the next variable. All variables before this are displayed correctly. The relevant code of the reader:
hdr.bytes = str2double(fread(fid,8,'*char')');
reserved = fread(fid,44);%#ok
hdr.records = str2double(fread(fid,8,'*char')');
if hdr.records == -1
beep
disp('There appears to be a problem with this file; it returns an out-of-spec value of -1 for ''numberOfRecords.''')
disp('Attempting to read the file with ''edfReadUntilDone'' instead....');
[hdr, record] = edfreadUntilDone(fname, varargin);
return
end
hdr.duration = str2double(fread(fid,8,'*char')');
The likely problem is that your character array s does not have 8 characters in it, but you expect there to be 8 when you read it from the file. Whatever the number of characters in the array is, that's how many values fwrite will write out to the file. Anything less than 8 characters and you'll end up reading part of the next piece of data when you read from the file.
One fix would be to pad s with blanks before writing it:
s = [blanks(8-numel(records)) records];
In addition, the syntax '*char' is only valid when using fread: the * indicates that the output class should be 'char' as well. It's unnecessary when using fwrite.
So I have a directory that has many .mat files:
apples.mat, oranges.mat, bananas.mat, grapes.mat, apricots.mat, pears.mat, pineapple.mat
All of these .mat files has a variable name "calories" assigned a value. How do I load all of these .mat files simultaneously in MATLAB and change the variables for each one from calories to calories_(fruit name) so that I can have all the variable values in the workspace to play with?
For example, I want to load apples.mat and change its variable name from calories to calories_apple, then load oranges.mat and change is variable name from calories_orange, etc. without doing it all manually.
I know it's something along the lines of creating a string with the different fruit names, and the indexing along the string to load a file and change its variable from variable to variable_%s with the %s indicating the fruit content generated along the loop. It's a big mess for me, however, I don't think I'm structuring it right. Would anyone care to help me out?
I would load each .mat file in sequence and put each of the corresponding calories as a separate field being all combined into a single struct for you to access. Given the directory of where these .mat files appear, do something like this:
%// Declare empty structure
s = struct()
folder = '...'; %// Place directory here
%// Get all MAT files in directory
f = dir(fullfile(folder, '*.mat'));
%// For each MAT file...
for idx = 1 : numel(f)
%// Get absolute path to MAT file - i.e. folder/file.mat
name = fullfile(folder, f(idx).name);
%// Load this MAT file into the workspace and get the calories variable
load(name, 'calories');
%// Get the name of the fruit, are all of the characters except the last 4 (i.e. .mat)
fruit_name = f(idx).name(1:end-4);
%// Place corresponding calories of the fruit in the structure
s.(['calories_' fruit_name]) = calories;
end
You can then access each of the calories like so using dot notation:
c = s.calories_apple;
d = s.calories_orange;
...
...
... and so on.
I am assuming that you do not mean to include the parenthesis in calories_(fruit name). I am also assuming there are no other .mat files in your current directory. This should do the trick:
theFiles = dir('*.mat');
for k = 1:length(theFiles)
load(theFiles(k).name, 'calories');
eval(['calories_' theFiles(k).name(1:length(theFiles(k).name)-4) ' = calories;'])
clear calories
end
Let me know if this helps or not.
EDIT
As, rayryeng points out. The use of eval is, apparently, a bad practice. So, if you are willing to change the way you are thinking about the problem, I suggest you use a structure. In which case, rayryeng's response would be an acceptable answer, even though it does not directly answers your original question.
I am trying to read all the images in the folder in MATLAB using this code
flst=dir(str_Expfold);
But it shows me output like this. which is not the sequence as i want.
Can anyone please tell me how can i read all of them in sequence?
for giving downmark, please explain the reason for that too.
By alphabetical order depth10 comes before depth2. If at all possible, when creating string + num type filenames, use a fixed width numerical part (e.g. depth01, depth02) - this tends to avoid sorting problems.
If you are stuck with the filenames you have, and know the filename pattern, though, you can not bother using dir at all and create your filename list in the correct order in the first place:
for n = 1:50
fname = sprintf('depth%d.png',n);
% code to read and process images goes here
end
From the Matlab forums, the dir command output sorting is not specified, but it seems to be purely alphabetical order (with purely I mean that it does not take into account sorter filenames first). Therefore, you would have to manually sort the names. The following code is taken from this link (you probably want to change the file extension):
list = dir(fullfile(cd, '*.mat'));
name = {list.name};
str = sprintf('%s#', name{:});
num = sscanf(str, 'r_%d.mat#');
[dummy, index] = sort(num);
name = name(index);
I have hundreds of data files, named file001~file400 for example, and should pick a numeric value from each. I know how to pick each number, but, since there are too many files, I need to loop the command and order all extracted numbers with regard to the number of corresponding file. I appreciate any help.
The problem is to loop over all files. This can be done in the following way:
for i = 1:400
filename = sprintf('file%03d',i);
// do the number picking, etc. using the filename.
end
EDIT: Per request, for filenames FT00100 to FT05320, we we make two small changes, one in the loop range and one in the first parameter to sprintf:
for i = 100:5320
filename = sprintf('FT%05d',i);
// do the number picking, etc. using the filename.
end
I am little stuck on a problem. I have tons of files generated daily and I need to sort them by file name and date. I need to do this so my MATLAB script can read them. I currently do this manually, but was wondering if there is a easier way in MATLAB to sort and copy files.
My file names look like:
data1_2009_12_12_9.10
data1_2009_12_12_9.20
data1_2009_12_12_9.30
data1_2009_12_12_9.40
data2_2009_12_12_9.10
data2_2009_12_12_9.20
data2_2009_12_12_9.30
data2_2009_12_12_9.40
data3_2009_12_12_9.10
data3_2009_12_12_9.20
data3_2009_12_12_9.30
data3_2009_12_12_9.40
...
and tons of files like this.
Addition to above problem :
There has to be a easier way to stitch the files together.
I mean copy file
' data1_2009_12_12_9.20' after file 'data1_2009_12_12_9.10' and so on ,...
such that i am left with a huge txt file in end named data1_2009_12_12 ( or what ever ). containing all the data stitched together.
Only way now i know to do is open all files with individual dlmread command in matlab and xls write one after another ( or more trivial way of copy paste manually )
Working in the field of functional imaging research, I've often had to sort large sets of files into a particular order for processing. Here's an example of how you can find files, parse the file names for certain identifier strings, and then sort the file names by a given criteria...
Collecting the files...
You can first get a list of all the file names from your directory using the DIR function:
dirData = dir('your_directory'); %# Get directory contents
dirData = dirData(~[dirData.isdir]); %# Use only the file data
fileNames = {dirData.name}; %# Get file names
Parsing the file names with a regular expression...
Your file names appear to have the following format:
'data(an integer)_(a date)_(a time)'
so we can use REGEXP to parse the file names that match the above format and extract the integer following data, the three values for the date, and the two values for the time. The expression used for the matching will therefore capture 6 "tokens" per valid file name:
expr = '^data(\d+)\_(\d+)\_(\d+)\_(\d+)\_(\d+)\.(\d+)$';
fileData = regexp(fileNames,expr,'tokens'); %# Find tokens
index = ~cellfun('isempty',fileData); %# Find index of matches
fileData = [fileData{index}]; %# Remove non-matches
fileData = vertcat(fileData{:}); %# Format token data
fileNames = fileNames(index); %# Remove non-matching file names
Sorting based on the tokens...
You can convert the above string tokens to numbers (using the STR2DOUBLE function) and then convert the date and time values to a date number (using the function DATENUM):
nFiles = size(fileData,1); %# Number of files matching format
fileData = str2double(fileData); %# Convert from strings to numbers
fileData = [fileData zeros(nFiles,1)]; %# Add a zero column (for the seconds)
fileData = [fileData(:,1) datenum(fileData(:,2:end))]; %# Format dates
The variable fileData will now be an nFiles-by-2 matrix of numeric values. You can sort these values using the function SORTROWS. The following code will sort first by the integer following the word data and next by the date number:
[fileData,index] = sortrows(fileData,1:2); %# Sort numeric values
fileNames = fileNames(index); %# Apply sort to file names
Concatenating the files...
The fileNames variable now contains a cell array of all the files in the given directory that match the desired file name format, sorted first by the integer following the word data and then by the date. If you now want to concatenate all of these files into one large file, you could try using the SYSTEM function to call a system command to do this for you. If you are using a Windows machine, you can do something like what I describe in this answer to another SO question where I show how you can use the DOS for command to concatenate text files. You can try something like the following:
inFiles = strcat({'"'},fileNames,{'", '}); %# Add quotes, commas, and spaces
inFiles = [inFiles{:}]; %# Create a single string
inFiles = inFiles(1:end-2); %# Remove last comma and space
outFile = 'total_data.txt'; %# Output file name
system(['for %f in (' inFiles ') do type "%f" >> "' outFile '"']);
This should create a single file total_data.txt containing all of the data from the individual files concatenated in the order that their names appear in the variable fileNames. Keep in mind that each file will probably have to end with a new line character to get things to concatenate correctly.
An alternative to what #gnovice suggested is to loop over the file names and use sscanf() to recover the different sections in the filenames you are interested in:
n = sscanf(filename, 'data%d_%d_%d_%d_%d.%d')
n(1) %# data number
n(2) %# the year
...
Example:
files = dir('data*'); %# list all entries beginning with 'data'
parts = zeros(length(files), 6); %# read all the 6 parts into this matrix
for i=1:length(files)
parts(i,:) = sscanf(files(i).name, 'data%d_%d_%d_%d_%d.%d')'; %'#transposed
end
[parts idx] = sortrows(parts, [6 1]); %# sort by one/multiple columns of choice
files = files(idx); %# apply the new order to the files struct
EDIT:
I just saw your edit about merging those files. That can be done easily from the shell. For example lets create one big file for all data from the year 2009 (assuming it makes sense to stack files on top of each other):
on Windows:
type data*_2009_* > 2009.backup
on Unix:
cat data*_2009_* > 2009.backup
In Matlab the function call
files = dir('.');
returns a structure (called files) with fields
name
date
bytes
isdir
datenum
You can use your usual Matlab techniques for manipulating files.names.