I have a main code named as process.m in which I define the path to 4 different .csv files for calculating values for each person. If I have a list of 30 persons and I don't want to define process.m as a function for each person, how can I do the processing for all the persons in one go. I want some idea by which process.m itself picks files for one person, then generate the results, then move to other person, pick his .csv files, generate the result and so on.
A breif outline of my code is attached here that would project the problem.
file1='Joseph_front.csv';
file2='Joseph_side.csv';
file3='Joseph_knee.csv';
file4='Joseph_back.csv';
A1=initiate2(file1); %initiate2 function reads the csv and perfoms some filtering operations on image in .csv format
A2=initiate2(file2);
A3=initiate2(file3);
A4=initiate2(file4);
%%omitted large part of code
cal(1) = p+q+r*s;
cal(2) = p+q+r+s;
cal(3) = p+q+r-s;
cal=cal'
%code to write the calculation in excel file
excelfile= 'test.xlsx';
xlswrite(excelfile,ValuesInInches,'Joseph_data',posColumn);
Describing more i want my code to process for 30 people all at once by selecting and picking the files itself, although i have done this operation by making the same code as a function for each person, but that is not very efficient as when I have to make a small change I have to make it in every function that means one change needs to be edited in 30 functions. Please suggest an efficient way to do it.
Note: All persons .csv files are named in the same manner and exist in the current folder.
I am assuming all of your files in one directory and there's no other files .
This portion of code will get the available filenames.
listFiles = dir('path of the directory');
filenames = strings; % an empty string array to save the filenames
j = 1;
for i = 1:1:length(listFiles)
if ~listFiles(i).isdir % to avoid the directory names
filenames(j,1) = listFiles(i).name;
j = j+1;
end
end
Now, there's 4 file for each person. So the loop should take 4 files at a time.
for ii = 1:4:length(filenames)
file1=filename(i);
file2=filename(i+1);
file3=filename(i+2);
file4=filename(i+3);
%% continue with your code
end
Related
I have a list of .txt datafiles to import. Suppose they are called like that
file100data.txt file101data.txt ... file109data.txt I want to import them all using readtable.
I tried using the for to specify a vector a = [0:9] through which matlab could loop the readtable command but I cannot make it work.
for a = [0:9]
T_a_ = readtable('file10_a_data.txt')
end
I know I cannot just put _a_ where I want the vector to loop through, so my question is how can I actually do it?
Thank you in advance!
Here is a solution that should work even if you have missing files in your folder (e.g. you have file100data.txt to file107data.txt, but you are missing file file108data.txt and file109data.txt):
files=dir('file10*data.txt'); %list all data files in your folder
nof=size(files,1); %number of files
for i=1:nof %loop over the number of files
table_index=files(i).name(7) %recover table index from data filename
eval(sprintf('T%s = readtable(files(i).name)', table_index)); %read table
end
Now, please note that is it generally regarded as poor practice to dynamically name variables in Matlab (see this post for example). You may want to resort to structures or cells to store your data.
You need to convert the value of a into a string and combine strings together, like this:
Tables = struct();
for a = 0:9
% note: using dynamic structure field names to store the imported tables
fname = ['file10_' num2str(a) '_data'];
Tables.(fname) = readtable([fname '.txt']);
end
I am trying to move my files and create a new folder to put those files there.
I have many .png files in my images folder in my MATLAB directory. I want to randomly choose 80% of them and move them to another folder called training folder in my matlab directory. Heres my code which is not working. it cant find the file to move :(
data_add = fullfile(cd,'images');
all_files = dir(data_add);
all_files = all_files(3:end);
num_files = numel(all_files);
image_order = randperm(num_files);
for k = 1:(image_order)*0.8
file_name = all_files(k).name;
file_add = all_files(k).folder;
file_to_move = fullfile('path_to_images\images',file_name);
mkdir training
movefile file_to_move training
end
A couple issues here:
Like Flynn comments, the call to mkdir training only needs to be made once, so you can place it before your loop.
You may be thinking about the variable image_order incorrectly when it comes to your for loop.
The call image_order = randperm(num_files); produces an array of randomly ordered indices from 1:num_files, which is helpful. However, the expression (image_order)*0.8 is actually multiplying each of these indices times 0.8, such that they are no longer valid, integer indices (aside from a few, like 8 which would become 1).
I think what you are attempting and wanting to do is this:
mkdir('training');
for k = 1:num_files*0.8
randK = image_order(k);
file_name = all_files(randK).name;
file_to_move = fullfile(data_add,file_name);
movefile(file_to_move, 'training'); % using function style
end
You may run into other issues next depending on where the folder training is located and where you are running your script from, but this should be closer to what you are to get, and at least locate the files for you.
This question already has answers here:
Dynamically name a struct variable in MATLAB
(2 answers)
Closed 6 years ago.
I have a folder full of csv data files that I want to import to MATLAB. The import function will be the same for each of them. Their names follow a pattern, e.g. 'a0-b0.csv', 'a0-b5.csv', 'a0-b10.csv', 'a0-b15.csv' ... 'a2-b0' etc.
I want to import all these files at once. I want to import them as tables with sensible names that correspond to the data they contain.
Doing something like
filepath = 'file/path/'
for (a = [0, 2])
for (b = [0:5:50])
filename = strcat(filepath, 'a', num2str(a), '-b', num2str(b), '.csv')
varname = strcat('a', num2str(a), '_b', num2str(b))
varname = importfile(filename, startrow, endrow)
end
end
made sense in concept to me.
As 'varname' itself is the variable, not the string it contains, this does not do what I want. I've seen a lot of answers to similar situations suggesting eval(), while simultaneously vehemently advocating against it.
eval() has side effects that make it annoying to implement. Everyone's intense aversion to it make me wonder if it's worth putting in the effort to try to implement it.
I can think of no other way to do this, however. I don't see how using arrays (the recommended alternative) would result in appropriate/relevant names.
Does anyone have a suggestion as to how to automatically import many csv files to tables in MATLAB with names that correspond to their filenames?
(Regarding duplicates: I haven't yet found a question that addresses the automatic creation of corresponding variable names, which is the main issue here)
As #transversality condition wrote in their comment, you should rethink you approach.
I'd suggest two steps:
If you are about to change files in and import htem again, create .mat file with already imported data and check hwther to rescan the folder or load the .mat file.
Decide whether you want to use struct class or cell class to contain your data.
The code for struct can be:
filepath = 'file/path/'; % define the data folder
FileNames=dir('filepath\*.csv'); % list all csv. files within filepath
N=numel(FileNames); % count the csv files
DATA(N)=struct('name','','data',[]); % preallocate DATA
for (ii=1:N)
Temp=regexp(FileNames(ii).name,'\.','split'); % separate file name and abbreviation
DATA(ii).name = Temp{1}; % save the file name
DATA(ii).data = importfile([filepath,'\',FileNames(ii).name], startrow, endrow); % save the data
end
The code for cell can be:
filepath = 'file/path/'; % define the data folder
FileNames=dir('filepath\*.csv'); % list all csv. files within filepath
N=numel(FileNames); % count the csv files
DATA=cell(2,N); % preallocate DATA
for (ii=1:N)
Temp=regexp(FileNames(ii).name,'\.','split'); % separate file name and abbreviation
DATA{1,ii} = Temp{1}; % save the file name
DATA{2,ii} = importfile([filepath,'\',FileNames(ii).name], startrow, endrow); % save the data
end
In both cases you have to find the appropriate line by, for example, DATA(find(strcmpi(DATA.name,'<name>))).data.
You can also use the cell approach to create struct. Suppose we've run the code for cell.
Then command
DataStruct=struct(DATA{:});
will allow you to access your data via Data.Struct.<filename> command directly.
I am new to MATLAB and try to run a loop within a loop. I define a variable ID beforehand, for example ID={'100'}. In my loop, I then want to go to the ID's directory and then load the matfile in there. However, whenever I load the matfile, suddenly the ID definition gets overridden by all possible IDs (all folders in the directory where also the ID 100 is). Here is my code - I also tried fullfile, but no luck so far:
ID={'100'}
for subno=1:length(ID) % first loop
try
for sessno=1:length(Session) % second loop, for each ID there are two sessions
subj_name = ([ID{subno} '_' Session{sessno} '_vectors_SC.mat']);
cd(['C:\' ID{subno} '\' Session{sessno}]);
load(subj_vec_name) % the problem occurs here, when loading, not before
end
end
end
When I then check the length of the IDs, it is now not 1 (one ID, namely 100), but there are all possible IDs within the directory where 100 also lies, and the loop iterates again for all other possible IDs (although it should stop after ID 100).
You should always specify an output to load to prevent over-writing variables in your workspaces and polluting your workspace with all of the contents of the file. There is an extensive discussion of some of the strange potential side-effects of not doing this here.
Chances are, you have a variable named ID inside of the .mat file and it over-writes the ID variable in your workspace. This can be avoided using the output of load.
The output to load will contain a struct which can be used to access your data.
data = load(subj_vec_name);
%// Access variables from file
id_from_file = data.ID;
%// Can still access ID from your workspace!
ID
Side Note
It is generally not ideal to change directories to access data. This is because if a user runs your script, they may start in a directory they want to be in but when your program returns, it dumps them somewhere unexpected.
Instead, you can use fullfile to construct a path to the file and not have to change folders. This also allows your code to work on both *nix and Windows systems.
subj_name = ([ID{subno} '_' Session{sessno} '_vectors_SC.mat']);
%// Construct the file path
filepath = fullfile('C:', ID{subno}, Session{sessno}, subj_name);
%// Load the data without changing directories
data = load(filepath);
With the command load(subj_vec_name) you are loading the complete mat file located there. If this mat file contains the variable "ID" it will overwrite your initial ID.
This should not cause your outer for-loop to execute more than once. The vector 1:length(ID) is created by the initial for-loop and should not get overwritten by subsequent changes to length(ID).
If you insert disp(ID) before and after the load command and post the output it might be easier to help.
I have data stored in the .tdms format, gathering the data of many sensors, measured every second, every day. A new tdms file is created every day, and stored in a folder per month. Using the convertTDMS function, I have converted these tdms files to mat files.
As there are some errors in some of the measurements(e.g. negative values which can not physically occur), I have performed some corrections by loading one mat file at a time, do the calculations and then save the data into the original .mat file.
However, when I try to do what I described above in a loop (so: load .mat in folder, do calculations on one mat file (or channel therein), save mat file, repeat until all files in the folder have been done), I end up running into trouble with the limitations of the save function: so far I save all variables (or am unable to save) in the workspace when using the code below.
for k = 1:nFiles
w{k,1} = load(wMAT{k,1});
len = length(w{k,1}.(x).(y).(z));
pos = find(w{k,1}.(x).(y).(z)(1,len).(y)<0); %Wind speed must be >0 m/s
for n = 1:length(pos)
w{k,1}.(x).(y).(z)(1,len).(y)(pos(n)) = mean([w{k,1}.(x).(y).(z)(1,len).(y)(pos(n)+1),...
w{k,1}.(x).(y).(z)(1,len).(y)(pos(n)-1)],2);
end
save( name{k,1});
%save(wMAT{k,1},w{k,1}.(x),w{k,1}.ConvertVer,w{k,1}.ChanNames);
end
A bit of background information: the file names are stored in a cell array wMAT of length nFiles in the folder. Each cell in the cell array wMAT stores the fullfile path to the mat files.
The data of the files is loaded and saved into the cell array w, also of length nFiles.
Each cell in "w" has all the data stored from the tdms to mat conversion, in the format described in the convertTDMS description.
This means: to get at the actual data, I need to go from the
cell in the cell array w{k,1} (my addition)
to the struct array "ConvertedData" (Structure of all of the data objects - part of convertTDMS)
to the struct array below called "Data" (convertTDMS)
to the struct array below called "MeasuredData" (convertTDMS) -> at this level, I can access the channels which store the data.
to finally access/manipulate the values stored, I have to select a channel, e.g. (1,len), and then go via the struct array to the actual values (="Data"). (convertTDMS)
In Matlab format, this looks like "w{1, 1}.ConvertedData.Data.MeasuredData(1, len).Data(1:end)" or "w{1, 1}.ConvertedData.Data.MeasuredData(1, len).Data".
To make typing easier, I took
x = 'ConvertedData';
y = 'Data';
z = 'MeasuredData';
allowing me to write instead:
w{k,1}.(x).(y).(z)(1,len).(y)
using the dot notation.
My goal/question: I want to load the values stored in a .mat file from the original .tdms files in a loop to a cell array (or if I can do better than a cell array: please tell me), do the necessary calculations, and then save each 'corrected' .mat file using the original name.
So far, I have gotten a multitude of errors from trying a variety of solutions, going from "getfieldnames", trying to pass the name of the (dynamically changing) variable(s), etc.
Similar questions which have helped me get in the right direction include Saving matlab files with a name having a variable input, Dynamically Assign Variables in Matlab and http://www.mathworks.com/matlabcentral/answers/4042-load-files-containing-part-of-a-string-in-the-file-name-and-the-load-that-file , yet the result is that I am still no closer than doing manual labour in this case.
Any help would be appreciated.
If I understand your ultimate goal correctly, I think you're pretty much there. I think you're trying to process your .mat files and that the loading of all of the files into a cell array is not a requirement, but just part of your solution? Assuming this is the case, you could just load the data from one file, process it, save it and then repeat. This way you only ever have one file loaded at a time and shouldn't hit any limits.
Edit
You could certainly make a function out of your code and then call that in a loop, passing in the file name to modify. Personally I'd probably do that as I think it's neater solution. If you don't want to do that though, you could just replace w{k,1} with w then each time you load a file w would be overwritten. If you wanted to explicitly clear variables you can use the clear command with a space separated list of variables e.g. clear w len pos, but I don't think that this is necessary.