I am really a newbie in matlab programming. I have a problem in coding to import multiple csv files into one from certain folder:
This is my code:
%% Importing multiple CSV files
myDir = uigetdir; %gets directory
myFiles = dir(fullfile(myDir,'*.csv')); %gets all csv files in struct
for k = 1:length(myFiles)
data{k} = csvread(myFiles{k});
end
I use the code uigetdir in order to be able to select data from any folder, because I try to make an automation program so it would be flexible to use by others. The code that I run only look for the directory and shows the list, but not for merging the csv files into one and read it in "import data". I want it to be merged and read as one file.
My merged file should look like this with semicolon delimited and consist of 47 csv files merged together (this picture is one of the csv file I have):
my merged file
I have been working for it a whole day but I find always error code. Please help me :(. Thank you very much in advance for your help.
As the error message states, you're attempting to reference myFiles as a cell array when it is not. The output of dir is a structure, which cannot be indexed like a cell array.
You want to do something like the following:
for k = 1:numel(myFiles)
filepath = fullfile(myFiles(k).folder, myFiles(k).name);
data{k} = csvread(filepath);
end
Related
The goal of my code is to look into a certain folder and create a new text file with a list of names of all the files that aren't empty in that folder written to a new file, and the list of names of all the empty files (no text) into another folder. My current code is only able to create a new text file with a list of names of all the files (regardless of its content) written to a new file. I want to know how to set up if statement regarding the content of the file (array).
function ListFile
dirName = '';
files = dir(fullfile(dirName,'*.txt'));
files = {files.name};
[fid,msg] = fopen(sprintf('output.txt'),'w+t');
assert(fid>=0,msg)
fprintf(fid,'%s\n',files{:});
fclose(fid);
EDIT: The linked solution in Stewie Griffin's comment is way better. Use this!
A simple approach would be to iterate all files, open them, and check their content. Caveat: If you have large files, this approach might be memory intensive.
A possible code for that could look like this:
function ListFile
dirName = '';
files = dir(fullfile(dirName, '*.txt'));
files = {files.name};
fidEmpty = fopen(sprintf('output_empty_files.txt'), 'w+t');
fidNonempty = fopen(sprintf('output_nonempty_files.txt'), 'w+t');
for iFile = 1:numel(files)
content = fileread(files{iFile})
if (isempty(content))
fprintf(fidEmpty, '%s\n', files{iFile});
else
fprintf(fidNonempty, '%s\n', files{iFile});
end
end
fclose(fidEmpty);
fclose(fidNonempty);
I have two non-empty files nonempty1.txt and nonempty2.txt as well as two empty files empty1.txt and empty2.txt. Running this code, I get the following outputs.
Debugging output from fileread:
content =
content =
content = Test
content = Another test
Content of output_empty_files.txt:
empty1.txt
empty2.txt
Content of output_nonempty_files.txt:
nonempty1.txt
nonempty2.txt
Matlab isn't really the optimal tool for this task (although it is capable). To generate the files you're looking for, a command line tool would be much more efficient.
For example, using GNU find you could do
find . -type f -not -empty -ls > notemptyfiles.txt
find . -type f -empty -ls > emptyfiles.txt
to create the text files you desire. Here's a link for doing something comparable using the windows command line. You could also call these functions from within Matlab if you want to using the system command. This would be much faster than iterating over the files from within Matlab.
My matlab function is in a folder that contains the main project and the other functions of the code. However, the data is stored in a folder withing the main one named "data" and inside the specific dataset that i want, for example "ded4" in this example. I can't figure out how to open the text file that I want without changing the file to the main folder. The code I have so far is:
function[Classify] = Classify(logDir)
%%%%logDir='ded014a04';
Directory = ['data/' logDir '/']
Filename = [logDir '-fixationsOffSet']
File_name = fullfile(Directory,Filename)
File = fopen(File_name,'r')
end
The code is in the 'dev' folder, I think my path is correct because when I do
open(File_name)
it opens.
Thanks for the help
If you want to open the file in the editor, use
open(File_name)
If you want to read data from the file, you can use
dlmread(File_name) % Read ASCII delimited file.
or
C = textscan(File,'FORMAT') % Read formatted data from text file or string.
or more low-level using fscanf, e.g., if the file contains three columns of integers you do the following: Read the values in column order, and transpose to match the appearance of the file: (from the help of fprintf)
fid = fopen('count.dat');
A = fscanf(fid,'%d',[3,inf])';
fclose(fid);
I want to read multiple files from a folder but this code does not work properly:
direction=dir('data');
for i=3:length(direction)
Fold_name=strcat('data\',direction(i).name);
filename = fullfile(Fold_name);
fileid= fopen(filename);
data = fread (fileid)';
end
I modified your algorithm to make it easier
Just use this form :
folder='address\datafolder\' ( provide your folder address where data is located)
then:
filenames=dir([folder,'*.txt']); ( whatever your data format is, you can specify it in case you have other files you do not want to import, in this example, i used .txt format files)
for k = 1 : numel(filenames)
Do your code
end
It should work. It's a much more efficient method, as it can apply to any folder without you worrying about names, number order etc... Unless you want to specify certain files with the same format within the folder. I would recommend you to use a separate folder to put your files in.
In case of getting access to all the files after reading:
direction=dir('data');
for i=3:length(direction)
Fold_name=strcat('data\',direction(i).name);
filename = fullfile(Fold_name);
fileid(i)= fopen(filename);
data{i-2} = fread (fileid(i))';
end
I have several data log files (here: 34) for those I have to calculate some certain values. I wrote a seperate function to publish the results of the calculation in a pdf file. But I only can publish one file after another, so it takes a while to publish all 34 files.
Now I want to automize that with a loop - importing the data, calculate the values and publish the results for every log file in a new pdf file. I want 34 pdf files for every log file at the end.
My problem is, that I couldn't find a way to rename the pdf files during publishing. The pdf file is always named after the script which is calculating the values. Obviously the pdf is overwritten within a loop. So at the end everything is calculated, but I only have the pdf from the last calculated log file.
There was this hacky solution to change the Matlab publish script, but since I don't have admin rights I can't use that:
"This is really hacky, but I would modify publish to accept a new option prefix. Replace line 93
[scriptDir,prefix] = fileparts(fullPathToScript);
with
if ~isfield(options, 'prefix')
[scriptDir,prefix] = fileparts(fullPathToScript);
else
[scriptDir,~] = fileparts(fullPathToScript);
prefix = options.prefix; end
Now you can set options.prefix to whatever filename you want. If you want to be really hardcore, make the appropriate modifications to supplyDefaultOptions and checkOptionFields as well."
Any suggestions?
Thanks in advance,
Martin
Here's one idea using movefile to rename the resultant published PDF on each iteration:
for i = 1:34
file = publish(files(i)); % Replace with your own command(s)
[pathStr,fileName,ext] = fileparts(file);
newFile = [pathStr filesep() fileName '_' int2str(i) ext]; % Example: append _# to each
[success,msg,msgid] = movefile(file,newFile);
if ~success
error(msgid,msg);
end
end
Also used are fileparts and filesep. See this question for other ways to rename and move files.
I have 4 folders in the same directory where each folder contains ~19 .xls files. I have written the code below to obtain the name of each of the folders and the name of each .xls file within the folders.
path='E:\Practice';
folder = path;
dirListing = dir(folder);
dirListing=dirListing(3:end);%first 2 are just pointers
for i=1:length(dirListing);
f{i} = fullfile(path, dirListing(i,1).name);%obtain the name of each folder
files{i}=dir(fullfile(f{i},'*.xls'));%find the .xls files
for j=1:length(files{1,i});
File_Name{1,i}{j,1}=files{1,i}(j,1).name;%find the name of each .xls file
end
end
Now I'm trying to import the data from excel into matlab by using xlsread. What I'm struggling with is knowing how to load the data into matlab within a loop where the excel files are in different directories (different folders).
This leaves me with a 1x4 cell named File_Name where each cell refers to a different folder located under 'path', and within each cell is then the name of the spreadsheets wanting to be imported. The size of the cells vary as the number of spreadsheets in each folder varies.
Any ideas?
thanks in advance
I'm not sure if I'm understanding your problem, but all you have to do is concatenate the strings that contain directory (f{}) and the file name. Modifying your code:
for i=1:length(dirListing);
f{i} = fullfile(path, dirListing(i,1).name);%obtain the name of each folder
files{i}=dir(fullfile(f{i},'*.xls'));%find the .xls files
for j=1:length(files{1,i});
File_Name{1,i}{j,1}=files{1,i}(j,1).name;%find the name of each .xls file
fullpath = [f{i} '/' File_Name{1,i}{j,1}];
disp(['Reading file: ' fullpath])
x = xlsread(fullpath);
end
end
This works on *nix systems. You may have to join the filenames with a '\' on Windows. I'll find a more elegant way and update this posting.
Edit: The command filesep gives the forward or backward slash, depending on your system. The following should give you the full path:
fullpath = [f{i} filesep File_Name{1,i}{j,1}];
Take a look at this helper function, written by a member of the matlab community.
It allows you to recursively search through directories to find files that match a certain pattern. This is a super handy function to use when looking to match files.
You should be able to find all your files in a single call to this function. Then you can loop through the results of the rdir function, loading the files one at a time into whatever data structure you want.