I hope this makes sense:
I have a series of .mat files that i want to use to train a weka classifier and I need to get these .mat files all into one big .CSV file to do so.
My files each contain an eigenvector which hast around 120 values.
Being brand spanking new to this enviroment, i wrote a simple script to concat all these vectos into one long vector, which doesnt really help me.
function c = csv()
for i = 1:99
if i>9
fname = strcat('Betas00', int2str(i));
else
fname = strcat('Betas000', int2str(i));
end
fext = strcat(fname, '.mat');
//fext
filename = sprintf("%s", fext);
load(filename);
dlmwrite('test.txt', fext, '-append');
dlmwrite('test.txt', ignmds, '-append');
end
Its not pretty, but can someone explain how i can use these basic concepts to do what i want? to get a CSV file which has 99 eigenvectors in it? Please? Pretty Please?
Hope that all makes sense!
It looks like you've done it already, what is left to do?
By the way you may want to replace your if with:
fname = strcat('Betas', num2str(i, '%-04.4d'));
Related
I have a piece of MATLAB code that works fine, but I wanted to know is there any faster way of performing the same task, where each .csv file is a 768*768 dimension matrix
Current code:
for k = 1:143
matFileName = sprintf('ang_thresholded%d.csv', k);
matData = load(matFileName);
imshow(matData)
end
Any help in this regard will be very helpful. Thank You!
In general, its better to separate the loading, computational and graphical stuff.
If you have enough memory, you should try to change your code to:
n_files=143;
% If you know the size of your images a priori:
matData=zeros( 768, 768,n_files); % prealocate for speed.
for k = 1:n_files
matFileName = sprintf('ang_thresholded%d.csv', k);
matData(:,:,k) = load(matFileName);
end
seconds=0.01;
for k=1:n_Files
%clf; %Not needed in your case, but needed if you want to plot more than one thing (hold on)
imshow(matData(:,:,k));
pause(seconds); % control "framerate"
end
Note the use of pause().
Here is another option using Matlab's data stores which are designed to work with large datasets or lots of smaller sets. The TabularTextDatastore is specifically for this kind of text based data.
Something like the following. However, note that since I don't have any test files it is sort of notional example ...
ttds = tabularTextDatastore('.\yourDirPath\*.csv'); %Create the data store
while ttds.hasdata %This turns false after reading the last file.
temp = read(ttds); %Returns a Matlab table class
imshow(temp.Variables)
end
Since it looks like your filenames' numbering is not zero padded (e.g. 1 instead of 001) then the file order might get messed up so that may need addressed as well. Anyway I thought this might be a good alternative approach worth considering depending on what else you want to do with the data and how much of it there might be.
I have to read a lot of text files in a folder which only have one matrix in them. So I found and tested that dlmread is good when I write a file's name. But I couldn't have MATLAB read them all.
How can I correct this small code and If there is better way, please write.
files = dir('C:\Users\ABC\Desktop\a\*.txt')
for k = 1:length(files)
fname= files(k).name
M = dlmread('fname') % This part is wrong
% REST OF CODE for each Matrix M
end
well fname really reads the name of file, 1.txt for example. If I write M = dlmread('1.txt') myself, it reads matrix to M, but if I write M = dlmread('fname') it does not.
How can I do this best?
you need:
for k=1:numel(files)
fname = fullfile('C:\Users\ABC\Desktop\a', files(k).name);
M = dlmread(fname);
% ...
end
Two things:
fname is a variable, not a literal string
since the files are not in the current directory, you have to specify the full path
I have 100 data files in a folder called "Experiment1", and I need to take all of the data from them and put them into a single matrix. Each data file contains 15 columns and 40 rows of data.
The order in which the files are in the folder is arbitrary. It doesn't matter in what order they get put into the combined matrix.
I've written some code using dlmread that will do the job:
for i = 1:100
%% Read in the relevant file.
filename = ['File_' int2str(i) '.dat']
Data = dlmread(fullfile(pwd, 'Experiment1',filename));
%% Put all the data I need in a separate matrix
NeededData(1+((i-1)*40):i+((i-1)*40)-i+40,1:15) = Data(:,1:15);
end
However, there are two things I don't like about my code.
The files have random names at present, and I'd need to manually change all their names to "File_1.dat", "File_2.dat", etc.
The code is cumbersome and hard to read.
How could I do things better?
Since you've fixed the problem of defining the name of the files to be read with dir, you can improve the way you add the read data (Data) to the output matrix (NeededData).
You can sumultaneously read the input files and add the data to the output matrix by inserting the call to dlmread directly in the assignment statement:
files=dir('*.dat');
n_files=length(files)
% Initialize the output matrix as empty
NeededData_0=[]
for i=1:n_files
% Simultaneously read input file and assign data to the output matrinx
NeededData_0=[NeededData_0;dlmread(files(i).name)]
end
In case you prefer working with the inides (as in your origina approach), since you know in advance that all the files have the same (40) number of rows) you can simplify the notation as follows:
files=dir('*.dat');
n_files=length(files)
% Define the number of rows in each inout file
n_rows=40;
% Define the number of colums in each inout file
n_col=15;
NeededData_2=nan(n_rows*n_files,n_col)
% Define the sequence of rows
r_list=1:n_rows:n_rows*n_files
for i=1:3
Data=dlmread(files(i).name)
NeededData_2(r_list(i):r_list(i)+n_rows-1,:)=Data
end
Hope this helps.
Using the suggestion to use dir present in the answers I have made the following code, which is clearly an improvement on my earlier effort. I would welcome further improvements, or suggestions for alternative approaches.
files = dir('*.dat');
for i = 1:length({files.name})
%% Read in the relevant file.
Data = dlmread(files(i).name);
%% Put all the data I need in a separate matrix
NeededData(1+((i-1)*40):i+((i-1)*40)-i+40,1:15) = Data(:,1:15);
end
I am quite new to data analysis, so if this is a rookie question, I'm sorry, I am learning as I go.
I have just started doing some work in variable star astronomy. I have about 100 files for every night of observation that all contain the same basic information (star coordinates, magnitude, etc.). I am loading all of the files into my workspace as arrays using a for-loop
files = dir('*.out');
for i=1:length(files)
eval(['load ' files(i).name ' -ascii']);
end
I'm only really interested in two columns in each file. Is there a way to extract a column and set it to a vector while this for-loop is running? I'm sure that it's possible, but the actual syntax for it is escaping me.
try using load as a function and save it's output to a variable
files = dir('*.out');
twoCols = {};
for ii=1:length(files)
data = load( files(ii).name, '-ascii' ); % load file into "data"
twoCols{ii} = data(:,1:2); % take only two columns
end
Now variable twoCols holds the two columns of each file in a different cell.
You have to assign the load result to a new variable. Then if lets say your variable is starsInfo you can use
onlyTwoFirst = starsInfo(:,1:2)
That means take all the rows, but only columns 1 and 2.
My raw CSV file looks like the 1st pic. And I wants to use Matlab read it into the format as the 2rd pic. I have over 1000 the same kind of CSV files, it will be painful if I do it by copy/paste. How can I do this? Any examples?
raw data:
output data:
First thing to realize is that a .csv file has a very simple format. Your above file is actually a plain text file with the following text on each line:
id,A001
height
a1,a2,a3
3,4,5
3,4,5
6,7,5
weight
a1,a2,a3
4,4,5
5,4,6
i6,7,5
So it is not all that hard for you to write your own parser in Matlab. You want to use commands like
fid = fopen('filename.csv','r');
L = fgetl(fid); % get a text line from the file
commas = find(L==','); % find where the commas are in the line
n1 = str2num(L(1:commas(1)-1); % convert the first comma-delimited number on line L
fidout - fopen('myfile.csv','w');
Lout = [ L(commas(2)+1:commas(3)-1) ', a1, a1'];
fwrite(fidout,Lout); % write a line out to the output file
fclose all; % close all open files.
It will seem slow at first reading the various values in to various variables, and then arranging them to write out the way you want them written out to your output file. But once you get rolling it will go pretty fast and you will find yourself with a pretty good understanding of what is in files, and you will know first hand what is involved in writing something like texscan.m or csvwrite.m and so on.
Good luck!