I have the following code to load a single matrix file in matlab
filename='1';
filetype='.txt';
filepath='D:\20170913\';
fidi = fopen(strcat(filepath,filename,filetype));
Datac = textscan(fidi, repmat('%f', 1, 640), 'HeaderLines',1,
'CollectOutput',1);
f1 = Datac{1};
sum(sum(f1))
how can I load many files, say 1-100.
Thanks in advance.
Just include everything in a for loop that loops from 1 up to N_files, which is the number of files that you have. I used the function num2str() to convert the index i to a string. I also included the matrix sum in an array file_sums and a loaded_matrices cell array that stores all the loaded matrices. If all the loaded matrices have a known and same dimensions, you could use a 2D array (ex. loaded_matrices = zeros(N_rows, N_columns, N_files); and then load the data as loaded_matrices(:,:,i) = Datac{1};).
% N_files - the number of files that you have
N_files = 100;
file_sums = zeros(1,N_files);
loaded_matrices = cell(1,N_files);
for i=1:1:N_files
filename=num2str(i);
filetype='.txt';
filepath='';
fidi = fopen(strcat(filepath,filename,filetype));
Datac = textscan(fidi, repmat('%f', 1, 640), 'HeaderLines',1,...
'CollectOutput',1);
loaded_matrices{i} = Datac{1};
file_sums(i) = sum(sum(loaded_matrices{i}));
end
Related
I am using Matlab 2019 and I saved a large number of different tables with the same name, each in a .mat-file with a different name. I'd like to load them and save them in a new table together.
I need a table that has the same variables (variable1, variable2, variable3), and numbers but an additional dimension for the different .mat-files. I.e. in the dummy code below the size of the table allData should be: 2 x 3 x 2.
If I simply concatenate them like this [Results1.Inpts; Results2.Inpts], then the table is 4 x 3 and I cannot distinguish the data from the different files.
Also I have a large number of .mat-files, so I would like to automate it like done here (https://www.mathworks.com/matlabcentral/answers/233745-loading-mat-files-with-same-variable-name), which I tried to do for my table below, but it didn't work.
Please find my dummy code below.
%% Create data and .mat files
clc;
clear;
close all;
format long;
Names = {'variable1','variable2','variable3'};
Numbers1 = [2, 3, 4; 3, 4, 5];
Numbers2 = [1, 4, 10; 2, 6, 3];
Inpts = array2table(Numbers1,'VariableNames',Names);
save('Results1.mat');
clear Inpts;
Inpts = array2table(Numbers2,'VariableNames',Names);
save('Results2.mat');
clear;
%% Analyse data
files = dir('*.mat');
allData = table(size(files));
for ii=1:length(files)
aFileData = load(files(ii).name);
allData{ii} = aFileData.Inpts;
end
I think you just need to add a column in each table before concatenation
allData = cell(numel(files),1); % Store the tables in a cell
for ii=1:length(files)
aFileData = load(files(ii).name);
% Add a column
aFileData.FileName(:) = {files(ii).name};
% Stick the modified table in the cell
allData{ii} = aFileData.Inpts;
end
% Merge tables
allData = vertcat( allData{:} );
If you really want to do a 3-dimensional array with the third dimension corresponding to files, you can't use a table, because tables are inherently two-dimensional. Instead, convert your table data back to a matrix after reading it, and then concatenate them along the third dimension.
allData = cell(numel(files), 1);
for i = 1:numel(files)
fileTable = load(files(i).name);
fileMat = table2array(fileTable);
allData{i} = fileMat;
end
data = cat(3, allData{:});
I have a loop that makes a 100x10 matrix in every iteration, i want to save all the matrices of this loop in one matrix. assuming that i have a loop with 5 iterations, i want to have a 500x10 matrix in the end (after appending all the 5 matrices of the loop).
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
....
s = zeros (100, 10);
%doing some stuffs
save('s_all', 's','-append');
end
I have used save('s_all', 's','-append');
but it doesn't append the matrices. How can i do that?
As you can read in the document:
save(filename,variables,'-append') adds new variables to an existing file. If a variable already exists in a MAT-file, then save overwrites it with the value in the workspace.
Therefore, save just adds a variable at the end of the .mat file, not to add at the end of a variable inside the .mat file.
Solution 1:
To write matrix into the file it would be better using dlmwrite likes the following:
dlmwrite(filename,s,'-append');
You can find more details here.
In a complete case you can do:
filename = 's_all.csv';
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
% ...
s = zeros (100, 10);
%doing some stuffs
dlmwrite(filename,s,'-append');
end
Solution 2:
The other solution is each time load the specified matrix, then attach the matrices into it, and then append to the file.
filename = 'file.mat';
% suppose originMatrix is an empty matrix or a matrix with columns size 10
for ii = 1:numfiles
load(filename,'originMatrix');
s = zeros (100, 10);
%doing some stuffs
originMatrix = [originMatrix; s];
save(filename,'originMatrix','-append');
end
Can someone help me to understand how I can save in matlab a group of .csv files, select only the columns in which I am interested and get as output a final file in which I have the average value of the y columns and standard deviation of y axes? I am not so good in matlab and so I kindly ask if someone to help me to solve this question.
Here what I tried to do till now:
clear all;
clc;
which_column = 5;
dirstats = dir('*.csv');
col3Complete=0;
col4Complete=0;
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K)=mean(col4(:));
end
col3Complete(1)=[];
col4Complete(1)=[];
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
m=mean(B,2);
C = reshape (col4Complete,[5000,K]);
S=std(C,0,2);
Now I know that I should compute mean and stdeviation inside for loop, using mean()function, but I am not sure how I can use it.
which_column = 5;
dirstats = dir('*.csv');
col3Complete=[]; % Initialise as empty matrix
col4Complete=[];
avgVal = zeros(length(dirstats),2); % initialise as columnvector
for K = 1:length(dirstats)
[num,txt,raw] = xlsread(dirstats(K).name);
col3=num(:,3);
col4=num(:,4);
col3Complete=[col3Complete;col3];
col4Complete=[col4Complete;col4];
avgVal(K,1)=mean(col4(:)); % 1st column contains mean
avgVal(K,2)=std(col4(:)); % 2nd column contains standard deviation
end
%columnavg = mean(col4Complete);
%columnstd = std(col4Complete);
% xvals = 1 : size(columnavg,1);
% plot(xvals, columnavg, 'b-', xvals, columnavg-columnstd, 'r--', xvals, columnavg+columstd, 'r--');
B = reshape(col4Complete,[5000,K]);
meanVals=mean(B,2);
I didn't change much, just initialised your arrays as empty arrays so you do not have to delete the first entry later on and made avgVal a column vector with the mean in column 1 and the standard deviation in column 1. You can of course add two columns if you want to collect those statistics for your 3rd column in the csv as well.
As a side note: xlsread is rather heavy for reading files, since Excel is horribly inefficient. If you want to read a structured file such as a csv, it's faster to use importdata.
Create some random matrix to store in a file with header:
A = rand(1e3,5);
out = fopen('output.csv','w');
fprintf(out,['ColumnA', '\t', 'ColumnB', '\t', 'ColumnC', '\t', 'ColumnD', '\t', 'ColumnE','\n']);
fclose(out);
dlmwrite('output.csv', A, 'delimiter','\t','-append');
Load it using csvread:
data = csvread('output.csv',1);
data now contains your five columns, without any headers.
I have a huge sparse matrix a and I want to save it in a .csv. I can not call full(a) because I do not have enough ram memory. So, calling dlmwrite with full(a) argument is not possible. We must note that dlmwrite is not working with sparse formatted matrices.
The .csv format is depicted below. Note that the first row and column with the characters should be included in the .csv file. The semicolon in the (0,0) position of the .csv file is necessary too.
;A;B;C;D;E
A;0;1.5;0;1;0
B;2;0;0;0;0
C;0;0;1;0;0
D;0;2.1;0;1;0
E;0;0;0;0;0
Could you please help me to tackle this problem and finally save the sparse matrix in the desired form?
You can use csvwrite function:
csvwrite('matrix.csv',a)
You could do this iteratively, as follows:
A = sprand(20,30000,.1);
delimiter = ';';
filename = 'filecontaininghugematrix.csv';
dims = size(A);
N = max(dims);
% create names first
idx = 1:26;
alphabet = dec2base(9+idx,36);
n = ceil(log(N)/log(26));
q = 26.^(1:n);
names = cell(sum(q),1);
p = 0;
for ii = 1:n
temp = repmat({idx},ii,1);
names(p+(1:q(ii))) = num2cell(alphabet(fliplr(combvec(temp{:})')),2);
p = p + q(ii);
end
names(N+1:end) = [];
% formats for writing
headStr = repmat(['%s' delimiter],1,dims(2));
headStr = [delimiter headStr(1:end-1) '\n'];
lineStr = repmat(['%f' delimiter],1,dims(2));
lineStr = ['%s' delimiter lineStr(1:end-1) '\n'];
fid = fopen(filename,'w');
% write header
header = names(1:dims(2));
fprintf(fid,headStr,header{:});
% write matrix rows
for ii = 1:dims(1)
row = full(A(ii,:));
fprintf(fid, lineStr, names{ii}, row);
end
fclose(fid);
The names cell array is quite memory demanding for this example. I have no time to fix that now, so think about this part yourself if it is really a problem ;) Hint: just write the header element wise, first A;, then B; and so on. For the rows, you can create a function that maps the index ii to the desired character, in which case the complete first part is not necessary.
I have three .mat files, A.mat, B.mat and C.mat. And the content of the .mat files look like this:
![enter image description here][1]
It is in r2 = [rs2,cs2,sortedValues2] format. How can I sort rs2(e.g, 3468, 3909...) of the three
.mat files together in increasing order, and count the number of appearance of each element of rs2?
Anyone can give me any suggestion?
Original
First, you're going to want load each files r2, then pull out it's rs(1, :) value into a column vector, then sort that column vector. Once you have the vector, use logical indexing to determine how many times each element repeats. Finally, attach those tow columns together to attach each element to it's corresponding count. The output vector will have duplicate rows, but I assume that won't be a problem.
allData = [load('A.mat',r2); load('B.mat',r2)l; load('C.mat',r2)];
colVector = allData(:, 1);
sortedVec = sort(colVector);
countVec = zeros(size(sortedVec));
for ii = 1:length(sortedVec)
countVec(ii) = sum(sortedVec==sortedVec(ii));
end
outputVec = [sortedVec, countVec]
Edit
Since your edited question is easy, and almost the same as your original, I'll answer it here. Most of the code is the same, you just need to get the data out of the cell array instead of the files. Like so:
colVector = [yourCellArray{:}];
sortedVec = sort(colVector);
countVec = zeros(size(sortedVec));
for ii = 1:length(sortedVec)
countVec(ii) = sum(sortedVec==sortedVec(ii));
end
outputVec = [sortedVec, countVec]
load A
rs2_1 = r2(:,1);
clearvars r2
load B
rs2_2 = r2(:,1);
clearvars r2
load C
rs2_3 = r2(:,1);
clearvars r2
% to combine
rs2_combined = [rs2_1;rs2_2;rs2_3];
% to sort
rs2_sorted = sort(rs2_combined);
% to count for appearance
rs2_count = hist(rs2_combined, min(rs2_combined):1:max(rs2_combined));
EDIT: using cell arrays
% recreate your situation
R = cell(11,10);
R = cellfun(#(x) [randi(3000,50,1),ones(50,1),ones(50,1)*-.008], R,'UniformOutput', false);
% extract rs2
r = cell2mat( reshape(R, [1,1,11,10]) );
rs2 = reshape( r(:,1,:,:), [50*11*10,1] );
% do what you want
rs2_sorted = sort(rs2);
rs2_count = hist(rs2, min(rs2):1:max(rs2));
note - I assumed you have 50x3 arrays. If merely 50x1, then reshape(R, [1,11,10]) and reshape( r, [50*11*10,1] ); also works.
hist put all numbers into different bins of different values. It's equivalent to do
rs2_scale = min(rs2_combined):1:max(rs2_combined);
rs2_count = zeros(1, length(rs2_scale));
for ii = 1:length(rs2_scale)
rs2_count(ii) = sum( rs2_combined == rs2_scale(ii) );
end
To remove zero-count numbers -
rs2_count(rs2_count==0) = [];
Then you can calculate the probability -
rs2_prob = rs2_count / sum(rs2_count);
Validate this answer by
>> sum(rs2_prob)
ans =
1.0000