I have a large number of text files that I have to read, find the max value for a certain column, and the corresponding time. The for loop for finding these values works fine, but my problem is writing a text file that shows the three variables I need (thisfilename, M, and wavetime) for each iteration of the for loop.
Output_FileName_MaxWaveHeights = ['C:\Users\jl44459\Desktop\QGIS_and_Basement\BASEMENT\Mesh_5_2045\Run_A\','MaxWaveHeights.txt'];
writefile = fopen(Output_FileName_MaxWaveHeights,'a');
dinfo = dir('*.dat');
for K = 1 : length(dinfo)
thisfilename = dinfo(K).name; %just the name of the file
fileID = fopen(thisfilename); %creates numerical ID for the file name
thisdata = textscan(fileID,'%f64%f64%f64%f64%f64%f64%f64',500,'HeaderLines',1); %load just this file
thisdataM = cell2mat(thisdata); %transforms file from cell array to matrix
[M,I] = max(thisdataM(:,5)); %finds max WSE and row it's in
wavetime = 2*(I-1); %converts column of max WSE to time
fprintf(writefile,'%s %8.4f %4.0f \r\n',thisfilename,M,wavetime);
fclose(fileID); %closes file to make space for next one
end
The text file ends up just giving me the values for one iteration instead of all of them. I was able to use displaytable as a workaround, but then I have problems writing "thisfilename", which includes non-numerical characters.
Although I am not able to reproduce the issue with the code provided, a possible solution might be to write to the file outside of the loop and to close the file afterwards:
Output_FileName_MaxWaveHeights = ['C:\Users\jl44459\Desktop\QGIS_and_Basement\BASEMENT\Mesh_5_2045\Run_A\','MaxWaveHeights.txt'];
writefile = fopen(Output_FileName_MaxWaveHeights,'a');
s = [];
dinfo = dir('*.dat');
for K = 1 : length(dinfo)
thisfilename = dinfo(K).name; %just the name of the file
fileID = fopen(thisfilename); %creates numerical ID for the file name
thisdata = textscan(fileID,'%f64%f64%f64%f64%f64%f64%f64',500,'HeaderLines',1); %load just this file
thisdataM = cell2mat(thisdata); %transforms file from cell array to matrix
[M,I] = max(thisdataM(:,5)); %finds max WSE and row it's in
wavetime = 2*(I-1); %converts column of max WSE to time
s = [s, fprintf(writefile,'%s %8.4f %4.0f \r\n',thisfilename,M,wavetime)];
fclose(fileID); %closes file to make space for next one
end
fprintf(writefile,s);
fclose(writefile);
Solved--it was simply me forgetting to close the output file after the loop. Thanks for the help!
Related
I gave the error Error using readtable (line 216) Input must be a row vector of characters or string scalar when I tried to run this code in Matlab:
clear
close all
clc
D = 'C:\Users\Behzad\Desktop\New folder (2)';
filePattern = fullfile(D, '*.xlsx');
file = dir(filePattern);
x={};
for k = 1 : numel(file)
baseFileName = file(k).name;
fullFileName = fullfile(D, baseFileName);
x{k} = readtable(fullFileName);
fprintf('read file %s\n', fullFileName);
end
% allDates should be out of the loop because it's not necessary to be in the loop
dt1 = datetime([1982 01 01]);
dt2 = datetime([2018 12 31]);
allDates = (dt1 : calmonths(1) : dt2).';
allDates.Format = 'MM/dd/yyyy';
% 1) pre-allocate a cell array that will store
% your tables (see note #3)
T2 = cell(size(x)); % this should work, I don't know what x is
% the x is xlsx files and have different sizes, so I think it should be in
% a loop?
% creating loop
for idx = 1:numel(x)
T = readtable(x{idx});
% 2) This line should probably be T = readtable(x(idx));
sort = sortrows(T, 8);
selected_table = sort (:, 8:9);
tempTable = table(allDates(~ismember(allDates,selected_table.data)), NaN(sum(~ismember(allDates,selected_table.data)),size(selected_table,2)-1),'VariableNames',selected_table.Properties.VariableNames);
T2 = outerjoin(sort,tempTable,'MergeKeys', 1);
% 3) You're overwriting the variabe T2 on each iteration of the i-loop.
% to save each table, do this
T2{idx} = fillmissing(T2, 'next', 'DataVariables', {'lat', 'lon', 'station_elevation'});
end
the x is each xlsx file from the first loop. my xlsx file has a different column and row size. I want to make the second loop process for all my xlsx files in the directory.
did you know what is the problem? and how to fix it?
Readtable has one input argument, a filename. It returns a table. In your code you have the following:
x{k} = readtable(fullFileName);
All fine, you are reading the tables and storing the contents in x. Later in your code you continue with:
T = readtable(x{idx});
You already read the table, what you wrote is basically T = readtable(readtable(fullFileName)). Just use T=x{idx}
I have a loop that makes a 100x10 matrix in every iteration, i want to save all the matrices of this loop in one matrix. assuming that i have a loop with 5 iterations, i want to have a 500x10 matrix in the end (after appending all the 5 matrices of the loop).
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
....
s = zeros (100, 10);
%doing some stuffs
save('s_all', 's','-append');
end
I have used save('s_all', 's','-append');
but it doesn't append the matrices. How can i do that?
As you can read in the document:
save(filename,variables,'-append') adds new variables to an existing file. If a variable already exists in a MAT-file, then save overwrites it with the value in the workspace.
Therefore, save just adds a variable at the end of the .mat file, not to add at the end of a variable inside the .mat file.
Solution 1:
To write matrix into the file it would be better using dlmwrite likes the following:
dlmwrite(filename,s,'-append');
You can find more details here.
In a complete case you can do:
filename = 's_all.csv';
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
% ...
s = zeros (100, 10);
%doing some stuffs
dlmwrite(filename,s,'-append');
end
Solution 2:
The other solution is each time load the specified matrix, then attach the matrices into it, and then append to the file.
filename = 'file.mat';
% suppose originMatrix is an empty matrix or a matrix with columns size 10
for ii = 1:numfiles
load(filename,'originMatrix');
s = zeros (100, 10);
%doing some stuffs
originMatrix = [originMatrix; s];
save(filename,'originMatrix','-append');
end
I have a huge sparse matrix a and I want to save it in a .csv. I can not call full(a) because I do not have enough ram memory. So, calling dlmwrite with full(a) argument is not possible. We must note that dlmwrite is not working with sparse formatted matrices.
The .csv format is depicted below. Note that the first row and column with the characters should be included in the .csv file. The semicolon in the (0,0) position of the .csv file is necessary too.
;A;B;C;D;E
A;0;1.5;0;1;0
B;2;0;0;0;0
C;0;0;1;0;0
D;0;2.1;0;1;0
E;0;0;0;0;0
Could you please help me to tackle this problem and finally save the sparse matrix in the desired form?
You can use csvwrite function:
csvwrite('matrix.csv',a)
You could do this iteratively, as follows:
A = sprand(20,30000,.1);
delimiter = ';';
filename = 'filecontaininghugematrix.csv';
dims = size(A);
N = max(dims);
% create names first
idx = 1:26;
alphabet = dec2base(9+idx,36);
n = ceil(log(N)/log(26));
q = 26.^(1:n);
names = cell(sum(q),1);
p = 0;
for ii = 1:n
temp = repmat({idx},ii,1);
names(p+(1:q(ii))) = num2cell(alphabet(fliplr(combvec(temp{:})')),2);
p = p + q(ii);
end
names(N+1:end) = [];
% formats for writing
headStr = repmat(['%s' delimiter],1,dims(2));
headStr = [delimiter headStr(1:end-1) '\n'];
lineStr = repmat(['%f' delimiter],1,dims(2));
lineStr = ['%s' delimiter lineStr(1:end-1) '\n'];
fid = fopen(filename,'w');
% write header
header = names(1:dims(2));
fprintf(fid,headStr,header{:});
% write matrix rows
for ii = 1:dims(1)
row = full(A(ii,:));
fprintf(fid, lineStr, names{ii}, row);
end
fclose(fid);
The names cell array is quite memory demanding for this example. I have no time to fix that now, so think about this part yourself if it is really a problem ;) Hint: just write the header element wise, first A;, then B; and so on. For the rows, you can create a function that maps the index ii to the desired character, in which case the complete first part is not necessary.
I am trying to copy the results to a matrix and want the output in a 32768*8 array. This is the code I am using, but it stops working after the last line.
As you can see for the first file ( i=1), the decimal data,T(32768*1) is converted to M(32768*8). Now I want this M to be stored for each iteration of i, without overwriting anything.
Files_list = getAllFiles('C:\Stellaris Measurements\Stellaris-LM4F120_all');
for i = 1:15000
B=num2str(cell2mat(Files_list(i)));
fid = fopen(B,'rb');
T= fread(fid,inf,'uint8','ieee-be');
total = numel(T);
%M=textread('C:\Users\admin\Workspace\STELLARIS-LM4F120_00_210214_104000_0001_temp_025.bin','%2c');
%M=dec2bin(M);
M= de2bi(T,8,'left-msb');
M = measure(i);
end
So, basically I want to create a martix for each of the measurement, which will store the converted binary results in a 32768*8 array.
Thanks!
BR,
\Kashif
I do matlab operation with two data file whose entries are complex numbers. For example,
fName = '1corr.txt';
f = dlmread('1EA.txt',',');
fid = fopen(fName);
tline = '';
Then I do matrix and other operations between these two files and write my output which I call 'modTrace' as:
modTrace
fileID = fopen('1out.txt','w');
v = [(0:(numel(modTrace)-1)).' real(modTrace(:)) ].';
fprintf(fileID,'%d %e\n',v);
The question is, if I have for example 100 pairs of such data files, like (2corr.txt, 2EA.txt), ....(50corr.txt, 50EA.txt) how can I generalize the input files and how to write all the output files at a time?
First of all, use sprintf to get your variable names depending on the current index.
corrName=sprintf('%dcorr.txt',idx);
EAName=sprintf('%dEA.txt',idx);
outName=sprintf('%dout.txt',idx);
This way, you have one variable (idx) which has to be changed.
Finally put everything into a loop:
n=100
for idx=1:n
corrName=sprintf('%dcorr.txt',idx);
EAName=sprintf('%dEA.txt',idx);
outName=sprintf('%dout.txt',idx);
f = dlmread(EAName,',');
fid = fopen(corrName);
tline = '';
modTrace
fileID = fopen(outName,'w');
v = [(0:(numel(modTrace)-1)).' real(modTrace(:)) ].';
fprintf(fileID,'%d %e\n',v);
end
Instead of hardcoding the number 100, you could also use n=numel(dir('*EA.txt')). It count's the files ending with EA.txt