I am trying to copy the results to a matrix and want the output in a 32768*8 array. This is the code I am using, but it stops working after the last line.
As you can see for the first file ( i=1), the decimal data,T(32768*1) is converted to M(32768*8). Now I want this M to be stored for each iteration of i, without overwriting anything.
Files_list = getAllFiles('C:\Stellaris Measurements\Stellaris-LM4F120_all');
for i = 1:15000
B=num2str(cell2mat(Files_list(i)));
fid = fopen(B,'rb');
T= fread(fid,inf,'uint8','ieee-be');
total = numel(T);
%M=textread('C:\Users\admin\Workspace\STELLARIS-LM4F120_00_210214_104000_0001_temp_025.bin','%2c');
%M=dec2bin(M);
M= de2bi(T,8,'left-msb');
M = measure(i);
end
So, basically I want to create a martix for each of the measurement, which will store the converted binary results in a 32768*8 array.
Thanks!
BR,
\Kashif
Related
I have a loop that makes a 100x10 matrix in every iteration, i want to save all the matrices of this loop in one matrix. assuming that i have a loop with 5 iterations, i want to have a 500x10 matrix in the end (after appending all the 5 matrices of the loop).
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
....
s = zeros (100, 10);
%doing some stuffs
save('s_all', 's','-append');
end
I have used save('s_all', 's','-append');
but it doesn't append the matrices. How can i do that?
As you can read in the document:
save(filename,variables,'-append') adds new variables to an existing file. If a variable already exists in a MAT-file, then save overwrites it with the value in the workspace.
Therefore, save just adds a variable at the end of the .mat file, not to add at the end of a variable inside the .mat file.
Solution 1:
To write matrix into the file it would be better using dlmwrite likes the following:
dlmwrite(filename,s,'-append');
You can find more details here.
In a complete case you can do:
filename = 's_all.csv';
for ii = 1:numfiles
str = fullfile(PathName,FileName{ii});
file_id = fopen(str);
data = fread (file_id)';
% ...
s = zeros (100, 10);
%doing some stuffs
dlmwrite(filename,s,'-append');
end
Solution 2:
The other solution is each time load the specified matrix, then attach the matrices into it, and then append to the file.
filename = 'file.mat';
% suppose originMatrix is an empty matrix or a matrix with columns size 10
for ii = 1:numfiles
load(filename,'originMatrix');
s = zeros (100, 10);
%doing some stuffs
originMatrix = [originMatrix; s];
save(filename,'originMatrix','-append');
end
I have a large number of text files that I have to read, find the max value for a certain column, and the corresponding time. The for loop for finding these values works fine, but my problem is writing a text file that shows the three variables I need (thisfilename, M, and wavetime) for each iteration of the for loop.
Output_FileName_MaxWaveHeights = ['C:\Users\jl44459\Desktop\QGIS_and_Basement\BASEMENT\Mesh_5_2045\Run_A\','MaxWaveHeights.txt'];
writefile = fopen(Output_FileName_MaxWaveHeights,'a');
dinfo = dir('*.dat');
for K = 1 : length(dinfo)
thisfilename = dinfo(K).name; %just the name of the file
fileID = fopen(thisfilename); %creates numerical ID for the file name
thisdata = textscan(fileID,'%f64%f64%f64%f64%f64%f64%f64',500,'HeaderLines',1); %load just this file
thisdataM = cell2mat(thisdata); %transforms file from cell array to matrix
[M,I] = max(thisdataM(:,5)); %finds max WSE and row it's in
wavetime = 2*(I-1); %converts column of max WSE to time
fprintf(writefile,'%s %8.4f %4.0f \r\n',thisfilename,M,wavetime);
fclose(fileID); %closes file to make space for next one
end
The text file ends up just giving me the values for one iteration instead of all of them. I was able to use displaytable as a workaround, but then I have problems writing "thisfilename", which includes non-numerical characters.
Although I am not able to reproduce the issue with the code provided, a possible solution might be to write to the file outside of the loop and to close the file afterwards:
Output_FileName_MaxWaveHeights = ['C:\Users\jl44459\Desktop\QGIS_and_Basement\BASEMENT\Mesh_5_2045\Run_A\','MaxWaveHeights.txt'];
writefile = fopen(Output_FileName_MaxWaveHeights,'a');
s = [];
dinfo = dir('*.dat');
for K = 1 : length(dinfo)
thisfilename = dinfo(K).name; %just the name of the file
fileID = fopen(thisfilename); %creates numerical ID for the file name
thisdata = textscan(fileID,'%f64%f64%f64%f64%f64%f64%f64',500,'HeaderLines',1); %load just this file
thisdataM = cell2mat(thisdata); %transforms file from cell array to matrix
[M,I] = max(thisdataM(:,5)); %finds max WSE and row it's in
wavetime = 2*(I-1); %converts column of max WSE to time
s = [s, fprintf(writefile,'%s %8.4f %4.0f \r\n',thisfilename,M,wavetime)];
fclose(fileID); %closes file to make space for next one
end
fprintf(writefile,s);
fclose(writefile);
Solved--it was simply me forgetting to close the output file after the loop. Thanks for the help!
very very new to Matlab and I'm having trouble reading a binary file into a matrix. The problem is I am trying to write the binary file into a two column matrix (which has 100000's of rows) where each column is a different format type.
I want column 1 to be in 'int8' format and column 2 to be a 'float'
This is my attempt so far:
FileID= fopen ('MyBinaryFile.example');
[A,count] = fread(FileID,[nrows, 2],['int8','float'])
This is not working because I get the error message 'Error using fread' 'Invalid Precision'
I will then go on to plot once I have successfully done this.
Probably a very easy solution to someone with matlab experience but I haven't been successful at finding a solution on the internet.
Thanks in advance to anyone who can help.
You should be aware that Matlab cannot hold different data type in a matrix (it can do so in a cell array but this is another topic). So there is no point trying to read your mixed type file in one go in one single matrix ... it is not possible.
Unless you want a cell array, you will have to use 2 different variables for your 2 columns of different type. Once this is established, there are many ways to read such a file.
For the purpose of the example, I had to create a binary file as you described. This is done this way:
%% // write example file
A = int8(-5:5) ; %// a few "int8" data
B = single(linspace(-3,1,11)) ; %// a few "float" (=single) data
fileID = fopen('testmixeddata.bin','w');
for il=1:11
fwrite(fileID,A(il),'int8');
fwrite(fileID,B(il),'single');
end
fclose(fileID);
This create a 2 column binary file, with first column: 11 values of type int8 going from -5 to +5, and second column: 11 values of type float going from -3 to 1.
In each of the solution below, the first column will be read in a variable called C, and the second column in a variable called D.
1) Read all data in one go - convert to proper type after
%% // Read all data in one go - convert to proper type after
fileID = fopen('testmixeddata.bin');
R = fread(fileID,'uint8=>uint8') ; %// read all values, most basic data type (unsigned 8 bit integer)
fclose(fileID);
R = reshape( R , 5 , [] ) ; %// reshape data into a matrix (5 is because 1+4byte=5 byte per column)
temp = R(1,:) ; %// extract data for first column into temporary variable (OPTIONAL)
C = typecast( temp , 'int8' ) ; %// convert into "int8"
temp = R(2:end,:) ; %// extract data for second column
D = typecast( temp(:) , 'single' ) ; %// convert into "single/float"
This is my favourite method. Specially for speed because it minimizes the read/seek operations on disk, and most post calculations are done in memory (much much faster than disk operations).
Note that the temporary variable I used was only for clarity/verbose, you can avoid it altogether if you get your indexing into the raw data right.
The key thing to understand is the use of the typecast function. And the good news is it got even faster since 2014b.
2) Read column by column (using "skipvalue") - 2 pass approach
%% // Read column by column (using "skipvalue") - 2 pass approach
col1size = 1 ; %// size of data in column 1 (in [byte])
col2size = 4 ; %// size of data in column 2 (in [byte])
fileID = fopen('testmixeddata.bin');
C = fread(fileID,'int8=>int8',col2size) ; %// read all "int8" values, skipping all "float"
fseek(fileID,col1size,'bof') ; %// rewind to beginning of column 2 at the top of the file
D = fread(fileID,'single=>single',col1size) ; %// read all "float" values, skipping all "int8"
fclose(fileID);
That works too. It works fine ... but probably much slower than above. Although it may be clearer code to read for someone else ... I find that ugly (and yet I've used this way for several years until I got to use the method above).
3) Read element by element
%% // Read element by element (slow - not recommended)
fileID = fopen('testmixeddata.bin');
C=[];D=[];
while ~feof(fileID)
try
C(end+1) = fread(fileID,1,'int8=>int8') ;
D(end+1) = fread(fileID,1,'single=>single') ;
catch
disp('reached End Of File')
end
end
fclose(fileID);
Talking about ugly code ... that does work too, and if you were writing C code it would be more than ok. But in Matlab ... please avoid ! (well, your choice ultimately)
Merging in one variable
If really you want all of that in one single variable, it could be a structure or a cell array. For a cell array (to keep matrix indexing style), simply use:
%% // Merge into one "cell array"
Data = { C , D } ;
Data =
[11x1 int8] [11x1 single]
I'm trying to write an octave program that will convert a .mat file to a .csv file. The .mat file has a matrix X and a column vector y. X is populated with 0s and 1s and y is populated with labels from 1 to 10. I want to take y and put it in front of X and write it as a .csv file.
Here is a code snippet of my first approach:
load(filename, "X", "y");
z = [y X];
basename = split{1};
csvname = strcat(basename, ".csv");
csvwrite(csvname, z);
The resulting file contains lots of really small decimal numbers, e.g. 8.560596795891285e-06,1.940359477121703e-06, etc...
My second approach was to loop through and manually write the values out to the .csv file:
load(filename, "X", "y");
z = [y X];
basename = split{1};
csvname = strcat(basename, ".csv");
csvfile = fopen(csvname, "w");
numrows = size(z, 1);
numcols = size(z, 2);
for i = 1:numrows
for j = 1:numcols
fprintf(csvfile, "%d", z(i, j));
if j == numcols
fprintf(csvfile, "\n");
else
fprintf(csvfile, ",");
end
end
end
fclose(csvfile);
That gave me a correct result, but took a really long time.
Can someone tell me either how to use csvwrite in a way that will write the correct values, or how to more efficiently manually create the .csv file.
Thanks!
The problem is that if y is of type char, your X vector gets converted to char, too. Since your labels are nothing else but numbers, you can simply convert them to numbers and save the data using csvwrite:
csvwrite('data.txt', [str2num(y) X]);
Edit Also, in the loop you save the numbers using integer conversion %d, while csvwrite writes doubles if your data is of type double. If the zeros are not exactly zeros, csvwrite will write them with scientific notation, while your loop will round them. Hence the different behavior.
Just a heads up your code isn't optimized for Matab / octave. Switch the for i and for j lines around.
Octave is in column major order so its not cache efficient to do what your doing. It will speed up the overall loop by making the change to probably an acceptable time
I have the following script for importing text files into matlab which include hourly data, where I am then trying to convert them into daily averages:
clear all
pathName = ...
TopFolder = pathName;
dirListing = dir(fullfile(TopFolder,'*.txt'));%Lists the folders in the directory specified by pathName.
for i = 1:length(dirListing);
SubFolder{i} = dirListing(i,1).name;%obtain the name of each folder in
%the specified path.
end
%import data
for i=1:length(SubFolder);
rawData1{i} = importdata(fullfile(pathName,SubFolder{i}));
end
%convert into daily averages
rawData2=cell2mat(rawData1);
%create one matrix for entire data set
altered=reshape(rawData2,24,(size(rawData2,2)*365));
%convert into daily values
altered=mean(altered)';
%take the average for each day
altered=reshape(altered,365,size(rawData2,2));
%convert back into original format
My problem lies in trying to convert the data back into the same format as 'rawData1' which was a cell for each variable (where each variable is denoted by 'SubFolder'. The reason for doing this is that all but one of the variables are vectors, where the remaining variable is a matrix (8760*11).
So, an example of this would be:
clear all
cell_1 = rand(8760,1);
cell_2 = rand(8760,1);
cell_3 = rand(8760,1);
cell_4 = rand(8760,1);
cell_5 = rand(8760,1);
cell_6 = rand(8760,11);
cell_7 = rand(8760,1);
cell_8 = rand(8760,1);
cell_9 = rand(8760,1);
data = {cell_1,cell_2,cell_3,cell_4,cell_5,cell_6,cell_7,cell_8,cell_9};
Where I need to convert each cell in 'data' from hourly values into daily averages (i.e. 365 rows).
Any advice would be much appreciated.
I think this does what you want.
data = cellfun(#(x) reshape(mean(reshape(x,24,[]))',365,[]),data,'uniformoutput',false);
However that is kind of confusing so I will explain a little.
This part mean(reshape(x,24,[]))' inside of the cellfun will reshape each cell in data into a 24 by 365, compute the mean, then turn it back into a single column. This works fine when the original data only has 1 column ... but for cell_6 with 11 columns it puts all the data end to end. So I added an addition reshape(...) wrapper around the mean(...) part to put it back into the original 11 columns ... or more precises N columns that are 365 rows in length.
Note: This is going to give you errors if you ever have data sets dimensions are not 8760 by X.