I have a huge arrays in matlab that I am concatenating into a table and then I am dumping that table into a .csv file. Shown below:
% Create manipulated results with noise table
busResultNoise = array2table(busResult_arr); %This command converts the array into a table
% Horizontally concatenate timestamp table and bus data measurements with
% noise table
busResultNoise = horzcat(timeStamp_bus_tab,busResultNoise); %adds the timestamps array
% Assign same variable to busResultNoise as in busResult table
busResultNoise.Properties.VariableNames= busResult.Properties.VariableNames;%adds column names
%Export csv
writetable(busResultNoise,'BusDataWithNoise.csv');
This works great especially if the table/array is small. Is there a faster way to write to .csv file (or may be a text file with commas in it)? I am not required to form a table as shown above but I am doing so because I thought adding timestamp column and adding column names is easier. Please suggest if there is a faster way to do it because this code chokes if the array/table is really large.
Related
I have text file in the single line, I am pasting it in the code section, how can I read it in MATLAB in the table or in cells. Currently, I am doing it with the strsplit function and then saving it in cells and to csv, but it is taking lot of time for a single file, as my original data have thousands of rows.
[{"ID":1,"Column_A":"red","Column_B":"apple","Column_C":0},{"ID":2,"Column_A":"orange","Column_B":"orange","Column_C":1},{"ID":2,"Column_A":"green","Column_B":"grapes","Column_C":3},{"ID":3,"Column_A":"purple","Column_B":"onion","Column_C":4}]
MY technique:
data=importdata('example.txt')
columns = strsplit(data{1,1},"},")
%For concatenation
check=[];
for i=1:159929 %total rows in my original file
check = [check ;strsplit(columns{1,i},{':',','})];
end
Unfortunately I am not too tech proficient and only have a basic MATLAB/programming background...
I have several csv data files in a folder, and would like to make a histogram plot of all of them simultaneously in order to compare them. I am not sure how to go about doing this. Some digging online gave a script:
d=dir('*.csv'); % return the list of csv files
for i=1:length(d)
m{i}=csvread(d(i).name); % put into cell array
end
The problem is I cannot now simply write histogram(m(i)) command, because m(i) is a cell type not a csv file type (I'm not sure I'm using this terminology correctly, but MATLAB definitely isn't accepting the former).
I am not quite sure how to proceed. In fact, I am not sure what exactly is the nature of the elements m(i) and what I can/cannot do with them. The histogram command wants a matrix input, so presumably I would need a 'vector of matrices' and a command which plots each of the vector elements (i.e. matrices) on a separate plot. I would have about 14 altogether, which is quite a lot and would take a long time to load, but I am not sure how to proceed more efficiently.
Generalizing the question:
I will later be writing a script to reduce the noise and smooth out the data in the csv file, and binarise it (the csv files are for noisy images with vague shapes, and I want to distinguish these shapes by setting a cut off for the pixel intensity/value in the csv matrix, such as to create a binary image showing these shapes). Ideally, I would like to apply this to all of the images in my folder at once so I can shift out which images are best for analysis. So my question is, how can I run a script with all of the csv files in my folder so that I can compare them all at once? I presume whatever technique I use for the histogram plots can apply to this too, but I am not sure.
It should probably be better to write a script which:
-makes a histogram plot and/or runs the binarising script for each csv file in the folder
-and puts all of the images into a new, designated folder, so I can sift through these.
I would greatly appreciate pointers on how to do this. As I mentioned, I am quite new to programming and am getting overwhelmed when looking at suggestions, seeing various different commands used to apparently achieve the same thing- reading several files at once.
The function csvread returns natively a matrix. I am not sure but it is possible that if some elements inside the csv file are not numbers, Matlab automatically makes a cell array out of the output. Since I don't know the structure of your csv-files I will recommend you trying out some similar functions(readtable, xlsread):
M = readtable(d(i).name) % Reads table like data, most recommended
M = xlsread(d(i).name) % Excel like structures, but works also on similar data
Try them out and let me know if it worked. If not please upload a file sample.
The function csvread(filename)
always return the matrix M that is numerical matrix and will never give the cell as return.
If you have textual data inside the .csv file, it will give you an error for not having the numerical data only. The only reason I can see for using the cell array when reading the files is if the dimensions of individual matrices read from each file are different, for example first .csv file contains data organised as 3xA, and second .csv file contains data organised as 2xB, so you can place them all into a single structure.
However, it is still possible to use histogram on cell array, by extracting the element as an array instead of extracting it as cell element.
If M is a cell matrix, there are two options for extracting the data:
M(i) and M{i}. M(i) will give you the cell element, and cannot be used for histogram, however M{i} returns element in its initial form which is numerical matrix.
TL;DR use histogram(M{i}) instead of histogram(M(i)).
I am using Matlab R2013b.
I have a 100x100 matrix which contains both numbers and strings. I converted it to a cell array (alldat) and wrote it to a csv file (blah.csv).
I then tried to append a single number to the top line of this csv file...which Matlab won't let me do.
cell2csv('blah.csv',alldat)
I can append the single number 'n' at the bottom of the matrix:
dlmwrite('blah.csv',n,'-append','delimiter',' ','roffset',1)
But it won't let me do it the other way around (so I can put the number in the first cell of the csv file, then have the matrix below it.
Can anyone advise?
I also tried outputting the cell array to a txt document using dlmwrite:
dlmwrite('blah.txt',alldat,'delimiter',' ');
And I kept getting this error:
Error using dlmwrite (line 113) The input cell array cannot be
converted to a matrix.
I often use tables for such tasks. Since you have a 100 x 100 array and not variables with different dimensions, it should be possible to adapt.
VarA={'12A3';123;'12B3'};
VarB={'45A6';456;'45B6'};
T=table(VarA,VarB);
writetable(T,'test.csv','WriteVariableNames',false)
T1=readtable('test.csv','ReadVariableNames',false)
You may want to use cell2table to create a table directly from your cell array, although it didn't work for me because it made some strange conversions from number to character.
I have 672 samples like these in a .txt file:
{
sleep:1360.36,
eat:4.36,
live:16.37,
travel:22.18,
work:22,
school:0.84,
vt:386.87
},
I want to put them in an excel file where {sleep, eat, live, travel, work, school, vt} are represented in a row and each sample represented in columns, with the correspondent number matching each. I've never dealt with text files following this format on matlab so I have no idea how to do this. Can anyone help me?
You can import data from Excel into Matlab using xlsread and export data using xlswrite. See the documentation
Syntax
xlswrite(filename,A,sheet,xlRange)
where A might be a cell array where the cells contain number of strings, sheet is the name of the Excel sheet and xlRange is the range in the Excel sheet (example: A1:B5).
Code example:
A = {'Column1', 'Column2', 'Column3'; 1, 2, 3};
xlswrite('example.xls', A, 'ExampleSheet', 'A1:B3');
Some hints:
If you know the number of rows and columsn of your data only at runtime but still want to give a range you must somehow assemble the range string yourself (rows are easy with sprintf, column names are more difficult (A, B, C, .., Z, AA, AB, ...))
If you do not have Excel on your computer, you will get csv files (see documentation)
Although each call to xlswrite returns quite fast, the system is still working. If another call to xlswrite comes too soon you might get unexpected (delay dependent) errors with no way to avoid them then to wait for sufficient time. I usually collect my data and then write everything to an excel file in one go.
very possible, you can do it in Matlab if you are familiar with it (although it is also quite easy to do in excel). To load in your file (no need to convert it, Matlab reads txt Files). you can do something like:
fileID = fopen('test2.txt'); %//Your file name
Input = textscan(fileID,'%s %f','Delimiter',':');
GoodRows = find(~isnan(Input{2} ));
column1 = Input{1}(GoodRows,:); //Column 1 is in the format of Cells (since you are storing Strings
column2 = Input{2}(GoodRows,:); //Column 2 is in the format of a Matrix, which will allow you to take numbers and do averages etc.
The cell and the matrix share indexes, so you can reformat your data eventually in to a Cell and export it in Matlab.
column1 =
'sleep'
'eat'
'live'
'travel'
'work'
'school'
'vt'
column2 =
1.0e+003 *
1.3604
0.0044
0.0164
0.0222
0.0220
0.0008
0.3869
==============EDIT===============
If you have multiple columns after the String, i.e.:
sleep,1.5,1.4,1.3
If you want to keep using textscan, you will need to specify how many columns there are. This is done by either:
Input = textscan(fileID,'%s %f %f %f %f','Delimiter',':'); //% add %f for each numeric column.
Or
Input = textscan(fileID,'%s repmat('%f',[1,N])]','Delimiter',':'); %// where N is the number of numeric columns you have.
I have 2D data for 10x10 matrices, and it looks like this
here is the table
However, data is updating and append every dt calculation, therefore I would like to reorganize and write it for specific columns , you may see this table in link
I use normally those codes to write
t=t+dt;
if ss==2000
dlmwrite('d:\Model_Results_Theta.txt', Tnew,'-append');
ss=0;
end
Could you recommend me any different way to organize the data based on the specific rows and columns using the matlab codes? Thanks in advance !!