I have a .mat file and the data structure is as below.
Data =
struct with fields:
axis1: [25626600×1 double]
axis2: [25626600×1 double]
axis3: [25626600×1 double]
datenum: [25626600×1 double]
sample_rate: 30
serial_number: 'MOS2D10171145'
I want this .mat file to be save as a csv file.
I tried this code but it gives me this error.
csvwrite('FileName.csv', Data);
Undefined function 'real' for input arguments of type 'struct'.
Error in dlmwrite (line 189)
str = sprintf('%.*g%+.*gi',precn,real(m(i,j)),precn,imag(m(i,j)));
Error in csvwrite (line 42)
dlmwrite(filename, m, ',', r, c);
as explained by Wolfie, you cannot do this direclty, because your sample_rate and your serial_number variables cannot go direclty in your CSV.
A solution could be to put these 2 variable in your CSV filename to keep them.
%% Parameters
cdSAVE='YOUR_PATH'; % PUT YOUR SAVING PATH
%% Variables (put your own)
axis1 = zeros(250,1);
axis2 = zeros(250,1)+1;
axis3 = zeros(250,1)+2;
datenum = zeros(250,1)+3;
sample_rate = 30;
serial_number = 'SERIAL_NUMBER';
%% Make a matrix with what you can
Matrix = zeros(length(axis1),4);
Matrix(:,1)=axis1;
Matrix(:,2)=axis2;
Matrix(:,3)=axis3;
Matrix(:,4)=datenum;
%% Convert to table (to have the headers written in your csv file)
T = array2table(Matrix);
T.Properties.VariableNames(1:4) = {'axis1','axis2','axis3','datenum'};
%% Create a the filename with the 2 other datas
fileName = char(strcat('Extract_CSV_Sample_rate_',string(sample_rate),'_Serial_Number_',serial_number,'.csv'));
%% Writing
fullPathFileName=char(strcat(cdSAVE,"\",fileName));
%% Choose what you want :
%% Here for just the matrix with csvwrite
csvwrite(fullPathFileName,Matrix);
%% Here with headers also with writetable
writetable(T,fullPathFileName)
And the CSV name will be : Extract_CSV_Sample_rate_30_Serial_Number_SERIAL_NUMBER.csv
Related
I gave the error Error using readtable (line 216) Input must be a row vector of characters or string scalar when I tried to run this code in Matlab:
clear
close all
clc
D = 'C:\Users\Behzad\Desktop\New folder (2)';
filePattern = fullfile(D, '*.xlsx');
file = dir(filePattern);
x={};
for k = 1 : numel(file)
baseFileName = file(k).name;
fullFileName = fullfile(D, baseFileName);
x{k} = readtable(fullFileName);
fprintf('read file %s\n', fullFileName);
end
% allDates should be out of the loop because it's not necessary to be in the loop
dt1 = datetime([1982 01 01]);
dt2 = datetime([2018 12 31]);
allDates = (dt1 : calmonths(1) : dt2).';
allDates.Format = 'MM/dd/yyyy';
% 1) pre-allocate a cell array that will store
% your tables (see note #3)
T2 = cell(size(x)); % this should work, I don't know what x is
% the x is xlsx files and have different sizes, so I think it should be in
% a loop?
% creating loop
for idx = 1:numel(x)
T = readtable(x{idx});
% 2) This line should probably be T = readtable(x(idx));
sort = sortrows(T, 8);
selected_table = sort (:, 8:9);
tempTable = table(allDates(~ismember(allDates,selected_table.data)), NaN(sum(~ismember(allDates,selected_table.data)),size(selected_table,2)-1),'VariableNames',selected_table.Properties.VariableNames);
T2 = outerjoin(sort,tempTable,'MergeKeys', 1);
% 3) You're overwriting the variabe T2 on each iteration of the i-loop.
% to save each table, do this
T2{idx} = fillmissing(T2, 'next', 'DataVariables', {'lat', 'lon', 'station_elevation'});
end
the x is each xlsx file from the first loop. my xlsx file has a different column and row size. I want to make the second loop process for all my xlsx files in the directory.
did you know what is the problem? and how to fix it?
Readtable has one input argument, a filename. It returns a table. In your code you have the following:
x{k} = readtable(fullFileName);
All fine, you are reading the tables and storing the contents in x. Later in your code you continue with:
T = readtable(x{idx});
You already read the table, what you wrote is basically T = readtable(readtable(fullFileName)). Just use T=x{idx}
I have a .mtx file which contains a vector which I am suppose to use for matrix vector multiplication. I tried to open the file using
fopen('filename') but this does not work, it returns a single number. I have also tried using readmtx but this gives me the following error: File size does not match inputs. Expected a file size of 232316 bytes. Instead the file size is
365 bytes. Could you please advise how I can open and work with this type of file in MATLAB.
fopen('filename') outputs only the fileID, which is typically an integer >3. This fileID can be used with e.g. fscanf(fileID) to get the content of the file.
For the .mtx format, where each line contains [row-number column-number matrix-entry], you could also do something like:
%% file
filename = 'my_matrix.mtx';
%% read Matrix
Mat = read_mtx(filename);
%% function
function Mat = read_mtx(filename)
% open file
fID = fopen(filename,'r');
M = fscanf(fID, '%f');
% reshape M vector into Nx3 matrix
M = reshape(M, [3, length(M)/3])';
% assemble final matrix
Mat = zeros(M(end,1),M(end,1));
for ii = 1:size(M,1)
Mat(M(ii,1),M(ii,2)) = M(ii,3);
end
end
I am using the fgetl command to read a .csv file but instead of returning the results I wanted as:
"HIST",1,1,27,PWH,"1"
it returned with additional space between each character:
" H I S T " , 1 , 1 , 2 7 , P W H , " 1 "
I know that I can replace the space with regexprep, but my file contains billions of lines so the added expression might consume considerably more time. I had a feeling that this is a unicode issue and someone pointed out the same issue when he used Java and it was related to unicode. I wonder if anyone knows a better way to deal with the problem in MATLAB?
Update:
It should be the unicode issue because the .csv file is an output from another program, and when I read it using fgetl the spaces are added. However, if I save the .csv file again using Excel and read the .csv file using fgetl again, it returns the results I want.
I am not able to provide an example because the .csv file is very large and I cannot make a small sample because when I open and save it from Excel, this problem is gone.
For the purpose of demonstration, let's consider a demo file - demo.csv:
"GIST",1,6,17,PWH,"1"
"FIST",0,4,72,WPH,"2"
"MIST",3,2,27,WHP,"3"
You have some options:
textscan (for any text file with a known structure):
fID = fopen('demo.csv');
C = textscan(fID,'%s%d%d%d%s%s','Delimiter',{',','"'},'MultipleDelimsAsOne',1);
fclose(fID);
Which results in:
C =
{3x1 cell} [3x1 int32] [3x1 int32] [3x1 int32] {3x1 cell} {3x1 cell}
Import helper + generate script (AKA overkill is an understatement):
Which results in:
%% Import data from text file.
% Script for importing data from the following text file:
%
% F:\demo.csv
%
% To extend the code to different selected data or a different text file, generate a
% function instead of a script.
% Auto-generated by MATLAB on 2016/04/20 19:51:32
%% Initialize variables.
filename = 'F:\demo.csv';
delimiter = ',';
%% Read columns of data as strings:
% For more information, see the TEXTSCAN documentation.
formatSpec = '%q%q%q%q%q%q%[^\n\r]';
%% Open the text file.
fileID = fopen(filename,'r');
%% Read columns of data according to format string.
% This call is based on the structure of the file used to generate this code. If an error
% occurs for a different file, try regenerating the code from the Import Tool.
dataArray = textscan(fileID, formatSpec, 'Delimiter', delimiter, 'ReturnOnError', false);
%% Close the text file.
fclose(fileID);
%% Convert the contents of columns containing numeric strings to numbers.
% Replace non-numeric strings with NaN.
raw = repmat({''},length(dataArray{1}),length(dataArray)-1);
for col=1:length(dataArray)-1
raw(1:length(dataArray{col}),col) = dataArray{col};
end
numericData = NaN(size(dataArray{1},1),size(dataArray,2));
for col=[2,3,4,6]
% Converts strings in the input cell array to numbers. Replaced non-numeric strings with
% NaN.
rawData = dataArray{col};
for row=1:size(rawData, 1);
% Create a regular expression to detect and remove non-numeric prefixes and suffixes.
regexstr = '(?<prefix>.*?)(?<numbers>([-]*(\d+[\,]*)+[\.]{0,1}\d*[eEdD]{0,1}[-+]*\d*[i]{0,1})|([-]*(\d+[\,]*)*[\.]{1,1}\d+[eEdD]{0,1}[-+]*\d*[i]{0,1}))(?<suffix>.*)';
try
result = regexp(rawData{row}, regexstr, 'names');
numbers = result.numbers;
% Detected commas in non-thousand locations.
invalidThousandsSeparator = false;
if any(numbers==',');
thousandsRegExp = '^\d+?(\,\d{3})*\.{0,1}\d*$';
if isempty(regexp(numbers, thousandsRegExp, 'once'));
numbers = NaN;
invalidThousandsSeparator = true;
end
end
% Convert numeric strings to numbers.
if ~invalidThousandsSeparator;
numbers = textscan(strrep(numbers, ',', ''), '%f');
numericData(row, col) = numbers{1};
raw{row, col} = numbers{1};
end
catch me
end
end
end
%% Split data into numeric and cell columns.
rawNumericColumns = raw(:, [2,3,4,6]);
rawCellColumns = raw(:, [1,5]);
%% Allocate imported array to column variable names
GIST = rawCellColumns(:, 1);
VarName2 = cell2mat(rawNumericColumns(:, 1));
VarName3 = cell2mat(rawNumericColumns(:, 2));
VarName4 = cell2mat(rawNumericColumns(:, 3));
PWH = rawCellColumns(:, 2);
VarName6 = cell2mat(rawNumericColumns(:, 4));
%% Clear temporary variables
clearvars filename delimiter formatSpec fileID dataArray ans raw col numericData rawData row regexstr result numbers invalidThousandsSeparator thousandsRegExp me rawNumericColumns rawCellColumns;
csvread (for numeric values only; which means it is not applicable here).
I happened to have the same issue. I opened a .csv file using textscan and it added 1 whitespace on both side of any character and I also noticed that when opening the variable storing the read data, the font was different than the usual in Matlab.
We managed to solve this issue by opening the '.csv' file into Notepad++ and changed the encoding to UTF-8. It solved the problem.
Hope it helps!
Seeking help from skillful Matlab users!
I'm kind of new to Matlab and hope somebody has the time to help me. I need to import some .txt-files from a directory. I have found a way to do this trough the import tool. There are some data using comma insted of dots, so importdata will not work, but the 'import data' tool does.
So i'm wondering (and hoping) if it is possible to edit the generated function to import all the files in the directory, in such a way as the single file is imported? I want each file to be imported as matrix variable (double). I want to import all the files in one process (loop). Also there are many files and they all have some 100 000 lines or so.
If someone see an easy way to do this i would appreciate the help. Please keep the explanation on a low level, as i'm quite novice. I get the following function using the 'import data' tool:
function Streaming0x00x00158D00000E04621709201405 = importfile1(filename, startRow, endRow)
%IMPORTFILE1 Import numeric data from a text file as a matrix.
% STREAMING0X00X00158D00000E04621709201405 = IMPORTFILE1(FILENAME) Reads
% data from text file FILENAME for the default selection.
%
% STREAMING0X00X00158D00000E04621709201405 = IMPORTFILE1(FILENAME,
% STARTROW, ENDROW) Reads data from rows STARTROW through ENDROW of text
% file FILENAME.
%
% Example:
% Streaming0x00x00158D00000E04621709201405 =
% importfile1('Streaming_0_x_0_0_x_00158D00000E0462_17-09-2014_05.32.24_part000.txt',
% 17, 137834);
%
% See also TEXTSCAN.
% Auto-generated by MATLAB on 2015/02/04 09:28:07
%% Initialize variables.
delimiter = ';';
if nargin<=2
startRow = 17;
endRow = inf;
end
%% Read columns of data as strings:
% For more information, see the TEXTSCAN documentation.
formatSpec = '%s%s%[^\n\r]';
%% Open the text file.
fileID = fopen(filename,'r');
%% Read columns of data according to format string.
% This call is based on the structure of the file used to generate this
% code. If an error occurs for a different file, try regenerating the code
% from the Import Tool.
textscan(fileID, '%[^\n\r]', startRow(1)-1, 'ReturnOnError', false);
dataArray = textscan(fileID, formatSpec, endRow(1)-startRow(1)+1, 'Delimiter', delimiter, 'ReturnOnError', false);
for block=2:length(startRow)
frewind(fileID);
textscan(fileID, '%[^\n\r]', startRow(block)-1, 'ReturnOnError', false);
dataArrayBlock = textscan(fileID, formatSpec, endRow(block)-startRow(block)+1, 'Delimiter', delimiter, 'ReturnOnError', false);
for col=1:length(dataArray)
dataArray{col} = [dataArray{col};dataArrayBlock{col}];
end
end
%% Close the text file.
fclose(fileID);
%% Convert the contents of columns containing numeric strings to numbers.
% Replace non-numeric strings with NaN.
raw = repmat({''},length(dataArray{1}),length(dataArray)-1);
for col=1:length(dataArray)-1
raw(1:length(dataArray{col}),col) = dataArray{col};
end
numericData = NaN(size(dataArray{1},1),size(dataArray,2));
for col=[1,2]
% Converts strings in the input cell array to numbers. Replaced non-numeric
% strings with NaN.
rawData = dataArray{col};
for row=1:size(rawData, 1);
% Create a regular expression to detect and remove non-numeric prefixes and
% suffixes.
regexstr = '(?<prefix>.*?)(?<numbers>([-]*(\d+[\.]*)+[\,]{0,1}\d*[eEdD]{0,1}[-+]*\d*[i]{0,1})|([-]*(\d+[\.]*)*[\,]{1,1}\d+[eEdD]{0,1}[-+]*\d*[i]{0,1}))(?<suffix>.*)';
try
result = regexp(rawData{row}, regexstr, 'names');
numbers = result.numbers;
% Detected commas in non-thousand locations.
invalidThousandsSeparator = false;
if any(numbers=='.');
thousandsRegExp = '^\d+?(\.\d{3})*\,{0,1}\d*$';
if isempty(regexp(thousandsRegExp, '.', 'once'));
numbers = NaN;
invalidThousandsSeparator = true;
end
end
% Convert numeric strings to numbers.
if ~invalidThousandsSeparator;
numbers = strrep(numbers, '.', '');
numbers = strrep(numbers, ',', '.');
numbers = textscan(numbers, '%f');
numericData(row, col) = numbers{1};
raw{row, col} = numbers{1};
end
catch me
end
end
end
%% Replace non-numeric cells with NaN
R = cellfun(#(x) ~isnumeric(x) && ~islogical(x),raw); % Find non-numeric cells
raw(R) = {NaN}; % Replace non-numeric cells
%% Create output variable
Streaming0x00x00158D00000E04621709201405 = cell2mat(raw);
If something is unclear, please comment.
All help is useful, thanks :)
If all files are the same, you can make a cell array of the filenames (NOT a standard array, they do not behave correctly on strings). Then you can loop over the cell array. For instance:
fname_arr = {'file1.txt','file2.txt'}; % your filenames go here
for k in length(fname_arr):
filename = fname_arr{k};
%% Open the text file.
fileID = fopen(filename,'r'); % start of the relevant part of your codeblock
<...> % omitting the stuff in the middle of the code
fclose(fileID) % end of the relevant part of your codeblock
allDataArray{k} = DataArray
end
Then allDataArray is a cell array whose kth element contains the DataArray obtained from file fname_arr{k}.
Ok, tried something here. Implemented this in my function:
output=(dir_output);
for k=1:length(output);
filename = output{k}.name;
%% Open the text file. fileID = fopen(filename,'r');
where 'dir_output' is a struct, containing all the file names in the Directory. Also put in:
%% Close the text file.
fclose(fileID);
allDataArray{k} = DataArray;
end
Get this as error:
>> function1 Undefined function or variable 'dir_output'. Error in function1 (line 30) output=(dir_output);
Why???
I need to read the following csv file in MATLAB:
2009-04-29 01:01:42.000;16271.1;16271.1
2009-04-29 02:01:42.000;2.5;16273.6
2009-04-29 03:01:42.000;2.599609;16276.2
2009-04-29 04:01:42.000;2.5;16278.7
...
I'd like to have three columns:
timestamp;value1;value2
I tried the approaches described here:
Reading date and time from CSV file in MATLAB
modified as:
filename = 'prova.csv';
fid = fopen(filename, 'rt');
a = textscan(fid, '%s %f %f', ...
'Delimiter',';', 'CollectOutput',1);
fclose(fid);
But it returs a 1x2 cell, whose first element is a{1}='ÿþ2', the other are empty.
I had also tried to adapt to my case the answers to these questions:
importing data with time in MATLAB
Read data files with specific format in matlab and convert date to matal serial time
but I didn't succeed.
How can I import that csv file?
EDIT After the answer of #macduff i try to copy-paste in a new file the data reported above and use:
a = textscan(fid, '%s %f %f','Delimiter',';');
and it works.
Unfortunately that didn't solve the problem because I have to process csv files generated automatically, which seems to be the cause of the strange MATLAB behavior.
What about trying:
a = textscan(fid, '%s %f %f','Delimiter',';');
For me I get:
a =
{4x1 cell} [4x1 double] [4x1 double]
So each element of a corresponds to a column in your csv file. Is this what you need?
Thanks!
Seems you're going about it the right way. The example you provide poses no problems here, I get the output you desire. What's in the 1x2 cell?
If I were you I'd try again with a smaller subset of the file, say 10 lines, and see if the output changes. If yes, then try 100 lines, etc., until you find where the 4x1 cell + 4x2 array breaks down into the 1x2 cell. It might be that there's an empty line or a single empty field or whatever, which forces textscan to collect data in an additional level of cells.
Note that 'CollectOutput',1 will collect the last two columns into a single array, so you'll end up with 1 cell array of 4x1 containing strings, and 1 array of 4x2 containing doubles. Is that indeed what you want? Otherwise, see #macduff's post.
I've had to parse large files like this, and I found I didn't like textscan for this job. I just use a basic while loop to parse the file, and I use datevec to extract the timestamp components into a 6-element time vector.
%% Optional: initialize for speed if you have large files
n = 1000 %% <# of rows in file - if known>
timestamp = zeros(n,6);
value1 = zeros(n,1);
value2 = zeros(n,1);
fid = fopen(fname, 'rt');
if fid < 0
error('Error opening file %s\n', fname); % exit point
end
cntr = 0
while true
tline = fgetl(fid); %% get one line
if ~ischar(tline), break; end; % break out of loop at end of file
cntr = cntr + 1;
splitLine = strsplit(tline, ';'); %% split the line on ; delimiters
timestamp(cntr,:) = datevec(splitLine{1}, 'yyyy-mm-dd HH:MM:SS.FFF'); %% using datevec to parse time gives you a standard timestamp vector
value1(cntr) = splitLine{2};
value2(cntr) = splitLine{3};
end
%% Concatenate at the end if you like
result = [timestamp value1 value2];