I have the following sample from a CSV file. Structure is:
Date ,Time(Hr:Min:S:mS), Value
2015:08:20,08:20:19:123 , 0.05234
2015:08:20,08:20:19:456 , 0.06234
I then would like to read this into a matrix in MATLAB.
Attempt :
Matrix = csvread('file_name.csv');
Also tried an attempt formatting the string.
fmt = %u:%u:%u %u:%u:%u:%u %f
Matrix = csvread('file_name.csv',fmt);
The problem is when the file is read the format is wrong and displays it differently.
Any help or advice given would be greatly appreciated!
EDIT
When using #Adriaan answer the result is
2015 -11 -9
8 -17 -1
So it seems that MATLAB thinks the '-' is the delimiter(separator)
Matrix = csvread('file_name.csv',1,0);
csread does not support a format specifier. Just enter the number of header rows (I took it to be one, as per example), and number of header columns, 0.
You file, however, contains non-numeric data. Thus import it with importdata:
data = importdata('file_name.csv')
This will get you a structure, data with two fields: data.data contains the numeric data, i.e. a vector containing your value. data.textdata is a cell containing the rest of the data, you need the first two column and extract the numerics from it, i.e.
for ii = 2:size(data.textdata,1)
tmp1 = data.textdata{ii,1};
Date(ii,1) = datenum(tmp1,'YYYY:MM:DD');
tmp2 = data.textdata{ii,2};
Date(ii,2) = datenum(tmp2,'HH:MM:SS:FFF');
end
Thanks to #Excaza it turns out milliseconds are supported.
Related
I am generating 2500 values in Matlab in format (time,heart_rate, resp_rate) by using below code
numberOfSeconds = 2500;
time = 1:numberOfSeconds;
newTime = transpose(time);
number0 = size(newTime, 1)
% generating heart rates
heart_rate = 50 +(70-50) * rand (numberOfSeconds,1);
intHeartRate = int64(heart_rate);
number1 = size(intHeartRate, 1)
% hist(heart_rate)
% generating resp rates
resp_rate = 50 +(70-50) * rand (numberOfSeconds,1);
intRespRate = int64(resp_rate);
number2 = size(intRespRate, 1)
% hist(heart_rate)
% joining time and sensor data
joinedStream = strcat(num2str(newTime),{','},num2str(intHeartRate),{','},num2str(intRespRate))
dlmwrite('/Users/amar/Desktop/geenrated/rate.txt', joinedStream,'delimiter','');
The data shown in the console is alright, but when I save this data to a .txt file, it contains extra spaces in beginning. Hence I am not able to parse the .txt file to generate input stream. Please help
Replace the last two lines of your code with the following. No need to use strcat if you want a CSV output file.
dlmwrite('/Users/amar/Desktop/geenrated/rate.txt', [newTime intHeartRate intRespRate]);
πβπ π πππ’π‘πππ π π’ππππ π‘ππ ππ¦ ππΎπ ππ π‘βπ π ππππππ π‘ πππ π¦ππ’π πππ π. πβππ πππ π€ππ ππ₯ππππππ π€βπ¦ π¦ππ’ πππ‘ π‘βπ π’πππ₯ππππ‘ππ ππ’π‘ππ’π‘.
The data written in the file is exactly what is shown in the console.
>> joinedStream(1) %The exact output will differ since 'rand' is used
ans =
cell
' 1,60,63'
num2str basically converts a matrix into a character array. Hence number of characters in its each row must be same. So for each column of the original matrix, the row with the maximum number of characters is set as a standard for all the rows with less characters and the deficiency is filled by spaces. Columns are separated by 2 spaces. Take a look at the following smaller example to understand:
>> num2str([44, 42314; 4, 1212421])
ans =
2Γ11 char array
'44 42314'
' 4 1212421'
I want to use fscanf for reading a text file containing 4 rows with an unknown number of columns. The newline is represented by two consecutive spaces.
It was suggested that I pass : as the sizeA parameter but it doesn't work.
How can I read in my data?
update: The file format is
String1 String2 String3
10 20 30
a b c
1 2 3
I have to fill 4 arrays, one for each row.
See if this will work for your application.
fid1=fopen('test.txt');
i=1;
check=0;
while check~=1
str=fscanf(fid1,'%s',1);
if strcmp(str,'')~=1;
string(i)={str};
end
i=i+1;
check=strcmp(str,'');
end
fclose(fid1);
X=reshape(string,[],4);
ar1=X(:,1)
ar2=X(:,2)
ar3=X(:,3)
ar4=X(:,4)
Once you have 'ar1','ar2','ar3','ar4' you can parse them however you want.
I have found a solution, i don't know if it is the only one but it works fine:
A=fscanf(fid,'%[^\n] *\n')
B=sscanf(A,'%c ')
Z=fscanf(fid,'%[^\n] *\n')
C=sscanf(Z,'%d')
....
You could use
rawText = getl(fid);
lines = regexp(thisLine,' ','split);
tokens = {};
for ix = 1:numel(lines)
tokens{end+1} = regexp(lines{ix},' ','split'};
end
This will give you a cell array of strings having the row and column shape or your original data.
To read an arbitrary line of text then break it up according the the formating information you have available. My example uses a single space character.
This uses regular expressions to define the separator. Regular expressions powerful but too complex to describe here. See the MATLAB help for regexp and regular expressions.
Here is my code. I have 59 CSV files in my directory, I have 11 variables in each, with first one date in quarter format. I need to tell MATLAB that first column has date format, because the code below imports it as a string variable.
ext = '.csv';
countries = dir(['*', ext]);
countryFiles = {countries.name};
countriesNames = strrep(countryFiles, ext, '');
%%
a = cell(length(countriesNames), 2);
a(:,1) = countriesNames(:)';
a(:,2) = cellfun(#(file) readtable(file, 'TreatAsEmpty', '.','Format'?), countryFiles(:), 'uni', 0);
As far as I understand this is option 'Format' with datenum in readtable... However, I can`t find useful information on it in helpfiles and whatever I try I get errors... Below how my data looks for each country. Data is 1980Q1-2014Q4.
So I have 59*2 cell array with first column representing countryNames and second column contains 59 140*11 tables. First I do not know how to access variable names within cell array`s table. Second problem is that if you try to define x to a column in that table and then datenum(x,'YYYYQQ') I get "Input expected to be a cell array, was table instead."
So, I need to extract from cell a tables in a{1,2}, a{2,2}, a{3,2} ... then convert those tables to cellarray using table2cell option. How to do it using loop for each country and then write it back to the cell?
The following code generates a similar dataset to what I am currently working with:
clear all
a = rand(131400,12);
DateTime=datestr(datenum('2011-01-01 00:01','yyyy-mm-dd HH:MM'):4/(60*24):...
datenum('2011-12-31 23:57','yyyy-mm-dd HH:MM'),...
'yyyy-mm-dd HH:MM');
DateTime=cellstr(DateTime);
header={'DateTime','temp1','temp2','temp4','temp7','temp10',...
'temp13','temp16','temp19','temp22','temp25','temp30','temp35'};
I'm trying to convert the outputs into one variable (called 'Data'), i.e. have header as the first row (1,:), 'DateTime' starting from row 2 (2:end,1) and running through each row, and finally having 'a' as the data (2:end,2:end) if that makes sense. So, 'DateTime' and 'header' are used as the heading for the rows and column respectively. Following this I need to save this into a tab delimited text file.
I hope I've been clear in expressing what I'm attempting.
An easy way, but might be not the fastest:
Data = [header; DateTime, num2cell(a)];
filename = 'test.txt';
dlmwrite(filename,1); %# no create text file, not Excel
xlswrite(filename,Data);
UPDATE:
It appears that xlswrite actually changes the format of DateTime values even if it writes to a text file. If the format is important here is the better and actually faster way:
filename = 'test.txt';
out = [DateTime, num2cell(a)];
out = out'; %# our cell array will be printed by columns, so we have to transpose
fid = fopen(filename,'wt');
%# printing header
fprintf(fid,'%s\t',header{1:end-1});
fprintf(fid,'%s\n',header{end});
%# printing the data
fprintf(fid,['%s\t', repmat('%f\t',1,size(a,2)-1) '%f\n'], out{:});
fclose(fid);
I had a similar question. but what i am trying now is to read files in .txt format into MATLAB. My problem is with the headers. Many times due to errors the system rewrites the headers in the middle of file and then MATLAB cannot read the file. IS there a way to skip it? I know i can skip reading some characters if i know what the character is.
here is the code i am using.
[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data');
file=[pathc c];
data= dlmread(file, ',', 1,4);
this way i let the user pick the file. My files are huge typically [ 86400 125 ]
so naturally it has 125 header fields or more depends on files.
Thanks
Because the files are so big i cannot copy , but its in format like
day time col1 col2 col3 col4 ...............................
2/3/2010 0:10 3.4 4.5 5.6 4.4 ...............................
..................................................................
..................................................................
and so on
With DLMREAD you can read only numeric data. It will not read date and time, as your first two columns contain. If other data are all numeric you can tell DLMREAD to skip first row and 2 columns on the right:
data = dlmread(file, ' ', 1,2);
To import also day and time you can use IMPORTDATA instead of DLMREAD:
A = importdata(file, ' ', 1);
dt = datenum(A.textdata(2:end,1),'mm/dd/yyyy');
tm = datenum(A.textdata(2:end,2),'HH:MM');
data = A.data;
The date and time will be converted to serial numbers. You can convert them back with DATESTR function.
It turns out that you can still use textscan. Except that you read everything as string. Then, you attempt to convert to double. 'str2double' returns NaN for strings, and since headers are all strings, you can identify header rows as rows with all NaNs.
For example:
%# find and open file
[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data');
file=[pathc c];
fid = fopen(file);
%# read all text
strData = textscan(fid,'%s%s%s%s%s%s','Delimiter',',');
%# close the file again
fclose(fid);
%# catenate, b/c textscan returns a column of cells for each column in the data
strData = cat(2,strData{:});
%# convert cols 3:6 to double
doubleData = str2double(strData(:,3:end));
%# find header rows. headerRows is a logical array
headerRowsL = all(isnan(doubleData),2);
%# since I guess you know what the headers are, you can just remove the header rows
dateAndTimeCell = strData(~headerRowsL,1:2);
dataArray = doubleData(~headerRowsL,:);
%# and you're ready to start working with your data