tab delimited text file from matlab - matlab

The following code generates a similar dataset to what I am currently working with:
clear all
a = rand(131400,12);
DateTime=datestr(datenum('2011-01-01 00:01','yyyy-mm-dd HH:MM'):4/(60*24):...
datenum('2011-12-31 23:57','yyyy-mm-dd HH:MM'),...
'yyyy-mm-dd HH:MM');
DateTime=cellstr(DateTime);
header={'DateTime','temp1','temp2','temp4','temp7','temp10',...
'temp13','temp16','temp19','temp22','temp25','temp30','temp35'};
I'm trying to convert the outputs into one variable (called 'Data'), i.e. have header as the first row (1,:), 'DateTime' starting from row 2 (2:end,1) and running through each row, and finally having 'a' as the data (2:end,2:end) if that makes sense. So, 'DateTime' and 'header' are used as the heading for the rows and column respectively. Following this I need to save this into a tab delimited text file.
I hope I've been clear in expressing what I'm attempting.

An easy way, but might be not the fastest:
Data = [header; DateTime, num2cell(a)];
filename = 'test.txt';
dlmwrite(filename,1); %# no create text file, not Excel
xlswrite(filename,Data);
UPDATE:
It appears that xlswrite actually changes the format of DateTime values even if it writes to a text file. If the format is important here is the better and actually faster way:
filename = 'test.txt';
out = [DateTime, num2cell(a)];
out = out'; %# our cell array will be printed by columns, so we have to transpose
fid = fopen(filename,'wt');
%# printing header
fprintf(fid,'%s\t',header{1:end-1});
fprintf(fid,'%s\n',header{end});
%# printing the data
fprintf(fid,['%s\t', repmat('%f\t',1,size(a,2)-1) '%f\n'], out{:});
fclose(fid);

Related

Reading Text file comma seperated on Matlab

I'm trying to read a comma separated text file that looks like the following:
2017-10-24,01:17:38,2017-10-24,02:17:38,+1.76,L,Meters
2017-10-24,02:57:31,2017-10-24,03:57:31,+1.92,H,Meters
2017-10-24,05:53:35,2017-10-24,06:53:35,+1.00,L,Meters
2017-10-24,10:45:01,2017-10-24,11:45:01,+2.06,H,Meters
2017-10-24,13:27:16,2017-10-24,14:27:16,+1.78,L,Meters
2017-10-24,15:07:16,2017-10-24,16:07:16,+1.92,H,Meters
2017-10-24,18:12:08,2017-10-24,19:12:08,+0.98,L,Meters
My code so far is:
LT_data = fopen('D:\Beach Erosion and Recovery\Bournemouth\Bournemouth Tidal Data\tidal_data_jtide.txt');% text file containing predicted low tide times
LT_celldata = textscan(LT_data,'%D %D %D %D %d ','delimiter',',')'
For mixed data types, I'd recommend readtable. This will read your data straight into a table object without having to specify a format spec or use fopen,
t = readtable( 'myFile.txt', 'ReadVariableNames', false, 'Delimiter', ',' );
Then you can easily manipulate the data
% Variable names in the table
t.Properties.VariableNames = {'Date1', 'Time1', 'Date2', 'Time2', 'Value', 'Dim', 'Units'};
% Create full datetime object columns from the date and time columns
t.DateTime1 = datetime( strcat(t.Date1,t.Time1), 'InputFormat', 'yyyy-MM-ddHH:mm:ss' );
If you do know the formats, you can specify the 'format' property within readtable and it will convert the data when reading.
This is working perfectly. The formatspec need to be edited.
file = 'D:\Beach Erosion and Recovery\Bournemouth\Bournemouth Tidal Data\tidal_data_jtide.txt'
fileID = fopen(file);
LT_celldata = textscan(fileID,'%D%D%D%D%d%[^\n\r]','delimiter',',')

Insert String in a CSV file matlab

how do I insert a string into a csv file in matlab. i used this code to write some data and create my csv file:
and here is the output of the code:
I'm trying to insert some text in the first 2 columns before the numerical data..
thanks in advance :)
There are several approaches are possible here.
Let's take a look at some of them:
If you need to add string to your csv file.
For example, I create some csv file like your:
q = [1 2 3 4 5 6 7];
csvwrite('csvlist4.csv',q,2,0);
All troubles is to add some string to csv - it's because we need to combine numeric and text data. There are no good functions for it in Matlab except low levels:
c = 'some big text';
fid = fopen('csvlist4.csv','r+');
fprintf(fid,c);
How it works: the data in csv is an array. I put data in 3rd row, so first and second is an empty but they have ','. When you use fprintf it will replace this , with your text. So if text is too long it will overwrite your data.
How to avoid this?
Easiest way - is to do it with help of xlswrite function. For your example:
txt = cell(size(Q))
txt{1} = 'this is your text'
X = [txt; num2cell(Q)]
xlswrite('new.xlsx',X)
Result:
Important moment here: number of cell for text must be the same as your data. But I filled with the text only first cell in my example.
And the one more way: read csv file, modify it's data and write to csv:
csvwrite('csvlist4.csv',a,2,0);
m = csvread('csvlist4.csv');
fid = fopen('csvlist4.csv','w');
c = 'your text'
fprintf(fid, c); fprintf(fid, '\n');
fclose(fid);
dlmwrite('csvlist4.csv', m(3:end,:), '-append');
Of course you can use cell array instead c and so on and so on.
Hope it helps!

MATLAB reading CSV file with timestamp and values

I have the following sample from a CSV file. Structure is:
Date ,Time(Hr:Min:S:mS), Value
2015:08:20,08:20:19:123 , 0.05234
2015:08:20,08:20:19:456 , 0.06234
I then would like to read this into a matrix in MATLAB.
Attempt :
Matrix = csvread('file_name.csv');
Also tried an attempt formatting the string.
fmt = %u:%u:%u %u:%u:%u:%u %f
Matrix = csvread('file_name.csv',fmt);
The problem is when the file is read the format is wrong and displays it differently.
Any help or advice given would be greatly appreciated!
EDIT
When using #Adriaan answer the result is
2015 -11 -9
8 -17 -1
So it seems that MATLAB thinks the '-' is the delimiter(separator)
Matrix = csvread('file_name.csv',1,0);
csread does not support a format specifier. Just enter the number of header rows (I took it to be one, as per example), and number of header columns, 0.
You file, however, contains non-numeric data. Thus import it with importdata:
data = importdata('file_name.csv')
This will get you a structure, data with two fields: data.data contains the numeric data, i.e. a vector containing your value. data.textdata is a cell containing the rest of the data, you need the first two column and extract the numerics from it, i.e.
for ii = 2:size(data.textdata,1)
tmp1 = data.textdata{ii,1};
Date(ii,1) = datenum(tmp1,'YYYY:MM:DD');
tmp2 = data.textdata{ii,2};
Date(ii,2) = datenum(tmp2,'HH:MM:SS:FFF');
end
Thanks to #Excaza it turns out milliseconds are supported.

Date Format in MATLAB

Here is my code. I have 59 CSV files in my directory, I have 11 variables in each, with first one date in quarter format. I need to tell MATLAB that first column has date format, because the code below imports it as a string variable.
ext = '.csv';
countries = dir(['*', ext]);
countryFiles = {countries.name};
countriesNames = strrep(countryFiles, ext, '');
%%
a = cell(length(countriesNames), 2);
a(:,1) = countriesNames(:)';
a(:,2) = cellfun(#(file) readtable(file, 'TreatAsEmpty', '.','Format'?), countryFiles(:), 'uni', 0);
As far as I understand this is option 'Format' with datenum in readtable... However, I can`t find useful information on it in helpfiles and whatever I try I get errors... Below how my data looks for each country. Data is 1980Q1-2014Q4.
So I have 59*2 cell array with first column representing countryNames and second column contains 59 140*11 tables. First I do not know how to access variable names within cell array`s table. Second problem is that if you try to define x to a column in that table and then datenum(x,'YYYYQQ') I get "Input expected to be a cell array, was table instead."
So, I need to extract from cell a tables in a{1,2}, a{2,2}, a{3,2} ... then convert those tables to cellarray using table2cell option. How to do it using loop for each country and then write it back to the cell?

skip reading headers in MATLAB

I had a similar question. but what i am trying now is to read files in .txt format into MATLAB. My problem is with the headers. Many times due to errors the system rewrites the headers in the middle of file and then MATLAB cannot read the file. IS there a way to skip it? I know i can skip reading some characters if i know what the character is.
here is the code i am using.
[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data');
file=[pathc c];
data= dlmread(file, ',', 1,4);
this way i let the user pick the file. My files are huge typically [ 86400 125 ]
so naturally it has 125 header fields or more depends on files.
Thanks
Because the files are so big i cannot copy , but its in format like
day time col1 col2 col3 col4 ...............................
2/3/2010 0:10 3.4 4.5 5.6 4.4 ...............................
..................................................................
..................................................................
and so on
With DLMREAD you can read only numeric data. It will not read date and time, as your first two columns contain. If other data are all numeric you can tell DLMREAD to skip first row and 2 columns on the right:
data = dlmread(file, ' ', 1,2);
To import also day and time you can use IMPORTDATA instead of DLMREAD:
A = importdata(file, ' ', 1);
dt = datenum(A.textdata(2:end,1),'mm/dd/yyyy');
tm = datenum(A.textdata(2:end,2),'HH:MM');
data = A.data;
The date and time will be converted to serial numbers. You can convert them back with DATESTR function.
It turns out that you can still use textscan. Except that you read everything as string. Then, you attempt to convert to double. 'str2double' returns NaN for strings, and since headers are all strings, you can identify header rows as rows with all NaNs.
For example:
%# find and open file
[c,pathc]=uigetfile({'*.txt'},'Select the data','V:\data');
file=[pathc c];
fid = fopen(file);
%# read all text
strData = textscan(fid,'%s%s%s%s%s%s','Delimiter',',');
%# close the file again
fclose(fid);
%# catenate, b/c textscan returns a column of cells for each column in the data
strData = cat(2,strData{:});
%# convert cols 3:6 to double
doubleData = str2double(strData(:,3:end));
%# find header rows. headerRows is a logical array
headerRowsL = all(isnan(doubleData),2);
%# since I guess you know what the headers are, you can just remove the header rows
dateAndTimeCell = strData(~headerRowsL,1:2);
dataArray = doubleData(~headerRowsL,:);
%# and you're ready to start working with your data