Im trying to convert the first two columns of a cell into a Matlab time. First column {1,1} is the date in YYYY-MM-DD format and the second is the time in HH:MM format.
Any ideas where I'm going wrong? My code:
file = 'D:\Beach Erosion and Recovery\Bournemouth\Bournemouth Tidal
Data\tidal_data_jtide.txt'
fileID = fopen(file);
LT_celldata = textscan(fileID,'%D%D%D%D%d%[^\n\r]','delimiter',',');
formattime = 'yyyy-mm-dd HH:MM'
date = LT_celldata{1,1};
time = LT_celldata{1,2};
date_time = datenum('date','time'); code
Screenshot below is LT_celldata{1,1} :
You can combine variables date and time with the following code:
date = datetime(LT_celldata{1,1},'InputFormat','yyyy-MM-dd');
time = datetime(LT_celldata{1,2},'InputFormat','HH:mm:ss','Format','HH:mm:ss');
myDatetime = datetime(date + timeofday(time),'Format','yyyy-MM-dd HH:mm:ss');
The code uses timeofday function to combine date and time information from the two different variables. You may find more information and examples at this documentation page.
Related
I have a time series data recorded in different time zones (sample figure attached). Please note that the time information is in UTC. I need to convert them to local time (AM/PM) automatically for a large dataset. I have attached sample data and any help in this regard is highly appreciated.
https://www.dropbox.com/s/i2eeja8a7em87v9/Sample.csv?dl=0
You can use the TimeZone property for MATLAB's datetime data type to do the conversion for you. The only complication is then getting MATLAB to "forget" the timezone info so that you can just have a simple dataset with your desired datetimes. I've achieved this going via datenum below:
Note that if you want to retain the timezone information then you can simplify this by not doing the extra conversion steps.
Note also you would replace my dummy data at the top with readtable or similar to bring in the data, but I've assumed that the source data is entirely cells of chars, i.e. you may need to be careful of readtable "cleverly" detecting a datetime column, this may mean you have to handle it differently in the loop or provide additional arguments to readtable to prevent this automation.
data = cell2table( { ...
'4/02/2021 16:25', 'Pacific/Auckland'
'4/02/2021 16:05', 'Pacific/Auckland'
'5/02/2021 9:17', 'America/Chicago'
'4/02/2021 22:27', 'Asia/Singapore'
}, 'VariableNames', {'timeStrUtc','timezone'} );
data.timeUtc(:) = NaT;
data.timeLocal(:) = NaT;
for ii = 1:height(data)
% Convert input string to UTC-timezoned datetime
zonedDateUtc = datetime( data.timeStrUtc{ii}, ...
'InputFormat', 'dd/MM/yyyy HH:mm', ...
'TimeZone', 'UTC' );
% Convert to a local datetime
zonedDateLocal = datetime( zonedDateUtc, 'TimeZone', data.timezone{ii} );
% Get rid of MATLAB's associated timezones by converting to datenum
zonedDateUtc = datenum( zonedDateUtc );
zonedDateLocal = datenum( zonedDateLocal );
% Assign output to table, without timezones so all utc/local
data.timeUtc(ii) = datetime( zonedDateUtc, 'ConvertFrom', 'datenum' );
data.timeLocal(ii) = datetime( zonedDateLocal, 'ConvertFrom', 'datenum' );
end
% Format table columns
data.timeUtc.Format = 'd/M/yyyy h:mm a';
data.timeLocal.Format = 'd/M/yyyy h:mm a';
You can do this using a combination of readtable and the datetime constructor. First use readtable to read the data as text (because we want more control over the datetime construction):
t = readtable('Sample.csv', 'DateTimeType', 'text', 'TextType', 'string');
This results in a Nx2 table with string data. To convert each row to a datetime, we need a form of the datetime constructor like this:
datetime("4/02/2021 16:25", "InputFormat", "MM/dd/uuuu HH:mm", "TimeZone", "UTC")
Please check the "InputFormat" used there, because it isn't entirely obvious whether you want "MM/dd/uuuu" or "dd/MM/uuuu".
We can add another variable to the table t like this:
% Make a UTC datetime
t.endTimeUtc = datetime(t.endTimeUtcSecs, 'InputFormat', 'MM/dd/uuuu HH:mm', 'TimeZone', 'UTC');
Next, we need to convert each datetime from being in UTC to being in the specified TimeZone. Because a datetime array has a single TimeZone for the entire array, we must create a cell array of results. So, here's a function that takes a datetime and a time zone, and returns a new datetime wrapped in a cell array.
fun = #(ts, z) {datetime(ts, 'TimeZone', z, 'Format', 'MM/dd/uuuu HH:mm Z')};
(That function also sets up the display format of the datetime array to include time zone information)
We want to apply that function to the new variable in the table, and the time zone variable. Like this:
zonedEndTime = rowfun(fun, t(:, {'endTimeUtc', 'recordingTimezone'}));
(Note that I got a few warnings because some of your TimeZone specifications are apparently ambiguous)
This gives me the following result:
>> head(zonedEndTime)
ans =
8×1 table
Var1
__________________________
{[04/03/2021 05:25 +1300]}
{[04/03/2021 05:05 +1300]}
{[05/02/2021 04:17 -0500]}
{[04/03/2021 06:27 +0800]}
{[05/02/2021 01:51 -0500]}
{[05/02/2021 07:42 +0200]}
{[05/02/2021 08:17 +0800]}
{[05/02/2021 01:22 -0400]}
After a data export I get a string variable with "2017/02/22 1320:35 +000 4".
Through:
compute #TS = char.index(Timestamp_1, " ").
string date (A10).
compute date = char.substr(Timestamp_1,1,#TS).
alter type date (A10 = SDATE10).
I manage to get the date in a separate variable.
The same:
string time (A8).
compute time = char.substr(Timestamp_2,#TS+1,7).
alter type time (A8 = TIME8).
doesn't work for the time because it is in the 'hhmm:ss' format. How do I change the string variable '1320:35' into a time variable '13:20:35'?
You can insert a ":" manually into the time string by using the concat function before you alter the type.
COMPUTE time = CONCAT(CHAR.SUBSTR(time,1,2),":",(CHAR.SUBSTR(time,3))).
Another approach would be to extract hours, minutes and seconds from Timestamp variable individually and to use these elements inside the TIME.HMS function:
COMPUTE #hh = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+1,2),F2).
COMPUTE #mm = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+3,2),F2).
COMPUTE #ss = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+6,2),F2).
COMPUTE time = TIME.HMS(#hh,#mm,#ss).
EXECUTE.
FORMATS time (TIME8).
Statistics has a datetime format and, new in V24, an ISO 8601 timestamp format, YMDHMS. If the string is converted to a regular date/time value, then the XDATE.DATE and XDATE.TIME functions can be used to extract the pieces. The Date and Time wizard may accommodate the format you have (not sure about that +000 4 part at the end).
I have some text files with dates of the form
2015-09-08 14:38:03
2015-09-08 14:38:03.1
2015-09-08 14:38:03.2
that I want to convert into Matlab date/time format. As you can see, the text strings have a different time format regarding the milliseconds. In the first case, no milliseconds are given; in the seconds case, milliseconds are given with one digit only. This gives a sampling rate of 20Hz for measuring data.
So far, only
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS');
is working, but of course misses the milliseconds. A conversion like
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
does not work as the milliseconds are either zero (full seconds) or have one digit after the '.' delimiter. A workaround like
x = datestr(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
i.e. converting the text string into a Matlab string (and giving it additional FFF/FF digits) and then converting that one into a date/time number works - but this is such time-cosuming that I cannot use it for my data. I have millions of time rows in different files.
Do you have any ideas?
Greetings, Arne
Thanks to Nick I found a way to solve it:
dataVelPres = readtable(fileName, ...
'ReadVariableNames', false, ...
'HeaderLines', 4 );
dataVelPres.Properties.VariableNames = {'date' 'velU' 'velV' 'velW' 'pres'};
dateMill = datetime(dataVelPres.date, 'inputformat', 'yyyy-MM-dd HH:mm:ss.S');
dateFull = datetime(dataVelPres.date, 'inputformat', 'yyyy-MM-dd HH:mm:ss');
dateNaT = isnat(dateMill);
dateMill(dateNaT) = dateFull(dateNaT);
dataVelPres.dateTime = dateMill;
dataVelPres.date = datenum(dataVelPres.dateTime); % Convert to number format if needed
This works with two tables - one for millisec. and one without - and puts both together, as both give NaT entries in case the inputformat does not match.
Is there a more elegant way?
You may try something like:
a='2015-09-08 14:38:03';
s=strsplit(a,{'-',':',' '})
x=datenum(cellfun(#str2num,s(1:end)))
I highly recommend using the new datetime object:
strings = {'2015-09-08 14:38:03', '2015-09-08 14:38:03.1', '2015-09-08 14:38:03.2'};
dates = {};
for d = strings
d = d{1};
try
dt = datetime(d, 'inputformat', 'yyyy-MM-dd HH:mm:ss.S');
catch
dt = datetime(d, 'inputformat', 'yyyy-MM-dd HH:mm:ss');
end
dates{end + 1} = dt;
end
>> dates
dates =
[08-Sep-2015 14:38:03] [08-Sep-2015 14:38:03] [08-Sep-2015 14:38:03]
>> dates{end}.Second
ans =
3.2000
It's also easy convert from a datetime object to a datenum:
>> x = [datetime('now'), datetime('yesterday')]
x =
10-Dec-2015 12:53:40 09-Dec-2015 00:00:00
>> datenum(x)
ans =
1.0e+05 *
7.3631 7.3631
>>
Is it possible to create a vector automatically or progressively with dates. I want to ask the user about the starting date and the final date and I'd to fill a vector with the dates and the ones between both.
so abstractly:
"what is the first date?" '...' firstDate = '...' --> "what is the final date?" '...' finalDate='...'
and following, I'd like to fill in a vector all of the dates between firstDate and finalDate. is this possible in Matlab, and how? Is there a function to use?
You can use linspace after the user input:
%// prompt and user input
prompt1 = 'What is the first date? (yyyy-MM-dd)\n';
prompt2 = 'What is the last date? (yyyy-MM-dd)\n';
startDate = datenum(input(prompt1,'s'),'yyyy-mm-dd')
endDate = datenum(input(prompt2,'s'),'yyyy-mm-dd')
% number of days
numdays = endDate-startDate
% array of dates
alldays = linspace(startDate,endDate,numdays)
DateString = datestr(alldays, 'mm/dd/yyyy')
For the input 1989-07-01 and 1989-07-07 it will return:
DateString =
07/01/1989
07/02/1989
07/03/1989
07/04/1989
07/05/1989
07/07/1989
Dates can be called using datevec, datestr, datenum etc. datenum will be your best bet, since you can then create your vector using
DateVector = datenum(firstDate):datenum(finalDate);
This will create a vector containing dates per day. The way back is to say datestr(DateVector)
I have an array of timestamps, these are just three examples:
time_1 = '23/01/2013 00:47:00'
time_2 = '04/02/2013 00:00:00'
time_3 = '27/01/2013 03:31:00'
I want to use 'datevec' on these timestamps:
tVec_1 = datevec(datenum(time_1,'dd/mm/yyyy HH:MM:SS'));
tVec_2 = datevec(datenum(time_2,'dd/mm/yyyy HH:MM:SS'));
tVec_3 = datevec(datenum(time_3,'dd/mm/yyyy HH:MM:SS'));
for the tVec_1 and tVec_3 timestamp it works nicely, but it crashes for the tVec_2:
Error using dtstr2dtvecmx
Failed on converting date string to date number.
Error in datevec (line 117)
y = dtstr2dtvecmx(t,icu_dtformat);
I suspect, that the problem comes from "zero time" which is not treated properly by my format 'dd/mm/yyyy HH:MM:SS' (if I remove the last "HH:MM:SS" then it works).
Unfortunately, I could not find anything useful on matlab website.
Suggestions are welcome!
Thanks!