I have a time series data recorded in different time zones (sample figure attached). Please note that the time information is in UTC. I need to convert them to local time (AM/PM) automatically for a large dataset. I have attached sample data and any help in this regard is highly appreciated.
https://www.dropbox.com/s/i2eeja8a7em87v9/Sample.csv?dl=0
You can use the TimeZone property for MATLAB's datetime data type to do the conversion for you. The only complication is then getting MATLAB to "forget" the timezone info so that you can just have a simple dataset with your desired datetimes. I've achieved this going via datenum below:
Note that if you want to retain the timezone information then you can simplify this by not doing the extra conversion steps.
Note also you would replace my dummy data at the top with readtable or similar to bring in the data, but I've assumed that the source data is entirely cells of chars, i.e. you may need to be careful of readtable "cleverly" detecting a datetime column, this may mean you have to handle it differently in the loop or provide additional arguments to readtable to prevent this automation.
data = cell2table( { ...
'4/02/2021 16:25', 'Pacific/Auckland'
'4/02/2021 16:05', 'Pacific/Auckland'
'5/02/2021 9:17', 'America/Chicago'
'4/02/2021 22:27', 'Asia/Singapore'
}, 'VariableNames', {'timeStrUtc','timezone'} );
data.timeUtc(:) = NaT;
data.timeLocal(:) = NaT;
for ii = 1:height(data)
% Convert input string to UTC-timezoned datetime
zonedDateUtc = datetime( data.timeStrUtc{ii}, ...
'InputFormat', 'dd/MM/yyyy HH:mm', ...
'TimeZone', 'UTC' );
% Convert to a local datetime
zonedDateLocal = datetime( zonedDateUtc, 'TimeZone', data.timezone{ii} );
% Get rid of MATLAB's associated timezones by converting to datenum
zonedDateUtc = datenum( zonedDateUtc );
zonedDateLocal = datenum( zonedDateLocal );
% Assign output to table, without timezones so all utc/local
data.timeUtc(ii) = datetime( zonedDateUtc, 'ConvertFrom', 'datenum' );
data.timeLocal(ii) = datetime( zonedDateLocal, 'ConvertFrom', 'datenum' );
end
% Format table columns
data.timeUtc.Format = 'd/M/yyyy h:mm a';
data.timeLocal.Format = 'd/M/yyyy h:mm a';
You can do this using a combination of readtable and the datetime constructor. First use readtable to read the data as text (because we want more control over the datetime construction):
t = readtable('Sample.csv', 'DateTimeType', 'text', 'TextType', 'string');
This results in a Nx2 table with string data. To convert each row to a datetime, we need a form of the datetime constructor like this:
datetime("4/02/2021 16:25", "InputFormat", "MM/dd/uuuu HH:mm", "TimeZone", "UTC")
Please check the "InputFormat" used there, because it isn't entirely obvious whether you want "MM/dd/uuuu" or "dd/MM/uuuu".
We can add another variable to the table t like this:
% Make a UTC datetime
t.endTimeUtc = datetime(t.endTimeUtcSecs, 'InputFormat', 'MM/dd/uuuu HH:mm', 'TimeZone', 'UTC');
Next, we need to convert each datetime from being in UTC to being in the specified TimeZone. Because a datetime array has a single TimeZone for the entire array, we must create a cell array of results. So, here's a function that takes a datetime and a time zone, and returns a new datetime wrapped in a cell array.
fun = #(ts, z) {datetime(ts, 'TimeZone', z, 'Format', 'MM/dd/uuuu HH:mm Z')};
(That function also sets up the display format of the datetime array to include time zone information)
We want to apply that function to the new variable in the table, and the time zone variable. Like this:
zonedEndTime = rowfun(fun, t(:, {'endTimeUtc', 'recordingTimezone'}));
(Note that I got a few warnings because some of your TimeZone specifications are apparently ambiguous)
This gives me the following result:
>> head(zonedEndTime)
ans =
8×1 table
Var1
__________________________
{[04/03/2021 05:25 +1300]}
{[04/03/2021 05:05 +1300]}
{[05/02/2021 04:17 -0500]}
{[04/03/2021 06:27 +0800]}
{[05/02/2021 01:51 -0500]}
{[05/02/2021 07:42 +0200]}
{[05/02/2021 08:17 +0800]}
{[05/02/2021 01:22 -0400]}
Related
Im trying to convert the first two columns of a cell into a Matlab time. First column {1,1} is the date in YYYY-MM-DD format and the second is the time in HH:MM format.
Any ideas where I'm going wrong? My code:
file = 'D:\Beach Erosion and Recovery\Bournemouth\Bournemouth Tidal
Data\tidal_data_jtide.txt'
fileID = fopen(file);
LT_celldata = textscan(fileID,'%D%D%D%D%d%[^\n\r]','delimiter',',');
formattime = 'yyyy-mm-dd HH:MM'
date = LT_celldata{1,1};
time = LT_celldata{1,2};
date_time = datenum('date','time'); code
Screenshot below is LT_celldata{1,1} :
You can combine variables date and time with the following code:
date = datetime(LT_celldata{1,1},'InputFormat','yyyy-MM-dd');
time = datetime(LT_celldata{1,2},'InputFormat','HH:mm:ss','Format','HH:mm:ss');
myDatetime = datetime(date + timeofday(time),'Format','yyyy-MM-dd HH:mm:ss');
The code uses timeofday function to combine date and time information from the two different variables. You may find more information and examples at this documentation page.
After a data export I get a string variable with "2017/02/22 1320:35 +000 4".
Through:
compute #TS = char.index(Timestamp_1, " ").
string date (A10).
compute date = char.substr(Timestamp_1,1,#TS).
alter type date (A10 = SDATE10).
I manage to get the date in a separate variable.
The same:
string time (A8).
compute time = char.substr(Timestamp_2,#TS+1,7).
alter type time (A8 = TIME8).
doesn't work for the time because it is in the 'hhmm:ss' format. How do I change the string variable '1320:35' into a time variable '13:20:35'?
You can insert a ":" manually into the time string by using the concat function before you alter the type.
COMPUTE time = CONCAT(CHAR.SUBSTR(time,1,2),":",(CHAR.SUBSTR(time,3))).
Another approach would be to extract hours, minutes and seconds from Timestamp variable individually and to use these elements inside the TIME.HMS function:
COMPUTE #hh = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+1,2),F2).
COMPUTE #mm = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+3,2),F2).
COMPUTE #ss = NUMBER(CHAR.SUBSTR(Timestamp_1,TS+6,2),F2).
COMPUTE time = TIME.HMS(#hh,#mm,#ss).
EXECUTE.
FORMATS time (TIME8).
Statistics has a datetime format and, new in V24, an ISO 8601 timestamp format, YMDHMS. If the string is converted to a regular date/time value, then the XDATE.DATE and XDATE.TIME functions can be used to extract the pieces. The Date and Time wizard may accommodate the format you have (not sure about that +000 4 part at the end).
Is there a way to convert time= 08/10/2014 23:34:02 to Epoch seconds (array of numbers) in MATLAB?
So you want the Unix standard which can be calculated as follows:
InputDate=datenum('20141008 233402','yyyymmdd HHMMSS');
UnixOrigin=datenum('19700101 000000','yyyymmdd HHMMSS');
EpochSecond=round((InputDate-UnixOrigin)*86400);
>> 1412811242
EDIT for the OP's date format:
MYSTRING = '08/10/2014 23:34:02';
InputDate = datenum(MYSTRING,'dd/mm/yyyy HH:MM:SS');
UnixOrigin=datenum('19700101 000000','yyyymmdd HHMMSS'); %//This can stay the same, unless you want to change it for consistency.
EpochSecond=round((InputDate-UnixOrigin)*86400);
>>1412811242
I have some text files with dates of the form
2015-09-08 14:38:03
2015-09-08 14:38:03.1
2015-09-08 14:38:03.2
that I want to convert into Matlab date/time format. As you can see, the text strings have a different time format regarding the milliseconds. In the first case, no milliseconds are given; in the seconds case, milliseconds are given with one digit only. This gives a sampling rate of 20Hz for measuring data.
So far, only
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS');
is working, but of course misses the milliseconds. A conversion like
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
does not work as the milliseconds are either zero (full seconds) or have one digit after the '.' delimiter. A workaround like
x = datestr(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
x = datenum(file.dateColumn, 'yyyy-mm-dd HH:MM:SS.FFF');
i.e. converting the text string into a Matlab string (and giving it additional FFF/FF digits) and then converting that one into a date/time number works - but this is such time-cosuming that I cannot use it for my data. I have millions of time rows in different files.
Do you have any ideas?
Greetings, Arne
Thanks to Nick I found a way to solve it:
dataVelPres = readtable(fileName, ...
'ReadVariableNames', false, ...
'HeaderLines', 4 );
dataVelPres.Properties.VariableNames = {'date' 'velU' 'velV' 'velW' 'pres'};
dateMill = datetime(dataVelPres.date, 'inputformat', 'yyyy-MM-dd HH:mm:ss.S');
dateFull = datetime(dataVelPres.date, 'inputformat', 'yyyy-MM-dd HH:mm:ss');
dateNaT = isnat(dateMill);
dateMill(dateNaT) = dateFull(dateNaT);
dataVelPres.dateTime = dateMill;
dataVelPres.date = datenum(dataVelPres.dateTime); % Convert to number format if needed
This works with two tables - one for millisec. and one without - and puts both together, as both give NaT entries in case the inputformat does not match.
Is there a more elegant way?
You may try something like:
a='2015-09-08 14:38:03';
s=strsplit(a,{'-',':',' '})
x=datenum(cellfun(#str2num,s(1:end)))
I highly recommend using the new datetime object:
strings = {'2015-09-08 14:38:03', '2015-09-08 14:38:03.1', '2015-09-08 14:38:03.2'};
dates = {};
for d = strings
d = d{1};
try
dt = datetime(d, 'inputformat', 'yyyy-MM-dd HH:mm:ss.S');
catch
dt = datetime(d, 'inputformat', 'yyyy-MM-dd HH:mm:ss');
end
dates{end + 1} = dt;
end
>> dates
dates =
[08-Sep-2015 14:38:03] [08-Sep-2015 14:38:03] [08-Sep-2015 14:38:03]
>> dates{end}.Second
ans =
3.2000
It's also easy convert from a datetime object to a datenum:
>> x = [datetime('now'), datetime('yesterday')]
x =
10-Dec-2015 12:53:40 09-Dec-2015 00:00:00
>> datenum(x)
ans =
1.0e+05 *
7.3631 7.3631
>>
I have some data files with Unix timestamps (in this case, number of milliseconds since Jan 1, 1970 00:00 UTC). I would like to convert these to human-friendly date/time strings (e.g. 31-Aug-2012 11:36:24) in Matlab. Is there an easy way to do this in Matlab, or am I better off using an external library (e.g. java.text.SimpleDateFormat)?
How about
date = datestr(unix_time/86400 + datenum(1970,1,1))
if unix_time is given in seconds, unix_time/86400 will give the number of days since Jan. 1st 1970. Add to that the offset used by Matlab's datenum (datenum(0000,1,1) == 1), and you have the amount of days since Jan. 1st, 0000. This can be easily converted to human-readable form by Matlab's datestr.
If you have milliseconds, just use
date = datestr(unix_time/86400/1000 + datenum(1970,1,1))
Wrapped in functions, these would be
function dn = unixtime_to_datenum( unix_time )
dn = unix_time/86400 + 719529; %# == datenum(1970,1,1)
end
function dn = unixtime_in_ms_to_datenum( unix_time_ms )
dn = unix_time_ms/86400000 + 719529; %# == datenum(1970,1,1)
end
datestr( unixtime_to_datenum( unix_time ) )
Newer versions of MATLAB (verified in R2015a) have a datetime type that is useful for working with and formatting dates and times. You can convert UNIX timestamps into a MATLAB datetime with
dt = datetime( unix_time, 'ConvertFrom', 'posixtime' );
and then use datestr as before for formatting as a string
datestr( dt )