I have data (240 x 9 dimensions) in which a DATETIME column has entries as shown below. I want to separate the date and time parts. Also, I need to calculate the time difference between an entry and a previous one. The date is formatted as yyyymmdd followed by the hhmmsssss. However, the last three entries of the seconds represent the decimal seconds i.e., 36200 means 36.2 seconds
Using MATLAB I have separated the date and time entries as character arrays as the original data was in the double format as below:
datestr: 240 x8 char e.g., '20190101'
timestr: 240 x9 char e.g., '001634200'
I want to convert the date into the standard format. I used
formatOut = 'yyyy/mm/dd';
date=datestr(datenum('20190101','yyyymmdd',1900),formatOut);
But getting this error : Index in position 1 exceeds array bounds (must not exceed 240).
Additionally, I need to convert the time into hh:mm:ss.sss format and then calculate the difference between a time and its previous entry.
Any help will be appreciated.
Related
This question already has answers here:
Counting values by day/hour with timeseries in MATLAB
(3 answers)
Closed 6 years ago.
I have a year's worth of data, the data is recorded one minute intervals each day of the year.
The date and time was imported from excel (in form 243.981944, then by adding 42004 (so will be for 2015) and formatting to date it becomes 31.8.15 23:34:00).
Importing to MATLAB it becomes
'31/08/2015 23:34:00'
I require the data for each day of the year to be at hourly intervals, so I need to sum the data recorded in each hour and divide that by the number of data recorded for that hour, giving me the hourly average.
For some reason the data in August actually increments in 2 minute intervals, data for every other month increments in one minute intervals.
ie
...
31/07/2015 23:57:00
31/07/2015 23:58:00
31/07/2015 23:59:00
31/08/2015 00:00:00
31/08/2015 00:02:00
31/08/2015 00:04:00
...
I'm not sure how I can find all the values for a specific date and hour in order to work out the averages. I was thinking of using a for loop to find the values on each day, but when I got down to writing code realised this wouldn't work the way I was thinking.
I presume there must be some kind of functions available that would allow for data to be filtered by the date and time?
edit:
So I tried the following but I get these errors.
dates is a 520000x1 cell array containing the dates form = formatIn.
formatIn = 'DD/MM/YYYY HH:MM:SS';
[~,M,D,H] = datevec(dates, formatIn);
Error using cnv2icudf (line 131) Unrecognized minute format.
Format string: DD/MM/YYYY HH:MM:SS.
Error in datevec (line 112) icu_dtformat = cnv2icudf(varargin{isdateformat});`
Assuming your data is in a matrix or cell-array of strings called A, and your other data is in a vector X. Let's say all the data is in the same year (so we can ignore years)
[~,M,D,H] = datevec(A, 'dd/mm/yyyy HH:MM:SS');
mean_A = accumarray([M, D, H+1], X, [], #mean);
Then data from February will be in
mean_A(2,:,:)
To look at the data, you may find the squeeze() function useful, e.g.
squeeze(mean_A(2,1:10,13:24))
shows the average for the hours after midday (by column) for the first ten days (by row) of February.
See also:
Counting values by day/hour with timeseries in MATLAB
I am trying to read time-coordinate data from a netCDF file using matlab. I have a netCDF file (which I created) that has a time variable in the format of a double corresponding to the number of hours from a specific time (see below).
Variable attributes:
double time(Time) ;
time:standard_name = "Time" ;
time:units = "hours since 2002-01-01 0:0:0" ;
time:calendar = "proleptic_gregorian" ;
When I read the time variable using ncread) into matlab, it just prints out an integer e.g.,1. However, if I use "ncdump" to explore the file, I see the time variable in it's coordinate time e.g., 2002-01-01 01.
Specifically: "ncdump -t -v time ncfile.nc"
I'm relatively new to matlab, and I was wondering if anyone knew if there was a similar, or an equally simple, way to read this time variable as its coordinate time into matlab, either as a string, or numerical date. Specifically, I would like to avoid having to parse the attribute string and code up a bunch of pointers and conditions to convert the integer data to an actual date.
Alternatively, should I just create a new time variable in these files that is just an array of dates as strings?
Any information is very much appreciated!
Thanks!
NetCDF stores time as an offset from an epoch. From your variable attribute, your epoch is 2002-01-01 0:0:0, and the time is hours since then. Matlab has a similar methodology called date numbers, although it is based off of days since an epoch (which they call pivot years). There are two functions that you should look into: datenum and datestr. The first converts a string into a date number and the other converts a date number into a date string.
You can convert your time variable into a compatible Matlab date number by dividing by 24 and then use the datestr function to format it however you like. Here is a simple example:
>> time = [1;2;3;4];
>> datestr(time./24+datenum('2002-01-01 0:0:0'))
ans =
01-Jan-2002 01:00:00
01-Jan-2002 02:00:00
01-Jan-2002 03:00:00
01-Jan-2002 04:00:00
Look at the Matlab help files associated with the two functions and you can format the date output however you like.
I have a matrix containing the date in a cell structure. I managed to convert the date (2nd column) using datenum(), but I am not sure how to add on the time (3rd column)
The data looks like this:
'IBM' 20090602 0 108.410000000000
'IBM' 20090602 500 108.560000000000
My code:
date = datenum(num2str(IBM(:,2)),'yyyymmdd')
Let's review your mistakes first:
You feed datenum with the string 'IBM(:, 2)' instead of the actual array. Discard the quotes.
datenum accepts strings, not numerical values.
A possible solution is converting the second column of your data into an array of strings, and feeding it into datenum, like so:
d = datenum(num2str(vertcat(IBM{:, 2})), 'yyyymmdd');
Note that this is, of course, possible only if the format of the date string is fixed in each row.
EDIT:
To add the values in the third column to the result of datenum, simply do the following:
d + vertcat(IBM{:, 3})
Where d is a column vector of date values obtained from datenum (I assume that you want to do basic addition, since you haven't specified the actual meaning of the timje values in the third column).
In one line, the complete answer would look like this:
datenum(num2str(vertcat(IBM{:, 2})), 'yyyymmdd') + vertcat(IBM{:, 3})
You can straight-up add the time values in when you're converting to datenum. Just convert from what I assumed were minutes (if they're in seconds, add in another *60 to the divisor) to days, which is what MATLAB uses for its datenum calculations.
timestamps = cellfun(#(x,y) datenum(num2str(x),'yyyymmdd')+y/(24*60),...
IBM(:,2),...
IBM(:,3),...
'UniformOutput',false)
I have some data that was given to me in excel and the time format is rather confusing. The fist column of the is the DateTime but with incorrect HH:MM and the second column is the correct hour of the day HH:MM:
time = {'01/01/2000 00:00',num2str(2300);
'01/01/2000 00:00',num2str(2400);
'01/01/2000 00:00',num2str(10);
'01/01/2000 00:00',num2str(100)};
However, when the time exceeds midnight, instead of being 00:10 the time is 10, and 01:00 is 100. How could I alter these to the correct format? i.e. from the example above I would like the outcome to be:
time = {'01/01/2000 23:00';
'01/01/2000 24:00';
'01/01/2000 00:10';
'01/01/2000 01:00'};
How could I achieve this?
Using sprintf in MATLAB, you can use the field width specifier:
where
Field width:
Minimum number of characters to print. Can be a number, or an asterisk (*) to refer to an argument in the input list. For example, the input list ('%12d', intmax) is equivalent to ('%*d', 12, intmax).
Thus your times should end up all looking like "XX:XX", with the leading zero added by the sprintf if it is missing and the colon added in.
Thanks to #Junuxx for the exact command: sprintf('%02i:%02i', hours, minutes)
To separate hours and minutes, you would obviously do time % 100 to get the minutes and integer divide by 100 to get the hours.
From there, you simply strcat or concatenate ["a" "b"] your two columns to get your desired result.
How can I convert this kind of data 08:00:43.771 given as string into a number specifying the number of milliseconds since midnight corresponding to this time instance?
I generally use the Matlab datenum outputs for timestamping in Matlab. Datenums are the number of days since 0/0/0000, expressed as a double (double precision numbers are precise to about 14 usec for contemporary dates).
Using datenums.
currentDateTime1 = datenum('08:00:43.771'); %Assumes today
currentDateTime2 = datenum('6/8/1975 08:00:43.771'); %Using an explicit date
millisecondsSinceMidnight = mod(currentDateTime1 ,1) *24*60*60*1000; %Mod 1 removes any day component
millisecondsSinceMidnight = mod(currentDateTime2 ,1) *24*60*60*1000; %Then this is just a unit conversion
For unusual string formats, use the extended form of datenum, which can accept a string format specifier.
Use 1000*etime(datevec('08:00:43.771'),datevec('0:00')) to give the number of milliseconds since midnight. etime gives the number of seconds between two date vectors, datevec converts strings to date vectors (assuming Jan 1 this year if only a time is given).