Assume the following timeseries (ts) with assigned values:
time val
15:00 4
15:45 7
17:12 2.3
17:50 2.9
Every value from a timestamp is valid until the next appears. Thus, from 15:00 to 15:45 the value is 4 or from 15:45 to 17:12 it is 2.3. Every new data point between these timestamps should have the same value. What i want is a new ts, with a constant time-interval and a pre-defined start point. Let's say the starting point is 15:00 and the interval should be 30 min. Normally, I could use the synchronize function - however, the function uses the interpolation method and this is not what i need here, since the values between the data points should not be interpolated, but be averaged if timestamps are overlapping.
The new ts should be like:
time val
15:00 4
15:30 5.5
16:00 7
16:30 7
17:00 4.18
The value for timestamp 15:30 is computed as = (4*15+7*15)/30, and so on. I have implemented a code, that is capable of fixing this by applying the trapz function with a lot of if statements. However, I was wondering if there are better/simpler solutions around, as a modified synchronize function, since I have more than 500.000 data points.
Thanks in advance
I managed to fix my problem by dividing all time steps into minute-values and afterwards applying the trapezoidal rule to get the sum of the area under the curve (AUC) and then the average by dividing with the applied minute interval.
AllValues = interp1(Time,Data,NewTime,'previous')';
[Xdata,Ydata] = stairs(NewTime,AllValues);
NewTS = timeseries(Xdata,Ydata);
TrapzSum = cumtrapz(NewTS.time,NewTS.data);
TrapzSum = TrapzSum(1:2:end);
NewResults = diff(TrapzSum(IndicesOfNewInterval))/MinInt;
Related
for my project, I need to calculate TOW (Time of week) in Simulink. I know this can be achieved through conversion of UTC time to GPS time.
I have written a simple m-file in Matlab which does the action for me in Matlab as follow:
date_gps_int = 10000*y + 100*m + d
date_gps_str = int2str(date_gps_int)
date_gps_str_to_serial = datenum(date_gps_str,'yyyymmdd')
date_str_format = datestr(date_gps_str_to_serial,'dd-mmmm-yyyy')
Num_Days = daysact('06-jan-1980',date_str_format)
Num_Weeks = Num_Days/7
TOW = Num_Weeks - 1024
My first intention was to use this as a function in simulink. But apparently because of 'datenum' and 'datestr' it is not possible, since simulink does not handle strings.
Now I am wondering if anyone can help me with this issue. Is there any way to calculate TOW from the UTC date in Matlab without using those predefined functions?
I also tried to write an algorithm for calculating number of days since '6 January 1980' and then calculating number of weeks by dividing that by 7. But since I am not very familiar with leap year calculation and I don't really know the formula for these kinds of calculations, my result differs from real TOW.
I would appreciate if anybody can help me on this.
There are three formats handled by Matlab for time: formatted date strings - what datestr outputs -, serial date - scalar double, what datenum outputs - and date vectors (see datevec). Conversion functions work with these three, and the most convenient way to convert individual variables (year, month, etc) to a date is to build a date vector [yyyy mm dd HH MM SS].
date_gps_str_to_serial = datenum([y m d 0 0 0]); % midnight on day y-m-d
date_Jan_6_1980 = datenum([1980 01 06 0 0 0]); % midnight on Jan 6th, 1980
Num_Days = date_gps_str_to_serial - date_Jan_6_1980;
Now, beware of leap seconds...
GPS time is computed form the time elapsed since Jan 6th 1980. Take the number of seconds elapsed since that day, as measured by the satellites' atomic clocks, divide by (24*3600) to get a number of days, the remainder is the time of the day (in seconds since midnight).
But, once in a while, the International Earth Rotation and Reference Systems Service will decide that a day will last one second longer to accommodate for the slowing of Earth rotation. It may happen twice a year, on June 30th or December 31st. The calculation of GPS time is wrong, because it does not take into account that some days last 86401 seconds (so dividing by 24*3600 does not work) and will advance by 1 second with respect to UTC each time this happens. There has been 18 such days since Jan 6th 1980, so one should subtract 18 seconds from GPS time to find UTC time. The next time a leap second may be added is June 2019.
I am working with a data set of 10,000s of variables which have been repeatedly measured since the 1980s. The first meassurements for each variable are not on the same date and the variables are irregularly measured - sometimes measurements are only a month apart, in a small number of cases they are decades apart.
I want to get the change in each variable per month.
So far I have a cell of dates of measurements,and interpolated rates of change between measurements (each cell represents a single variable in either, and I've only posted the first 5 cells in each array)
DateNumss= {[736614;736641;736669] [736636;736666] 736672 [736631;736659;736685] 736686}
LinearInterpss={[17.7777777777778;20.7142857142857;0] [0.200000000000000;0] 0 [2.57142857142857;2.80769230769231;0]}
How do I get monthly sums of the interpolated change in variable?
i.e.
If the first measurement for a variable is made on the January 1st, and the linearly interpolated change between that an the next measurement is 1 per day; and the next measurement is on Febuary the 5th and the corresponding linearly interpolated change is 2; then January has a total change of 1*31 (31 days at 1) and febuary has a total change of 1*5+2*23 (5 days at 1, 23 days at 2).
You would need the points in the serial dates that correspond with the change of a month.
mat(:,1)=sort(repmat(1980:1989,[1,12]));
mat(:,2)=repmat(1:12,[1,size(mat,1)/12]);
mat(:,3)=1;
monthseps=datenum(mat);
This gives you a list of all 120 changes of months in the eighties.
Now you want, for each month, the change per day, and sum it. If you take the original data it is easier, since you can just interpolate each day's value using matlab. If you only have the "LinearInterpss" you need to map it on the days using interp1 with the method 'previous'.
for ct = 2:length(monthseps)
days = monthseps(ct-1):(monthseps(ct)-1); %days in the month
%now we need each day assigned a certain change. This value depends on your "LinearInterpss". interp1 with method 'previous' searches LineairInterpss for the last value.
vals = interp1(DateNumss,LinearInterpss,days,'previous');
sum(vals); %the sum over the change in each day is the total change in a month
end
This question already has answers here:
Counting values by day/hour with timeseries in MATLAB
(3 answers)
Closed 6 years ago.
I have a year's worth of data, the data is recorded one minute intervals each day of the year.
The date and time was imported from excel (in form 243.981944, then by adding 42004 (so will be for 2015) and formatting to date it becomes 31.8.15 23:34:00).
Importing to MATLAB it becomes
'31/08/2015 23:34:00'
I require the data for each day of the year to be at hourly intervals, so I need to sum the data recorded in each hour and divide that by the number of data recorded for that hour, giving me the hourly average.
For some reason the data in August actually increments in 2 minute intervals, data for every other month increments in one minute intervals.
ie
...
31/07/2015 23:57:00
31/07/2015 23:58:00
31/07/2015 23:59:00
31/08/2015 00:00:00
31/08/2015 00:02:00
31/08/2015 00:04:00
...
I'm not sure how I can find all the values for a specific date and hour in order to work out the averages. I was thinking of using a for loop to find the values on each day, but when I got down to writing code realised this wouldn't work the way I was thinking.
I presume there must be some kind of functions available that would allow for data to be filtered by the date and time?
edit:
So I tried the following but I get these errors.
dates is a 520000x1 cell array containing the dates form = formatIn.
formatIn = 'DD/MM/YYYY HH:MM:SS';
[~,M,D,H] = datevec(dates, formatIn);
Error using cnv2icudf (line 131) Unrecognized minute format.
Format string: DD/MM/YYYY HH:MM:SS.
Error in datevec (line 112) icu_dtformat = cnv2icudf(varargin{isdateformat});`
Assuming your data is in a matrix or cell-array of strings called A, and your other data is in a vector X. Let's say all the data is in the same year (so we can ignore years)
[~,M,D,H] = datevec(A, 'dd/mm/yyyy HH:MM:SS');
mean_A = accumarray([M, D, H+1], X, [], #mean);
Then data from February will be in
mean_A(2,:,:)
To look at the data, you may find the squeeze() function useful, e.g.
squeeze(mean_A(2,1:10,13:24))
shows the average for the hours after midday (by column) for the first ten days (by row) of February.
See also:
Counting values by day/hour with timeseries in MATLAB
I am creating the limits for an equally spaced time series and I need to be able to change the time interval (1min, 5min, 10min, 15min, 30min, 60min etc.). My bounds are opening and closing time of the market. The stocks I am working on trades from 17.00 to 16.15 of the day after.
Here is what I am using:
timevec=datenum(2013,1,1,17,0:1*interval:1395,0)';
% It creates a time vector from 1-1-2013 17.00.00 to 1-2-2013 16.15.00
% spaced by "1min*interval"
The formula to be used is pretty simple but a problem arise if I need to use 10min or 30min as the result would be:
(10min)
02-Jan-2013 15:50:00
02-Jan-2013 16:00:00
02-Jan-2013 16:10:00
(30min)
02-Jan-2013 15:30:00
02-Jan-2013 16:00:00
What I would like to have is an extra interval 16:20:00 for the ten min case and 16:30:00 for the 30min case. The only solution I can come up with is moving the bound to 16:30 and adding an if statement to remove the extra observations in case they are not needed or keep the bound at 16:15:00 and adding an if statement to add the extra observation in case they are needed.
Is there anyway to do a one-line able to treat these two cases?
Matlab creates ranges such that all values are inside the limits. If you want to add one additional value right outside the limits, you can modify the upper limit by adding almost one interval length to the end:
step = 15;
1:step:100+0.99*step
ans =
1 16 31 46 61 76 91 106
I was hoping someone that is good with math and loops could help me out. I'm writing a program in Objective C where I need to come up with a way to do a cycle. If you don't know Objective C I would appreciate any help in pseudo code just to help me figure this out.
What I need is a scale that is based on two dates. I know this will be some sort of loop but not sure how to figure it out.
For instance, lets say that the first date is 5/25/1976 and the second date is 9/25/2009. Every 25 days there will be a "peak" so it's value will be 100. If I divide 23 in half I get 12 (rounded) so it would be the opposite or "valley" so it's numerical value would be 0. In other words on the 23rd day it would be at 100 but then on the 24th day it would start going back down and then bottom out 12 days later and then start the cycle back up and top out again at 23 days.
What I need to be able to do is find the numerical value for any given date in between any two given dates.
Thanks for any help you can offer!
value = 100*cos(2*pi*(numDays/25))
Or something like that.
Calculate the difference in days (optionally in fractional days too) between the starting point and the day you want the value for.
Divide by the cycle period (could be 23 or 25 according to the question).
Take the fractional part.
Apply the correct periodic function - for example, either sin() or cos(), appropriately scaled for the trigonometric functions (multiply the fraction by 2π).
You could simulate the shape by values out of a table describing the values indexed on days into the period (so you would use waveform[Δt mod period] to determine the value).
The NSDate class has a method timeIntervalSinceDate that will give you then number of seconds between two dates. You could calculate the number of days between two dates like this:
- (double) daysBetweenStart:(NSDate*)start end:(NSDate*)end
{
return [start timeIntervalSinceDate:end] / 86400.0; // seconds in a day
}
You could use this to compute a step function based on that:
- (double) someDescriptiveFunctionName:(NSDate*)date fromDate:(NSDate*)start
{
double days = [self daysBetweenStart:start end:date];
if ((int) days % 23 == 0)
return 100.0;
else
return 0.0;
}
This function returns 100.0 if the given date is between 23 and 24 days from the start, and 0.0 otherwise. You could substitute 23 for whatever period you like. I'm not sure if this is what you wanted, so clarify your question if it wasn't.
Disclaimer: This is Cocoa. Hopefully it's the same as iPhone Cocoa?