python xarray resample between selected dates - date

I have the following xarray:
<xarray.Dataset>
Dimensions: (DATE: 14245, x: 214, y: 173)
Coordinates:
* DATE (DATE) datetime64[ns] 1980-01-01T12:00:00 ... 2018-1...
* x (x) float64 6.168e+05 6.171e+05 ... 6.698e+05 6.701e+05
* y (y) float64 5.113e+06 5.113e+06 ... 5.156e+06 5.156e+06
transverse_mercator int64 0
Data variables:
precipitation (DATE, y, x) float32 nan nan nan nan ... nan nan nan
Attributes:
CDI: Climate Data Interface version 1.9.9 (https://mpimet.mpg.de...
Conventions: CF-1.5
Title: Daily total precipitation Trentino-South Tyrol 250-meter re...
Created on: Fri Feb 26 21:30:51 2021
history: Fri Feb 26 23:31:30 2021: cdo -z zip -mergetime DAILYPCP_19...
CDO: Climate Data Operators version 1.9.9 (https://mpimet.mpg.de...
I would like to resample it but not accordingly the standard year as:
da.resample(time="Y").sum()
but using user defined time windows. Basically, I would like to sum between the 1st of October to the 30th September of the next year for all my time window. I supposed that I could do something with "da.sel()" but that seems to me neither straightforward nor elegant.

Related

How to convert datetimes to days of the year?

Suppose DT is a vector of type datetime. How can I get their days of the year?
This is an approximation, but I can't find the right function:
month(DT) * 30 + day(DT)
You can use the Matlab's day function with a vector of datetime elements (see the doc)
t = [datetime('yesterday');datetime('today');datetime('tomorrow')]; % vector of datetime
day( t, 'dayofyear')
that will give the Day-of-year number, from 1 to 365 or 366, depending on the year.
ans =
334
335
336
You can use datetime and between as follows:
d = datetime; %Read current time (just for the example).
jan_1st = datetime(d.Year, 1, 1); %Create datetime object of January fist in year of d.
day_in_year = between(jan_1st, d, 'Days'); %Get day in year.
The result is a calendarDuration object: 334d.
For converting to scalar, use split:
day_in_year = split(day_in_year, 'Days') + 1; % Add one in case the first day of the year is day 1 and not 0
Applying the solution on a vector of datetime may require a for loop due to the usage of datetime(d.Year, 1, 1).

MATLAB get date from week number and year

In Matlab 2016a, I have a vector of week numbers:
weekNum = [20 21 22 23];
Is there a way to convert the week numbers to their respective dates in 2018 starting on Sunday?
You can do this using the functions days and weekday. Starting with the first day of the year, you can subtract days to find the previous Sunday (if it isn't Sunday already), then multiply your vector weekNum by 7 to add offsets to that date for each desired week:
weekNum = [20 21 22 23];
t = datetime(2018, 1, 1, 'Format', 'dd-MMMM-yyyy'); % First day of the year
t = t - days(weekday(t)-1) + days((weekNum-1).*7);
And this gives you the following array of Sunday datetimes:
t =
1×4 datetime array
13-May-2018 20-May-2018 27-May-2018 03-June-2018
And you can confirm it works using week:
>> week(t)
ans =
20 21 22 23 % Same as weekNum
NOTE: The first week is usually defined as the week that includes January 1st, and can include days from the previous year. This means you could end up with a date in 2017 for a Sunday in week 1.
% 1st day of year
D = datetime(2018,1,1);
% datetime objects representing the Sunday of the specified weeks
Dt = D + (20:23)*7-7-(weekday(D)-1);
This assumes that week 1 is defined to be the first week for which at least one day is in 2018. If you use a different definition, adjust your indexes accordingly.
What I love about Matlab is that, sometimes, you can delegate computational tasks to the underlying Java framework, if you know it a little bit:
year = 2018;
week = [20 21 22 23];
dow = 1;
sep = repmat('-',4,1);
chain = [repmat(num2str(year),4,1) sep num2str(week.','%d') sep repmat(num2str(dow),4,1)];
df = java.text.SimpleDateFormat('yyyy-w-u');
for i = 1:4
date = 719529 + (df.parse(chain(i,:)).getTime() / 1000 / 3600 / 24);
disp(datestr(date,'dd-mm-yyyy'));
end
Output:
13-05-2018
20-05-2018
27-05-2018
03-06-2018

How to implement linear interpolation in Matlab - Coordinates and dates

I have a CSV file contains data of Hurricane location coordinates.
I'm new to Matlab so I'm not sure how to treat correctly date and hour cells, especially when they are displayed unconventionally.
I need to apply linear interpolation so I can get the date for every 30 minutes.
Let's assume you read the data in as numerical values
Now you have some matrix like so:
data = [20130928 0 21.1 50.0
20130928 600 22.2 50.3
20130928 1200 23.3 50.6
20130928 1800 24.2 50.6];
To convert the first two columns to datetime values, we could do this:
% Concatenate first two columns, including making all times 4 digits by 0 padding
fulltime = [num2str(data(:,1)), num2str(data(:,2), '%.4u')]
% Use datetime to convert (cell) times to dates with given format
dates = datetime(cellstr(fulltime),'inputformat', 'yyyyMMddHHmm');
>> dates = 28-Sep-2013 00:00:00
28-Sep-2013 06:00:00
28-Sep-2013 12:00:00
28-Sep-2013 18:00:00
Now we can easily interpolate. First create an array of times we want to use:
% Data value every 30 mins
interpdates = dates(1):hours(0.5):dates(end)
Then use interp1
interpolateddata = interp1(dates, data(:,3:4), interpdates);
>> interpolateddata = 21.1000 50.0000
21.1917 50.0250
21.2833 50.0500
21.3750 50.0750
...
24.1250 50.6000
24.2000 50.6000

Print dates on the x-axis from datafile by gnuplot

I have data file in this represetation:
07/06/2011 19,06
08/06/2011 31,9099
09/06/2011 31,5
10/06/2011 35
11/06/2011 30
12/06/2011 24,99
13/06/2011 24,5
14/06/2011 20,99
15/06/2011 20
etc. ( I have 1900 records in datafile... )
I'm using these operations:
set title "financial data"
set grid x y
set lmargin 9
set rmargin 2
plot 'data.dat' using 2
I would like to see on the x-axis dates, but I only see on the x-axis some count of records from datafile ( no dates ). I wouldn't like to see each date on x-axis. I want to see about 10 dates regularly on the x-axis:
example 10 dates:
x-axis = Jun 2011, Jul 2011, Aug 2011, Sep 2011,...
How can I do this?
You must have your x-data interpreted as time. This is done with
set xdata time
set timefmt '%d/%m/%Y'
set xtics 30*24*60*60
set grid
plot 'data.dat' using 1:2
The part set xtics 30*24*60*60 sets the distance between two major tics to be one month. I don't exactly know, how this is done internally, but with the following example file data.dat
01/01/2012 2
01/02/2012 4
01/03/2012 3
01/04/2012 5
the results seems to be correct:
For your requirements you may want to change the format of the x-axis with e.g.
set format x '%b %Y'
to display Jun 2011 ...

plotting the mean and standard deviation of data when there is more than one piece of data collected at a particular point

I have a matrix, in one columnn is the day of year and in the other is the data associated with that day of year. On some days of the year there are multiple data points, while others there is one or none. This makes it difficult to plot the information, what I would like to do is plot the data based on the mean and standard deviation of the data. So if data was collected three times on the 320th day of the year then the mean and standard deviation of these three data points would be found out and then when plotted the mean line would go through the mean and the standard deviation would represent error bars. So just say the data is:
DOY DATA
30, 12
30, 10
30, 8
120, 6
110, 5
I'd Like to transform it to:
DOY DATA STD
30, 10, 2
120, 6, 0
110, 5, 0
I then wish to plot this data with the standard deviation representing error bars.
How would I go about this?
Thanks
You can use Matlab's dataset to get easy grouping -
>> doy = [30 30 30 120 110]';
>> data = [12 10 8 6 5]';
The next line creates a dataset object with two columns, called "doy" and "data"
>> ds = dataset(doy, data);
This line says to calculate group statistics, using "doy" as the grouping variable, and computing the mean and std for each group. It also gives you the number of variables in each group in the column GroupCount.
>> grpstats(ds, 'doy', {'mean', 'std'})
ans =
doy GroupCount mean_data std_data
30 30 3 10 2
110 110 1 5 0
120 120 1 6 0
You could also use accumarray especially if you don't have the stats toolbox:
doy = [30 30 30 120 110]';
data = [12 10 8 6 5]';
[~,ind,subs] = unique(DOY);
means = accumarray(subs, data, size(ind), #mean);
stds = accumarray(subs, data, size(ind), #std);
final = [DOY(ind), means, stds]