I've run across an issue with date-time conventions in a file served via OPeNDAP. In particular this problem arises from the GrADS DODS Server (GDS). The GDS can serve files via reading a data descriptor (ascii file) that contains the specifics of the binary file (GDS is not limited to netCDF as input). The data descriptor file defines the start time and increment. Somehow, GDS converts this to a convention (?) of "days since 1-1-1 00:00:00". I've discovered an interesting issue with this.
As an example, the data served at http://apdrc.soest.hawaii.edu:80/dods/public_data/satellite_product/ASCAT/daily are defined to start on 03 March 2009. The OPeNDAP info page has the following for time:
time: Array of 64 bit Reals [time = 0..1141]
grads_dim: "t"
grads_mapping: "linear"
grads_size: "1142"
grads_min: "00z03mar2009"
grads_step: "1dy"
units: "days since 1-1-1 00:00:0.0"
long_name: "time"
minimum: "00z03mar2009"
maximum: "00z17apr2012"
resolution: 1.0
So it correctly gets the first time value. I've tested this in several client tools, including GrADS, Ferret, panoply, and IDV; all correctly recognize the first time value as March 3, 2009.
The trouble arises from the OPeNDAP time of "days since 1-1-1", which gets returned as 733470 (try an ncdump on the above file). As far as I can tell, this is actually March 5th, 2009. The result here is tools like Matlab and EDC get the initial date as March 5.
I'm wondering if anyone can shed some light on this? Thanks in advance,
Jim
Unless the other tools use the 'minimum' attribute somehow, I cant see how the first time element can be 03-Mar-2009.
However, March 5th, 2009 checks out if use the information in the units field:
Here's what I see in MATLAB:
>> ncdisp('http://apdrc.soest.hawaii.edu:80/dods/public_data/satellite_product/ASCAT/daily','time')
Source:
http://apdrc.soest.hawaii.edu:80/dods/public_data/satellite_product/ASCAT/daily
Format:
64bit
Dimensions:
time = 1142
Variables:
time
Size: 1142x1
Dimensions: time
Datatype: double
Attributes:
grads_dim = 't'
grads_mapping = 'linear'
grads_size = '1142'
grads_min = '00z03mar2009'
grads_step = '1dy'
units = 'days since 1-1-1 00:00:0.0'
long_name = 'time'
minimum = '00z03mar2009'
maximum = '00z17apr2012'
resolution = 1
Lets see what MATLAB's reference date is for numbers:
matlabRefDate = datestr(0)
matlabRefDate =
00-Jan-0000
The date number for this data source is since 1-1-1 so:
dataRefDate = '01-Jan-0001';
dataRefDateNum = datenum(dataRefDate)
dataRefDateNum =
367
Lets read the data and update the reference:
>> time = ncread('http://apdrc.soest.hawaii.edu:80/dods/public_data/satellite_product/ASCAT/daily','time');
>> datestr(time(1))
ans =
03-Mar-2008
>> correctedTime = time+dataRefDateNum;
>> datestr(correctedTime(1))
ans =
05-Mar-2009
I think I may have found the explanation:
From http://aa.usno.navy.mil/data/docs/JulianDate.php
The Julian date for CE 2009 March 3 00:00:00.0 UT is
JD 2454893.500000
The Julian date for CE 1 January 1 00:00:00.0 UT is
JD 1721423.500000
So the time of the ASCAT start since the year 1 origin is 733470 days. This agrees with
the GrADS/GDS accounting. The answer all comes down to reference dates. I suspect that what netCDF4 calls the "standard" calendar is this USNO version that switches from Gregorian to Julian, and this is not what matlab does.
It's probably better practice, therefore, not to reference to Jan 1, 0001 but rather something closer to present thus avoiding the potential two day difference between old and new calendars. However, if a data service is providing it (as is the case here), you're kind of stuck with it.
Eric Firing has a nice summary here:
http://matplotlib.org/api/dates_api.html
Related
I am using Tableau Server version 10.4.3
I have a dimension rTime which has string value. the entries in rTime is of like this
May 10, 2019 8:59:56.303 PM UTC
I want to check whether the rTime is today or not. I cannot use DateParse since my server doesn't have this functionality.
I created a calculated field CheckrTime with below content :
STR(LEFT(SPLIT([rTime],':',1),LEN(SPLIT([rTime],':',1))-2))
When I am dragging CheckrTime into workspace area, the output is coming in below format which is what I wanted :
May 10, 2019
When I am checking ISDATE("May 10, 2019") (a normal string), it is outputting TRUE as expected but when I am checking ISDATE(CheckrTime) it is outputting FALSE . Why?
The reason I am checking above thing is I am looking to use DATEDIFF function of tableau in this way:
DATEDIFF('day', DATE(CheckrTime), TODAY()) == 0
NOTE
If someone is wondering ,I have taken care of the level of granularity.
If you have a better solution then the one I am following, please do tell me.
This is working for me. I would expect May 10 <> May 16 (today) and therefore return false. However, when I change your example to today's date it does in fact come back as true.
You could also try this formula for the date LEFT([rTime],FINDNTH([rTime],' ',3)). It is slightly less complicated but will give you the same answer.
Calculated field (date type) depends the locale used which defines date format. Are you able to use Date function?
In Tableau website there is a example using english locale
For string 06May2017
DATE (LEFT([Original Date], 2) + "/" + MID([Original Date],3,3) + "/" + RIGHT([Original Date],4))
Above mentioned highligts / character between digits which is depending on locale
I am using python's dateutil module to parse recurring rules in my calendar. A problem arises with the following rrule:
from dateutil.rrule import rrulestr
def test():
rrule = 'FREQ=MONTHLY;INTERVAL=1;BYMONTHDAY=30;UNTIL=20180331T2359'
dtstart = datetime.datetime(2018, 1, 1, 18, 0)
dates = list(rrulestr(rrule + ';UNTIL=', dtstart = dtstart ))
This results in the following output (missing February):
datetime: 2018-01-30 18:00:00
datetime: 2018-03-30 18:00:00
Is this a bug in dateutil module and how should I fix it? Or am I doing something wrong?
Per my answer on this equivalent question, this is a deliberate feature of the iCalendar RFC that dateutil is implementing, because dateutil implements RFC 2445 and does not support all (or most) of the features of the updated RFC 5545. The relevant section of RFC 2445:
Recurrence rules may generate recurrence instances with an invalid date (e.g., February 30) or nonexistent local time (e.g., 1:30 AM on a day where the local time is moved forward by an hour at 1:00 AM). Such recurrence instances MUST be ignored and MUST NOT be counted as part of the recurrence set.
February is missing because 2018-02-30 is an invalid date (it's actually the example specified in the RFC).
One thing to note is that this pull request implements the functionality you want, but it is (as of this writing) currently blocked waiting for support of SKIP in BYWEEKNO. After that is merged, you will be able to modify your RRULE:
rrule = ('FREQ=MONTHLY;INTERVAL=1;BYMONTHDAY=30;UNTIL=20180331T2359;'+
'SKIP=BACKWARD;RSCALE=GREGORIAN')
Until then, your best option may be to use a BYMONTHDAY=28 and then add a relativedelta(day=30) to the result, e.g.:
from dateutil.rrule import rrule, MONTHLY
from dateutil.relativedelta import relativedelta
def end_of_month(dtstart, until):
rr = rrule(freq=MONTHLY, interval=1, bymonthday=28,
dtstart=dtstart, until=until)
for dt in rr:
yield dt + relativedelta(day=30)
This works because the 28th exists in all months (so the rrule will always generate it) and relativedelta has the "fall backwards at end of month" behavior that you are looking for. To be 100% safe, you can choose bymonthday=1 instead, it is equivalent in this case.
I want to split a string variable that comes from a gps, like this:
2013-08-13T14:33:29.000Z
into:
year = 2013 month = 08
I searched for a long time.
I've tried many things but haven't had any succeeded in getting anything to work.
Any ideas?
Here is a basic example of how to do this in Python.
This is neither the most efficient nor the cleanest way to do it, but this illustrates how do split strings and such in a manner that is relevant to a beginning Python programmer.
gpsstring = '2013-08-13T14:33:29.000Z'
year = gpsstring.split('T')[0].split('-')[0]
month = gpsstring.split('T')[0].split('-')[1]
day = gpsstring.split('T')[0].split('-')[2]
hour = gpsstring.split('T')[1].split(':')[0]
minute = gpsstring.split('T')[1].split(':')[1]
second = gpsstring.split('T')[1].split(':')[2].split('.')[0]
Essentially each variable is being set by splitting the gpsstring. We know where to split the gpsstring because the data you've provided is a standard timestamp interpreted from the NMEA Timestamp.
Edit - the Timezone info is the end of the string (the 000Z in this case) and can also be grabbed as follows:
timezone = gpsstring.split('T')[1].split(':')[2].split('.')[1]
Make sense?
I would like to calculate precisely the months between two dates to achieve this I do something like :
DateTimeZone ZONE = DateTimeZone.forID("Europe/London");
String DATE_FORMAT = "dd/MM/yyyy";
DateTimeFormatter FORMATTER = DateTimeFormat.forPattern(DATE_FORMAT).withZone(ZONE);
LocalDate dateTime = FORMATTER.parseLocalDate("28/05/2013");
LocalDate dateTime6MonthAfter = FORMATTER.parseLocalDate("28/02/2014");
Period todayUntilEndOfContract = new Period(dateTime,dateTime6MonthAfter);
todayUntilEndOfContract.getMonths() +"M/"+ todayUntilEndOfContract.getWeeks() +"W/"+ todayUntilEndOfContract.getDays() +"D/");
So this give me precisely 9 month between 28/05/2013 and 28/02/2014 BUT!!!
when I calculate the dates (29, 30, 31)/05/2013 with 28/02/2014 it always give me 9 month normally it should say 8M/3W/(6,5,4)D/ why is it always 9M/0W/0D please...?
Thanks a lot
Your issue is that you are expecting something a little different than what is provided. If I ask you the question "what is 30th January plus one month?" then there are a number of different answers which are valid under different assumptions. In Joda's case the answer is "28th February" (or 29th if a leap year).
Although you are asking for month-based information I would suggest that you obtain the number of days instead and use that as a basis, as it is probably closer to what you need:
int days = Days.daysBetween(dateTime, dateTime6MonthAfter).getDays();
You can always use this number to feed back in to your code and obtain different values to fit your requirements.
My problem can be easily created by the scenario below:
//create a gregorian calendar object that set the date and time as 4th June 2012 at 10:30PM
Calendar calendar = new GregorianCalendar(2012, 6, 4, 22, 30);
//when I print out these:
System.out.println(calendar.get(Calendar.DAY_OF_WEEK));
System.out.println(calendar.get(Calendar.MINUTE));
System.out.println(calendar.get(Calendar.HOUR));
System.out.println(calendar.get(Calendar.DATE));
System.out.println(calendar.get(Calendar.MONTH));
System.out.println(calendar.get(Calendar.YEAR));
//output reads as:
4
30
10
4
6
2012
//so does calendar.get(Calendar.DAY_OF_WEEK) == calendar.get(Calendar.DATE) ???
Just so that everyone is clear the 4th of June 2012 is a Monday, so shouldn't calendar.get(Calendar.DAY_OF_WEEK) return 0 as part of the first day of the week?
Thank for your all your help and concerns, please also verify the source that you are referring to.
user1442080
The month in java Calendar classes is 0-based. So June is month number 5.
You actually created an object representing July 4th, which happens to be a Wednesday, which is the fourth day of that week.
One should always look for values returned by Calendar.{field} e.g. like Calendar.SUNDAY, Calendar.MONDAY, Calendar.JANUARY, Calendar.MARCH etc. and so on.
This is because, the values returned by Calendar.{field} depends upon the TimeZone specified while creating Calendar instance.
You can try this by creating two calendar instances with different timezones:
Calendar.getInstance("BST")
and
Calendar.getInstance() // default timezone
and now try getting calendar.get(Calendar.DAY_OF_WEEK) which will return different integer values for these two instances.