Python dateutil - bug with choosing closest day with BYMONTHDAY - python-dateutil

I am using python's dateutil module to parse recurring rules in my calendar. A problem arises with the following rrule:
from dateutil.rrule import rrulestr
def test():
rrule = 'FREQ=MONTHLY;INTERVAL=1;BYMONTHDAY=30;UNTIL=20180331T2359'
dtstart = datetime.datetime(2018, 1, 1, 18, 0)
dates = list(rrulestr(rrule + ';UNTIL=', dtstart = dtstart ))
This results in the following output (missing February):
datetime: 2018-01-30 18:00:00
datetime: 2018-03-30 18:00:00
Is this a bug in dateutil module and how should I fix it? Or am I doing something wrong?

Per my answer on this equivalent question, this is a deliberate feature of the iCalendar RFC that dateutil is implementing, because dateutil implements RFC 2445 and does not support all (or most) of the features of the updated RFC 5545. The relevant section of RFC 2445:
Recurrence rules may generate recurrence instances with an invalid date (e.g., February 30) or nonexistent local time (e.g., 1:30 AM on a day where the local time is moved forward by an hour at 1:00 AM). Such recurrence instances MUST be ignored and MUST NOT be counted as part of the recurrence set.
February is missing because 2018-02-30 is an invalid date (it's actually the example specified in the RFC).
One thing to note is that this pull request implements the functionality you want, but it is (as of this writing) currently blocked waiting for support of SKIP in BYWEEKNO. After that is merged, you will be able to modify your RRULE:
rrule = ('FREQ=MONTHLY;INTERVAL=1;BYMONTHDAY=30;UNTIL=20180331T2359;'+
'SKIP=BACKWARD;RSCALE=GREGORIAN')
Until then, your best option may be to use a BYMONTHDAY=28 and then add a relativedelta(day=30) to the result, e.g.:
from dateutil.rrule import rrule, MONTHLY
from dateutil.relativedelta import relativedelta
def end_of_month(dtstart, until):
rr = rrule(freq=MONTHLY, interval=1, bymonthday=28,
dtstart=dtstart, until=until)
for dt in rr:
yield dt + relativedelta(day=30)
This works because the 28th exists in all months (so the rrule will always generate it) and relativedelta has the "fall backwards at end of month" behavior that you are looking for. To be 100% safe, you can choose bymonthday=1 instead, it is equivalent in this case.

Related

Same unixtime yields different date time in joda than the correct date time

I read from a very old post here on stackoverflow that joda is a possible solution to convert Unix timestamp.
import org.joda.time._
new DateTime(1511544070).toString("yyyy-MM-dd")
I got 1970-01-18 for this case, however, this is wrong because the date should be
according to this online converter: 11/24/2017 # 5:21pm (UTC)
It is possible the online converter is correct because the sample unix timestamp 1511544070 is from a dataset that date range is November 25 to December 03, 2017, the dataset is from China time which is 8 hours ahead of UTC, meaning 11/24/2017 # 5:21pm (UTC) is actually 11/25/2017 # 1:21am (Beijing Time)
Where can I get a working library or is there a working library that can get the same result like the online converter?
You can do that using java.time:
import java.time.{ LocalDateTime, ZoneOffset }
import java.time.format.DateTimeFormatter
LocalDateTime.ofEpochSecond(1511544070, 0, ZoneOffset.UTC)
.format(DateTimeFormatter.ofPattern("yyyy-MM-dd # h:mm a"))
Looking at the documentation for joda-time we see that a DateTime can take a Long specifying the milliseconds since 1 Jan 1970. However, you seem to be providing a value in seconds. Joda-time is actually calculating it correctly, since since 1511544070/(1000*3600*24) equals 17.49 days, i.e. 1970-01-18.
To get the expected result multiply with 1000:
new DateTime(1511544070*1000).toString("yyyy-MM-dd")
To get the time in another timezone, add withZone() as follows (for Shanghai/Beijing):
new DateTime(1511544070*1000).toString("yyyy-MM-dd")
.withZone(DateTimeZone.forID("Asia/Shanghai"))

RRULE for first Advent

I am currently trying to setup my own holiday iCalendar to which I can subscribe on, since I don't want to depend on 3rd party services.
I am currently trying to make the the VEVENTs for Christmas. The 2nd, 3rd and 4th advent, as well as the Christmas holidays are straight forward, however I have big issues to model the 1st advent.
Specifically, the problem is that the first advent can be in November and December (27th November to 3rd Devember)
How can I make a recurring event (or, more specifically, the RRULE) to cover all cases for the 1st advent?
What I've tried
My first idea was this:
FREQ=YEARLY;INTERVAL=1;BYMONTH=11,12;BYMONTHDAY=27,28,29,30,1,2,3;BYDAY=SU
The idea was to just pick the one Sunday in between 27th November and 3rd December. This does of course not work because BYMONTH expands the search to all days in November and December, and BYMONTHDAY limits the search to those days in both months. I.e. November 1st, November 2nd, ... December 27th, December 28th, ..., which is of course not what I want.
Next, I tried to use BYYEARDAY=331,332,333,334,335,336,337 instead of BYMONTHDAY and BYMONTH, but unfortunately my webdav server (Nextcloud, which uses Sabre as far as I know. I got an error message "Invalid BYYEARDAY rule") does not support this.
My next idea was to use multiple RRULEs -- at least I did not see any passage in the RFC stating that only one RRULE is allowed at most. So I ended up with:
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=SU;BYMONTHDAY=27,28,29,30;BYMONTH=11
RRULE:FREQ=YEARLY;INTERVAL=1;BYDAY=SU;BYMONTHDAY=1,2,3;BYMONTH=12
Didn't work as well. My last resort was to create two separate VEVENTs, one with the first RRULE above and one with the second RRULE above, but otherwise identical. This worked, but it left me confused.
Is there no better solution? How would you do it?
unfortunately my webdav server (Nextcloud, which uses Sabre as far as I know. I got an error message "Invalid BYYEARDAY rule") does not support this.
Well I think you should raise a bug report, because as far I can tell your solutions are correct and RFC-compliant.
I tried your solutions 2 and 3 with different libs (my own lib php-rrule, and rrule.js) and both options seems to work just fine.
FREQ=YEARLY;BYDAY=SU;BYYEARDAY=331,332,333,334,335,336,337
or combining 2
FREQ=YEARLY;INTERVAL=1;BYDAY=SU;BYMONTHDAY=27,28,29,30;BYMONTH=11
FREQ=YEARLY;INTERVAL=1;BYDAY=SU;BYMONTHDAY=1,2,3;BYMONTH=12
will both produce:
2018-12-02
2019-12-01
2020-11-29
2021-11-28
2022-11-27
2023-12-03
2024-12-01
2025-11-30
2026-11-29
2027-11-28
which according to Google and Wikipedia are the correct dates for the 1st Advent Sunday in the next 10 years.
Side note
at least I did not see any passage in the RFC stating that only one RRULE is allowed at most.
While not strictly forbidden, in RFC 5545 it's literally written everytime RRULE is mentioned:
;
; The following is OPTIONAL,
; but SHOULD NOT occur more than once.
;
rrule
Appendix A even states in the "New restrictions":
2. The "RRULE" property SHOULD NOT occur more than once in a
component.
That being said, multiple RRULE is a great feature, I don't know why they restricted it.
If I'm not mistaken, the first Advent is always the fifth last Sunday in a year. So the following rule should do the trick:
FREQ=YEARLY;BYDAY=-5SU
See the next 10 results: http://recurrence-expansion-service.appspot.com/reaas?dtstart=20181202&rrule=FREQ%3DYEARLY%3BBYDAY%3D-5SU&max_instances=10
Or to put it another way:
FREQ=YEARLY;BYDAY=SU;BYSETPOS=-5

Is this a valid time?

I get the following datetime String from a backend system: 2014-06-10+02:00.
Is this a valid datetime? There is no information about the time (I get only the date) but there is a time offset.
If it is valid according to which standard is this valid and what is the UTC time?
Thanks a lot
This is a valid date, not a date-time.
An offset-from-UTC is relevant to a date. For any given moment the date varies around the globe by time zone. For example, a few minutes after midnight in Paris France is a new day while still “yesterday” in Montréal Québec.
By the way, an offset-from-UTC is not a time zone. A time zone is a history of changes (past, present, and future) to the offset used by a particular region. A time zone has a name in format of continent/region such as America/Montreal.
With a date and an offset, you can determine the range of all moments occurring in that day, all the points on the timeline.
Example code in Java.
ZoneOffset offset = ZoneOffset.parse( "+02:00" );
LocalDate ld = LocalDate.parse( "2014-06-10" ) ;
OffsetDateTime odt = OffsetDateTime.of( ld , LocalTime.MIN , offset );
https://docs.oracle.com/javase/8/docs/api/java/time/format/DateTimeFormatter.html
The output is valid Date but not a valid Time as per ISO Date Specification. Please see ISO_OFFSET_DATE.
UTC (Coordinated Universal Time) is a time standard is defined by International Telecommunications Union.
If it is valid according to which standard is this valid and what is
the UTC time?
You have asked three questions in this line and the answer to these questions are as follows:
Is it valid?
Yes, it is a valid date string.
You have already mentioned in your question that it does not have a time part; rather, it has a (timezone) offset of +02:00 hours. So, it is just a valid date string, not a date-time string.
Which standard is this?
This is ISO 8601.
What is the UTC time?
A date starts with the start-of-the-day time which, in most cases, is 00:00 hours. However, for the timezones that observe DST, it may not be the case. Such timezones have generally one hour difference in the timezone offset between with and without DST.
Your string has a fixed (timezone) offset (+02:00); rather than a timezone itself (e.g. Africa/Cairo) and therefore, in this case, the start of the day is always 00:00 hours.
So, it can be written as 2014-06-10'T'00:00:00+02:00. As soon as you represent it in this way, I am sure you must have already guessed that it is equivalent to 2014-06-09'T'22:00:00Z where Z is the timezone designator for zero-timezone offset. It stands for Zulu and specifies the Etc/UTC timezone (which has the timezone offset of +00:00 hours).
Enough talking, let's write some code.
import java.time.OffsetDateTime;
import java.time.ZoneOffset;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeFormatterBuilder;
import java.time.temporal.ChronoField;
import java.util.Locale;
public class Main {
public static void main(String[] args) {
String str = "2014-06-10+02:00";
DateTimeFormatter dtf = new DateTimeFormatterBuilder()
.appendPattern("u-M-d['T'[H[:m[:s]]]]XXX")
.parseDefaulting(ChronoField.HOUR_OF_DAY, 0)
.parseDefaulting(ChronoField.MINUTE_OF_HOUR, 0)
.parseDefaulting(ChronoField.SECOND_OF_MINUTE, 0)
.toFormatter(Locale.ENGLISH);
OffsetDateTime odt = OffsetDateTime.parse(str, dtf);
System.out.println(odt);
OffsetDateTime odtUtc = odt.withOffsetSameInstant(ZoneOffset.UTC);
// The default format omits second and fraction-of-second if they are zero
System.out.println(odtUtc);
// Custom format
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("uuuu-MM-dd'T'HH:mm:ssXXX", Locale.ENGLISH);
System.out.println(formatter.format(odtUtc));
}
}
Output:
2014-06-10T00:00+02:00
2014-06-09T22:00Z
2014-06-09T22:00:00Z
Learn more about the the modern date-time API* from Trail: Date Time.
* For any reason, if you have to stick to Java 6 or Java 7, you can use ThreeTen-Backport which backports most of the java.time functionality to Java 6 & 7. If you are working for an Android project and your Android API level is still not compliant with Java-8, check Java 8+ APIs available through desugaring and How to use ThreeTenABP in Android Project.
yes it is correct date format.There are lot of place jerusalam..etc with the +2 hour you can find it in your system.

How to set day month and year values for a Date or DataAndTime object in Smalltalk (Pharo / Squeak)

I'm trying to find a a pre-defined method for objects of Class Date or Class DateAndTime, that allows me to create an new Date (or a new DateAndTime) by supplying three integers: one integer for the day of the month (1-31); one for the month (1 - 12); and a four-digit integer for the year.
( The closest I've found so far is Integer>>asYear )
Is there a method that can set all three parameters at once?
If I understand you correctly, you are trying to create an instance of Date from three integers representing the day, month and year of said date.
When facing a question like this you can browse the class, Date in this case, and check its protocol for instance creation methods. In Pharo there are several methods in this category, but it is easy so see (I think) that #year:month:day looks like a good candidate. So, you can try it. Just evaluate the expression
Date year: 2015 month: 12 day: 31
and see what happens (you can inspect or print it to see the result).
You will also find #newDay:month:year as another good candidate. You could try it too. Or you could just see that it sends our previous message and thus it is just a synonymous (which is present for compatibility with other dialects that support the Smalltalk-80/ANSI specification).
String>>asDate
allows for date-formatted Strings to be converted into Dates
e.g. (in a Workspace)
aDateString := String new . " prints as '' "
aDateString := '1984-11-03' . " prints as '1984-11-03' "
aDate := Date new . " prints as 1 January 1901 "
aDate := aDateString asDate . "prints as 3 November 1984"
The comment in the String>>asDate method states that "Many allowed forms, see Date>>#readFrom:"
Date>>readFrom: says
"Read a Date from the stream in any of the forms:
<day> <month> <year> (15 April 1982; 15-APR-82; 15.4.82; 15APR82)
<month> <day> <year> (April 15, 1982; 4/15/82)
<year>-<month>-<day> (1982-04-15) (ISO8601)"
Also relevant: String>>asDateAndTime
n.b. a DateAndTime object is an exact instant in time.
Date, Year, Month, Week and Schedule are all subclasses of Class Timespan, so they all have a start moment, a duration, and an end moment.

Noda Time Date Comparison

I am new to Noda Time and I basically want to compare if a date has expired or not. In my case I have an object with the date it was created, represented by a LocalDate and the amount of months it's valid as an int, so I wanted to do a simple:
if ( Now > (dateCreated + validMonths) ) expired = true;
But I can't find in the Noda Time documentation the proper way to get the Now Date (they only show how to get the Now Time as SystemClock.Instance.Now) and the proper way to handle time comparisons.
For example if today is January 1st 2015 and the document was created in December 1st 2014, and it was valid for one month, today it expires its one month validity.
I miss methods such as isBefore() and isAfter() to compare dates and times. Simple overloads of the < > operators could also be very helpful.
EDIT:
1 - Sorry, there are < > operators to compare dates.
2 - I solve my problem using this code (not tested yet!):
...
LocalDate dateNow = this.clock.Now.InZone(DateTimeZoneProviders.Tzdb.GetSystemDefault()).LocalDateTime.Date;
LocalDate dateExpiration = DataASO.PlusMonths(validity);
return (dateNow < dateExpiration);
To get the current date, you need to specify which time zone you're in. So given a clock and a time zone, you'd use:
LocalDate today = clock.Now.InZone(zone).Date;
While you can use SystemClock.Instance, it's generally better to inject an IClock into your code, so you can test it easily.
Note that in Noda Time 2.0 this will be simpler, using ZonedClock, where it will just be:
LocalDate today = zonedClock.GetCurrentDate();
... but of course you'll need to create a ZonedClock by combining an IClock and a DateTimeZone. The fundamentals are still the same, it's just a bit more convenient if you're using the same zone in multiple places. For example:
// These are IClock extension methods...
ZonedClock zonedClock = SystemClock.Instance.InTzdbSystemDefaultZone();
// Or...
ZonedClock zonedClock = SystemClock.Instance.InZone(specificZone);