Azure Synapse Pipeline Date Expression - Last Monday - azure-data-factory

I have the following azure function that is supposed to retrieve the date of the previous Monday. It works fine except for if the current date is a monday. I need the function to still retrieve the previous monday date if it is Monday or Tuesday. This is due to the time not being updated until middle of day tuesday.
#{formatDateTime( subtractFromTime( utcNow(), sub(dayOfWeek(utcNow()),1), 'Day' ), 'yyyy-MM-dd 00:00:00' )}
I am still learning Azure synapse so I am not sure if I can write an IF statement that accomplishes this or if there is a better way to write it.

I have reproduced the above and able to get the desired result by using the below dynamic content.
#if(greater(dayOfWeek(utcnow()),1),formatDateTime(addDays(subtractFromTime(utcnow(),dayOfWeek(utcnow()),'Day'),1),'yyyy/MM/dd'),formatDateTime(addDays(subtractFromTime(utcnow(),dayOfWeek(utcnow()),'Day'),-6),'yyyy/MM/dd'))
The day of week for a monday from last sunday is 1, so I am checking the current date's day of week is greater than monday or not.
If it is greater (Today is not a Monday), then I am giving last Monday by adding 1 day to the last Sunday.
If it is not, then I am subtracting -6 from the last Sunday which is the previous Monday.
This will work for all days, but we need to change the condition and the number which we are adding and subtracting as per the day we required.
Result:
If you want the result to be Monday even though current date is Monday or Tuesday, then give the expression for Tuesday also in the above condition using or.

Related

Find last day of the previous week in Teradata?

I want to find out the previous weeks's last day in Teradata SQL using Sunday as the last day of the week. For example, today is Friday 1/27, so the last week ended on Sunday (1/22) and I would want to return 2023-01-22.
Other examples:
If current date is '2023-01-02', then the output I require is '2023-01-01'
If current date is '2023-01-18', then the output I require is '2023-01-15'
With Hive query I would use:
date_sub(current_date, cast(date_format(current_date, 'u') as int));
What would the equivalent be in Teradata? I've tried using the code below but it seems to return the date of the closest Sunday instead of the date of the previous Sunday.
SELECT ROUND(current_date, 'd') (FORMAT 'yyyy-mm-dd');
There are several ways:
Probably the best one is one of the built-in functions to return the previous xxxday <= the input date:
Td_Sunday(Current_Date - 1)
Or the function to return the next xxxday > input date:
Next_Day(Current_Date - 8, 'sun')
Truncating is least understandable:
Trunc(Current_Date, 'IW') -1
TRUNC supports three variations, only IW is usable, but restricted to Monday as week start:
IW: the Monday of the ISO week
WW: the same day of the week as January 1st of the year
W: the same day of the week as the first day of the month
You can use the trunc function to return the first day of the a week, month, ect.
select trunc(current_date -7 ,'IW')
Current date today is 2023-01-27. This will return 2023-01-15, the previous Sunday.
EDIT: Sorry, meant to use the ISO week. As Dnoeth points out, the regular week option doesn't work consistently (which I didn't know, never used it for this before). Anyhoo, his answer is better than mine...

RRULE for every other week except last week of month

I need an RRULE for every other Saturday except the last week of the month. I tried to create one with a weekly frequency, but didn't know how to apply an exception for the last week of the month:
RRULE:FREQ=MONTHLY;INTERVAL=1;BYDAY=SA
I also tried creating a rule with a monthly frequency, which allowed me to skip the last week of the month, but I didn't know how to make it every other week:
RRULE:FREQ=MONTHLY;INTERVAL=1;BYDAY=SA;BYSETPOS=-2,-3,-4,-5
I looked through the specification, but I'm not seeing anything that makes this possible.
The secret to the solution here is that BYMONTHDAY can be negative like BYSETPOS. This allows one to exclude the last 7 days of each month even though the number of days in the month varies. Your DTSTART should be on a SATURDAY, so one doesn't really need the BYDAY=SA
This rrule works in google calendar (if DTSTART is on Saturday and generally for any every 2nd week but not the last week rule for any day of week specified by the DTSTART):
RRULE:FREQ=WEEKLY;INTERVAL=2;BYMONTHDAY=-8,-9,-10,-11,-12,-13,-14,-15,-16,-17,-18,-19,-20,-21,-22,-23,-24,-25,-26,-27,-28,-29,-30,-31
See test calendar:
https://calendar.google.com/calendar/u/0?cid=ZXBwdWE4N2RwZm5xODVic3JydDJzaXFsY3NAZ3JvdXAuY2FsZW5kYXIuZ29vZ2xlLmNvbQ
Spec: https://datatracker.ietf.org/doc/html/rfc2445#page-43
and
https://icalevents.com/2447-need-to-know-the-possible-combinations-for-repeating-dates-an-ical-cheatsheet/ for the expansion & contraction (limiting) rules & combinations

Trigger Azure Data Factory Pipeline on second Friday from end of month and on Tues, Wed and Thurs of the same week

I need a Pipeline to trigger on the Monthend week (Tues, Wed, Thurs and Friday) of a month.
Monthend is defined as,
"Last but one" Friday Or
Second Friday from the end of the month.
For Example, For month of June 2021, 18th is the Monthend (Orange color as shown in the image)
Calendar Image
If its just on Monthend i.e. Second Friday from the end of calendar month, its easy. Just use Occurrance as -2 and day as Friday in the Scheduled trigger and add to a pipeline to trigger,
"schedule": {
"monthlyOccurrences": [
{
"day": "Friday",
"occurrence": -2
}
]
}
but I also need to run on the Tues, Wed and Thurs of the same week, which I find it difficult as these weekdays can be second or third from the end of the calendar month. For example: For June 2021, as shown in the image, I also need to run on 15th (Third Tuesday from the end of calendar month), 16th (Third Wednesday from the end of calendar month), 17th (Second Thursday from the end of calendar month).
Can you let me know if this can be implemented using triggers of Azure data factory? If not, any otherways of implementing? Thank You!
The scheduled trigger alone is not capable of that logic (as of 2021-05-04). Easiest solution would be to use some other scheduling application.
For a purely Data-Factory solution, schedule the trigger for all the days the desired days could possibly occur on. Then modify the pipeline to do logic to determine whether the current day is actually one of the desired days.
Implementation details and sample code
The logic:
Find the last day of the month (First of the next month less 1 day).
Subtract a week so you are in the second-to-last week
Loop over [0,-1,-2,-3,-4,-5,-6] as number of days to add to the date. This produces the dates of each day of the week.
Use the dayOfWeek function to change the date into which day of the week it is
Filter to get the Friday date
Ask whether today is between the Friday date and Friday date - 3 days

RRULE for the weekend including the first sunday of a month

We have a event every year on the weekend (Fr-Su) that includes the first Sunday of June. How would I create a iCalendar Event that expresses these three days (whole day events)?
Creating a rule for the first Sunday is easy. But for the Saturday and Friday, I did not succeed to create a rule that counts backwards (RFC 5545 says INTERVAL, COUNT must be positive]. Moreover I could not think of a different expression that would start from the Friday - it could be the last Friday of May, but also the first in June.
The RRRULE specification in RFC 5545 is lacking in this regard. The INTERVAL and COUNT values are for the repeating events, not the event itself. I've come across a similar issue when trying to define the USA day "Black Friday", the day after the 4th Thursday in November (Friday after Thanksgiving). The 4th Friday in November could occur the day after the 4th Thursday, or the prior week. There is not a way that I have found to make a RRULE for this situation.
I believe you will need to code the events individually instead of using a recurring rule.
RRULE:FREQ=YEARLY;BYDAY=FR;BYMONTH=5,6;BYSETPOS=2;BYMONTHDAY=-2,-1,1,2,3,4,5,6,7' seems to do the trick.
How can I write an ICS file for the Friday before the first Saturday of the month? led me to the right track: with 'BYMONTHDAY' I can count backwards from the end of the month.
The Friday before the first Sunday of the next month can be the last or the second to last day of the previous month, or up to the 5th day of the month. If I include May and June, I will get a set that includes the day. 'BYSETPOS' allows me to choose the second of the found Fridays. To always have the second in the set being my desired day, I include the 6th and 7th day of the month, which gives me a stable first Friday in May. Possibly matched additional Fridays in June are discarded by 'BYSETPOS' anyway.
Extending this to Saturday is simple, and the first Sunday of June is trivial.
I developed the rule with rrule.js
https://jakubroztocil.github.io/rrule/#/rfc/RRULE:BYDAY=FR;BYMONTH=5,6;BYSETPOS=2;BYMONTHDAY=-2,-1,1,2,3,4,5,6,7

Recurring calendar event on First of the month

I have a recurring calendar event on 1st of each month, when 1st falls on Sat/Sun is it possible for iCal to schedule it for last working day Fri?
If I understood you correctly the following RRULE should do the trick:
FREQ=MONTHLY;BYDAY=1MO,1TU,1WE,1TH,1FR,-1FR;BYMONTHDAY=1,-1,-2
See the next 100 instances
It basically does two things:
Iterate the first weekday that falls on the 1st day of the month
Iterate the last Friday of each month that falls on the last or 2nd last day of the month (in which cases the 1st of the next month falls on Saturday or Sunday).
A slightly shorter version that should yield the same results is
FREQ=MONTHLY;BYDAY=1MO,1TU,1WE,1TH,FR;BYMONTHDAY=1,-1,-2
This just iterates every Friday, but only keeps the ones that fall on the first, last and 2nd last day of each month. Though it looks like the recurrence expansion service above disagrees. That's probably a bug.
I'm assuming you mean "the last working day Friday of the previous month", in which case I don't think it possible. It would mean that some months have 2 occurrences when others have 0, which doesn't really work.
You could easily do the first working day of the month (the 1st, or the first Monday):
FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYSETPOS=1
Or, you could do always the last working day of the month:
FREQ=MONTHLY;BYDAY=MO,TU,WE,TH,FR;BYSETPOS=-1