Fetch week start date and week end date from Date - pyspark

Question

I need to fetch week start date and week end date from a given date, taking into account that the week starts from Sunday and ends on Saturday.
I referred this post but this takes monday as starting day of week. Is there any inbuilt function in spark which can take care of this?

Answer 1

Find out the day of the week and Use selectExpr to iterate through columns , and making Sunday as week start date
from pyspark.sql import functions as F
df_b = spark.createDataFrame([('1','2020-07-13')],[ "ID","date"])
df_b = df_b.withColumn('day_of_week', F.dayofweek(F.col('date')))
df_b = df_b.selectExpr('*', 'date_sub(date, day_of_week-1) as week_start')
df_b = df_b.selectExpr('*', 'date_add(date, 7-day_of_week) as week_end')
df_b.show()
+---+----------+-----------+----------+----------+
| ID| date|day_of_week|week_start| week_end|
+---+----------+-----------+----------+----------+
| 1|2020-07-13| 2|2020-07-12|2020-07-18|
+---+----------+-----------+----------+----------+
Update in Spark SQL
Crete a Temporary view from the data-frame first
df_a.createOrReplaceTempView("df_a_sql")
Code here
%sql
select *, date_sub(date,dayofweek-1) as week_start,
date_sub(date, 7-dayofweek) as week_end
from
(select *, dayofweek(date) as dayofweek
from df_a_sql) T
Output
+---+----------+-----------+----------+----------+
| ID| date|day_of_week|week_start| week_end|
+---+----------+-----------+----------+----------+
| 1|2020-07-13| 2|2020-07-12|2020-07-18|
+---+----------+-----------+----------+----------+

Answer 2

Perhaps this is helpful -
Load the test data
val df = spark.sql("select cast('2020-07-12' as date) as date")
df.show(false)
df.printSchema()
/**
* +----------+
* |date |
* +----------+
* |2020-07-15|
* +----------+
*
* root
* |-- date: date (nullable = true)
*/
week starting from SUNDAY and ending SATURDAY
// week starting from SUNDAY and ending SATURDAY
df.withColumn("week_end", next_day($"date", "SAT"))
.withColumn("week_start", date_sub($"week_end", 6))
.show(false)
/**
* +----------+----------+----------+
* |date |week_end |week_start|
* +----------+----------+----------+
* |2020-07-12|2020-07-18|2020-07-12|
* +----------+----------+----------+
*/
week starting from MONDAY and ending SUNDAY
// week starting from MONDAY and ending SUNDAY
df.withColumn("week_end", next_day($"date", "SUN"))
.withColumn("week_start", date_sub($"week_end", 6))
.show(false)
/**
* +----------+----------+----------+
* |date |week_end |week_start|
* +----------+----------+----------+
* |2020-07-12|2020-07-19|2020-07-13|
* +----------+----------+----------+
*/
week starting from TUESDAY and ending MONDAY
// week starting from TUESDAY and ending MONDAY
df.withColumn("week_end", next_day($"date", "MON"))
.withColumn("week_start", date_sub($"week_end", 6))
.show(false)
/**
* +----------+----------+----------+
* |date |week_end |week_start|
* +----------+----------+----------+
* |2020-07-12|2020-07-13|2020-07-07|
* +----------+----------+----------+
*/

Answer 3

Find out the start date and end date of week in pyspark dataframe. Monday being the first day of week.
def add_start_end_week(dataframe, timestamp_col, StartDate, EndDate):
""""
Function:
Get the start date and the end date of week
args
dataframe: spark dataframe
column_name: timestamp column based on which we have to calculate the start date and end date
StartDate: start date column name of week
EndDate: end date column name of week
"""
dataframe = dataframe.withColumn(
'day_of_week', dayofweek(col(timestamp_col)))
# start of the week (Monday as first day)
dataframe = dataframe.withColumn('StartDate',when(col("day_of_week")>1, \
expr("date_add(date_sub({},day_of_week-1),1)".format(timestamp_col))). \
otherwise(expr("date_sub({},6)".format(timestamp_col))))
#End of the Week
dataframe = dataframe.withColumn('EndDate',when(col("day_of_week")>1, \
expr("date_add(date_add({},7-day_of_week),1)".format(timestamp_col))). \
otherwise(col("{}".format(timestamp_col))))
return dataframe
Validate the above function:
df = spark.createDataFrame([('2021-09-26',),('2021-09-25',),('2021-09-24',),('2021-09-23',),('2021-09-22',),('2021-09-21',),('2021-09-20',)], ['dt'])
dataframe = df.withColumn('day_of_week', dayofweek(col('dt')))
# start of the week (Monday as first day)
dataframe = dataframe.withColumn('StartDate',when(col("day_of_week")>1,expr("date_add(date_sub(dt,day_of_week-1),1)")).otherwise(expr("date_sub(dt,6)")))
#End of the Week
dataframe = dataframe.withColumn('EndDate',when(col("day_of_week")>1,expr("date_add(date_add(dt,7-day_of_week),1)")).otherwise(col("dt")))