Hello I'm trying to grouping a django query by hour excluding minutes.
Let me show a example:
[...]
date_example = models.DateTimeField("My Example", blank=True, null=True)
quantity = models.IntegerField("Quantity", default=1)
[...]
In my head this example should work:
ModelExample.objects.values("date_example__hour").annotate(total_quantity=Sum("quantity"))
But the output takes into account minutes and seconds.
Is there a way to group by hour excluding minutes and seconds?
Example: the group has 7 hours inside (7:00, 7:20, 7:45)
You can use ExtractHour [Django-doc] for that:
from django.db.models import Sum
from django.db.models import ExtractHour
ModelExample.objects.values(
hour=ExtractHour('date_example')
).annotate(
total_quantity=Sum('quantity')
).order_by('hour')
Related
I am trying to create a list of the last days of each month for the past n months from the current date but not including current month
I tried different approaches:
def last_n_month_end(n_months):
"""
Returns a list of the last n month end dates
"""
return [datetime.date.today().replace(day=1) - datetime.timedelta(days=1) - datetime.timedelta(days=30*i) for i in range(n_months)]
somehow this partly works if each every month only has 30 days and also not work in databricks pyspark. It returns AttributeError: 'method_descriptor' object has no attribute 'today'
I also tried the approach mentioned in Generate a sequence of the last days of all previous N months with a given month
def previous_month_ends(date, months):
year, month, day = [int(x) for x in date.split('-')]
d = datetime.date(year, month, day)
t = datetime.timedelta(1)
s = datetime.date(year, month, 1)
return [(x - t).strftime('%Y-%m-%d')
for m in range(months - 1, -1, -1)
for x in (datetime.date(s.year, s.month - m, s.day) if s.month > m else \
datetime.date(s.year - 1, s.month - (m - 12), s.day),)]
but I am not getting it correctly.
I also tried:
df = spark.createDataFrame([(1,)],['id'])
days = df.withColumn('last_dates', explode(expr('sequence(last_day(add_months(current_date(),-3)), last_day(add_months(current_date(), -1)), interval 1 month)')))
I got the last three months (Sep, oct, nov), but all of them are the 30th but Oct has Oct 31st. However, it gives me the correct last days when I put more than 3.
What I am trying to get is this:
(last days of the last 4 months not including last_day of current_date)
daterange = ['2022-08-31','2022-09-30','2022-10-31','2022-11-30']
Not sure if this is the best or optimal way to do it, but this does it...
Requires the following package since datetime does not seem to have anyway to subtract months as far as I know without hardcoding the number of days or weeks. Not sure, so don't quote me on this....
Package Installation:
pip install python-dateutil
Edit: There was a misunderstanding from my end. I had assumed that all dates were required and not just the month ends. Anyways hope the updated code might help. Still not the most optimal, but easy to understand I guess..
# import datetime package
from datetime import date, timedelta
from dateutil.relativedelta import relativedelta
def previous_month_ends(months_to_subtract):
# get first day of current month
first_day_of_current_month = date.today().replace(day=1)
print(f"First Day of Current Month: {first_day_of_current_month}")
# Calculate and previous month's Last date
date_range_list = [first_day_of_current_month - relativedelta(days=1)]
cur_iter = 1
while cur_iter < months_to_subtract:
# Calculate First Day of previous months relative to first day of current month
cur_iter_fdom = first_day_of_current_month - relativedelta(months=cur_iter)
# Subtract one day to get the last day of previous month
cur_iter_ldom = cur_iter_fdom - relativedelta(days=1)
# Append to the list
date_range_list.append(cur_iter_ldom)
# Increment Counter
cur_iter+=1
return date_range_list
print(previous_month_ends(3))
Function to calculate date list between 2 dates:
Calculate the first of current month.
Calculate start and end dates and then loop through them to get the list of dates.
I have ignored the date argument, since I have assumed that it will be for current date. alternatively it can be added following your own code which should work perfectly.
# import datetime package
from datetime import date, timedelta
from dateutil.relativedelta import relativedelta
def gen_date_list(months_to_subtract):
# get first day of current month
first_day_of_current_month = date.today().replace(day=1)
print(f"First Day of Current Month: {first_day_of_current_month}")
start_date = first_day_of_current_month - relativedelta(months=months_to_subtract)
end_date = first_day_of_current_month - relativedelta(days=1)
print(f"Start Date: {start_date}")
print(f"End Date: {end_date}")
date_range_list = [start_date]
cur_iter_date = start_date
while cur_iter_date < end_date:
cur_iter_date += timedelta(days=1)
date_range_list.append(cur_iter_date)
# print(date_range_list)
return date_range_list
print(gen_date_list(3))
Hope it helps...Edits/Comments are welcome - I am learning myself...
I just thought a work around I can use since my last codes work:
df = spark.createDataFrame([(1,)],['id'])
days = df.withColumn('last_dates', explode(expr('sequence(last_day(add_months(current_date(),-3)), last_day(add_months(current_date(), -1)), interval 1 month)')))
is to enter -4 and just remove the last_date that I do not need days.pop(0) that should give me the list of needed last_dates.
from datetime import datetime, timedelta
def get_last_dates(n_months):
'''
generates a list of lastdates for each month for the past n months
Param:
n_months = number of months back
'''
last_dates = [] # initiate an empty list
for i in range(n_months):
last_dates.append((datetime.today() - timedelta(days=i*30)).replace(day=1) - timedelta(days=1))
return last_dates
This should give you a more accurate last_days
Given this domain:
class Burger{
String description
Date dateCreated
}
Currently, I have this namedQuery
queryOnDateCreated {Date dateArgument ->
eq 'dateCreated', dateArgument
}
I need a query that allows me find all the objects in the domain Burger with a specific dateCreated only taking into accountYear, Month and day (of month), while ignoring hours, minutes, seconds, miliseconds.
After some additional research, I found a solution which I'm going to share in case it helps someone else:
The named query needs to be as follows:
queryOnDateCreated {Date dateArgument ->
def dateArgumentIntervalEnd = DateUtils.addMilliseconds(dateArgument + 1, - 1)
between 'dateCreated', dateArgument, dateArgumentIntervalEnd
}
Explanation:
The "between" criteria returns every object in the domain whose date is between the interval given.
Since dateArgument is a Date created only with Year, Month and Day, it's time should be 00:00:00:000 (the first moment of the day).
Furthermore, "dateArgument + 1" holds the value of the next day (at the same time), which is why the substraction of 1 millisecond is required, that way "dateArgumentIntervalEnd" will hold the value of the same Year, Month and Day of "dateArgument" but the time will be 23:59:59:999 holding an interval of the whole day.
I'm using BigQuery standart SQL and i need to find the difference between 2 timestamps, in minutes.
For example:
Timestamp1 = '2016-10-10 09:40:00' |
Timestamp2 = '2016-10-10 09:50:00'
I want to return the difference:
Timestamp2-Timestamp1 = 10
I found how to do it with Legacy SQL, but it doesn't help:
https://cloud.google.com/bigquery/query-reference#datediff
Thank you !
check for TIMESTAMP_DIFF function
SELECT
TIMESTAMP "2016-10-10 09:50:00" as first_timestamp,
TIMESTAMP "2016-10-10 09:40:00" as second_timestamp,
TIMESTAMP_DIFF(TIMESTAMP "2016-10-10 09:50:00",
TIMESTAMP "2016-10-10 09:40:00", MINUTE) AS minutes;
I have one table with dates and another table where there is rather weekly data. My weeks start at Tuesday and the second table's date is supposed to determine the week (basically the Tuesday before the date is the start of the week; alternatively that date is an example day in that week).
How can I join the dates to information about weeks?
Here is the setup:
from datetime import datetime as dt
import pandas as pd
df=pd.DataFrame([dt(2016,2,3), dt(2016,2,8), dt(2016,2,9), dt(2016,2,15)])
df_week=pd.DataFrame([(dt(2016,2,4),"a"), (dt(2016,2,11),"b")], columns=["week", "val"])
# note the actual start of the weeks are the Tuesdays: 2.2., 9.2.
# I expect a new column df["val"]=["a", "a", "b", "b"]
I've seen pandas date_range, but I cannot see how to do that from there.
You're looking for DatetimeIndex.asof:
This will give you the closest index up to the day in df:
df_week.set_index('week', inplace=True)
df_week.index.asof(df['day'][1])
You can now use it to select the corresponding value:
df_week.loc[df_week.index.asof(df['day'][1])]
Finally, apply it to the entire dataframe:
df = pd.DataFrame([dt(2016,2,8), dt(2016,2,9), dt(2016,2,15)], columns=['day'])
df['val'] = df.apply(lambda row: df_week.loc[df_week.index.asof(row['day'])]['val'], axis=1)
I removed the first value from df because I didn't want to deal with edge cases.
Result:
day val
0 2016-02-08 a
1 2016-02-09 a
2 2016-02-15 b
recently I asked how to convert calendar weeks into a list of dates and received a great and most helpful answer:
convert calendar weeks into daily dates
I tried to apply the above method to create a list of dates based on a column with "year - month". Alas i cannot make out how to account for the different number of days in different months.
And I wonder whether the package lubridate 'automatically' takes leap years into account?
Sample data:
df <- data.frame(YearMonth = c("2016 - M02", "2016 - M06"), values = c(28,60))
M02 = February, M06 = June (M11 would mean November, etc.)
Desired result:
DateList Values
2016-02-01 1
2016-02-02 1
ect
2016-02-28 1
2016-06-01 2
etc
2016-06-30 2
Values would something like
df$values / days_in_month()
Thanks a million in advance - it is honestly very much appreciated!
I'll leave the parsing of the line to you.
To find the last day of a month, assuming you have GNU date, you can do this:
year=2016
month=02
last_day=$(date -d "$year-$month-01 + 1 month - 1 day" +%d)
echo $last_day # => 29 -- oho, a leap year!
Then you can use a for loop to print out each day.
thanks to answer 6 at Add a month to a Date and answer for (how to extract number with leading 0) i got an idea to solve my own question using lubridate. It might not be the most elegant way, but it works.
sample data
data <- data_frame(mon=c("M11","M02"), year=c("2013","2014"), costs=c(200,300))
step 1: create column with number of month
temp2 <- gregexpr("[0-9]+", data$mon)
data$monN <- as.numeric(unlist(regmatches(data$mon, temp2)))
step 2: from year and number of month create a column with the start date
data$StartDate <- as.Date(paste(as.numeric(data$year), formatC(data$monN, width=2, flag="0") ,"01", sep = "-"))
step 3: create a column EndDate as last day of the month based on startdate
data$EndDate <- data$StartDate
day(data$EndDate) <- days_in_month(data$EndDate)
step 4: apply answer from Apply seq.Date using two dataframe columns to create daily list for respective month
data$id <- c(1:nrow(data))
dataL <- setDT(data)[,list(datelist=seq(StartDate, EndDate, by='1 day'), costs= costs/days_in_month(EndDate)) , by = id]