Sut:
create table meter.materialized_quarters
(
id int4 not null generated by default as identity,
tm timestamp without time zone
,constraint pk_materialized_quarters primary key (id)
--,constraint uq_materialized_quarters unique (tm)
);
Then setup data:
insert into meter.materialized_quarters (tm)
select GENERATE_SERIES ('1999-01-01', '2030-10-30', interval '15 minute');
And check data:
select count(*),tm
from meter.materialized_quarters
group by tm
having count(*)> 1
Some results:
count|tm |
-----+-----------------------+
2|1999-10-31 02:00:00.000|
2|1999-10-31 02:15:00.000|
2|1999-10-31 02:30:00.000|
2|1999-10-31 02:45:00.000|
2|2000-10-29 02:00:00.000|
2|2000-10-29 02:15:00.000|
2|2000-10-29 02:30:00.000|
2|2000-10-29 02:45:00.000|
2|2001-10-28 02:00:00.000|
2|2001-10-28 02:15:00.000|
2|2001-10-28 02:30:00.000|
....
Details:
select * from meter.materialized_quarters where tm = '1999-10-31 01:45:00';
Result:
id |tm |
-----+-----------------------+
29092|1999-10-31 01:45:00.000|
As I see, 29092 is maximum series of nonduplicated data generated by: GENERATE_SERIES with 15 minutes interval.
How to fill table (meter.materialized_quarters) from 1999 to 2030?
One solution is:
insert into meter.materialized_quarters (tm)
select GENERATE_SERIES ('1999-01-01', '1999-10-31 01:45:00', interval '15 minute');
then:
insert into meter.materialized_quarters (tm)
select GENERATE_SERIES ('1999-10-31 02:00:00.000', '2000-10-29 00:00:00.000', interval '15 minute');
and again, and again.
Or
with bad as (
select count(*),tm
from meter.materialized_quarters
group by tm
having count(*)> 1
)
, ids as (
select mq1.id, mq2.id as iddel
from meter.materialized_quarters mq1 inner join bad on bad.tm = mq1.tm inner join meter.materialized_quarters mq2 on bad.tm = mq2.tm
where mq1.id<mq2.id
)
delete from meter.materialized_quarters
where id in (select iddel from ids);
Is there more 'elegant' way?
EDIT.
I see the problem.
xxxx-10-29 02:00:00 - summer time become winter time.
select GENERATE_SERIES ('1999-10-31 01:45:00', '1999-10-31 02:00:00', interval '15 minute');
Your problem is the conversion from timestamp WITH time zone which is returned by generate_series() and your column which is defined as timestamp WITHOUT time zone.
1999-10-31 is the day where daylight savings time changes (at least in some countries)
If you change your column to timestamp WITH time zone your code works without any modification.
Example
If you want to stick with timestamp WITHOUT timestamp you need to convert the value returned by generate_series()
insert into materialized_quarters (tm)
select g.tm at time zone 'UTC' --<< change to the time zone you need
from GENERATE_SERIES ('1999-01-01', '2030-10-30', interval '15 minute') as g(tm)
Example
I have a table like follows:
id start_date end_date
1 2020-01-01 2020-05-01
2 2020-03-01 2021-04-02
I need to be able to split the rows by financial year e.g. 2020-04-01 -> 2021-03-31)
So the result of the query would be as follows:
id start_date end_date
1 2020-01-01 2020-03-31
1 2020-04-01 2020-05-01
2 2020-03-01 2020-03-31
2 2020-04-01 2021-03-31
2 2021-04-01 2021-04-02
Actually another post helped me resolve this: Date split-up based on Fiscal Year
DROP TABLE your_table;
CREATE TABLE your_table (id int, start_date date, end_date date);
INSERT INTO your_table VALUES (1, '2020-01-01', '2020-05-01');
INSERT INTO your_table VALUES (2, '2020-03-01', '2021-04-02');
SELECT
id,
GREATEST(start_date, ('01-04-'||series.year)::date) AS year_start,
LEAST(end_date, ('31-03-'||series.year + 1)::date) AS year_end
FROM
(SELECT
id,
start_date,
end_date,
generate_series(
date_part('year', your_table.start_date - INTERVAL '3 months')::int,
date_part('year', your_table.end_date - INTERVAL '3 months')::int)
FROM your_table) AS series(id, start_date, end_date, year)
ORDER BY
start_date;
Result:
"id","year_start","year_end"
1,"2020-01-01","2020-03-31"
1,"2020-04-01","2020-05-01"
2,"2020-03-01","2020-03-31"
2,"2020-04-01","2021-03-31"
2,"2021-04-01","2021-04-02"
I’m building a booking system where a user will set their availability eg: I’m available Monday’s from 9am to 11am, Tuesdays from 9am to 5pm etc… and need to generate a list of time slots 15mins apart from their availability.
I have the following table (but am flexible to changing this):
availabilities(day_of_week text, start_time: time, end_time: time)
which returns records like:
‘Monday’ | 09:00:00 | 11:00:00
‘Monday’ | 13:00:00 | 17:00:00
‘Tuesday’ | 08:00:00 | 17:00:00
So I’m trying to build a stored procedure to generate a list of time slots so far I've got this:
create or replace function timeslots ()
return setof timeslots as $$
declare
rec record;
begin
for rec in select * from availabilities loop
/*
convert 'Monday' | 09:00:00 | 11:00:00 into:
2020-02-03 09:00:00
2020-02-03 09:15:00
2020-02-03 09:30:00
2020-02-03 09:45:00
2020-02-03 10:00:00
and so on...
*/
return next
end loop
$$ language plpgsql stable;
I return a setof instead of a table as I'm using Hasura and it needs to return a setof so I just create a blank table.
I think I'm on the right track but am currently stuck on:
how do I create a timestamp from 'Monday' 09:00:00 for the next monday as I only care about timeslots from today onwards?
how do I convert 'Monday' | 09:00:00 | 11:00:00 into a list of time slots 15 mins apart?
how do I create a timestamp from 'Monday' 09:00:00 for the next monday
as I only care about timeslots from today onwards?
You can use date_trunc for this (see this question for more info):
SELECT date_trunc('week', current_date) + interval '1 week';
From the docs re week:
The number of the ISO 8601 week-numbering week of the year. By
definition, ISO weeks start on Mondays
So taking this value and adding a week gives next Monday (you may need to ammend this behaviour based upon what you want to do if today is monday!).
how do I convert 'Monday' | 09:00:00 | 11:00:00 into a list of time
slots 15 mins apart?
This is a little tricker; generate_series will give you the timeslots but the trick is getting it into a result set. The following should do the job (I have included your sample data; change the values bit to refer to your table) - dbfiddle :
with avail_times as (
select
date_trunc('week', current_date) + interval '1 week' + case day_of_week when 'Monday' then interval '0 day' when 'Tuesday' then interval '1 day' end + start_time as start_time,
date_trunc('week', current_date) + interval '1 week' + case day_of_week when 'Monday' then interval '0 day' when 'Tuesday' then interval '1 day' end + end_time as end_time
from
(
values
('Monday','09:00:00'::time,'11:00:00'::time),
('Monday','13:00:00'::time,'17:00:00'::time),
('Tuesday','08:00:00'::time,'17:00:00'::time)
) as availabilities (day_of_week,
start_time,
end_time) )
select
g.ts
from
(
select
start_time,
end_time
from
avail_times) avail,
generate_series(avail.start_time, avail.end_time - interval '1ms', '15 minutes') g(ts);
A few notes:
The CTE avail_times is used to simplify things; it generates two columns (start_time and end_time) which are the full timestamps (so including the date). In this example the first row is "2020-02-03 09:00:00, 2020-02-03 11:00:00" (I'm running this on 2020-02-02 so 2020-02-03 is next Monday).
The way I'm converting 'monday' etc to a day of the week is a bit of a hack (and I have not bothered to do the full week); there is probably a better way but storing the day of week as an integer would make this simpler.
I subtract 1ms from the end time because I'm assuming you dont want this in the result set.
The main query is using a LATERAL Subquery. See this question for more info.
Aditional Question
how to adjust this so I can pass in a start and end date so I can get
time slots for a particular period
You could do something like the following (just adjust the dates CTE to return whatever days you want to include; you could convert to a function or just pass the dates in as parameters).
Note that as #Belayer mentions my original solution did not cater for shifts over midnight so this addresses that too.
with dates as (
select
day
from
generate_series('2020-02-20'::date, '2020-03-10'::date, '1 day') as day ),
availabilities as (
select
*
from
(
values (1,'09:00:00'::time,'11:00:00'::time),
(1,'13:00:00'::time,'17:00:00'::time),
(2,'08:00:00'::time,'17:00:00'::time),
(3,'23:00:00'::time,'01:00:00'::time)
) as availabilities
(day_of_week, -- 1 = monday
start_time,
end_time) ) ,
avail_times as (
select
d.day + start_time as start_time,
case
end_time > start_time
when true then d.day
else d.day + interval '1 day' end + end_time as end_time
from
availabilities a
inner join dates d on extract(ISODOW from d.day) = a.day_of_week )
select
g.ts
from
(
select
start_time,
end_time
from
avail_times) avail,
generate_series(avail.start_time, avail.end_time - interval '1ms', '15 minutes') g(ts)
order by
g.ts;
The following uses much of the techniques mentioned by #Brits. They present some very good information, so I'll not repeat but suggest you review it (and the links).
I do however take a slightly different approach. First a couple table changes. I use the ISO day of week 1-7 (Monday-Sunday) rather than the day name. The day name is easily extracted for the dater later.
Also I use interval instead to time for start and end times. ( A time data type works for most scenarios but there is one it doesn't (more later).
One thing your description does not make clear is whether the ending time is included it the available time or not. If included the last interval would be 11:00-11:15. If excluded the last interval is 10:45-11:00. I have assumed to excluded it. In the final results the end time is to be read as "up to but not including".
-- setup
create table availabilities (weekday integer, start_time interval, end_time interval);
insert into availabilities (weekday , start_time , end_time )
select wkday
, start_time
, end_time
from (select *
from (values (1, '09:00'::interval, '11:00'::interval)
, (1, '13:00'::interval, '17:00'::interval)
, (2, '08:00'::interval, '17:00'::interval)
, (3, '08:30'::interval, '10:45'::interval)
, (4, '10:30'::interval, '12:45'::interval)
) as v(wkday,start_time,end_time)
) r ;
select * from availabilities;
The Query
It begins with a CTE (next_week) generates a entry for each day of the week beginning Monday and the appropriate ISO day number for it. The main query joins these with the availabilities table to pick up times for matching days. Finally that result is cross joined with a generated timestamp to get the 15 minute intervals.
-- Main
with next_week (wkday,tm) as
(SELECT n+1, date_trunc('week', current_date) + interval '1 week' + n*interval '1 day'
from generate_series (0, 6) n
)
select to_char(gdtm,'Day'), gdtm start_time, gdtm+interval '15 min' end_time
from ( select wkday, tm, start_time, end_time
from next_week nw
join availabilities av
on (av.weekday = nw.wkday)
) s
cross join lateral
generate_series(start_time+tm, end_time+tm- interval '1 sec', interval '15 min') gdtm ;
The outlier
As mentioned there is one scenario where a time data type does not work satisfactory, but you may not nee it. What happens when a shift worker says they available time is 23:00-01:30. Believe me when a shift worker goes to work at 22:00 of Friday, 01:30 is still Friday night, even though the calendar might not agree. (I worked that shift for many years.) The following using interval handles that issue. Loading the same data as prior with an addition for the this case.
insert into availabilities (weekday, start_time, end_time )
select wkday
, start_time
, end_time + case when end_time < start_time
then interval '1 day'
else interval '0 day'
end
from (select *
from (values (1, '09:00'::interval, '11:00'::interval)
, (1, '13:00'::interval, '17:00'::interval)
, (2, '08:00'::interval, '17:00'::interval)
, (3, '08:30'::interval, '10:45'::interval)
, (5, '23:30'::interval, '02:30'::interval) -- Friday Night - Saturday Morning
) as v(wkday,start_time,end_time)
) r
;
select * from availabilities;
Hope this helps.
I have a table in my postgres database which has a column of dates. I want to search which of those dates is missing - for example:
date
2016-11-09 18:30:00
2016-11-09 19:00:00
2016-11-09 20:15:00
2016-11-09 22:20:00
2016-11-09 23:00:00
Here, |2016-11-09 21:00:00| is missing. After sorting my generated series if my table has an entry between two slots (slot of 1 hr interval) i need to remove that.
I want to make a query with generate_series that returns me the date which is missing. Is this possible?.
sample query that i used to generate series.
SELECT t
FROM generate_series(
TIMESTAMP WITH TIME ZONE '2016-11-09 18:00:00',
TIMESTAMP WITH TIME ZONE '2016-11-09 23:00:00',
INTERVAL '1 hour'
) t
EXCEPT
SELECT tscol
FROM mytable;
But this query is not removing 2016-11-09 18:30:00,2016-11-09 20:15:00 etc. cuz i used except.
This is not a gaps-and-island problem. You just want to find the 1 hour intervals for which no record exist in the table.
EXCEPT does not work here because it does equality comparison, while you want to check if a record exists or not within a range.
A typical solution for this is to use a left join antipattern:
select dt
from generate_series(
timestamp with time zone '2016-11-09 18:00:00',
timestamp with time zone '2016-11-09 23:00:00',
interval '1 hour'
) d(dt)
left join mytable t
on t.tscol >= dt and t.tscol < dt + interval '1 hour'
where t.tscol is null
You can also use not exists:
select dt
from generate_series(
timestamp with time zone '2016-11-09 18:00:00',
timestamp with time zone '2016-11-09 23:00:00',
interval '1 hour'
) d(dt)
where not exists (
select 1
from mytable t
where t.tscol >= dt and t.tscol < dt + interval '1 hour'
)
In this demo on DB Fiddle, both queries return:
| dt |
| :--------------------- |
| 2016-11-09 21:00:00+00 |
Have two dates - '2018-05-01' and '2018-06-01'. I would like to expand this window to the past by day difference of those dates.
SELECT * FROM data
WHERE
start_time > CAST('2018-05-01' AS timestamptz) - INTERVAL '30 DAY'
AND start_time < CAST('2018-06-01' AS timestamptz)
How can I replace INTERVAL '30 DAY' with number of days between given dates without explicitly defining number of days? I know to calculate day difference:
date_part('day',age('2018-05-01', '2018-06-01'))
But not sure how to incorporate into the substraction. Dates and days between them will change.
You can use date_trunc('mon', some_date_expression) to round down to the start of a month:
select date_trunc('mon', now() - '3 mon'::interval) as date_begin
, date_trunc('mon', now() - '1 day'::interval) as date_end
;
Result
date_begin | date_end
------------------------+------------------------
2018-03-01 00:00:00+01 | 2018-06-01 00:00:00+02
(1 row)
You can simply subtract the difference from the start date:
with t (start_date, end_date) as (
values (date '2018-05-01', date '2018-06-01')
)
select start_date - (end_date - start_date) as new_start,
end_date
from t;
returns
new_start | new_end
-----------+-----------
2018-03-31 | 2018-06-01