I have a database table that contains a start visdate and an end visdate. If a date is within this range the asset is marked available. Assets belong to a user. My query takes in a date range (start and end date). I need to return data so that for a date range it will query the database and return a count of assets for each day in the date range that assets are available.
I know there are a few examples, I was wondering if it's possible to just execute this as a query/common table expression rather than using a function or a temporary table. I'm also finding it quite complicated because the assets table does not contain one date which an asset is available on. I'm querying a range of dates against a visibility window. What is the best way to do this? Should I just do a separate query for each day in the date range I'm given?
Asset Table
StartvisDate Timestamp
EndvisDate Timestamp
ID int
User Table
ID
User & Asset Join table
UserID
AssetID
Date | Number of Assets Available | User
11/11/14 5 UK
12/11/14 6 Greece
13/11/14 4 America
14/11/14 0 Italy
You need to use a set returning function to generate the needed rows. See this related question:
SQL/Postgres datetime division / normalizing
Example query to get you started:
with data as (
select id, start_date, end_date
from (values
(1, '2014-12-02 14:12:00+00'::timestamptz, '2014-12-03 06:45:00+00'::timestamptz),
(2, '2014-12-05 15:25:00+00'::timestamptz, '2014-12-05 07:29:00+00'::timestamptz)
) as rows (id, start_date, end_date)
)
select data.id,
count(data.id)
from data
join generate_series(
date_trunc('day', data.start_date),
date_trunc('day', data.end_date),
'1 day'
) as days (d)
on days.d >= date_trunc('day', data.start_date)
and days.d <= date_trunc('day', data.end_date)
group by data.id
id | count
----+-------
1 | 2
2 | 1
(2 rows)
You'll want to convert it to using ranges instead, and adapt it to your own schema and data, but it's basically the same kind of query as the one you want.
Related
I have two tables and I am trying to find data gaps in them where the dates do not overlap.
Item Table:
id unique start_date end_date data
1 a 2019-01-01 2019-01-31 X
2 a 2019-02-01 2019-02-28 Y
3 b 2019-01-01 2019-06-30 Y
Plan Table:
id item_unique start_date end_date
1 a 2019-01-01 2019-01-10
2 a 2019-01-15 'infinity'
I am trying to find a way to produce the following
Missing:
item_unique from to
a 2019-01-11 2019-01-14
b 2019-01-01 2019-06-30
step-by-step demo:db<>fiddle
WITH excepts AS (
SELECT
item,
generate_series(start_date, end_date, interval '1 day') gs
FROM items
EXCEPT
SELECT
item,
generate_series(start_date, CASE WHEN end_date = 'infinity' THEN ( SELECT MAX(end_date) as max_date FROM items) ELSE end_date END, interval '1 day')
FROM plan
)
SELECT
item,
MIN(gs::date) AS start_date,
MAX(gs::date) AS end_date
FROM (
SELECT
*,
SUM(same_day) OVER (PARTITION BY item ORDER BY gs)
FROM (
SELECT
item,
gs,
COALESCE((gs - LAG(gs) OVER (PARTITION BY item ORDER BY gs) >= interval '2 days')::int, 0) as same_day
FROM excepts
) s
) s
GROUP BY item, sum
ORDER BY 1,2
Finding the missing days is quite simple. This is done within the WITH clause:
Generating all days of the date range and subtract this result from the expanded list of the second table. All dates that not occur in the second table are keeping. The infinity end is a little bit tricky, so I replaced the infinity occurrence with the max date of the first table. This avoids expanding an infinite list of dates.
The more interesting part is to reaggregate this list again, which is the part outside the WITH clause:
The lag() window function take the previous date. If the previous date in the list is the last day then give out true (here a time changing issue occurred: This is why I am not asking for a one day difference, but a 2-day-difference. Between 2019-03-31 and 2019-04-01 there are only 23 hours because of daylight saving time)
These 0 and 1 values are aggregated cumulatively. If there is one gap greater than one day, it is a new interval (the days between are covered)
This results in a groupable column which can be used to aggregate and find the max and min date of each interval
Tried something with date ranges which seems to be a better way, especially for avoiding to expand long date lists. But didn't come up with a proper solution. Maybe someone else?
I'm using PostgreSql. I have a query that gives values for a specific date range. How can i get the columns as zero when there is no data for a specific date.
Example: the date range is '2018-06-10' to '2018-06-16', and there is no data for the dates 2018-06-15 and 2018-06-16. My existing query returns data for 10th,11th,12th,13th and 14th but not 15th and 16th. How can i get 15th and 16th dates with columns as zeroes.
here's my query:
SELECT 40 AS clientId,
0 AS accountId,
0 AS searchEngineId,
'present' AS dateType,
SUM(em.clicks) AS clicks,
To_date(em.DATE, 'YYYY-MM-dd') AS date
FROM emailreportstats em
WHERE em.clientid = 40
AND em.accountid IN ( 13, 14 )
AND em.DATE BETWEEN '2018-06-10' AND '2018-06-16'
GROUP BY em.DATE,
em.datetype;
Code to execute query:
Query query = em.createQuery(pSqlQuery, EmailReportStats.class);
emailReportStatses = query.getResultList();
You need a calendar table. You can generate one on the fly (available only for this query) using generate_series function. Then you would outer join your emailreportstats to that table, so that you would have all dates covered even if they are not present in your email table.
I've added coalesce() wrap to your calculated clicks column and also swapped the input for date column.
This is as far as I can go having no table structures and sample data to test it.
SELECT
40 AS clientId,
0 AS accountId,
0 AS searchEngineId,
'present' AS dateType,
COALESCE(SUM(em.clicks),0) AS clicks,
calendar.dt AS date
FROM (
SELECT dt::date as dt
FROM generate_series('2018-06-10', '2018-06-16', '1 day') g(dt) -- input your dates here
) calendar
LEFT JOIN emailreportstats em ON
calendar.dt = em.date::date
WHERE
em.clientid = 40
AND em.accountid IN ( 13, 14 )
GROUP BY
calendar.dt,
em.datetype
;
I have a table with a create date called created_at and a delete date called delete_at for each record. If the record was deleted, the field save that date; it's a logic delete.
I need to count the active records in a specific month. To understand what is an active record for me, let's see an example:
For this example we'll use this hypothetical record:
id | created_at | deleted_at
1 | 23-01-2014 | 05-06-2014
This record is active for every days between its creation date and delete date. Including that last. So if I need count the active record for March, in this case, this record must be counted in every days of that month.
I have a query (really easy to do) that show the actives records for a specific month, but my principal problem is how to count that actives for each day in that month.
SELECT
date_trunc('day', created_at) AS dia_creacion,
date_trunc('day', deleted_at) AS dia_eliminacion
FROM
myTable
WHERE
created_at < TO_DATE('01-04-2014', 'DD-MM-YYYY')
AND (deleted_at IS NULL OR deleted_at >= TO_DATE('01-03-2014', 'DD-MM-YYYY'))
Here you are:
select
TO_DATE('01-03-2014', 'DD-MM-YYYY') + g.i,
count( case (TO_DATE('01-03-2014', 'DD-MM-YYYY') + g.i) between created_at and coalesce(deleted_at, TO_DATE('01-03-2014', 'DD-MM-YYYY') + g.i)
when true then 1
else null
end)
from generate_series(0, TO_DATE('01-04-2014', 'DD-MM-YYYY') - TO_DATE('01-03-2014', 'DD-MM-YYYY')) as g(i)
left join myTable on true
group by 1
order by 1;
You can add more specific condition for joining only relevant records from myTable, but even without it gives you idea how to achieve counting as desired.
I am creating a Customer table and i want one of the attributes to be Expiry Date of credit card.I want the format to be 'Month Year'. What data type should i use? i want to use date but the format is year/month/day. Is there any other way to restrict format to only Month and year?
You can constrain the date to the first day of the month:
create table customer (
cc_expire date check (cc_expire = date_trunc('month', cc_expire))
);
Now this fails:
insert into customer (cc_expire) values ('2014-12-02');
ERROR: new row for relation "customer" violates check constraint "customer_cc_expire_check"
DETAIL: Failing row contains (2014-12-02).
And this works:
insert into customer (cc_expire) values ('2014-12-01');
INSERT 0 1
But it does not matter what day is entered. You will only check the month:
select
date_trunc('month', cc_expire) > current_date as valid
from customer;
valid
-------
t
Extract year and month separately:
select extract(year from cc_expire) "year", extract(month from cc_expire) "month"
from customer
;
year | month
------+-------
2014 | 12
Or concatenated:
select to_char(cc_expire, 'YYYYMM') "month"
from customer
;
month
--------
201412
Use either
char(5) for two-digit years, or
char(7) for four-digit years.
Code below assumes two-digit years, which is the form that matches all my credit cards. First, let's create a table of valid expiration dates.
create table valid_expiration_dates (
exp_date char(5) primary key
);
Now let's populate it. This code is just for 2013. You can easily adjust the range by changing the starting date (currently '2013-01-01'), and the "number" of months (currently 11, which lets you get all of 2013 by adding from 0 to 11 months to the starting date).
with all_months as (
select '2013-01-01'::date + (n || ' months')::interval months
from generate_series(0, 11) n
)
insert into valid_expiration_dates
select to_char(months, 'MM') || '/' || to_char(months, 'YY') exp_date
from all_months;
Now, in your data table, create a char(5) column, and set a foreign key reference from it to valid_expiration_dates.exp_date.
While you're busy with this, think hard about whether "exp_month" might be a better name for that column than "exp_date". (I think it would.)
As another idea you could essentially create some brief utilities to do this for you using int[]:
CREATE OR REPLACE FUNCTION exp_valid(int[]) returns bool LANGUAGE SQL IMMUTABLE as
$$
SELECT $1[1] <= 12 AND (select count(*) = 2 FROM unnest($1));
$$;
CREATE OR REPLACE FUNCTION first_invalid_day(int[]) RETURNS date LANGUAGE SQL IMMUTABLE AS
$$
SELECT (to_date($1[2]::text || $1[1]::text, CASE WHEN $1[2] < 100 THEN 'YYMM' ELSE 'YYYYMM' END) + '1 month'::interval)::date;
$$;
These work:
postgres=# select exp_valid('{04,13}');
exp_valid
-----------
t
(1 row)
postgres=# select exp_valid('{13,04}');
exp_valid
-----------
f
(1 row)
postgres=# select exp_valid('{04,13,12}');
exp_valid
-----------
f
(1 row)
Then we can convert these into a date:
postgres=# select first_invalid_day('{04,13}');
first_invalid_day
-------------------
2013-05-01
(1 row)
This use of arrays does not violate any normalization rules because the array as a whole represents a single value in its domain. We are storing two integers representing a single date. '{12,2}' is December of 2002, while '{2,12}' is Feb of 2012. Each represents a single value of the domain and is therefore perfectly atomic.
Requirements
I have data table that saves data in date ranges.
Each record is allowed to overlap previous record(s) (record has a CreatedOn datetime column).
New record can define it's own date range if it needs to hence can overlap several older records.
Each new overlapping record overrides settings of older records that it overlaps.
Result set
What I need to get is get per day data for any date range that uses record overlapping. It should return a record per day with corresponding data for that particular day.
To convert ranges to days I was thinking of numbers/dates table and user defined function (UDF) to get data for each day in the range but I wonder whether there's any other (as in better* or even faster) way of doing this since I'm using the latest SQL Server 2008 R2.
Stored data
Imagine my stored data looks like this
ID | RangeFrom | RangeTo | Starts | Ends | CreatedOn (not providing data)
---|-----------|----------|--------|-------|-----------
1 | 20110101 | 20110331 | 07:00 | 15:00
2 | 20110401 | 20110531 | 08:00 | 16:00
3 | 20110301 | 20110430 | 06:00 | 14:00 <- overrides both partially
Results
If I wanted to get data from 1st January 2011 to 31st May 2001 resulting table should look like the following (omitted obvious rows):
DayDate | Starts | Ends
--------|--------|------
20110101| 07:00 | 15:00 <- defined by record ID = 1
20110102| 07:00 | 15:00 <- defined by record ID = 1
... many rows omitted for obvious reasons
20110301| 06:00 | 14:00 <- defined by record ID = 3
20110302| 06:00 | 14:00 <- defined by record ID = 3
... many rows omitted for obvious reasons
20110501| 08:00 | 16:00 <- defined by record ID = 2
20110502| 08:00 | 16:00 <- defined by record ID = 2
... many rows omitted for obvious reasons
20110531| 08:00 | 16:00 <- defined by record ID = 2
Actually, since you are working with dates, a Calendar table would be more helpful.
Declare #StartDate date
Declare #EndDate date
;With Calendar As
(
Select #StartDate As [Date]
Union All
Select DateAdd(d,1,[Date])
From Calendar
Where [Date] < #EndDate
)
Select ...
From Calendar
Left Join MyTable
On Calendar.[Date] Between MyTable.Start And MyTable.End
Option ( Maxrecursion 0 );
Addition
Missed the part about the trumping rule in your original post:
Set DateFormat MDY;
Declare #StartDate date = '20110101';
Declare #EndDate date = '20110501';
-- This first CTE is obviously to represent
-- the source table
With SampleData As
(
Select 1 As Id
, Cast('20110101' As date) As RangeFrom
, Cast('20110331' As date) As RangeTo
, Cast('07:00' As time) As Starts
, Cast('15:00' As time) As Ends
, CURRENT_TIMESTAMP As CreatedOn
Union All Select 2, '20110401', '20110531', '08:00', '16:00', DateAdd(s,1,CURRENT_TIMESTAMP )
Union All Select 3, '20110301', '20110430', '06:00', '14:00', DateAdd(s,2,CURRENT_TIMESTAMP )
)
, Calendar As
(
Select #StartDate As [Date]
Union All
Select DateAdd(d,1,[Date])
From Calendar
Where [Date] < #EndDate
)
, RankedData As
(
Select C.[Date]
, S.Id
, S.RangeFrom, S.RangeTo, S.Starts, S.Ends
, Row_Number() Over( Partition By C.[Date] Order By S.CreatedOn Desc ) As Num
From Calendar As C
Join SampleData As S
On C.[Date] Between S.RangeFrom And S.RangeTo
)
Select [Date], Id, RangeFrom, RangeTo, Starts, Ends
From RankedData
Where Num = 1
Option ( Maxrecursion 0 );
In short, I rank all the sample data preferring the newer rows that overlap the same date.
Why do it all in DB when you can do it better in memory
This is the solution (I eventually used) that seemed most reasonable in terms of data transferred, speed and resources.
get actual range definitions from DB to mid tier (smaller amount of data)
generate in memory calendar of a certain date range (faster than in DB)
put those DB definitions in (much easier and faster than DB)
And that's it. I realised that complicating certain things in DB is not not worth it when you have executable in memory code that can do the same manipulation faster and more efficient.