I have the following tables
Location table
[ID] [int] IDENTITY(1,1) NOT NULL
Package table
[ID] [int] IDENTITY(1,1) NOT NULL
PackageObservation table
[PackageID] int
[LocationID] int
[Date] datetime
[Quantity] int
For a given location I want to select packages where the last observation of the package was at the location
What is the Transact SQL?
I think it involves a common table expression but I cant figure it out.
More information.
The following almost does it, but I don't really want to assume that the identity field is in date order
select max(id) ,packageid
from packageobservation o1
where not exists (
select o2.id from packageobservation o2
where o2.[date] > o1.[date] )
group by packageid
You can use following SQL statement:
DECLARE #locationID int = 1
SELECT po.PackageID, MAX(po.[Date]) AS DateAtLocation
FROM PackageObservation po
WHERE po.LocationID=#locationID
AND NOT EXISTS (SELECT * FROM PackageObservation po2
WHERE po2.PackageID = po.PackageID AND
po2.LocationID <> po.LocationID AND
po2.[Date] >= po.[Date] )
GROUP BY po.PackageID
For better speed you can also add combined index on [LocationID],[PackageID] and [Date].
Seems to me that using CTE is not necessary here.
Related
This is my trivial test table,
create table test (
id int not null generated always as identity,
first_name. varchar,
primary key (id),
unique(first_name)
);
As an alternative to insert-into-on-conflict sentences, I was trying to use the coalesce laziness to execute a select whenever possible or an insert, only when select fails to find a row.
coalesce laziness is described in documentation. See https://www.postgresql.org/docs/current/functions-conditional.html
Like a CASE expression, COALESCE only evaluates the arguments that are needed to determine the result; that is, arguments to the right of the first non-null argument are not evaluated. This SQL-standard function provides capabilities similar to NVL and IFNULL, which are used in some other database systems.
I also want to get back the id value of the row, having being inserted or not.
I started with:
select coalesce (
(select id from test where first_name='carlos'),
(insert into test(first_name) values('carlos') returning id)
);
but an error syntax error at or near "into" was found.
See it on this other DBFiddle
https://www.db-fiddle.com/f/t7TVkoLTtWU17iaTAbEhDe/0
Then I tried:
select coalesce (
(select id from test where first_name='carlos'),
(with r as (
insert into test(first_name) values('carlos') returning id
) select id from r
)
);
Here I am getting a WITH clause containing a data-modifying statement must be at the top level error that I don't understand, as insert is the first and only sentence within the with.
I am testing this with DBFiddle and PostgreSQL 13. The source code can be found at
https://www.db-fiddle.com/f/hp8T1iQ8eS4wozDCBhBXDw/5
Different method: chained CTEs:
CREATE TABLE test
( id INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY PRIMARY KEY
, first_name VARCHAR UNIQUE
);
WITH sel AS (
SELECT id FROM test WHERE first_name = 'carlos'
)
, ins AS (
INSERT INTO test(first_name)
SELECT 'carlos'
WHERE NOT EXISTS (SELECT 1 FROM test WHERE first_name = 'carlos')
RETURNING id
)
, omg AS (
SELECT id FROM sel
UNION ALL
SELECT id FROM ins
)
SELECT id
FROM omg
;
It seems that the returning value from the insert into clause is not equivalent in nature to the scalar query of a select clause. So I try encapsulating the insert into into an SQL function and it worked.
create or replace function insert_first_name(
_first_name varchar
) returns int
language sql as $$
insert into test (first_name)
values (_first_name)
returning id;
$$;
select coalesce (
(select id from test where first_name='carlos'),
(select insert_first_name('carlos'))
);
See it on https://www.db-fiddle.com/f/73rVXgqGfrG4VmjrAk6Z3i/2
This is a refinement on #wildplasser accepted answer. it avoids comparing first_name twice and uses coalesce instead of union all. Kind of an selsert in just one sentence.
with sel as (
select id from test where first_name = 'carlos'
)
, ins as (
insert into test(first_name)
select 'carlos'
where (select id from sel) is null
returning id
)
select coalesce (
(select id from sel),
(select id from ins)
);
See it at https://www.db-fiddle.com/f/goRh4TyAebTkEZFHk6WbtK/6
I'm using NOT EXISTS during a DELETE statement in a stored procedure and the not exists is not being applied to the data.
Using the following example data:
CREATE TABLE Region
(
RegionID INT IDENTITY(1,1)
,RegionName VARCHAR(25)
)
GO
INSERT INTO Region(RegionName)
VALUES ('East Coast')
,('Mid West')
,('West Coast')
GO
CREATE TABLE Customer
(
CustomerID INT IDENTITY(1,1)
,FirstName VARCHAR(5)
,Region INT
)
GO
INSERT INTO Customer(FirstName,Region)
VALUES('Tom',1)
,('Mike',2)
,('Jean',3)
GO
CREATE TABLE Orders
(
OrderID INT IDENTITY(1,1)
,CustomerID INT
,OrderAmount INT
,OrderDate DATE
)
GO
INSERT INTO Orders(CustomerID,OrderAmount,OrderDate)
VALUES(1,10,'2018-11-30')
,(2,12,'2018-11-30')
,(2,15,'2018-12-01')
,(2,8,'2018-12-02')
,(2,11,'2018-12-03')
,(3,13,'2018-12-01')
,(3,20,'2018-12-03')
GO
Using that data I'm trying to create a procedure that does the following:
CREATE PROCEDURE udsp_GetOrdersOfXAmount #OrderAmount INT, #RegionID INT = 0
AS
BEGIN
DECLARE #ProcedureTemp TABLE
(
OrderID INT
,CustomerID INT
,OrderAmount INT
,OrderDate DATE
)
INSERT INTO #ProcedureTemp(OrderID,CustomerID,OrderAmount,OrderDate)
SELECT * FROM Orders WHERE OrderAmount >= #OrderAmount
--Do several other UPDATES/ DELETES to #ProcedureTemp
--This is where the issue lies
IF #RegionID > 0
BEGIN
DELETE T FROM #ProcedureTemp T
WHERE NOT EXISTS
(
SELECT *
FROM Customer C
JOIN #ProcedureTemp T ON T.CustomerID = C.CustomerID
WHERE C.Region = #RegionID
)
END
SELECT * FROM #ProcedureTemp
END
GO
If you execute the procedure with the #RegionID parameter populated, you will see the procedure is not honoring the filter by region.
E.G.
EXEC udsp_GetOrdersOfXAmount 10,3
However, if you run the sub query used in the DELETE statement as its own query, you will see the WHERE clause logic provided is working. I don't understand why the it isn't working when used with NOT EXISTS in the DELETE statement.
DECLARE #OrderAmount INT = 10, #RegionID INT = 3
DECLARE #ProcedureTemp TABLE
(
OrderID INT
,CustomerID INT
,OrderAmount INT
,OrderDate DATE
)
INSERT INTO #ProcedureTemp(OrderID,CustomerID,OrderAmount,OrderDate)
SELECT * FROM Orders WHERE OrderAmount >= #OrderAmount
SELECT *
FROM Customer C
JOIN #ProcedureTemp T ON T.CustomerID = C.CustomerID
WHERE C.Region = #RegionID
Thank you in advance for any help you can provide.
You don't need the join in the inner query.
The fact that you are using the same alias for the outer query and the inner one is confusing to me, I'm guessing SQL Server also should have a problem with it.
Try writing it like this:
DELETE T
FROM #ProcedureTemp T
WHERE NOT EXISTS
(
SELECT *
FROM Customer C
-- You already have the T from the outer statement
WHERE T.CustomerID = C.CustomerID
AND C.Region = #RegionID
)
I have an issue that I'm not sure how to overcome. I need to filter a my groups in crystal reports based on a field within the group. Not only that I need it to filter the groups based on if there are two different values in this field within the same group. For example, say I have a table of issues and votes for a council. There is one entry per vote on an issue containing the issue name and the vote cast (either yes, no, or abstain). I will group the table by issue name and I want to filter the groups to show only the issues with a vote split between yes and no (i.e. no abstains and not unanimous). How do I go about doing this?
The data you're feeding into the top-level of the grouping needs to be pre-aggregated so as to show which Votes have multiple different responses. You'd need to do this in whatever your back-end data source is. If I were to do it in SQL, e.g., given the tables:
create table dbo.Issues (
IssueID int identity(1,1) not null ,
constraint pkc_Issues primary key clustered ( IssueID ) ,
IssueText varchar(1000) )
--Note - not putting anything unique on VoterName because there may be 2 Joe Blows in the voter population.
create table dbo.Voters (
VoterID int identity(1,1) not null ,
constraint pkc_Voters primary key clustered ( VoterID ) ,
VoterName varchar(512) not null ) )
create table dbo.Votes (
VoteID int identity(1,1) not null ,
constraint pkn_Votes primary key nonclustered ( VoteID ) ,
VoterID int not null ,
constraint fk_VoterID#Votes foreign key ( VoterID ) references dbo.Voters ( VoterID ) ,
IssueID int not null ,
constraint fk_IssueID#Votes foreign key ( IssueID ) references dbo.Issues ( IssueID ) ,
constraint uci_IssueID_VoterID#Votes unique clustered ( IssueID , VoterID ) ,
VoteResponse varchar(16) null )
I'd pull the data using multiple steps (but feel free to do subqueries, if you think that's more understandable):
select IssueID , Count(VoteResponse) as ResponseCount
into #hasMultiple from (select distinct IssueID , VoteResponse from Votes)
I'd then join back to that, to feed to Crystal:
select dbo.Issues.IssueID ,
dbo.Issues.IssueText ,
cast(case when #hasMultiple.ResponseCount > 1 then 1 else 0 end as bit) as HasMultiple ,
dbo.Votes.VoteID ,
dbo.Votes.VoterID ,
dbo.Votes.VoteResponse ,
dbo.Voters.VoterName
from dbo.Issues
inner join dbo.Votes
on dbo.Issues.IssueID = dbo.Votes.IssueID
left join dbo.Voters
on dbo.Votes.VoterID = dbo.Voters.VoterID
left join #hasMultiple
on dbo.Issues.IssueID = #hasMultiple.IssueID
I'm using PostgreSQL 9.0 and I have a table with just an artificial key (auto-incrementing sequence) and another unique key. (Yes, there is a reason for this table. :)) I want to look up an ID by the other key or, if it doesn't exist, insert it:
SELECT id
FROM mytable
WHERE other_key = 'SOMETHING'
Then, if no match:
INSERT INTO mytable (other_key)
VALUES ('SOMETHING')
RETURNING id
The question: is it possible to save a round-trip to the DB by doing both of these in one statement? I can insert the row if it doesn't exist like this:
INSERT INTO mytable (other_key)
SELECT 'SOMETHING'
WHERE NOT EXISTS (SELECT * FROM mytable WHERE other_key = 'SOMETHING')
RETURNING id
... but that doesn't give the ID of an existing row. Any ideas? There is a unique constraint on other_key, if that helps.
Have you tried to union it?
Edit - this requires Postgres 9.1:
create table mytable (id serial primary key, other_key varchar not null unique);
WITH new_row AS (
INSERT INTO mytable (other_key)
SELECT 'SOMETHING'
WHERE NOT EXISTS (SELECT * FROM mytable WHERE other_key = 'SOMETHING')
RETURNING *
)
SELECT * FROM new_row
UNION
SELECT * FROM mytable WHERE other_key = 'SOMETHING';
results in:
id | other_key
----+-----------
1 | SOMETHING
(1 row)
No, there is no special SQL syntax that allows you to do select or insert. You can do what Ilia mentions and create a sproc, which means it will not do a round trip fromt he client to server, but it will still result in two queries (three actually, if you count the sproc itself).
using 9.5 i successfully tried this
based on Denis de Bernardy's answer
only 1 parameter
no union
no stored procedure
atomic, thus no concurrency problems (i think...)
The Query:
WITH neworexisting AS (
INSERT INTO mytable(other_key) VALUES('hello 2')
ON CONFLICT(other_key) DO UPDATE SET existed=true -- need some update to return sth
RETURNING *
)
SELECT * FROM neworexisting
first call:
id|other_key|created |existed|
--|---------|-------------------|-------|
6|hello 1 |2019-09-11 11:39:29|false |
second call:
id|other_key|created |existed|
--|---------|-------------------|-------|
6|hello 1 |2019-09-11 11:39:29|true |
First create your table ;-)
CREATE TABLE mytable (
id serial NOT NULL,
other_key text NOT NULL,
created timestamptz NOT NULL DEFAULT now(),
existed bool NOT NULL DEFAULT false,
CONSTRAINT mytable_pk PRIMARY KEY (id),
CONSTRAINT mytable_uniq UNIQUE (other_key) --needed for on conflict
);
you can use a stored procedure
IF (SELECT id FROM mytable WHERE other_key = 'SOMETHING' LIMIT 1) < 0 THEN
INSERT INTO mytable (other_key) VALUES ('SOMETHING')
END IF
I have an alternative to Denis answer, that I think is less database-intensive, although a bit more complex:
create table mytable (id serial primary key, other_key varchar not null unique);
WITH table_sel AS (
SELECT id
FROM mytable
WHERE other_key = 'test'
UNION
SELECT NULL AS id
ORDER BY id NULLS LAST
LIMIT 1
), table_ins AS (
INSERT INTO mytable (id, other_key)
SELECT
COALESCE(id, NEXTVAL('mytable_id_seq'::REGCLASS)),
'test'
FROM table_sel
ON CONFLICT (id) DO NOTHING
RETURNING id
)
SELECT * FROM table_ins
UNION ALL
SELECT * FROM table_sel
WHERE id IS NOT NULL;
In table_sel CTE I'm looking for the right row. If I don't find it, I assure that table_sel returns at least one row, with a union with a SELECT NULL.
In table_ins CTE I try to insert the same row I was looking for earlier. COALESCE(id, NEXTVAL('mytable_id_seq'::REGCLASS)) is saying: id could be defined, if so, use it; whereas if id is null, increment the sequence on id and use this new value to insert a row. The ON CONFLICT clause assure
that if id is already in mytable I don't insert anything.
At the end I put everything together with a UNION between table_ins and table_sel, so that I'm sure to take my sweet id value and execute both CTE.
This query needs to search for the value other_key only once, and is a "search this value" not a "check if this value not exists in the table", that is very heavy; in Denis alternative you use other_key in both types of searches. In my query you "check if a value not exists" only on id that is a integer primary key, that, for construction, is fast.
Minor tweak a decade late to Denis's excellent answer:
-- Create the table with a unique constraint
CREATE TABLE mytable (
id serial PRIMARY KEY
, other_key varchar NOT NULL UNIQUE
);
WITH new_row AS (
-- Only insert when we don't find anything, avoiding a table lock if
-- possible.
INSERT INTO mytable ( other_key )
SELECT 'SOMETHING'
WHERE NOT EXISTS (
SELECT *
FROM mytable
WHERE other_key = 'SOMETHING'
)
RETURNING *
)
(
-- This comes first in the UNION ALL since it'll almost certainly be
-- in the query cache. Marginally slower for the insert case, but also
-- marginally faster for the much more common read-only case.
SELECT *
FROM mytable
WHERE other_key = 'SOMETHING'
-- Don't check for duplicates to be removed
UNION ALL
-- If we reach this point in iteration, we needed to do the INSERT and
-- lock after all.
SELECT *
FROM new_row
) LIMIT 1 -- Just return whatever comes first in the results and allow
-- the query engine to cut processing short for the INSERT
-- calculation.
;
The UNION ALL tells the planner it doesn't have to collect results for de-duplication. The LIMIT 1 at the end allows the planner to short-circuit further processing/iteration once it knows there's an answer available.
NOTE: There is a race condition present here and in the original answer. If the entry does not already exist, the INSERT will fail with a unique constraint violation. The error can be suppressed with ON CONFLICT DO NOTHING, but the query will return an empty set instead of the new row. This is a difficult problem because getting that info from another transaction would violate the I in ACID.
Running the following query (SQL Server 2000) the execution plan shows that it used an index seek and Profiler shows it's doing 71 reads with a duration of 0.
select top 1 id from table where name = '0010000546163' order by id desc
Contrast that with the following with uses an index scan with 8500 reads and a duration of about a second.
declare #p varchar(20)
select #p = '0010000546163'
select top 1 id from table where name = #p order by id desc
Why is the execution plan different? Is there a way to change the second method to seek?
thanks
EDIT
Table looks like
CREATE TABLE [table] (
[Id] [int] IDENTITY (1, 1) NOT NULL ,
[Name] [varchar] (13) COLLATE Latin1_General_CI_AS NOT NULL)
Id is primary clustered key
There is a non-unique index on Name and a unique composite index on id/name
There are other columns - left them out for brevity
Now you've added the schema, please try this. SQL Server treats length differences as different data types and will convert the varchar(13) column to match the varchar(20) variable
declare #p varchar(13)
If not, what about collation coercien? Is the DB or server different to the column?
declare #p varchar(13) COLLATE Latin1_General_CI_AS NOT NULL
If not, add this before and post results
SET SHOWPLAN_TEXT ON
GO
If the name column is NVARCHAR then u need your parameter to be also of the same type. It should then pick it up by index seek.
declare #p nvarchar(20)
select #p = N'0010000546163'
select top 1 id from table where name = #p order by id desc