I've noticed a strange behaviour with my query raw that I want to perform. In my query, I want to take only matches that happened BEFORE a given date, however the query returns matches that started at the same time that I give to the query. However, when I hard-code the same date in the query, the match is not included, which is the behaviour I want. Tools I'm using:
"#prisma/client": "^4.4.0",
"prisma": "^4.4.0",
"#nestjs/core": "^9.0.0",
PostgreSQL 15.0 (Ubuntu 15.0-1.pgdg20.04+1) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0, 64-bit
I'm attaching a picture of a log from the query that is performed in the case it fails with parameters.
//! THIS INCLUDES MATCH I DON'T WANT
const date = parseISO("2022-04-16T15:00:00.000Z")
const count = await this.prisma.$queryRaw`
SELECT "Match"."id", SUM("Stats"."cornerKicks") as "sumOfCorners"
FROM "Match"
INNER JOIN "TeamMatchStatistics" as "Stats" ON "Match".id = "Stats"."matchId"
INNER JOIN "Season" ON "Match"."seasonId" = "Season".id
WHERE "Match"."awayTeamId" = 125
-- HERE IS THE CONDITION
AND "Match"."startDateTime" < ${date}
AND "Match"."statusDescriptionShort" IN ('FT', 'PEN', 'AET')
AND "Season"."start" <= ${date}
AND "Season"."end" >= ${date}
GROUP BY "Match"."id"
`
//! THIS WORKS FINE
const date = parseISO("2022-04-16T15:00:00.000Z")
const count = await this.prisma.$queryRaw`
SELECT "Match"."id", SUM("Stats"."cornerKicks") as "sumOfCorners"
FROM "Match"
INNER JOIN "TeamMatchStatistics" as "Stats" ON "Match".id = "Stats"."matchId"
INNER JOIN "Season" ON "Match"."seasonId" = "Season".id
WHERE "Match"."awayTeamId" = 125
-- HERE IS THE CONDITION
AND "Match"."startDateTime" < '2022-04-16T15:00:00.000Z'
AND "Match"."statusDescriptionShort" IN ('FT', 'PEN', 'AET')
AND "Season"."start" <= ${date}
AND "Season"."end" >= ${date}
GROUP BY "Match"."id"
`
I can't duplicate this locally, It happens only on the production DB. I tried adding ::timestamptz after the date, but with no successful result.
Related
SELECT_QUERY = `SELECT * FROM events WHERE c_id = ? AND start_time > ? and
end_time < ?`
query := sr.db.Raw(SELECT_QUERY, request.GetCId(), startTime, endTime)
var v = request.GetVIds()
if len(v) > 0 {
query = query.Where(` v_id IN (?) `, v)
} //Only this block introduces first ) after end_time
var c = request.GetStatus().String()
if len(c) > 0 {
query = query.Where( " status = ? ", c) // this introduces the other opening brace //after AND
}
Following is the query generated and found in logs
SELECT * FROM events WHERE c_id = 1 AND start_time > '2020-04-16 18:42:00' and
end_time < '2020-04-16 18:45:50' ) AND ( v_id IN (1,2)) AND ( status = 'STATUS_MIDDLE_CLASS' ORDER BY start_time DESC LIMIT 5 OFFSET 1
The other solution in stackoverflow and internet article doesn't help.
PS: Is it because I mix db.Raw( ) and query.Where() ?
Changing ? to $1 doesn't fix the issue.
Basically a few things fixed the issue.
1) Mixing Raw and query.Where was one issue.
After making the Raw query to sr.db.Where
2)
SELECT_QUERY = `SELECT * FROM events WHERE c_id = ? AND start_time > ? and
end_time < ?`
already has select * from. And then using query := sr.db.Raw(SELECT_QUERY, request.GetCId(), startTime, endTime) introduces nested select *.
So, changed SELECT_QUERY as follows
SELECT_QUERY = `events WHERE c_id = ? AND start_time > ? and
end_time < ?`
solved the issue.
I found a workaround to the error which I received when I tried to add a timestamp filed in Go/Gorm solution with PostgreSQL equivalent to default: now() which is default: time.Now().Format(time.RFC3339)
I received the error because I use AutoMigrate() to create the tables in PostgreSQL. The one problem I found is when trying to use the default value of a function instead of a string (which one can use for a fixed timestamp).
So I had to go into DataGrid (since I use JetBrains, but you can use any PostgreSQL admin tool like pgAdmin) and manually add the timestamp field with default of now() or merely update the existing field to add the default of now(). Then the error goes away when doing your next build in Go.
When running the following query on Postgres 10 using a connection pool in a concurrent manner, the results return in n * the length of time it takes to run one invocation.
Tested in Racket using threads and Node with promises. Both return ~ the same timings in milliseconds.
1 invocation:
8322
10 invocations:
82432, 82260, 82025, 82260, 82432, 82103, 82025, 82556, 82040, 82119
Here is the query:
WITH contact_searches as (select
md.geo_point as md_point, md.sf_id as md_name, md.sf_id as md_sf_id,
md.freehold_tenure as md_fh_ten,
md.freehold_search_price md_fh_sp, md.leasehold_tenure as md_lh_ten,
md.leasehold_search_price as md_lh_sp,
md.special_tenure_search_price as md_st_sp, md.geo_point as point,
ss.freehold_tenure as ss_fh_ten, ss.leasehold_tenure as ss_lh_ten,
ss.price_from as ss_pf, ss.price_to as ss_pt, md.marketing_trades as md_trades,
ss.trades as ss_trades,
ss.sf_id as ss_sf_id, ss.person_id as person_id, ss.areas as areas
from saved_searches ss
inner join marketing_details md on md.marketing_trades && ss.trades
where md.sf_id = ANY ($1)
and
((md.freehold_tenure is not null and ss.freehold_tenure ='true'
and (md.freehold_search_price is null or md.freehold_search_price >= ss.price_from)
and (md.freehold_search_price is null or md.freehold_search_price <= ss.price_to))
or
(md.leasehold_tenure is not null and ss.leasehold_tenure ='true'
and (md.leasehold_search_price is null or md.leasehold_search_price >= ss.price_from)
and (md.leasehold_search_price is null or md.leasehold_search_price <= ss.price_to))
or
(md.special_tenure_search_price is not null
and (md.special_tenure_search_price >= ss.price_from)
and (md.special_tenure_search_price <= ss.price_to))
)
), contact_ids as (select distinct person_id as sf_id
from contact_searches inner join areas a on st_contains(a.polygon, contact_searches.point)
where a.name = ANY (contact_searches.areas))
select c.sf_id, hasoptedoutofemail, preferred_contact_method, cco_email_opt_out, cf_email_opt_out, valid_email(c)
from contacts c inner join contact_ids on c.sf_id = contact_ids.sf_id where c.applicant_status = 'Live'
Does anyone have any insight as to whether complex queries don't run in parralell or whether there is a config change I could make or anything else that could help?
I have a query:
SELECT DISTINCT ON (analytics_staging_v2s.event_type, sent_email_v2s.recipient, sent_email_v2s.sent) sent_email_v2s.id, sent_email_v2s.user_id, analytics_staging_v2s.event_type, sent_email_v2s.campaign_id, sent_email_v2s.recipient, sent_email_v2s.sent, sent_email_v2s.stage, sent_email_v2s.sequence_id, people.role, people.company, people.first_name, people.last_name, sequences.name as sequence_name
FROM "sent_email_v2s"
LEFT JOIN analytics_staging_v2s ON sent_email_v2s.id = analytics_staging_v2s.sent_email_v2_id
JOIN people ON sent_email_v2s.person_id = people.id
JOIN sequences on sent_email_v2s.sequence_id = sequences.id
JOIN users ON sent_email_v2s.user_id = users.id
WHERE "sent_email_v2s"."status" = 1
AND "people"."person_type" = 0
AND (sent_email_v2s.sequence_id = 1888) AND (sent_email_v2s.sent >= '2016-03-18')
AND "users"."team_id" = 1
When I run EXPLAIN ANALYZE on it, I get:
Then, if I change that to the following (Just removing the (sent_email_v2s.sent >= '2016-03-18')) as follows:
SELECT DISTINCT ON (analytics_staging_v2s.event_type, sent_email_v2s.recipient, sent_email_v2s.sent) sent_email_v2s.id, sent_email_v2s.user_id, analytics_staging_v2s.event_type, sent_email_v2s.campaign_id, sent_email_v2s.recipient, sent_email_v2s.sent, sent_email_v2s.stage, sent_email_v2s.sequence_id, people.role, people.company, people.first_name, people.last_name, sequences.name as sequence_name
FROM "sent_email_v2s"
LEFT JOIN analytics_staging_v2s ON sent_email_v2s.id = analytics_staging_v2s.sent_email_v2_id
JOIN people ON sent_email_v2s.person_id = people.id
JOIN sequences on sent_email_v2s.sequence_id = sequences.id
JOIN users ON sent_email_v2s.user_id = users.id
WHERE "sent_email_v2s"."status" = 1
AND "people"."person_type" = 0
AND (sent_email_v2s.sequence_id = 1888) AND "users"."team_id" = 1
when I run EXPLAIN ANALYZE on this query, the results are:
EDIT:
The results above from today are about as I expected. When I ran this last night, however, the difference created by including the timestamp filter was about 100x slower (0.5s -> 59s). The EXPLAIN ANALYZE from last night showed all of the time increase to be attributed to the first unique/sort operation in the query plan above.
Could there be some kind of caching issue here? I am worried now that there might be something else going on (transiently) that might make this query take 100x longer since it happened at least once.
Any thoughts are appreciated!
There are two update queries and 1st update query execute successfully but 2nd update query is not execute and show the following message:
Msg 512, Level 16, State 1, Line 10
Subquery returned more than 1 value. This is not permitted when the subquery follows =, !=, <, <= , >, >= or when the subquery is used as an expression. The statement has been terminated.
1st update query:
update dbo.TblPrePostApproval
set
dbo.TblPrePostApproval.PAApprovedDate = (select dbo.TblMasterInfo.AppRefDate
from dbo.TblMasterInfo
Where dbo.TblMasterInfo.Appid = dbo.TblPrePostApproval.Appid),
dbo.TblPrePostApproval.PAApprovedTenor = '36',
dbo.TblPrePostApproval.PAApprovedAmt = (select dbo.TblMasterInfo.AppReqeustAmt
from dbo.TblMasterInfo
where dbo.TblPrePostApproval.Appid = dbo.TblMasterInfo.AppID),
dbo.TblPrePostApproval.PADisbBr = (select dbo.TblMasterInfo.AppSourceBrName
from dbo.TblMasterInfo
where dbo.TblPrePostApproval.Appid = dbo.TblMasterInfo.AppID)
2nd update query
update dbo.TblPrePostApproval
set
dbo.TblPrePostApproval.PAApprovedDate = (select dbo.TestPost.PADate
from dbo.TestPost
Where dbo.TestPost.Appid = dbo.TblPrePostApproval.Appid),
dbo.TblPrePostApproval.PAApprovedTenor = (select dbo.TestPost.PATenor
from dbo.TestPost
Where dbo.TestPost.Appid = dbo.TblPrePostApproval.Appid),
dbo.TblPrePostApproval.PAApprovedAmt = (select dbo.TestPost.PAAmt
from dbo.TestPost
where dbo.TestPost.Appid = dbo.TblPrePostApproval.AppID),
dbo.TblPrePostApproval.PADisbBr = (select dbo.TestPost.PABr
from dbo.TestPost
where dbo.TestPost.Appid = dbo.TblPrePostApproval.AppID)
Where is my problem? Pls any one suggest me.
One of your subqueries (I guess on line 10) is returning more than one row, so it can't check to see if it equals anything, because it's a set, not a value. Just change your query to be more specific. Try adding LIMIT 0, 1 to the end of the subqueries, or TOP (1) after the the SELECT in each subquery.
Why don't you use JOINs for your update? Much easier to read and understand!
Query #1:
UPDATE ppa
SET
PAApprovedDate = info.AppRefDate,
PAApprovedTenor = '36',
PAApprovedAmt = info.AppReqeustAmt,
PADisbBr = info.AppSourceBrName
FROM
dbo.TblPrePostApproval ppa
INNER JOIN
dbo.TblMasterInfo.TblMasterInfo info ON info.Appid = ppa.Appid
Query #2:
UPDATE ppa
SET
PAApprovedDate = tp.PADate,
PAApprovedTenor = tp.PATenor,
PAApprovedAmt = tp.PAAmt,
PADisbBr = tp.PABr
FROM
dbo.TblPrePostApproval ppa
INNER JOIN
dbo.TestPost tp ON tp.Appid = ppa.AppID
I have an UPDATE query in SQL Server Management Studio 2012 that is not giving me the expected results. I believe it is because I had to add the IJDATE field to the SELECT and GROUP BY and therefore it is not summing correctly. I am trying to update the current month/year for each vendor. Is there a better way to write this UPDATE query?
UPDATE S
SET S.PurchaseDlr = I.PurchaseDlr
FROM (SELECT
IJVEND,
IJDATE,
SUM(IJQTY * IJCOST) AS PurchaseDlr
FROM
dbo.S2K_IJ
WHERE
IJTYPE IN ('I','ID')
GROUP BY
IJVEND,
IJDATE) I
INNER JOIN PurchaseDollars S on I.IJVEND = S.Vendor AND S.MonthNum = MONTH(I.IJDATE) AND S.Year = YEAR(I.IJDATE)
You are grouping by date, but it looks like you want to group by month and year:
UPDATE S
SET S.PurchaseDlr = I.PurchaseDlr
FROM (SELECT
IJVEND,
MONTH(IJDATE) AS Month,
YEAR(IJDATE) AS Year,
SUM(IJQTY * IJCOST) AS PurchaseDlr
FROM
dbo.S2K_IJ
WHERE
IJTYPE IN ('I','ID')
GROUP BY
IJVEND,
MONTH(IJDATE),
YEAR(IJDATE)) I
INNER JOIN PurchaseDollars S
ON I.IJVEND = S.Vendor
AND S.MonthNum = I.Month
AND S.Year = I.Year