Strange MariaDB Issue LEFT JOIN SUM with multi GROUP BY Column - group-by

I Have strange issue on maria db in this query:
SELECT
tr_sales_delivery_detail.sales_order_id,
tr_sales_delivery_detail.row_id,
tr_sales_delivery_detail.qty_delivery,
so.qty AS so_qty
FROM tr_sales_delivery_detail
LEFT JOIN (
SELECT tr_sales_order_detail.item_id,
tr_sales_order_detail.row_id,
tr_sales_order_detail.sales_order_id,
SUM(tr_sales_order_detail.qty) AS qty FROM tr_sales_order_detail
GROUP BY tr_sales_order_detail.item_id, tr_sales_order_detail.row_id, tr_sales_order_detail.sales_order_id
) so
ON so.item_id = tr_sales_delivery_detail.item_id
AND so.row_id = tr_sales_delivery_detail.row_id
AND so.sales_order_id = tr_sales_delivery_detail.sales_order_id
WHERE tr_sales_delivery_detail.sales_delivery_id = '10923';
return :
sales_order_id
row_id
Qty Delivery
Qty
11062
10
1
1
11062
11
1
null
, but when I concat the group by it return the correct value
SELECT
tr_sales_delivery_detail.sales_order_id,
tr_sales_delivery_detail.row_id,
tr_sales_delivery_detail.qty_delivery ,
so.qty AS so_qty
FROM tr_sales_delivery_detail
LEFT JOIN (
SELECT tr_sales_order_detail.item_id,
tr_sales_order_detail.row_id,
tr_sales_order_detail.sales_order_id,
SUM(tr_sales_order_detail.qty) AS qty FROM tr_sales_order_detail
GROUP BY concat(tr_sales_order_detail.item_id, tr_sales_order_detail.row_id, tr_sales_order_detail.sales_order_id)
) so
ON so.item_id = tr_sales_delivery_detail.item_id
AND so.row_id = tr_sales_delivery_detail.row_id
AND so.sales_order_id = tr_sales_delivery_detail.sales_order_id
WHERE tr_sales_delivery_detail.sales_delivery_id = '10923';
return
sales_order_id
row_id
Qty Delivery
Qty
11062
10
1
1
11062
11
1
1
Can someone explain what happen with first query? does it mariadb bug?
after some research, I found it related with column index key
column item_id, row_id, and sales_order_id have index key.
when I remove index key for column sales_order_id, it return normal.
data type for each column (item_id, row_id, sales_order_id) is :
bigint(20)
Can someone explain does my index is wrong or it related to bug ?
It is return null only if the next row has same sales_order_id value.
I want to remove concat usage in group by, because its affect the performance
I Think its related to this bug https://jira.mariadb.org/browse/MDEV-26337

Related

postgresql - removing duplicates

I have 4 tables that i did a UNION with and the total count is roughly about 13 mils for an service_id field and 790k for a distinct service_id field.
cust_id
service_id
1
0423
2
0456
3
0423
When i did a count using
SELECT COUNT(service_id) as full_count, service_id
FROM temp
GROUP BY service_id
HAVING COUNT(1) > 1
I got service_id 0423 x 2
However, when i did
SELECT COUNT(*) as full_count, COUNT(DISTINCT cust_id) as dist_cust_id
FROM temp
I got the result of
full_count
dist_cust_id
3
3
My question is then how do i remove duplicate service_id and keeping 1 and knowing the one i keep is a valid one. Some tables do have a last_update column and some don't which adding to the complexity.
I have tried doing the SELF JOIN of the same table where a.cust_id <> b.cust_id AND a.service_id = b.service_id to identify which service_id has two or more cust_id. Not sure how I can keep the right one.
Thanks

How to include and exclude ids in once query postgresql

I use PostgreSQL 13.3
I'm trying to think how I can make include/exclude in query at the same time
I have include_system_ids [1,5] and exclude_system_ids [3]
There's one big table - records
system_records table
record
system_id
1
1
1
5
1
3
2
1
2
5
If a record contains an exclusive identifier, then it should not be included in the final selection. I had some several tries, but I didn't get a necessary result
Awaiting result: record with id 2
Fact result: 1, 2
My variants
select r.id from records r
left join (select record_id from system_records
where system_id in (1,5)
) include_ids on r.id = include_ids
left join (select record_id from system_records
where system_id not in (3)
) exclude_ids on r.id = exclude_ids.id
Honestly, I don't understand how I can do it((
Is there anyone who can help me
Maybe this query could be a solution (result here)
with x as (select record,string_agg(system_id::varchar,',') as sys_id from records group by record)
select records.*
from records,x
where records.record = x.record
and x.sys_id = '1,5'

Inner join removed from the SQL query

I have a below SQL query to get the three records for notifying purpose.
SELECT orders.msg
FROM orders
INNER JOIN
(
SELECT id
FROM orders
WHERE type_id = 12
ORDER BY id DESC LIMIT 3 OFFSET 0
) AS items
ON orders.id = items.id;
When trying to make the query optimized, i made the changes as below.
SELECT orders.msg
FROM orders
WHERE type_id = 12
ORDER BY id DESC LIMIT 3 OFFSET 0;
Is the modified query seems to be OK or did i miss anything here or any other way of doing is there??
The simplified version on the bottom looks logically identical, to me, to the one on top:
SELECT msg
FROM orders
WHERE type_id = 12
ORDER BY id DESC LIMIT 3;
Note that the above query could benefit from the following index:
CREATE INDEX idx ON orders (type_id, id, msg);
This index would completely cover the WHERE, ORDER BY, and SELECT clauses.
You can try this also:
SELECT orders.msg
FROM orders
WHERE orders.id
IN (
SELECT id
FROM orders
WHERE type_id = 12
ORDER BY id
DESC LIMIT 3 OFFSET 0
)

Getting NULL values in JOINED table with LIMIT

There are many similar questions which I've learned from, but my result set isn't returning the expected results.
My Objective:
Build a query that will return a result set containing all rows in table demo1 with user_id = "admin", and the only row of table demo2 with user_id = "admin". Each row in demo2 has a unique user_id so there's always only one row with "admin" as user_id.
However, I don't want demo2 data to wastefully repeat on every subsequent row of demo1. I only want the first row of the result set to contain demo2 data as non-null values. Null values for demo2 columns should only be returned for rows 2+ in the result set.
Current Status:
Right now my query is returning the appropriate columns (all demo1 and all demo2) but
all the data returned from demo2 is null.
Demo1:
id user_id product quantity warehouse
1 admin phone 3 A
2 admin desk 1 D
3 k45 chair 5 B
Demo2:
id user_id employee job country
1 admin james tech usa
2 c39 cindy tech spain
Query:
SELECT *
from demo1
left join (SELECT * FROM demo2 WHERE demo2.user_id = 'X' LIMIT 1) X
on (demo1.user_id = x.user_id)
WHERE demo1.user_id = 'admin'
Rationale:
The subquery's LIMIT 1 was my attempt to retrieve demo2 values for row 1 only, thinking the rest would be null. Instead, all values are null.
Current Result:
id user_id product quantity warehouse id employee job country
1 admin phone 3 A null null null null
2 admin desk 1 D null null null null
Desired Result:
id user_id product quantity warehouse id employee job country
1 admin phone 3 A 1 james tech usa
2 admin desk 1 D null null null null
I've tried substituting left join for left inner join, right join, full join, but nothing returns the desired result.
Your join is going to bring through ANY records that satisfies the join condition for your two tables. There is no changing that.
But you could suppress subsequent records in your result set from displaying the matching demo2 record that satisfied the join condition AFTER it's joined:
SELECT demo1.id ,
demo1.user_id,
demo1.product,
demo1.quantity,
demo1.warehouse
CASE WHEN ROW_NUMBER() OVER (PARTITION BY demo1.user_id ORDER BY demo1.id) = 1 THEN demo2.id END as demo2_id,
CASE WHEN ROW_NUMBER() OVER (PARTITION BY demo1.user_id ORDER BY demo1.id) = 1 THEN demo2.employee END AS demo2_employee,
CASE WHEN ROW_NUMBER() OVER (PARTITION BY demo1.user_id ORDER BY demo1.id) = 1 THEN demo2.job END as demo2_job,
CASE WHEN ROW_NUMBER() OVER (PARTITION BY demo1.user_id ORDER BY demo1.id) = 1 THEN demo2.country END as demo2_country
from demo1
left join demo2
on demo1.user_id = demo2.user_id
AND demo2.user_id = 'X'
WHERE demo1.user_id = 'admin'
That's just a quick rewrite of your original sql with the addition CASE expressions included.
That being said, this sql will produce no results for demo2 since the demo2.user_id can't satisfy both conditions in this query:
The join condition demo1.user_id = demo2.user_id with the where predicate of demo1.user_id = 'admin'
Also hold the value X.
It's either admin and satisfies your first join condition, but fails your second. Or it's X and satisfies your second condition, but nor your first.
Here is another nice approach:
sqlfiddle

Update Multiple Columns in One Statement Based On a Field with the Same Value as the Column Name

Not sure if this is possible without some sort of Dynamic SQL or a Pivot (which I want to stay away from)... I have a report that displays total counts for various types/ various status combinations... These types and statuses are always going to be the same and present on the report, so returning no data for a specific combination yields a zero. As of right now there are only three caseTypes (Vegetation, BOA, and Zoning) and 8 statusTypes (see below).
I am first setting up the skeleton of the report using a temp table. I have been careful to name the temp table columns the same as what the "statusType" column will contain in my second table "#ReportData". Is there a way to update the different columns in "#FormattedData" based on the value of the "statusType" column in my second table?
Creation of Formatted Table (for report):
CREATE TABLE #FormattedReport (
caseType VARCHAR(50)
, underInvestigation INT NOT NULL DEFAULT 0
, closed INT NOT NULL DEFAULT 0
, closedDPW INT NOT NULL DEFAULT 0
, unsubtantiated INT NOT NULL DEFAULT 0
, currentlyMonitored INT NOT NULL DEFAULT 0
, judicialProceedings INT NOT NULL DEFAULT 0
, pendingCourtAction INT NOT NULL DEFAULT 0
, other INT NOT NULL DEFAULT 0
)
INSERT INTO #FormattedReport (caseType) VALUES ('Vegetation')
INSERT INTO #FormattedReport (caseType) VALUES ('BOA')
INSERT INTO #FormattedReport (caseType) VALUES ('Zoning')
Creation of Data Table (to populate #FormattedReport):
SELECT B.Name AS caseType, C.Name AS StatusType, COUNT(*) AS Amount
INTO #ReportData
FROM table1 A
INNER JOIN table2 B ...
INNER JOIN table3 C ...
WHERE ...
GROUP BY B.Name, C.Name
CURRENT Update Statement (Currently will be 1 update per column in #FormattedReport):
UPDATE A SET underInvestigation = Amount FROM #ReportData B
INNER JOIN #FormattedReport A ON B.CaseType LIKE CONCAT('%', A.caseType, '%')
WHERE B.StatusType = 'Under Investigation'
UPDATE A SET closed = Amount FROM #ReportData B
INNER JOIN #FormattedReport A ON B.CaseType LIKE CONCAT('%', A.caseType, '%')
WHERE B.StatusType = 'Closed'
...
REQUESTED Update Statement: Would like to have ONE update statement knowing which column to update when "#ReportData.statusType" is the same as a "#FormattedData" column's name. For my "other" column, I'll just do that one manually using a NOT IN.
Assuming I understand the question, I think you can use conditional aggregation for this:
;WITH CTE AS
(
SELECT CaseType
,SUM(CASE WHEN StatusType = 'Under Investigation' THEN Amount ELSE 0 END) As underInvestigation
,SUM(CASE WHEN StatusType = 'Closed' THEN Amount ELSE 0 END) As closed
-- ... More of the same
FROM #ReportData
GROUP BY CaseType
)
UPDATE A
SET underInvestigation = B.underInvestigation
,closed = b.closed
-- more of the same
FROM #FormattedReport A
INNER JOIN CTE B
ON B.CaseType LIKE CONCAT('%', A.caseType, '%')