I have got massive table (over 95 000 000 records) in MSSQL database
id
configuration_id
equipment_group_id
name
price
1
1
100
item1
10
2
1
100
item2
20
3
1
100
item3
30
4
2
100
item1
10
5
2
100
item2
20
6
2
100
item3
30
7
3
100
item1
10
8
3
100
item2
20
9
3
100
item3
31
I am going to identify duplicated group of records.
Configuration 1 Group
id
configuration_id
equipment_group_id
name
price
1
1
100
item1
10
2
1
100
item2
20
3
1
100
item3
30
Configuration 2 Group
id
configuration_id
equipment_group_id
name
price
4
2
100
item1
10
5
2
100
item2
20
6
2
100
item3
30
Configuration 3 Group
id
configuration_id
equipment_group_id
name
price
7
3
100
item1
10
8
3
100
item2
20
9
3
100
item3
31
in my logic Group 1 and Group 2 are duplicates
has the same number of records
has the same content in fields equipment_group_id, name, price
Group 1 and Group 3 are NOT duplicates because there is at least one different element (last record has price 31, not 30)
How to construct a query to find all groups that are duplicated (not records) across the table?
Performance of this query for 95M records will probably not be ideal, but this should do the trick.
Find Exact Matches of Groups Containing Multiple Rows
DROP TABLE IF EXISTS #Config
CREATE TABLE #Config
(id int
,configuration_id int
,equipment_group_id int
,name VARCHAR(100)
,price INT
)
INSERT INTO #Config
VALUES
(1,1,100,'item1',10)
,(2,1,100,'item2',20)
,(3,1,100,'item3',30)
,(4,2,100,'item1',10)
,(5,2,100,'item2',20)
,(6,2,100,'item3',30)
,(7,3,100,'item1',10)
,(8,3,100,'item2',20)
,(9,3,100,'item3',31)
;WITH cte_ConfigCount AS (
SELECT *,ConfigTotalRowCnt = COUNT(*) OVER (PARTITION BY A.configuration_id) /*Counts how many rows in each config*/
FROM #Config AS A
)
SELECT
A.configuration_id
,B.configuration_id
,TextDescription = CONCAT('Config #',A.configuration_id,' matches Config #',B.configuration_id)
,A.ConfigTotalRowCnt
,RowsMatch = COUNT(*)
FROM cte_ConfigCount AS A
INNER JOIN cte_ConfigCount AS B
ON A.configuration_id < B.configuration_id /*Don't join to self and only join 1 way (so don't have one row with A-B and another row with B-A)*/
AND a.equipment_group_id = B.equipment_group_id
AND A.name = B.name
AND A.price = B.price
GROUP BY A.configuration_id,A.ConfigTotalRowCnt,B.configuration_id
HAVING A.ConfigTotalRowCnt = COUNT(*) /*Only return where the total row for the config matches the rows where the configs match*/
I'm using Entity Framework and I have 3 linked tables
image
I can create query like this
purchase_number album_name purchase_amount purchase_price
1 name_1 5 1000
1 name_2 10 2000
2 name_1 3 1000
2 name_3 7 1500
3 name_2 2 2000
How can I create query like this using LINQ
purchase_number purchase_price(purchase_price * purchase_amount)
1 25000
2 13500
3 4000
Where q is your original query:
var result = q
.GroupBy(x=>x.purchase_number)
.Select(x=>new {
purchase_number = x.Key,
purchase_price = x.Sum(z=>z.purchase_amount*z.purchase_price)
});
Optionally doing an orderby at the end to guarantee the order if that is a requirement.
I have a table that looks like this:
GroupID UserID Value
1 1 10
1 2 20
1 3 30
1 4 40
1 5 45
1 6 49
1 7 80
1 8 90
2 1 2
2 2 24
2 3 34
2 4 48
2 5 56
3 1 etc.
3 2
3 3
3 4
4 1
4 2
4 3
I am trying to write a LEAD function that will give me the midpoint between each value. To do this I have written the following:
SELECT
[GroupID]
, [UserID]+0.5
, (LEAD ([Value], 1) OVER (ORDER BY GroupID, UserID) + [Value])/2 as [Value]
from dbo.myTable
The problem with this function is that when it gets to the last User in the group, it gives me a bad value because it's taking the [Value] on the current row and the value from the next row.
What I want to do is stop it when it reaches the maximum UserID for each Group. In other words, when it gets to GroupID = 1 and UserID = 8, it should end and start at the next Group. I do not want a row that looks like this:
GroupID UserID Value
1 8.5 46
I could run a DELETE statement after I INSERT the rows into the original table, but I don't have anything to identify when a row is the "maximum" User for it's Group. Ideally, I would like to somehow tell the lead statement not to calculate it in the first place.
I would like a query that will show a sum of columns with a default value for missing data. For example assume I have a table as follows:
type_lookup:
id name
-----------
1 self
2 manager
3 peer
And a table as follows
data:
id type_lookup_id value
--------------------------
1 1 1
2 1 4
3 2 9
4 2 1
5 2 9
6 1 5
7 2 6
8 1 2
9 1 1
After running a query I would like a result set as follows:
type_lookup_id value
----------------------
1 13
2 25
3 0
I would like all rows in type_lookup table to be included in the result set - even if they don't appear in the data table.
It's a bit hard to read your data layout, but something like the following should do the trick:
SELECT tl.type_lookup_id, tl.name, sum(da.type_lookup_id) how_much
from type_lookup tl
left outer join data da
on da.type_lookup_id = tl.type_lookup_id
group by tl.type_lookup_id, tl.name
order by tl.type_lookup_id
[EDIT]
...subsequently edited by changing count() to sum().
I have a table like this
UserID Score Date
5 6 2010-1-1
7 8 2010-1-2
5 4 2010-1-3
6 3 2010-1-4
7 4 2010-1-5
6 1 2010-1-6
I would like to get a table like this
UserID Score RunningTotal Date
5 6 6 2010-1-1
5 4 10 2010-1-3
6 3 3 2010-1-4
6 1 4 2010-1-6
7 8 8 2010-1-2
7 4 12 2010-1-5
Thanks!
Unlike Oracle, PostgreSQL and even MySQL, SQL Server has no efficient way to calculate running totals.
If you have few scores per UserID, you can use this:
SELECT userId,
(
SELECT SUM(score)
FROM scores si
WHERE si.UserID = so.UserID
AND si.rn <= so.rn
)
FROM (
SELECT *,
ROW_NUMBER() OVER (PARTITION BY UserID) AS rn
FROM scores
) so
, however, this will be very inefficient for larger tables.
For larger tables, you could benefit from using (God help me) a cursor.
Would something like this work for you...?
SELECT UserID, Score,
(SELECT SUM(Score)
FROM TableName innerTable
WHERE innerTable.UserID = outerTable.userID
AND innerTable.Date <= outerTable.date) AS RunningTotal
FROM TableName outerTable
This assumes, though, that a user cannot have more than one score per day. (What is your PK?)