I have following 2 collection schema
images:{
imageId:"string", avgRating:{ rating1:decimal,rating2:decimal}, ratingCount:int}
}
ratings:{
imageId:"string", rating1:decimal, rating2:decimal
}
//here rating1 nd rating2 are ratings for different features(just according to my requirements)
so I am calculating avg as follows
db.images.aggregate([
{
$match: {
imageId: "someid",
},
},
{
$lookup:
{
from: "ratings",
let: {id: '$imageId'},
pipeline: [
{
$match: {
{
$eq: ['$imageId','$$id']
},
},
},{
$group:
{
_id: 0,
aggRating1: {$avg: "$rating1"},
aggRating2: {$avg: '$rating2'},
count: {$sum: 1}
}
},
{$project: {_id: 0,count:1,aggRating1:1,aggRating2:1}},
],
as: "rating"
}
},
{
$set: {
ratingCount: '$count',
'avgRating.rating1':'$review.aggRating1'
'avgRating.rating2':'$review.aggRating2'
}
},
]);
I am getting results like this
"data":[
{
"_id": "somedocId",
"imageId":"someid",
"ratingCount": 10,
"avgRating": {
"aggRating1": [
"rating1":{
"$numberDecimal": "3.25"
}],
"aggRating2": [
"rating2":{
"$numberDecimal": "3.25"
}]
},
"rating":[
{
"aggRating1": {
"$numberDecimal": "3.25"
},
"aggRating2": {
"$numberDecimal": "3.25"
},
"count": 10
}
],
}
]
So if u see when I set the aggRating1 and aggRating2 from rating lookup I got, it converts to array. But in rating it is an object. Idk why is that happening.
So how do i get just the decimal value of the avg results? and not like above? :/
Related
I'm trying to get a list of current holders at specific times from a collection. My collection looks like this:
[
{
"time": 1,
"holdings": [
{ "owner": "A", "tokens": 2 },
{ "owner": "B", "tokens": 1 }
]
},
{
"time": 2,
"holdings": [
{ "owner": "B", "tokens": 2 }
]
},
{
"time": 3,
"holdings": [
{ "owner": "A", "tokens": 3 },
{ "owner": "B", "tokens": 1 },
{ "owner": "C", "tokens": 1 }
]
},
{
"time": 4,
"holdings": [
{ "owner": "C", "tokens": 0 }
]
}
]
tokens show the current holdings of an owner if the holdings have changed to the last document. I would like to change the collection so that holdings always includes the full current holdings for any point in time.
At time: 1, the holdings are: A: 2, B: 1.
At time: 2, the holdings are: A: 2, B: 2. The collections does not include A's holdings however, because they haven't changed. So what I'd like to get is:
[
{
"time": 1,
"holdings": [
{ "owner": "A", "tokens": 2 },
{ "owner": "B", "tokens": 1 }
]
},
{
"time": 2,
"holdings": [
{ "owner": "A", "tokens": 2 }, // merged from prev doc.
{ "owner": "B", "tokens": 2 }
]
},
{
"time": 3,
"holdings": [
{ "owner": "A", "tokens": 3 },
{ "owner": "B", "tokens": 1 },
{ "owner": "C", "tokens": 1 }
]
},
{
"time": 4,
"holdings": [
{ "owner": "A", "tokens": 3 }, // merged from prev
{ "owner": "B", "tokens": 1 }, // merged from prev
{ "owner": "C", "tokens": 0 }
]
}
]
From what I understand $mergeObjects does that, but I don't understand how I can merge all previous docs in order up to the current doc for each doc. So I'm looking for a way to combine setWindowFields with mergeObjects I think.
This is a nice challenge.
So far, I got this complicated solution:
Get all of our timestamps in all of our documents. This is the purpose of the first 4 steps. $setWindowFields is used to accumulate this data.
$group by owner and calculate the empty timestamps as wantedTimes- next 5 steps.
$set empty timestamps with tokens: null to be filled with actual data and $unwind to separate - next 3 steps
Use $setWindowFields to find the last known token for each owner at each timestamp.
Fill this last known state for documents with unknown token - 2 steps
$group and format answer:
db.collection.aggregate([
{
$setWindowFields: {
sortBy: {time: 1},
output: {
allTimes: {$addToSet: "$time", window: {documents: ["unbounded", "current"]}
}
}
}
},
{
$setWindowFields: {
sortBy: {time: -1},
output: {
allTimes: {$addToSet: "$allTimes", window: {documents: ["unbounded", "current"]}
}
}
}
},
{
$set: {
allTimes: {
$reduce: {
input: "$allTimes",
initialValue: [],
in: {"$concatArrays": ["$$value", "$$this"]}
}
}
}
},
{$set: {allTimes: {$setIntersection: "$allTimes"}}},
{$unwind: "$holdings"},
{$sort: {time: 1}},
{$group: { _id: "$holdings.owner",
tokens: {$push: {tokens: "$holdings.tokens", time: "$time"}},
times: {$push: "$time"}, firstTime: {$first: "$time"},
allTimes: {$first: "$allTimes"}}
},
{
$addFields: {
wantedTimes: {
$filter: {
input: "$allTimes",
as: "item",
cond: {$gte: ["$$item", "$firstTime"]}
}
}
}
},
{
$project: {
tokens: 1,
wantedTimes: {$setDifference: ["$wantedTimes", "$times"]}
}
},
{
$set: {
data: {
$map: {
input: "$wantedTimes",
as: "item",
in: {time: "$$item", tokens: null}
}
}
}
},
{$project: {tokens: {"$concatArrays": ["$tokens", "$data"]}}},
{$unwind: "$tokens"},
{
$setWindowFields: {
partitionBy: "$_id",
sortBy: {"tokens.time": 1},
output: {
lastTokens: {
$push: "$tokens.tokens",
window: {documents: ["unbounded", "current"]}
}
}
}
},
{
$set: {
lastTokens: {
$filter: {
input: "$lastTokens",
as: "item",
cond: {$ne: ["$$item", null]}
}
}
}
},
{
$set: {
"tokens.tokens": {$ifNull: ["$tokens.tokens", {$last: "$lastTokens"}]}
}
},
{
$group: {
_id: "$tokens.time",
holdings: {$push: {owner: "$_id", tokens: "$tokens.tokens" }}
}
},
{$project: {time: "$_id", holdings: 1, _id: 0}},
{$sort: {time: 1}}
])
Playground example
From a performance perspective I recommend you split it into 2 calls, the first will be a quick findOne just to get the maximum time value in the collection.
Once you have that value the pipeline can be much leaner:
const maxItem = await db.collection.findOne({}).sort({ time: -1 });
db.collection.aggregate([
{
$unwind: "$holdings"
},
{
$group: {
_id: "$holdings.owner",
times: {
$push: {
time: "$time",
tokens: "$holdings.tokens"
}
},
minTime: {
$min: "$time"
}
}
},
{
$addFields: {
times: {
$reduce: {
input: {
$range: [
"$minTime",
maxItem.time + 1 // this is max time
]
},
initialValue: {
values: [],
lastIndex: 0
},
in: {
values: {
"$concatArrays": [
"$$value.values",
[
{
$cond: [
{
$in: [
"$$this",
"$times.time"
]
},
{
"$arrayElemAt": [
"$times",
"$$value.lastIndex"
]
},
{
"$mergeObjects": [
{
tokens: 0
},
{
"$arrayElemAt": [
"$times",
{
$subtract: [
"$$value.lastIndex",
1
]
}
]
},
{
time: "$$this"
}
]
}
]
}
]
]
},
lastIndex: {
$cond: [
{
$in: [
"$$this",
"$times.time"
]
},
{
$sum: [
"$$value.lastIndex",
1
]
},
"$$value.lastIndex"
]
}
}
}
}
}
},
{
$unwind: "$times.values"
},
{
$group: {
_id: "$times.values.time",
holdings: {
$push: {
owner: "$_id",
tokens: "$times.values.tokens"
}
}
}
},
{
$project: {
_id: 0,
time: "$_id",
holdings: 1
}
},
{
$sort: {
time: 1
}
}
])
This is still quite a heavy query as it requires to $unwind and $group the entire collection, however there is no workaround this due to the requirements. if the collection is too big for this approach I recommend iteration owner by owner, or time by time and doing separate updates accordingly.
Mongo Playground
If you don't care about performance at all and want it in a single query you can still use the same pipeline, you will have to first extract the max time in the collection, this will require you to add an initial $group stage, like so:
db.collection.aggregate([
{
$group: {
_id: null,
maxTime: {
$max: "$time"
},
roots: {
$push: "$$ROOT"
}
}
},
{
$unwind: "$roots"
},
{
$replaceRoot: {
newRoot: {
"$mergeObjects": [
"$roots",
{
maxTime: "$maxTime"
}
]
}
}
},
... same pipeline ...
])
I'm trying to count my "$attendance.status" with aggregation mongodb.
I've get my data with relations. then i want to count by relation columns like 'present', 'off', etc.
code
Employee.aggregate([
{
$lookup: {
from: "Attendance",
let: { employeeId: "$_id" },
pipeline: [
{
$match: {
$and: [
{ $expr: { $eq: ["$employeeId", "$$employeeId"] } },
{ isApproved: true },
{
createdAt: {
$gte: startOfMonth.toDate(),
$lte: endOfMonth.toDate(),
},
},
],
},
},
],
as: "attendance",
},
},
{
$project: {
_id: 1,
username: 1,
name: 1,
attendance: 1,
present: { $sum: { $eq: ["$attendance.status", "present"] } },
},
},
]);
But why cannot count my column?
i use $eq, with $sum then count the result. but the result is 0
{
"username": "Ethyl",
"name": "Kuhn",
"id": "614d43cde735f3e601dea165",
"attendance": [
{
"_id": "614d43cde735f3e601dea16f",
"status": "present",
"start": "2021-09-24T03:19:41.645Z",
"employeeId": "614d43cde735f3e601dea165",
"isApproved": true
},
],
"present": 0,
"sick": 0,
"off": 0,
},
I have two mongo aggregate pipelines that output results. Now I want combine these two pipelines to have a singular output.
Please find below sample collection.
[
{
_id: "ddfdfdfdggfgfgsg",
rate: "3323",
quantity_packs: "343",
shop_name: "Whole Foods",
sku: "20"
manufacturer_name: "Unilever"
},
{
_id: "ddfdfdfsdsds",
rate: "434",
quantity_packs: "453",
shop_name: "Carrefour",
sku: "200"
manufacturer_name: "Unilever"
},
{
_id: "dfdfdgcvgfgfvvv",
rate: "343",
quantity_packs: "23",
shop_name: "Target",
manufacturer_name: "Beirsdorf"
sku: "34"
}
]
Please find below my queries.
First Query
db.collection.aggregate([
{
$match: {
manufacturer_name: {
$in: [ "unilever" ]
}
}
},
{
$group: {
_id: {
"Shop Name": "$shop_name"
},
"total_sku": {
"$addToSet": "$sku"
},
"annual_cost": {
$sum: {
$cond: [
{
$eq: ["$manufacturer_name", "unilever"]
},
{
"$toDouble": "$rate"
},
0
]
}
},
"annual_qty": {
$sum: {
"$toDouble": "$annual_qty"
}
}
}
},
{
$project: {
"sku count": {
"$size": "$total_sku"
},
"Annual Cost WO GST": {
$multiply: [ "$annual_cost", "$annual_qty" ]
},
}
},
])
Result of First Query
[
{
_id: { 'Hospital Name': '7AM mart' },
'sku count': 29,
'Annual Cost WO GST': 79968887.67999999
},
{
_id: { 'Shop Name': 'Apex' },
'sku count': 20,
'Annual Cost WO GST': 1779192666.96
}
]
Second Query
db.collection.aggregate([
{
$match: {
$expr: {
$ne: ["$manufacturer_name", "unilever"]
}
}
},
{
$group: {
_id: {
"Shop Name": "$shop_name"
},
"annual_cost_wo_gst_wo_manu": {
$sum: {
"$toDouble": "$rate"
}
},
"annual_qty": {
$sum: {
"$toDouble": "$annual_qty"
}
}
}
},
{
$project: {
"Ann Cost For Other Manufacturers": {
$multiply: ["$annual_cost_wo_gst_wo_manu", "$annual_qty"]
},
}
}
])
Result of Second Query
[
{
_id: { 'Hospital Name': 'Apex' },
'Ann Cost For Other Manufacturers': 25246715130525.273
},
{
_id: { 'Hospital Name': '7AM Mart' },
'Ann Cost For Other Manufacturers': 1347701834351.495
}
]
As mentioned above, I somehow want to combine to results by correctly mapping the items.
Intended Result
[
{
_id: { 'Hospital Name': '7AM mart' },
'sku count': 29,
'Annual Cost WO GST': 79968887.67999999
'Ann Cost For Other Manufacturers': 1347701834351.495
},
{
_id: { 'Shop Name': 'Apex' },
'sku count': 20,
'Annual Cost WO GST': 1779192666.96
'Ann Cost For Other Manufacturers': 25246715130525.273
}
]
Your 2 queries do not quite produce your stated outputs. Nevertheless, you could first perform uncorrelated $lookup to perform your second query, storing the result of your secondary query in a field/object. Then you can continue your first query. Finally extract the result of secondary query from the previously stored field/object.
Here is a Mongo playground with some modifications to your original examples for your reference.
I'm writing a query to calculate multiple metrics for each user in my DB.
I've calculated all of the metrics, and have a structure like this
{
"metric1": [{"user_id": 1, "val": 13},{"user_id": 2, "val": 100}],
"metric2": [{"user_id": 2, "val": 29},{"user_id": 1, "val": 123}],
"metric3": [{"user_id": 1, "val": 46},{"user_id": 2, "val": 111]
}
I'm trying to convert the above into this structure
{
"user_id": [1,2],
"metric1": [13, 100],
"metric2": [29,123],
"metric3": [46,111]
}
So that I can display a table showing each user and the three metrics (one metric per column, and one user per row).
considering that your data is what you've said:
{
"metric1": [
{"id1": 1}, {"id2": 2}
],
"metric2": [
{"id2": 22}, {"id1": 11}
],
"metric3": [
{"id2": 222}, {"id1": 111}
]
}
all you've to do is using $unwind to be able to break the array and then $objectToArray to have access to keys
db.blah.aggregate([
{ $unwind: '$metric1' },
{ $unwind: '$metric2' },
{ $unwind: '$metric3' },
{ $project: {'metric1': { $objectToArray: '$metric1' }, 'metric2': { $objectToArray: '$metric2' }, 'metric3': { $objectToArray: '$metric3' }} },
{ $sort: { 'metric1.k' : -1} },
{ $sort: { 'metric2.k' : -1} },
{ $sort: { 'metric3.k' : -1} },
{ $unwind: '$metric1' },
{ $unwind: '$metric2' },
{ $unwind: '$metric3' },
{ $group: {
_id: null,
user_id: { $addToSet: '$metric1.k' },
metric1: { $addToSet: '$metric1.v' },
metric2: { $addToSet: '$metric2.v' },
metric3: { $addToSet: '$metric3.v' },
} },
{ $project: { _id: 0 } }
]).pretty()
which results
{
"user_id" : [
"id1",
"id2"
],
"metric1" : [
1,
2
],
"metric2" : [
11,
22
],
"metric3" : [
111,
222
]
}
I am trying to aggregate a batch of documents. There are two fields in the documents I would like to $push. However, lets say they are "_id" and "A" fields, I only want $push "_id" and "A" if "A" is $gt 0.
I tried two approaches.
First one.
db.collection.aggregate([{
"$group":{
"field": {
"$push": {
"$cond":[
{"$gt":["$A", 0]},
{"id": "$_id", "A":"$A"},
null
]
}
},
"secondField":{"$push":"$B"}
}])
But this will push a null value to "field" and I don't want it.
Second one.
db.collection.aggregate([{
"$group":
"field": {
"$cond":[
{"$gt",["$A", 0]},
{"$push": {"id":"$_id", "A":"$A"}},
null
]
},
"secondField":{"$push":"$B"}
}])
The second one simply doesn't work...
Is there a way to skip the $push in else case?
ADDED:
Expected documents:
{
"_id":objectid(1),
"A":2,
"B":"One"
},
{
"_id":objectid(2),
"A":3,
"B":"Two"
},
{
"_id":objectid(3),
"B":"Three"
}
Expected Output:
{
"field":[
{
"A":"2",
"_id":objectid(1)
},
{
"A":"3",
"_id":objectid(2)
},
],
"secondField":["One", "Two", "Three"]
}
You can use "$$REMOVE":
This system variable was added in version 3.6 (mongodb docs)
db.collection.aggregate([{
$group:{
field: {
$push: {
$cond:[
{ $gt: ["$A", 0] },
{ id: "$_id", A:"$A" },
"$$REMOVE"
]
}
},
secondField:{ $push: "$B" }
}
])
In this way you don't have to filter nulls.
This is my answer to the question after reading the post suggested by #Veeram
db.collection.aggregate([{
"$group":{
"field": {
"$push": {
"$cond":[
{"$gt":["$A", 0]},
{"id": "$_id", "A":"$A"},
null
]
}
},
"secondField":{"$push":"$B"}
},
{
"$project": {
"A":{"$setDifference":["$A", [null]]},
"B":"$B"
}
}])
One more option is to use $filter operator:
db.collection.aggregate([
{
$group : {
_id: null,
field: { $push: { id: "$_id", A : "$A"}},
secondField:{ $push: "$B" }
}
},
{
$project: {
field: {
$filter: {
input: "$field",
as: "item",
cond: { $gt: [ "$$item.A", 0 ] }
}
},
secondField: "$secondField"
}
}])
On first step you combine your array and filter them on second step
$group: {
_id: '$_id',
tasks: {
$addToSet: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.id', ''],
},
'',
],
},
then: '$$REMOVE',
else: {
id: '$tasks.id',
description: '$tasks.description',
assignee: {
$cond: {
if: {
$eq: [
{
$ifNull: ['$tasks.assignee._id', ''],
},
'',
],
},
then: undefined,
else: {
id: '$tasks.assignee._id',
name: '$tasks.assignee.name',
thumbnail: '$tasks.assignee.thumbnail',
status: '$tasks.assignee.status',
},
},
},
},
},
},
},
}