I want to filter my documents by sum of decimal field in array of objects, but didn't find anything good enough. for example I have documents like below:
[
{
"id": 1,
"limit": NumberDecimal("100000"),
"requests": [
{
"money": NumberDecimal("50000"),
"user": "user1"
}
]
},
{
"id": 2,
"limit": NumberDecimal("100000"),
"requests": [
{
"money": NumberDecimal("100000"),
"user": "user2"
}
]
},
{
"id": 1,
"limit": null,
"requests": [
{
"money": NumberDecimal("50000"),
"user": "user1"
},
{
"money": NumberDecimal("50000"),
"user": "user3"
}
]
},
]
description by documents fields:
limit - maximum amount of money, that I have
requests - array of objects, where money it's how much money user get from limit (if user1 get 50000 money there remainder it's 50000, limit - sum(requests.money))
I am making query in mongodb from scala projects:
get all documents where limit equal to null
get all documents where I have x remainder money (x like input value)
first case it's more easy than second one, I know how I can get sum of requests.money: I am doing it by this query:
db.campaign.aggregate([
{$project: {
total: {$sum: ["$requests.money"]}
}}
])
scala filter part
Filters.or(
Filters.equal("limit", null),
Filters.expr(Document(s""" {$$project: {total: {$$sum: ["$$requests.money"]}}}"""))
)
But I don't want to store it and get as result, I want to filter by this condition x (money which I want to get by some user) limit >= sum(requests.money) + x. And by this filter I want to get all filtered documents.
Example:
x = 50000
and output must be like this:
[
{
"id": 1,
"limit": NumberDecimal("100000"),
"requests": [
{
"money": NumberDecimal("50000"),
"user": "user1"
}
]
},
{
"id": 1,
"limit": null,
"requests": [
{
"money": NumberDecimal("50000"),
"user": "user1"
},
{
"money": NumberDecimal("50000"),
"user": "user3"
}
]
},
]
You have to use an aggregation pipeline like this:
db.campaign.aggregate([
{
$set: {
remainder: {
$subtract: [ "$limit", { $sum: "$requests.money" } ]
}
}
},
{
"$match": {
$or: [
{ limit: null },
{ remainder: { $gte: 0 } }
]
}
},
{ $unset: "remainder" }
])
Mongo Playground
This one is also possible, but more difficult to read:
db.campaign.aggregate([
{
"$match": {
$or: [
{ limit: null },
{
$expr: {
$gt: [
{ $subtract: [ "$limit", { $sum: "$requests.money" } ] },
0
]
}
}
]
}
}
])
Related
How can I get all the sum of fields in an array in Mongoose?
I want to sum up all the amounts in the payments array.
DB:
[
{
"_id": 0,
"name": "shoe",
"payments": [
{
"type": "a",
"amount": 10
},
{
"type": "b",
"amount": 15
},
{
"type": "a",
"amount": 15
},
]
},
{
"_id": 0,
"name": "shirt",
"payments": [
{
"type": "a",
"amount": 5
},
{
"type": "b",
"amount": 20
},
]
}
]
Expected result:
{
"amountSum": 65
}
There is a shorter and most likely faster solution:
db.collection.aggregate([
{
$group: {
_id: null,
amountSum: { $sum: { $sum: "$payments.amount" } }
}
}
])
$group - Group all documents.
1.1. $sum - Sum the value returned from 1.1.1 for the amountSum field.
1.1.1. $reduce - As payments is an array of objects, sum all the amount for the elements and transform the result from the array to number.
db.collection.aggregate([
{
$group: {
_id: null,
amountSum: {
$sum: {
$reduce: {
input: "$payments",
initialValue: 0,
in: {
$sum: [
"$$value",
"$$this.amount"
]
}
}
}
}
}
}
])
Demo # Mongo Playground
In my collection, I have documents which contains an array (called values) of objects, which has id and val fields. The data looks like this
[
{
values: [
{
"id": "123",
"val": true
},
{
"id": "456",
"val": true
},
{
"id": "789",
"val": false
},
]
},
{
values: [
{
"id": "123",
"val": true
},
{
"id": "123",
"val": true
},
{
"id": "123",
"val": false
},
]
},
{
values: [
{
"id": "234",
"val": false
},
{
"id": "567",
"val": false
}
]
}
]
I want to query this data by val to ensure it is true, and there may be cases where I want to ensure that an array has 2 instances of val's that is true. I am able to achieve this with the following query:
db.collection.find({
values: {
"$elemMatch": {
val: true
}
},
$expr: {
$gte: [
{
$reduce: {
input: "$values",
initialValue: 0,
in: {
$sum: [
"$$value",
{
$cond: [
{
$eq: [
"$$this.val",
true
]
},
1,
0
]
}
]
}
}
},
2
]
}
})
The results of the query above give me the following result:
[
{
"values": [
{
"id": "123",
"val": true
},
{
"id": "456",
"val": true
},
{
"id": "789",
"val": false
}
]
},
{
"values": [
{
"id": "123",
"val": true
},
{
"id": "123",
"val": true
},
{
"id": "123",
"val": false
}
]
}
]
The issue with this now, is that I want to ensure that there are no duplicate id's in the results of the $reduce'd list. I have researched the docs and $group looks promising, however I am unsure on how to integrate this into the query.
I want the result to look like the following:
[
{
"values": [
{
"id": "123",
"val": true
},
{
"id": "456",
"val": true
},
{
"id": "789",
"val": false
}
]
}
]
Here is a MongoPlayground link with all of the above in it.
Please Note
I have simplified the code examples here to get to the core of the problem. In my actual use case, there are many values of different data types, which means that using an $elemMatch is the best way to go for me.
After looking at Takis's answer, it made me realise that using $setUnion is an alternative way to find the distinct values of a field. Using this, I was able to rework my query to achieve what I want.
What I have done to achieve this is to have a $cond operator within the $expr operator. I pass in the original $reduce as the condition to see if x amount of val's appear within the document. If it succeeds, then I am ensuring that the size of the union of values.id's (i.e. all the unique id's within the current document values array) is at least x amount. If this condition is satisfied, the document will be returned. If not, then it falls back to the value false, i.e. it will not return the current document.
The query is as follows:
db.collection.find({
values: {
"$elemMatch": {
val: true
}
},
$expr: {
$cond: [
{
$gte: [
{
$reduce: {
input: "$values",
initialValue: 0,
in: {
$sum: [
"$$value",
{
$cond: [
{
$eq: [
"$$this.val",
true
]
},
1,
0
]
}
]
}
}
},
2 // x amount of instances
]
},
{
$gte: [
{
"$size": {
$setUnion: [
"$values.id"
]
}
},
2 // x amount of instances
]
},
false
]
}
})
Here is a MongoPlayground link showing it in action.
Query
reduce is fine, but maybe 2 filters are simpler here
first filter to have true count >=2
second filter to not have duplicate id, by comparing the values.id length, with the length of the set values.id (keep it only if same size else => duplicate), it works using paths on arrays = arrays like values.id is array of ids
*if this is slow for your use case maybe it can be faster
Playmongo
aggregate(
[{"$match":
{"$expr":
{"$and":
[{"$gte":
[{"$size": {"$filter": {"input": "$values.val", "cond": "$$this"}}},
2]},
{"$eq":
[{"$size": "$values.id"},
{"$size": {"$setUnion": ["$values.id", []]}}]}]}}}])
Suppose I have document as userDetails:
[
{
"roles": [
"author",
"reader"
],
"completed_roles": ["author", "reader"],
"address": {
"current_address": {
"city": "abc"
}
},
"is_verified": true
},
{
"roles": [
"reader"
],
"completed_roles": ["reader"],
"address": {
"current_address": {
"city": "abc"
}
},
"is_verified": true
},
{
"roles": [
"author"
],
"completed_roles": [],
"address": {
"current_address": {
"city": "xyz"
}
},
"is_verified": false
}
]
I want to fetch sum for all roles which has author based on city, total_roles_completed and is_verified.
So the O/P should look like:
[
{
"_id": {
"city": "abc"
},
"total_author": 1,
"total_roles_completed": 1,
"is_verified": 1
},
{
"_id": {
"city": "xyz"
},
"total_author": 1,
"total_roles_completed": 0,
"is_verified": 0
}
]
Basic O/P required:
Filter the document based on author in role (other roles may be present in role but author must be present)
Sum the author based on city
sum on basis of completed_profile has "author"
Sum on basis of documents if they are verified.
For this I tried as:
db.userDetails.aggregate([
{
$match: {
roles: {
$eleMatch: {
$eq: "author"
}
}
}
},
{
$unwind: "$completed_roles"
},
{
"$group": {
_id: { city: "$address.current_address.city"},
total_authors: {$sum: 1},
total_roles_completed: {
$sum: {
$cond: [
{
$eq: ["$completed_roles","author"]
}
]
}
},
is_verified: {
$sum: {
$cond: [
{
$eq: ["$is_verified",true]
}
]
}
}
}
}
]);
But the sum is incorrect. Please let me know where I made mistake. Also, if anyone needs any further information please let me know.
Edit: I figured that because of unwind it is giving me incorrect value, if I remove the unwind the sum is coming correct.
Is there any other way by which I can calculate the sum of total_roles_completed for each city?
If I've understood correctly you can try this query:
First $match to get only documents where roles contains author.
And then $group by the city (the document is not a valid JSON so I assume is address:{"current_addres:{city:"abc"}}). This $group get the authors for each city and also: $sum 1 if "author" is in completed_roles and check if is verified.
Here I don't know the way to know if the author is verified (I don't know if can be true in one document and false in other document. If is the same value over all documents you can use $first to get the first is_verified value). But I decided to use $allElementsTrue in a $project stage, so this only will be true if is_verified is true in all documents grouped by $group.
db.collection.aggregate([
{
"$match": {
"roles": "author"
}
},
{
"$group": {
"_id": "$address.current_address.city",
"total_author": {
"$sum": 1
},
"total_roles_completed": {
"$sum": {
"$cond": {
"if": {
"$in": [
"author",
"$completed_roles"
]
},
"then": 1,
"else": 0
}
}
},
"is_verified": {
"$addToSet": "$is_verified"
}
}
},
{
"$project": {
"_id": 0,
"city": "$_id",
"is_verified": {
"$allElementsTrue": "$is_verified"
},
"total_author": 1,
"total_roles_completed": 1
}
}
])
Example here
The result from this query is:
[
{
"city": "xyz",
"is_verified": false,
"total_author": 1,
"total_roles_completed": 0
},
{
"city": "abc",
"is_verified": true,
"total_author": 2,
"total_roles_completed": 2
}
]
I have a question regarding querying data in MongoDB. Here is my sample data:
{
"_id": 1,
"category": "fruit",
"userId": 1,
"name": "Banana"
},
{
"_id": 2,
"category": "fruit",
"userId": 2,
"name": "Apple"
},
{
"_id": 3,
"category": "fresh-food",
"userId": 1,
"name": "Fish"
},
{
"_id": 4,
"category": "fresh-food",
"userId": 2,
"name": "Shrimp"
},
{
"_id": 5,
"category": "vegetable",
"userId": 1,
"name": "Salad"
},
{
"_id": 6,
"category": "vegetable",
"userId": 2,
"name": "carrot"
}
The requirements:
If the category is fruit, returns all the records match
If the category is NOT fruit, returns maximum 10 records of each category grouped by user
The category is known and stable, so we can hard-coded in our query.
I want to get it done in a single query. So the result expected should be:
{
"fruit": [
... // All records of
],
"fresh-food": [
{
"userId": 1,
"data": [
// Top 10 records of user 1 with category = "fresh-food"
]
},
{
"userId": 2,
"data": [
// Top 10 records of user 2 with category = "fresh-food"
]
},
...
],
"vegetable": [
{
"userId": 1,
"data": [
// Top 10 records of user 1 with category = "vegetable"
]
},
{
"userId": 2,
"data": [
// Top 10 records of user 2 with category = "vegetable"
]
},
]
}
I've found the guideline to group by each group using $group and $slice, but I can't apply the requirement number #1.
Any help would be appreciated.
You need to use aggregation for this
$facet to categorize incoming data, we categorized into two. 1. Fruit and 2. non_fruit
$match to match the condition
$group first group to group the data based on category and user. Second group to group by its category only
$objectToArray to make the object into key value pair
$replaceRoot to make the non_fruit to root with fruit
Here is the code
db.collection.aggregate([
{
"$facet": {
"fruit": [
{ $match: { "category": "fruit" } }
],
"non_fruit": [
{
$match: {
$expr: {
$ne: [ "$category", "fruit" ]
}
}
},
{
$group: {
_id: { c: "$category", u: "$userId" },
data: { $push: "$$ROOT" }
}
},
{
$group: {
_id: "$_id.c",
v: {
$push: {
uerId: "$_id.u",
data: { "$slice": [ "$data", 3 ] }
}
}
}
},
{ $addFields: { "k": "$_id", _id: "$$REMOVE" } }
]
}
},
{ $addFields: { non_fruit: { "$arrayToObject": "$non_fruit" } }},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [ "$$ROOT", "$non_fruit" ]
}
}
},
{ $project: { non_fruit: 0 } }
])
Working Mongo playground
I have a collection with documents like this:
[
{
"user_id": 1,
"prefs": [
"item1",
"item2",
"item3",
"item4"
]
},
{
"user_id": 2,
"prefs": [
"item2",
"item5",
"item3"
]
},
{
"user_id": 3,
"prefs": [
"item4",
"item3",
"item7"
]
}
]
What I want is to write an aggregation which will get a user_id and producer a list containing all users mapped to the number of same prefs in their lists. for example if I run the aggregation for user_id = 1, I have to get:
[
{
"user_id": 2,
"same": 1
},
{
"user_id": 3,
"same": 2
}
]
You cannot write any query here with input as simple as "user_id": 1 here, but you can retrieve the document for that user and then get a comparison of that data to the other documents you are retrieving:
var doc = db.collection.findOne({ "user_id": 1 });
db.collection.aggregate([
{ "$match": { "user_id": { "$ne": 1 } } },
{ "$project": {
"_id": 0,
"user_id": 1
"same": { "$size": { "$setIntersection": [ "$prefs", doc.prefs ] } }
}}
])
Which is one approach, but also not that much different to comparing each document in the client:
function intersect(a,b) {
var t;
if (b.length > a.length) t = b, b = a, a = t;
return a.filter(function(e) {
if (b.indexOf(e) != -1) return true;
});
}
var doc = db.collection.findOne({ "user_id": 1 });
db.collection.find({ "user_id": { "$ne": 1 } }).forEach(function(mydoc) {
printjson({
"user_id": mydoc.user_id,
"same": intersect(mydoc.prefs, doc.prefs).length
});
});
It's the same thing. You are not really "aggregating" anything here but just making comparisons of one documents content against the other. Of course you can ask the aggregation framework to do something like "filter" out anything that does not have a similar match:
var doc = db.collection.findOne({ "user_id": 1 });
db.collection.aggregate([
{ "$match": { "user_id": { "$ne": 1 } } },
{ "$project": {
"_id": 0,
"user_id": 1
"same": { "$size": { "$setIntersection": [ "$prefs", doc.prefs ] } }
}},
{ "$match": { "same": { "$gt": 0 } }}
])
Though actually that would be more efficient to remove any documents with a zero count before doing the projection:
var doc = db.collection.findOne({ "user_id": 1 });
db.collection.aggregate([
{ "$match": { "user_id": { "$ne": 1 } } },
{ "$redact": {
"$cond": {
"if": { "$gt": [
{ "$size": { "$setIntersection": [ "$prefs", doc.prefs ] } },
0
]},
"then": "$$KEEP",
"else": "$$PRUNE"
}
}},
{ "$project": {
"_id": 0,
"user_id": 1
"same": { "$size": { "$setIntersection": [ "$prefs", doc.prefs ] } }
}}
])
And at least then that would make some sense to do the server processing.
But otherwise, it's all pretty much the same, with possibly a "little" more overhead on the client working out the "intersection" here.