This is my collection in my db:
{
"items": [
{
"id": "1",
"audit": [
{
"validFrom": ISODate("2021-01-20T14:24:57.483Z"),
"validTo": ISODate("2024-01-20T14:24:57.483Z")
}
]
},
{
"id": "1",
"audit": [
{
"validFrom": ISODate("2021-01-19T14:24:57.483Z"),
"validTo": ISODate("2024-01-19T14:24:57.483Z")
}
]
}
]
}
Part 1:
I wanted to query validFrom. And while querying, I want to display that specific audit element alone. I tried these queries:
This query returned only the first element that matched the condition
db.Balances.find({"items.audit.validto":{"$lte": ISODate("2024-01-20T14:24:57.483Z")}},{"items.$":1})
This query returned all data of that collection alone irrespective of the filter
db.Balances.find({""items.audit.validto":{"$lte": ISODate("2024-01-20T14:24:57.483Z")}},{"items":1})
Part 2:
After getting the desired result, I want to display the audit list alone instead of the entire item list
Expected Output:
"audit": [
{
"validFrom": ISODate("2021-01-20T14:24:57.483Z"),
"validTo": ISODate("2024-01-20T14:24:57.483Z")
}
]
This is one way of doing it using an aggregation pipeline.
Unwind the items array.
Filter out the elements matching the criteria.
Filter out the elements in the audit array matching the criteria.
db.collection.aggregate([
{
"$unwind": "$items"
},
{
"$match": {
"items.audit.validTo": {
"$lte": ISODate("2024-01-20T14:24:57.483Z")
}
}
},
{
"$project": {
"audit": {
"$filter": {
"input": "$items.audit",
"as": "elem",
"cond": {
"$lte": [
"$$elem.validTo",
ISODate("2024-01-20T14:24:57.483Z")
]
}
}
},
_id: 0
}
}
])
Playground link.
Related
I have this simplified MongoDB document and would like to change something because there is quite a lot of redundant data. This field "activeUsersLookup" is the result of aggregation which returns data I'd like to put inside the first users array.
First id:
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a" matches
id from activeUsersLookup the same story is with user IDs.
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"lastactive": {
"$date": {
"$numberLong": "1637922656000"
}
}
},
{
"_id": "4972ba13-6f4e-4943-be07-15802e22e0dd",
"lastactive": {
"$date": {
"$numberLong": "1653286066000"
}
}
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"lastactive": {
"$date": {
"$numberLong": "1558623982000"
}
}
}
],
"activeUsersLookup": [
{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"activities": 2
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"activities": 1
}
],
"sumOfActivities": 3
}
]
}]
So more or less the final document should look like this:
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"lastactive": {
"$date": {
"$numberLong": "1637922656000"
}
},
"activities": 2
},
{
"_id": "4972ba13-6f4e-4943-be07-15802e22e0dd",
"lastactive": {
"$date": {
"$numberLong": "1653286066000"
}
},
"activities": 0
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"lastactive": {
"$date": {
"$numberLong": "1558623982000"
}
},
"activities": 1
},
"sumOfActivities": 3
]
}]
I've tried with:
{
$addFields: {
'licenses.activities': '$activeUsersLookup.users.activities'
}
}
But this gives me an empty array so I must be doing something wrong.
The next stage would be to sum all those activities as sumOfActivities and the last stage would be unset activeUsersLookup.
What magic tricks must I do to have the needed result? :)
I don't think the expected result you posted for the "sumOfActivities": 3 in the users array is valid.
Assume that you are trying to achieve the result as below:
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [...],
"sumOfActivities": 3
}]
The query is a bit long:
$set - Set activeUsersLookup field as object.
1.1. $first - Get the first document from 1.2.
1.2. $filter - Filter document(s) from activeUsersLookup by matching _id for the document in activeUsersLookup with _id (root document).
$set
2.1. - Set users array.
2.1.1. $map - Iterate the documents in users array and return a new array.
2.1.2. $mergeObjects - Merge current documents with the documents with activities field.
2.1.3. $ifNull - Set activities as 0 if no result returned from 2.1.4.
2.1.4. $getField - Get the activities field from the result 2.1.5.
2.1.5. $first - Get the first document from the result 2.1.6.
2.1.6. $filter - Filter the activeUsersLookup.users documents by matching _id for the document (users array) with _id for the current document.
2.2. Set sumOfActivities field.
$unset - Remove activeUsersLookup field.
db.collection.aggregate([
{
$set: {
activeUsersLookup: {
$first: {
$filter: {
input: "$activeUsersLookup",
cond: {
$eq: [
"$$this._id",
"$_id"
]
}
}
}
}
}
},
{
$set: {
users: {
$map: {
input: "$users",
as: "user",
in: {
$mergeObjects: [
"$$user",
{
activities: {
"$ifNull": [
{
"$getField": {
"field": "activities",
"input": {
$first: {
$filter: {
input: "$activeUsersLookup.users",
cond: {
$eq: [
"$$this._id",
"$$user._id"
]
}
}
}
}
}
},
0
]
}
}
]
}
}
},
sumOfActivities: "$activeUsersLookup.sumOfActivities"
}
},
{
$unset: "activeUsersLookup"
}
])
Sample Mongo Playground
I have this Collection in mongodb
{
"_id" : "777",
"someKey" : "someValue",
"someArray" : [
{
"name" : "name1",
"someNestedArray" : [
{
"name" : "value"
},
{
"name" : "delete me"
}
]
}
]
}
I want to find document based on someArray.someNestedArray.name
but i can't find any useful link all search result about update nested array
i am trying this but return nothing
db.mycollection.find({"someArray.$.someNestedArray":{"$elemMatch":{"name":"1"}}})
db.mycollection.find({"someArray.$.someNestedArray.$.name":"1"})
and Some thing else
how can i find by element in double nested array mongodb?
In the simplest sense this just follows the basic form of "dot notation" as used by MongoDB. That will work regardless of which array member the inner array member is in, as long as it matches a value:
db.mycollection.find({
"someArray.someNestedArray.name": "value"
})
That is fine for a "single field" value, for matching multiple-fields you would use $elemMatch:
db.mycollection.find({
"someArray": {
"$elemMatch": {
"name": "name1",
"someNestedArray": {
"$elemMatch": {
"name": "value",
"otherField": 1
}
}
}
}
})
That matches the document which would contain something with a a field at that "path" matching the value. If you intended to "match and filter" the result so only the matched element was returned, this is not possible with the positional operator projection, as quoted:
Nested Arrays
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Modern MongoDB
We can do this by applying $filter and $map here. The $map is really needed because the "inner" array can change as a result of the "filtering", and the "outer" array of course does not match the conditions when the "inner" was stripped of all elements.
Again following the example of actually having multiple properties to match within each array:
db.mycollection.aggregate([
{ "$match": {
"someArray": {
"$elemMatch": {
"name": "name1",
"someNestedArray": {
"$elemMatch": {
"name": "value",
"otherField": 1
}
}
}
}
}},
{ "$addFields": {
"someArray": {
"$filter": {
"input": {
"$map": {
"input": "$someArray",
"as": "sa",
"in": {
"name": "$$sa.name",
"someNestedArray": {
"$filter": {
"input": "$$sa.someNestedArray",
"as": "sn",
"cond": {
"$and": [
{ "$eq": [ "$$sn.name", "value" ] },
{ "$eq": [ "$$sn.otherField", 1 ] }
]
}
}
}
}
},
},
"as": "sa",
"cond": {
"$and": [
{ "$eq": [ "$$sa.name", "name1" ] },
{ "$gt": [ { "$size": "$$sa.someNestedArray" }, 0 ] }
]
}
}
}
}}
])
Therefore on the "outer" array the $filter actually looks at the $size of the "inner" array after it was "filtered" itself, so you can reject those results when the whole inner array does in fact match noting.
Older MongoDB
In order to "project" only the matched element, you need the .aggregate() method:
db.mycollection.aggregate([
// Match possible documents
{ "$match": {
"someArray.someNestedArray.name": "value"
}},
// Unwind each array
{ "$unwind": "$someArray" },
{ "$unwind": "$someArray.someNestedArray" },
// Filter just the matching elements
{ "$match": {
"someArray.someNestedArray.name": "value"
}},
// Group to inner array
{ "$group": {
"_id": {
"_id": "$_id",
"name": "$someArray.name"
},
"someKey": { "$first": "$someKey" },
"someNestedArray": { "$push": "$someArray.someNestedArray" }
}},
// Group to outer array
{ "$group": {
"_id": "$_id._id",
"someKey": { "$first": "$someKey" },
"someArray": { "$push": {
"name": "$_id.name",
"someNestedArray": "$someNestedArray"
}}
}}
])
That allows you to "filter" the matches in nested arrays for one or more results within the document.
You can also try something like below:
db.collection.aggregate(
{ $unwind: '$someArray' },
{
$project: {
'filteredValue': {
$filter: {
input: "$someArray.someNestedArray",
as: "someObj",
cond: { $eq: [ '$$someObj.name', 'delete me' ] }
}
}
}
}
)
I'm a complete beginner in mongodb . Actually I'm trying to find all the documents containing null or nothing for example documents like {
"_id" : "abc"
} for deleting them from collection.
But even after searching a lot of SO questions I couldn't get any solution .So, how can I do this ? and sorry if I'm ignoring anything.
I don't know how to do it in a single operation, but you can try something like this:
db["collectionName"].find({_id: {$exists: true}}).forEach(function(doc) {
if (Object.keys(doc).length === 1) {
// ..delete this document db["collectionName"].remove({_id: doc._id})
}
})
One possible solution is to get a list of the _id values of those null field documents and then remove them. This can be significantly efficient considering that you only execute two queries instead of looping through the whole collection (this can potentially affect your db performance especially with large collections).
Consider running the following aggregate pipeline to get those ids:
var ids = db.collection.aggregate([
{ "$project": {
"hashmaps": { "$objectToArray": "$$ROOT" }
} },
{ "$project": {
"keys": "$hashmaps.k"
} },
{ "$redact": {
"$cond": [
{
"$eq":[
{
"$ifNull": [
{ "$arrayElemAt": ["$keys", 1] },
0
]
},
0
]
},
"$$KEEP",
"$$PRUNE"
]
} },
{ "$group": {
"_id": null,
"ids": { "$push": "$_id" }
} }
]).toArray()[0]["ids"];
Removing the documents
db.collection.remove({ "_id": { "$in": ids } });
The other approach is similar to the above in that you would need two queries; the first which returns a list of all the top level fields in the collection and the last removes the documents from the collection which do not have those fields altogether.
Consider running the following queries:
/*
Run an aggregate pipeline operation to get a list
of all the top-level fields in the collection
*/
var fields = db.collection.aggregate([
{ "$project": {
"hashmaps": { "$objectToArray": "$$ROOT" }
} },
{ "$project": {
"keys": "$hashmaps.k"
} },
{ "$group": {
"_id": null,
"fields": { "$addToSet": "$keys" }
} },
{ "$project": {
"fields": {
"$setDifference": [
{
"$reduce": {
"input": "$fields",
"initialValue": [],
"in": { "$setUnion" : ["$$value", "$$this"] }
}
},
["_id"]
]
}
}
}
]).toArray()[0]["fields"];
The second query looks for the existence of all the fields except the _id one. For example, suppose your collection has documents with the keys _id, a, b and c, the query
db.collection.find({
"a" : { "$exists": false },
"b" : { "$exists": false },
"c" : { "$exists": false }
});
matches documents that do not contain the all the three fields a, b AND c:
So if you have a list of the top level fields in your collection then all you need is to construct the above query document. Use reduce method on the array for this:
// Construct the above query
var query = fields.reduce(function(acc, curr) {
acc[curr] = { "$exists": false };
return acc;
}, {});
Then use the query to remove the documents as
db.collection.remove(query);
I have this collection:
{
"name": "Leonardo",
"height": "180",
"weapon": "sword",
"favorite_pizza": "Hawai"
},
{
"name": "Donatello",
"height": "181",
"weapon": "stick",
"favorite_pizza": "Pepperoni"
},
{
"name": "Michelangelo",
"height": "182",
"weapon": "nunchucks",
"favorite_pizza": "Bacon"
},
{
"name": "Raphael",
"height": "183",
"weapon": "sai",
"favorite_pizza": "Margherita"
}
With using one query I want this result (ordered by height):
{
"name": "Leonardo",
"height": "180",
"weapon": "sword",
"favorite_pizza": "Hawai"
},
{
"name": "Donatello",
},
{
"name": "Michelangelo",
},
{
"name": "Raphael",
}
So the query needs to first get the document which has smallest height field and then get all contents of that document, then it needs to get all other documents and return only name field of those documents, while ordering those documents by height.
Change your height to numeric for correct sorting and you can try below aggregation in 3.4 pipeline.
The query $sorts the document by "height" ascending followed by $group to create two fields, "first" field which has the smallest height record ($$ROOT to access the whole document) and "allnames" to record all names.
$project with $slice + $concatArrays to replace the "allnames" array first element with the smallest height document and get the updated array.
$unwind with $replaceRoot to promote all the docs to top level.
db.colname.aggregate([
{"$sort":{
"height":1
}},
{"$group":{
"_id":null,
"first":{"$first":"$$ROOT"},
"allnames":{"$push":{"name":"$name"}}
}},
{"$project":{
"data":{"$concatArrays":[["$first"],{"$slice":["$allnames",1,{"$size":"$allnames"}] } ]}
}},
{"$unwind":"$data"},
{"$replaceRoot":{"newRoot":"$data"}}
])
Just for completeness reasons...
#Veeram's answer is probably the better choice (I have a feeling it should be faster and easier to understand) but you can achieve the same result using a slightly simpler $group stage followed by slightly more complex $project stage using $reduce:
collection.aggregate([{
$sort: {
"height": 1
}
}, {
$group: {
"_id":null,
"allnames": {
$push: "$$ROOT"
}
}
}, {
$project: {
"data": {
$reduce: {
input: "$allnames",
initialValue: null,
in: {
$cond: [{
$eq: [ "$$value", null ] // if it's the first time we come here
},
[ "$$this" ], // we include the entire document
{
$concatArrays: [ // else we concat
"$$value", // the already concatenated values
[ { "name": "$$this.name" } ] // with the "name" of the currently looked at document
]
}]
}
}
}
}
}, {
$unwind: "$data"
}, {
$replaceRoot: {
"newRoot": "$data"
}
}])
Alternatively - as pointed out by #Veeram in the comment below - , it's possible to write the $reduce in this way:
$project: {
"data": {
$reduce: {
input: { "$slice": [ "$allnames", 1, { $size: "$allnames" } ] }, // process everything in the "allnames" array except for the first item
initialValue: { "$slice": [ "$allnames", 1 ] }, // start with the first item
in: { $concatArrays: [ "$$value", [ { "name": "$$this.name" } ] ]} // and keep appending the "name" field of all other items only
}
}
}
I have following json structure in mongo collection-
{
"students":[
{
"name":"ABC",
"fee":1233
},
{
"name":"PQR",
"fee":345
}
],
"studentDept":[
{
"name":"ABC",
"dept":"A"
},
{
"name":"XYZ",
"dept":"X"
}
]
},
{
"students":[
{
"name":"XYZ",
"fee":133
},
{
"name":"LMN",
"fee":56
}
],
"studentDept":[
{
"name":"XYZ",
"dept":"X"
},
{
"name":"LMN",
"dept":"Y"
},
{
"name":"ABC",
"dept":"P"
}
]
}
Now I want to calculate following output.
if students.name = studentDept.name
so my result should be as below
{
"name":"ABC",
"fee":1233,
"dept":"A",
},
{
"name":"XYZ",
"fee":133,
"dept":"X"
}
{
"name":"LMN",
"fee":56,
"dept":"Y"
}
Do I need to use mongo aggregation or is it possible to get above given output without using aggregation???
What you are really asking here is how to make MongoDB return something that is actually quite different from the form in which you store it in your collection. The standard query operations do allow a "limitted" form of "projection", but even as the title on the page shared in that link suggests, this is really only about "limiting" the fields to display in results based on what is present in your document already.
So any form of "alteration" requires some form of aggregation, which with both the aggregate and mapReduce operations allow to "re-shape" the document results into a form that is different from the input. Perhaps also the main thing people miss with the aggregation framework in particular, is that it is not just all about "aggregating", and in fact the "re-shaping" concept is core to it's implementation.
So in order to get results how you want, you can take an approach like this, which should be suitable for most cases:
db.collection.aggregate([
{ "$unwind": "$students" },
{ "$unwind": "$studentDept" },
{ "$group": {
"_id": "$students.name",
"tfee": { "$first": "$students.fee" },
"tdept": {
"$min": {
"$cond": [
{ "$eq": [
"$students.name",
"$studentDept.name"
]},
"$studentDept.dept",
false
]
}
}
}},
{ "$match": { "tdept": { "$ne": false } } },
{ "$sort": { "_id": 1 } },
{ "$project": {
"_id": 0,
"name": "$_id",
"fee": "$tfee",
"dept": "$tdept"
}}
])
Or alternately just "filter out" the cases where the two "name" fields do not match and then just project the content with the fields you want, if crossing content between documents is not important to you:
db.collection.aggregate([
{ "$unwind": "$students" },
{ "$unwind": "$studentDept" },
{ "$project": {
"_id": 0,
"name": "$students.name",
"fee": "$students.fee",
"dept": "$studentDept.dept",
"same": { "$eq": [ "$students.name", "$studentDept.name" ] }
}},
{ "$match": { "same": true } },
{ "$project": {
"name": 1,
"fee": 1,
"dept": 1
}}
])
From MongoDB 2.6 and upwards you can even do the same thing "inline" to the document between the two arrays. You still want to reshape that array content in your final output though, but possible done a little faster:
db.collection.aggregate([
// Compares entries in each array within the document
{ "$project": {
"students": {
"$map": {
"input": "$students",
"as": "stu",
"in": {
"$setDifference": [
{ "$map": {
"input": "$studentDept",
"as": "dept",
"in": {
"$cond": [
{ "$eq": [ "$$stu.name", "$$dept.name" ] },
{
"name": "$$stu.name",
"fee": "$$stu.fee",
"dept": "$$dept.dept"
},
false
]
}
}},
[false]
]
}
}
}
}},
// Students is now an array of arrays. So unwind it twice
{ "$unwind": "$students" },
{ "$unwind": "$students" },
// Rename the fields and exclude
{ "$project": {
"_id": 0,
"name": "$students.name",
"fee": "$students.fee",
"dept": "$students.dept"
}},
])
So where you want to essentially "alter" the structure of the output then you need to use one of the aggregation tools to do. And you can, even if you are not really aggregating anything.