mongodb - find previous and next document in aggregation framework - mongodb

After applying a long pipeline to my collection I can obtain something like this:
{
{
"_id": "main1",
"title": "First",
"code": "C1",
"subDoc": {
"active": true,
"sub_id": "main1sub1",
"order": 1
}
},
{
"_id": "main2",
"title": "Second",
"code": "C2",
"subDoc": {
"active": true,
"sub_id": "main2sub1",
"order": 1
}
},
{
"_id": "main3",
"title": "Third",
"code": "C3",
"subDoc": {
"active": false,
"sub_id": "main3sub1",
"order": 1
}
}
}
The documents are already in the correct order. Now I have to find the document immediately preceding or following the one corresponding to a given parameter. For example, if I know { "code" : "C2" } I have to retrieve the previous document (example document with "code" : "C1").
I only need to get that document, not the others.
I know how to do it using the find () method and applying sort () and limit () in sequence, but I want to get the document directly in the aggregation pipeline, adding the necessary stages to do it.
I've tried some combinations of $ indexOfArray and $ arrayElemAt, but the first problem I encounter is that I don't have an array, it's just documents.
The second problem is that the parameter I know might sometimes be inside the subdocument, for example {"sub_id": "main3sub1"}, and again I should always get the previous or next parent document as a response (in the example, the pipeline should return document "main2" as previous document)
I inserted the collection in mongoplayground to be able to perform the tests quickly:
mongoplayground
Any idea?

If you want to retrieve only the previous document, use the following query:
First Approach:
Using $match,$sort,$limit
db.collection.aggregate([
{
$match: {
code: {
"$lt": "C2"
}
}
},
{
"$sort": {
code: -1
}
},
{
$limit: 1
}
])
MongoDB Playground
Second Approach:
As specified by # Wernfried Domscheit,
Converting to array and then using $arrayElemAt
db.collection.aggregate([
{
$group: {
_id: null,
data: {
$push: "$$ROOT"
}
}
},
{
$addFields: {
"index": {
$subtract: [
{
$indexOfArray: [
"$data.code",
"C2"
]
},
1
]
}
}
},
{
$project: {
_id: 0,
data: {
$arrayElemAt: [
"$data",
"$index"
]
}
}
},
{
$replaceRoot: {
newRoot: "$data"
}
}
])
MongoDB Playground

Related

MongoDB - Move some fields from the array into another array

I have this simplified MongoDB document and would like to change something because there is quite a lot of redundant data. This field "activeUsersLookup" is the result of aggregation which returns data I'd like to put inside the first users array.
First id:
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a" matches
id from activeUsersLookup the same story is with user IDs.
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"lastactive": {
"$date": {
"$numberLong": "1637922656000"
}
}
},
{
"_id": "4972ba13-6f4e-4943-be07-15802e22e0dd",
"lastactive": {
"$date": {
"$numberLong": "1653286066000"
}
}
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"lastactive": {
"$date": {
"$numberLong": "1558623982000"
}
}
}
],
"activeUsersLookup": [
{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"activities": 2
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"activities": 1
}
],
"sumOfActivities": 3
}
]
}]
So more or less the final document should look like this:
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [
{
"_id": "eaa946da-2708-443e-ab4c-b6db357050ca",
"lastactive": {
"$date": {
"$numberLong": "1637922656000"
}
},
"activities": 2
},
{
"_id": "4972ba13-6f4e-4943-be07-15802e22e0dd",
"lastactive": {
"$date": {
"$numberLong": "1653286066000"
}
},
"activities": 0
},
{
"_id": "6c4a62ce-c6c6-430f-a0cd-d348ec77dbb2",
"lastactive": {
"$date": {
"$numberLong": "1558623982000"
}
},
"activities": 1
},
"sumOfActivities": 3
]
}]
I've tried with:
{
$addFields: {
'licenses.activities': '$activeUsersLookup.users.activities'
}
}
But this gives me an empty array so I must be doing something wrong.
The next stage would be to sum all those activities as sumOfActivities and the last stage would be unset activeUsersLookup.
What magic tricks must I do to have the needed result? :)
I don't think the expected result you posted for the "sumOfActivities": 3 in the users array is valid.
Assume that you are trying to achieve the result as below:
[{
"_id": "80b1565a-faf4-4e68-9bd6-8344060e8d3a",
"users": [...],
"sumOfActivities": 3
}]
The query is a bit long:
$set - Set activeUsersLookup field as object.
1.1. $first - Get the first document from 1.2.
1.2. $filter - Filter document(s) from activeUsersLookup by matching _id for the document in activeUsersLookup with _id (root document).
$set
2.1. - Set users array.
2.1.1. $map - Iterate the documents in users array and return a new array.
2.1.2. $mergeObjects - Merge current documents with the documents with activities field.
2.1.3. $ifNull - Set activities as 0 if no result returned from 2.1.4.
2.1.4. $getField - Get the activities field from the result 2.1.5.
2.1.5. $first - Get the first document from the result 2.1.6.
2.1.6. $filter - Filter the activeUsersLookup.users documents by matching _id for the document (users array) with _id for the current document.
2.2. Set sumOfActivities field.
$unset - Remove activeUsersLookup field.
db.collection.aggregate([
{
$set: {
activeUsersLookup: {
$first: {
$filter: {
input: "$activeUsersLookup",
cond: {
$eq: [
"$$this._id",
"$_id"
]
}
}
}
}
}
},
{
$set: {
users: {
$map: {
input: "$users",
as: "user",
in: {
$mergeObjects: [
"$$user",
{
activities: {
"$ifNull": [
{
"$getField": {
"field": "activities",
"input": {
$first: {
$filter: {
input: "$activeUsersLookup.users",
cond: {
$eq: [
"$$this._id",
"$$user._id"
]
}
}
}
}
}
},
0
]
}
}
]
}
}
},
sumOfActivities: "$activeUsersLookup.sumOfActivities"
}
},
{
$unset: "activeUsersLookup"
}
])
Sample Mongo Playground

MongoDB Aggregate Query to find the documents with missing values

I am having a huge collection of objects where the data is stored for different employees.
{
"employee": "Joe",
"areAllAttributesMatched": false,
"characteristics": [
{
"step": "A",
"name": "house",
"score": "1"
},
{
"step": "B",
"name": "car"
},
{
"step": "C",
"name": "job",
"score": "3"
}
]
}
There are cases where the score for an object is completely missing and I want to find out all these details from the database.
In order to do this, I have written the following query, but seems I am going wrong somewhere due to which it is not displaying the output.
I want the data in the following format for this query, so that it is easy to find out which employee is missing the score for which step and which name.
db.collection.aggregate([
{
"$unwind": "$characteristics"
},
{
"$match": {
"characteristics.score": {
"$exists": false
}
}
},
{
"$project": {
"employee": 1,
"name": "$characteristics.name",
"step": "$characteristics.step",
_id: 0
}
}
])
You need to use $exists to check the existence
playground
You can use $ifNull to handle both cases of 1. the score field is missing 2. score is null.
db.collection.aggregate([
{
"$unwind": "$characteristics"
},
{
"$match": {
$expr: {
$eq: [
{
"$ifNull": [
"$characteristics.score",
null
]
},
null
]
}
}
},
{
"$group": {
_id: null,
documents: {
$push: {
"employee": "$employee",
"name": "$characteristics.name",
"step": "$characteristics.step",
}
}
}
},
{
$project: {
_id: false
}
}
])
Here is the Mongo playground for your reference.

MongoDb aggregation framework to group elements of inner array

I'm using MongoDB aggregation framework trying to transform each document:
{
"all": [
{
"type": "A",
"id": "1"
},
{
"type": "A",
"id": "1"
},
{
"type": "B",
"id": "2"
},
{
"type": "A",
"id": "3"
}
]
}
into this:
{
"unique_type_A": [ "3", "1" ]
}
(final result is a collection of n documents with unique_type_A field)
The calculation consists of returning in an array all the uniques types of entities of type A.
I got stuck with $group step, anyone knows how to do it?
To apply this logic to each document, you can use the following;
db.collection.aggregate([
{
$unwind: "$all"
},
{
$match: {
"all.type": "A"
}
},
{
$group: {
_id: {
"type": "$all.type",
"oldId": "$_id"
},
unique_type_A: {
$addToSet: "$all.id"
}
}
},
{
$project: {
_id: 0
}
}
])
Where we first $unwind, to be able to filter and play with each member of all array. Then we just filter the non type:"A" members. The $group stage has the difference with a complex _id, where we utilize the _id of $unwind result, which refers back to the original document, so that we can group the results per original document. Collecting the id from all array with $addToSet to keep only unique values, and voilĂ !
And here is the result per document;
[
{
"unique_type_A": [
"3",
"1"
]
},
{
"unique_type_A": [
"4",
"11",
"5"
]
}
]
Check the code interactively on Mongoplayground

MongoDB: single find request to return data from different documents with different fields

I have this collection:
{
"name": "Leonardo",
"height": "180",
"weapon": "sword",
"favorite_pizza": "Hawai"
},
{
"name": "Donatello",
"height": "181",
"weapon": "stick",
"favorite_pizza": "Pepperoni"
},
{
"name": "Michelangelo",
"height": "182",
"weapon": "nunchucks",
"favorite_pizza": "Bacon"
},
{
"name": "Raphael",
"height": "183",
"weapon": "sai",
"favorite_pizza": "Margherita"
}
With using one query I want this result (ordered by height):
{
"name": "Leonardo",
"height": "180",
"weapon": "sword",
"favorite_pizza": "Hawai"
},
{
"name": "Donatello",
},
{
"name": "Michelangelo",
},
{
"name": "Raphael",
}
So the query needs to first get the document which has smallest height field and then get all contents of that document, then it needs to get all other documents and return only name field of those documents, while ordering those documents by height.
Change your height to numeric for correct sorting and you can try below aggregation in 3.4 pipeline.
The query $sorts the document by "height" ascending followed by $group to create two fields, "first" field which has the smallest height record ($$ROOT to access the whole document) and "allnames" to record all names.
$project with $slice + $concatArrays to replace the "allnames" array first element with the smallest height document and get the updated array.
$unwind with $replaceRoot to promote all the docs to top level.
db.colname.aggregate([
{"$sort":{
"height":1
}},
{"$group":{
"_id":null,
"first":{"$first":"$$ROOT"},
"allnames":{"$push":{"name":"$name"}}
}},
{"$project":{
"data":{"$concatArrays":[["$first"],{"$slice":["$allnames",1,{"$size":"$allnames"}] } ]}
}},
{"$unwind":"$data"},
{"$replaceRoot":{"newRoot":"$data"}}
])
Just for completeness reasons...
#Veeram's answer is probably the better choice (I have a feeling it should be faster and easier to understand) but you can achieve the same result using a slightly simpler $group stage followed by slightly more complex $project stage using $reduce:
collection.aggregate([{
$sort: {
"height": 1
}
}, {
$group: {
"_id":null,
"allnames": {
$push: "$$ROOT"
}
}
}, {
$project: {
"data": {
$reduce: {
input: "$allnames",
initialValue: null,
in: {
$cond: [{
$eq: [ "$$value", null ] // if it's the first time we come here
},
[ "$$this" ], // we include the entire document
{
$concatArrays: [ // else we concat
"$$value", // the already concatenated values
[ { "name": "$$this.name" } ] // with the "name" of the currently looked at document
]
}]
}
}
}
}
}, {
$unwind: "$data"
}, {
$replaceRoot: {
"newRoot": "$data"
}
}])
Alternatively - as pointed out by #Veeram in the comment below - , it's possible to write the $reduce in this way:
$project: {
"data": {
$reduce: {
input: { "$slice": [ "$allnames", 1, { $size: "$allnames" } ] }, // process everything in the "allnames" array except for the first item
initialValue: { "$slice": [ "$allnames", 1 ] }, // start with the first item
in: { $concatArrays: [ "$$value", [ { "name": "$$this.name" } ] ]} // and keep appending the "name" field of all other items only
}
}
}

Get Distinct list of two properties using MongoDB 2.4

I have an article collection:
{
_id: 9999,
authorId: 12345,
coAuthors: [23456,34567],
title: 'My Article'
},
{
_id: 10000,
authorId: 78910,
title: 'My Second Article'
}
I'm trying to figure out how to get a list of distinct author and co-author ids out of the database. I have tried push, concat, and addToSet, but can't seem to find the right combination. I'm on 2.4.6 so I don't have access to setUnion.
Whilst $setUnion would be the "ideal" way to do this, there is another way that basically involved "switching" between a "type" to alternate which field is picked:
db.collection.aggregate([
{ "$project": {
"authorId": 1,
"coAuthors": { "$ifNull": [ "$coAuthors", [null] ] },
"type": { "$const": [ true,false ] }
}},
{ "$unwind": "$coAuthors" },
{ "$unwind": "$type" },
{ "$group": {
"_id": {
"$cond": [
"$type",
"$authorId",
"$coAuthors"
]
}
}},
{ "$match": { "_id": { "$ne": null } } }
])
And that is it. You may know the $const operation as the $literal operator from MongoDB 2.6. It has always been there, but was only documented and given an "alias" at the 2.6 release.
Of course the $unwind operations in both cases produce more "copies" of the data, but this is grouping for "distinct" values so it does not matter. Just depending on the true/false alternating value for the projected "type" field ( once unwound ) you just pick the field alternately.
Also this little mapReduce does much the same thing:
db.collection.mapReduce(
function() {
emit(this.authorId,null);
if ( this.hasOwnProperty("coAuthors"))
this.coAuthors.forEach(function(id) {
emit(id,null);
});
},
function(key,values) {
return null;
},
{ "out": { "inline": 1 } }
)
For the record, $setUnion is of course a lot cleaner and more performant:
db.collection.aggregate([
{ "$project": {
"combined": {
"$setUnion": [
{ "$map": {
"input": ["A"],
"as": "el",
"in": "$authorId"
}},
{ "$ifNull": [ "$coAuthors", [] ] }
]
}
}},
{ "$unwind": "$combined" },
{ "$group": {
"_id": "$combined"
}}
])
So there the only real concerns are converting the singular "authorId" to an array via $map and feeding an empty array where the "coAuthors" field is not present in the document.
Both output the same distinct values from the sample documents:
{ "_id" : 78910 }
{ "_id" : 23456 }
{ "_id" : 34567 }
{ "_id" : 12345 }