I have a collection with documents which look like this:
{
"campaignType" : 1,
"allowAccessControl" : true,
"userId" : "108028399"
}
I'd like to query this collection using aggregation framework and have a result which looks like this:
{
"campaignType" : ["APPLICATION"],
"allowAccessControl" : "true",
"userId" : "108028399",
}
You will notice that:
campaignType field becomes and array
the numeric value was mapped to a string
Can that be done using aggregation framework?
I tried looking at $addToSet and $push but had no luck.
Please help.
Thanks
In either case here it is th $cond operator from the aggregation framework that is your friend. It is a "ternary" operator, which means it evaluates a condition for true|false and then returns the result based on that evaluation.
So for modern versions from MongoDB 2.6 and upwards you can $project with usage of the $map operator to construct the array:
db.campaign.aggregate([
{ "$project": {
"campaignType": {
"$map": {
"input": { "$literal": [1] },
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$campaignType", 1 ] },
"APPLICATION",
false
]
}
}
},
"allowAcessControl" : 1,
"userId": 1
}}
])
Or generally in most versions you can simply use the $push operator in a $group pipeline stage:
db.campaign.aggregate([
{ "$group": {
"_id": "$_id",
"campaignType": {
"$push": {
"$cond": [
{ "$eq": [ "$campaignType", 1 ] },
"APPLICATION",
false
]
}
},
"allowAccessControl": { "$first": "$allowAccessControl" },
"userId": { "first": "$userId" }
}}
])
But the general concept if that you use "nested" expressions with the $cond operator in order to "test" and return some value that matches your "mapping" condition and do that with another operator that allows you to produce an array.
Related
The query returns the order in which elements are placed in their collection, ignoring the order of the initial array. This affects the function of our system. Is there any extra command to put it in the correct order? Is there any workaround available?
Here follows a simple example:
Collection1 Document
{
"_id":ObjectId("5c781752176c512f180048e3"),
"Name":"Pedro",
"Classes":[
{"ID": ObjectId("5c7af2b2f6f6e47c9060d7ce") },
{"ID": ObjectId("5c7af2bcf6f6e47c9060d7cf") },
{"ID": ObjectId("5c7af2aaf6f6e47c9060d7cd") }
]
}
Collection2 Documents
{
"_id":ObjectId("5c7af2aaf6f6e47c9060d7cd"),
"variable1":"A"
},
{
"_id": ObjectId("5c7af2b2f6f6e47c9060d7ce"),
"variable1":"B"
},
{
"_id": ObjectId("5c7af2bcf6f6e47c9060d7cf"),
"variable1":"C"
}
The query:
aggregate(
pipeline = '[
{"$match": {"_id": {"$oid": "5c781752176c512f180048e3"}}},
{"$lookup": {"from": "collection2", "localField": "Classes.ID", "foreignField": "_id", "as": "Collection2_doc"}}
]'
)
Returns:
Result's order:
[
{
"_id":ObjectId("5c7af2aaf6f6e47c9060d7cd"),
"variable1":"A"
},
{
"_id": ObjectId("5c7af2b2f6f6e47c9060d7ce"),
"variable1":"B"
},
{
"_id": ObjectId("5c7af2bcf6f6e47c9060d7cf"),
"variable1":"C"
}
]
Expected order (first document array order):
[
{
"_id": ObjectId("5c7af2b2f6f6e47c9060d7ce"),
"variable1":"B"
},
{
"_id": ObjectId("5c7af2bcf6f6e47c9060d7cf"),
"variable1":"C"
},
{
"_id":ObjectId("5c7af2aaf6f6e47c9060d7cd"),
"variable1":"A"
}
]
Are there any extra command ex. $sort that could be used to return it respecting the original arrays order?
This is "by design" of the $lookup implementation. What actually happens "under the hood" is MongoDB internall converts the arguments in the $lookup to the new expressive format using $expr and $in. Even in versions prior to when this expressive form was implemented, the internal mechanics for an "array of values" was really much the same.
The solution here is to maintain a copy of the original array as a reference for reordering the "joined" items:
collection.aggregate([
{"$match": {"_id": ObjectId("5c781752176c512f180048e3") }},
{"$lookup": {
"from": "collection2",
"let": { "classIds": "$Classes.ID" },
"pipeline": [
{ "$match": {
"$expr": { "$in": [ "$_id", "$$classIds" ] }
}},
{ "$addFields": {
"sort": {
"$indexOfArray": [ "$$classIds", "$_id" ]
}
}},
{ "$sort": { "sort": 1 } },
{ "$addFields": { "sort": "$$REMOVE" }}
],
"as": "results"
}}
])
Or by the legacy $lookup usage:
collection.aggregate([
{"$match": {"_id": ObjectId("5c781752176c512f180048e3") }},
{"$lookup": {
"from": "collection2",
"localField": "Classes.ID",
"foreignField": "_id",
"as": "results"
}},
{ "$unwind": "$results" },
{ "$addFields": {
"sort": {
"$indexOfArray": [ "$Classes.ID", "$results._id" ]
}
}},
{ "$sort": { "_id": 1, "sort": 1 } },
{ "$group": {
"_id": "$_id",
"Name": { "$first": "$Name" },
"Classes": { "$first": "$Classes" },
"results": { "$push": "$results" }
}}
])
Both variants produce the same output:
{
"_id" : ObjectId("5c781752176c512f180048e3"),
"Name" : "Pedro",
"Classes" : [
{
"ID" : ObjectId("5c7af2b2f6f6e47c9060d7ce")
},
{
"ID" : ObjectId("5c7af2bcf6f6e47c9060d7cf")
},
{
"ID" : ObjectId("5c7af2aaf6f6e47c9060d7cd")
}
],
"results" : [
{
"_id" : ObjectId("5c7af2b2f6f6e47c9060d7ce"),
"variable1" : "B"
},
{
"_id" : ObjectId("5c7af2bcf6f6e47c9060d7cf"),
"variable1" : "C"
},
{
"_id" : ObjectId("5c7af2aaf6f6e47c9060d7cd"),
"variable1" : "A"
}
]
}
The general concept being to use $indexOfArray in comparison with the _id value from the "joined" content to find it's "index" position in the original source array from "$Classes.ID". The different $lookup syntax variants have different approaches to how you access this copy and how you basically reconstruct.
The $sort of course sets the order of actual documents, either being inside the pipeline processing for the expressive form, or via the exposed documents of $unwind. Where you used $unwind you would then $group back to the original document form.
NOTE: The usage examples here depend on MongoDB 3.4 for the $indexOfArray at least and the $$REMOVE aligns with MongoDB 3.6 as would the expressive $lookup.
There are other approaches to re-ordering the array for prior releases, but these are demonstrated in more detail on Does MongoDB's $in clause guarantee order. Realistically the bare minimum you should presently be running as a production MongoDB version is the 3.4 release.
See Support Policy under MongoDB Server for the full details of supported releases and end dates.
I have this Collection in mongodb
{
"_id" : "777",
"someKey" : "someValue",
"someArray" : [
{
"name" : "name1",
"someNestedArray" : [
{
"name" : "value"
},
{
"name" : "delete me"
}
]
}
]
}
I want to find document based on someArray.someNestedArray.name
but i can't find any useful link all search result about update nested array
i am trying this but return nothing
db.mycollection.find({"someArray.$.someNestedArray":{"$elemMatch":{"name":"1"}}})
db.mycollection.find({"someArray.$.someNestedArray.$.name":"1"})
and Some thing else
how can i find by element in double nested array mongodb?
In the simplest sense this just follows the basic form of "dot notation" as used by MongoDB. That will work regardless of which array member the inner array member is in, as long as it matches a value:
db.mycollection.find({
"someArray.someNestedArray.name": "value"
})
That is fine for a "single field" value, for matching multiple-fields you would use $elemMatch:
db.mycollection.find({
"someArray": {
"$elemMatch": {
"name": "name1",
"someNestedArray": {
"$elemMatch": {
"name": "value",
"otherField": 1
}
}
}
}
})
That matches the document which would contain something with a a field at that "path" matching the value. If you intended to "match and filter" the result so only the matched element was returned, this is not possible with the positional operator projection, as quoted:
Nested Arrays
The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single value
Modern MongoDB
We can do this by applying $filter and $map here. The $map is really needed because the "inner" array can change as a result of the "filtering", and the "outer" array of course does not match the conditions when the "inner" was stripped of all elements.
Again following the example of actually having multiple properties to match within each array:
db.mycollection.aggregate([
{ "$match": {
"someArray": {
"$elemMatch": {
"name": "name1",
"someNestedArray": {
"$elemMatch": {
"name": "value",
"otherField": 1
}
}
}
}
}},
{ "$addFields": {
"someArray": {
"$filter": {
"input": {
"$map": {
"input": "$someArray",
"as": "sa",
"in": {
"name": "$$sa.name",
"someNestedArray": {
"$filter": {
"input": "$$sa.someNestedArray",
"as": "sn",
"cond": {
"$and": [
{ "$eq": [ "$$sn.name", "value" ] },
{ "$eq": [ "$$sn.otherField", 1 ] }
]
}
}
}
}
},
},
"as": "sa",
"cond": {
"$and": [
{ "$eq": [ "$$sa.name", "name1" ] },
{ "$gt": [ { "$size": "$$sa.someNestedArray" }, 0 ] }
]
}
}
}
}}
])
Therefore on the "outer" array the $filter actually looks at the $size of the "inner" array after it was "filtered" itself, so you can reject those results when the whole inner array does in fact match noting.
Older MongoDB
In order to "project" only the matched element, you need the .aggregate() method:
db.mycollection.aggregate([
// Match possible documents
{ "$match": {
"someArray.someNestedArray.name": "value"
}},
// Unwind each array
{ "$unwind": "$someArray" },
{ "$unwind": "$someArray.someNestedArray" },
// Filter just the matching elements
{ "$match": {
"someArray.someNestedArray.name": "value"
}},
// Group to inner array
{ "$group": {
"_id": {
"_id": "$_id",
"name": "$someArray.name"
},
"someKey": { "$first": "$someKey" },
"someNestedArray": { "$push": "$someArray.someNestedArray" }
}},
// Group to outer array
{ "$group": {
"_id": "$_id._id",
"someKey": { "$first": "$someKey" },
"someArray": { "$push": {
"name": "$_id.name",
"someNestedArray": "$someNestedArray"
}}
}}
])
That allows you to "filter" the matches in nested arrays for one or more results within the document.
You can also try something like below:
db.collection.aggregate(
{ $unwind: '$someArray' },
{
$project: {
'filteredValue': {
$filter: {
input: "$someArray.someNestedArray",
as: "someObj",
cond: { $eq: [ '$$someObj.name', 'delete me' ] }
}
}
}
}
)
I am struggling with an aggregation in mongodb. I have the following type of documents:
{
"_id": "xxxx",
"workHome": true,
"commute": true,
"tel": false,
"weekend": true,
"age":39
},
{
"_id": "yyyy",
"workHome": false,
"commute": true,
"tel": false,
"weekend": true,
"age":32
},
{
"_id": "zzzz",
"workHome": false,
"commute": false,
"tel": false,
"weekend": false,
"age":27
}
Out of this I want to generate an aggregation by the total number of fields that are "true" in the document. There are a total of 4 boolean fields in the document so I want the query to group them together to generate the following output (as examples from e.g. a collection with 100 documents in total):
0:20
1:30
2:10
3:20
4:20
This means: There is 20 documents out of 100 with 'all false', 30 documents with '1x true', 10 documents with '2x true' etc. up to a total of 'all 4 are true'.
Is there any way to do this with an $aggregate statement? Right now I am trying to $group by the $sum of 'true' values but don't find a way to get the conditional query to work.
So assuming that the data is consistent with all the same fields as "workHome", "commute", "tel" and "weekend", then you would proceed with a "logical" evaluation such as this:
db.collection.aggregate([
{ "$project": {
"mapped": { "$map": {
"input": ["A","B","C","D"],
"as": "el",
"in": { "$cond": [
{ "$eq": [ "$$el", "A" ] },
"$workHome",
{ "$cond": [
{ "$eq": [ "$$el", "B" ] },
"$commute",
{ "$cond": [
{ "$eq": [ "$$el", "C" ] },
"$tel",
"$weekend"
]}
]}
]}
}}
}},
{ "$unwind": "$mapped" },
{ "$group": {
"_id": "$_id",
"size": { "$sum": { "$cond": [ "$mapped", 1, 0 ] } }
}},
{ "$group": {
"_id": "$size",
"count": { "$sum": 1 }
}},
{ "$sort": { "_id": 1 } }
])
From your simple sample this gives:
{ "_id" : 0, "count" : 1 }
{ "_id" : 2, "count" : 1 }
{ "_id" : 3, "count" : 1 }
To break this down, first the $map operator here transposes the values of the fields to an array of the same lenght as the fields themselves. This is done my comparing each element of the "input" to an expected value via $cond and either returning the true condtion where a match, or moving on to the next condition embedded in the false part of this "ternary" operator. This is done until all logical matches are met and results in an array of values from the fields like so, for the first document:
[true,true,false,true]
The next step is to $unwind the array elements for further comparison. This "de-normalizes" into separate documents for each array element, and is usually required in aggregation pipelines when processing arrays.
Once that is done a $group pipeline stage is invoked, in order to assess the "total" of those elements with a true value. The same $cond ternary is used to transform the logical true/falsecondtions into numeric values here and fed to the $sum accumulator for addition.
Since the "grouping key" provided in _id in the $group is the original document _id value, the current totals are per document for those fields that are true. In order to get totals on the "counts" over the whole collection ( or selection ) then the futher $group stage is invoked with the grouping key being the returned "size" of the matched true results from each document.
The $sum accumulator used there simply adds 1 for each match on the grouping key, thus "counting" the number of occurances of each match count.
Finally $sort by the number of matches "key" in to produce some order to the results.
For the record, this is so much nicer with the upcoming release of MongoDB ( as of writing ) which includes the $filter operator:
db.collection.aggregate([
{ "$group": {
"_id": {
"$size": {
"$filter": {
"input": { "$map": {
"input": ["A","B","C","D"],
"as": "el",
"in": { "$cond": [
{ "$eq": [ "$$el", "A" ] },
"$workHome",
{ "$cond": [
{ "$eq": [ "$$el", "B" ] },
"$commute",
{ "$cond": [
{ "$eq": [ "$$el", "C" ] },
"$tel",
"$weekend"
]}
]}
]}
}},
"as": "el",
"cond": {
"$eq": [ "$$el", true ]
}
}
}
},
"count": { "$sum": 1 }
}},
{ "$sort": { "_id": 1 } }
])
So now just "two" pipeline stages doing the same thing as the original statement that will work from MongoDB 2.6 and above.
Therefore if your own application is in "development" itself, or you are otherwise curious, then take a look at the Development Branch releases where this functionality is available now.
I have a document like this:
{fax: '8135551234', cellphone: '8134441234'}
Is there a way to project (without a group stage) this document into this:
{
phones: [{
type: 'fax',
number: '8135551234'
}, {
type: 'cellphone',
number: '8134441234'
}]
}
I could probably use a group stage operator for this, but I'd rather not if there's any other way, because my query also projects several other fields, all of which would require a $first just for the group stage.
Hope that's clear. Thanks in advance!
MongoDB 2.6 Introduces the the $map operator which is an array transformation operator which can be used to do exactly this:
db.phones.aggregate([
{ "$project": {
"phones": { "$map": {
"input": { "$literal": ["fax","cellphone"] },
"as": "el",
"in": {
"type": "$$el",
"number": { "$cond": [
{ "$eq": [ "$$el", "fax" ] },
"$fax",
"$cellphone"
]}
}
}}
}}
])
So your document now looks exactly like you want. The trick of course to to create a new array with members "fax" and "cellphone", then transform that array with the new document fields by matching those values.
Of course you can also do this in earlier versions using $unwind and $group in a similar fashion, but just not as efficiently:
db.phones.aggregate([
{ "$project": {
"type": { "$const": ["fax","cellphone"] },
"fax": 1,
"cellphone": 1
}},
{ "$unwind": "$type" },
{ "$group": {
"_id": "_id",
"phones": { "$push": {
"type": "$type",
"number": { "$cond": [
{ "$eq": [ "$type", "fax" ] },
"$fax",
"$cellphone"
]}
}}
}}
])
Of course it can be argued that unless you are doing some sort of aggregation then you may as well just post process the collection results in code. But this is an alternate way to do that.
I have a MongoDB collection indicators/
It returns statistical data such as:
/indicators/population
{
id: "population"
data : [
{
country : "A",
value : 100
},
{
country : "B",
value : 150
}
]
}
I would like to be able to limit the response to specific countries.
MongoDB doesn't seem to support this, so should I:
Restructure the MongoDB collection setup to allow this via native find()
Extend my API so that it allows filtering of the data array before returning to client
Other?
This is actually a very simple operation that just involves "projection" using the positional $ operator in order to match a given condition. In the case of a "singular" match that is:
db.collection.find(
{ "data.country": "A" },
{ "data.$": 1 }
)
And that will match the first element in the array which matches the condition as given in the query.
For more than one match, you need to invoke the aggregation framework for MongoDB:
db.collection.agggregate([
// Match documents that are possible first
{ "$match": {
"data.country": "A"
}},
// Unwind the array to "de-normalize" the documents
{ "$unwind": "$data" },
// Actually filter the now "expanded" array items
{ "$match": {
"data.country": "A"
}},
// Group back together
{ "$group": {
"_id": "$_id",
"data": { "$push": "$data" }
}}
])
Or with MongoDB 2.6 or greater, a little bit cleaner, or at least without the $unwind:
db.collection.aggregate({
// Match documents that are possible first
{ "$match": {
"data.country": "A"
}},
// Filter out the array in place
{ "$project": {
"data": {
"$setDifference": [
{
"$map": {
"input": "$data",
"as": "el",
"in": {
"$cond": [
{ "$eq": [ "$$el.country", "A" },
"$$el",
false
]
}
}
},
[false]
]
}
}}
])
If my understanding of the problem is ok, then you can use :
db.population.find({"population.data.country": {$in : ["A", "C"]}});