Are there computed fields in MongoDB? [duplicate] - mongodb

This question already has answers here:
MongoDB - The argument to $size must be an Array, but was of type: EOO / missing
(3 answers)
Closed 5 years ago.
Are there computed fields in MongoDB?
In SQL I can do:
SELECT A+B AS C FROM MYTABLE WHERE C>10
Can I do something similar in MongoDB?
UPDATE
I did with projection:
db.segments.aggregate(
[
{
$project: {
"_id": 1,
numberOfRestrictions: { $size: "$Speed Restrictions" }
}
}
]
)
and it works.
Unfortunately, further pipelining does not:
db.segments.aggregate(
[
{
$project: {
"_id": 1,
numberOfRestrictions: { $size: "$Speed Restrictions" }
}
},
{
$match: {
"numberOfRestrictions": {
"$gt": 1
}
}
}
]
)
Latter causes error
The argument to $size must be an Array, but was of type: EOO

Yes. It is called aggregation pipelines. Specifically, you need to use a $project stage to create the C field, and then use a $match stage to find all documents which match the criterion.
Example
Let's create some documents first:
for( var i = 1; i <=10; i++){
db.agg.insert({a:i,b:i})
}
Which results in a collection looking like this:
> db.agg.find()
{ "_id" : ObjectId("56c1b5561a3b578f37a99d4d"), "a" : 1, "b" : 1 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d4e"), "a" : 2, "b" : 2 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d4f"), "a" : 3, "b" : 3 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d50"), "a" : 4, "b" : 4 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d51"), "a" : 5, "b" : 5 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d52"), "a" : 6, "b" : 6 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d53"), "a" : 7, "b" : 7 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d54"), "a" : 8, "b" : 8 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d55"), "a" : 9, "b" : 9 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d56"), "a" : 10, "b" : 10 }
Finding all documents for which C > 10
db.agg.aggregate([
// You need to include all fields you want to have
// in the resulting document within the $project stage
{ "$project":{ a:1, b:1, c:{ "$add": ["$a","$b"] }}},
{ "$match":{ c:{ "$gt":10 }}}
])
Returns the following result:
{ "_id" : ObjectId("56c1b5561a3b578f37a99d52"), "a" : 6, "b" : 6, "c" : 12 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d53"), "a" : 7, "b" : 7, "c" : 14 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d54"), "a" : 8, "b" : 8, "c" : 16 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d55"), "a" : 9, "b" : 9, "c" : 18 }
{ "_id" : ObjectId("56c1b5561a3b578f37a99d56"), "a" : 10, "b" : 10, "c" : 20 }

There is an operator called $expr that enables you to use aggregation framework operator inside the find() query.
For instance, the SQL query
SELECT A+B AS C FROM MYTABLE WHERE C>10
can be translated to a mongo query as
db.segments.find({
"$expr": {
"$gt": [
{ "$add": [ "$A", "$B" ] },
10
]
}
})
And for checking an array length it's similar
db.segments.find({
"$expr": {
"$gt": [
{ "$size": "$SpeedRestrictions" },
10
]
}
})
With the aggregation framework it's also possible to use $expr within a $match pipeline step:
db.segments.aggregate([
{ "$match": {
"$expr": {
{ "$gt": [
{ "$size": "$SpeedRestrictions" },
10
] }
}
} }
])
And if the $expr operator is not available, for backwards compatibility one can use $redact as
db.segments.aggregate([
{ "$redact": {
"$cond": [
{ "$gt": [
{ "$size": "$SpeedRestrictions" },
10
] },
"$$KEEP",
"$$PRUNE"
]
} }
])
The other approach is to use the $addFields pipeline operator for creating the computed fields and the $match operator for filtering documents based on that computed field:
db.collection.aggregate([
{ "$addFields": { "C": { "$add": [ "$A", "$B" ] } } },
{ "$match": { "C": { "$gt": 10 } } }
])

Related

MongoDB aggregation without duplication

I have the following records:
{ "_id" : 1, "c" : 120, "b" : [ { "f1" : 10 }, { "f1" : 10 } ] }
{ "_id" :2, "c" : 5, "b" : [ { "f1" : 10 }, { "f1" : 10 } ] }
I need the output this way:
{ "_id" : 1, 'total':140}
{ "_id" :2, 'total':25 }
where total = sum of value in 'c' with sum of values in f1 for same record.
When i unwind the field 'b' it creates two documents with same id and hence data is duplicated and when i sum it up, i get:
db.test2.aggregate([
{'$unwind':'$b'},
{'$project':{'total':{'$add':['$c','$b.f1']}}},
{'$group':{'_id':'$_id', 'total':{'$sum':'$total'}}}
])
outputs:
{ "_id" : 1, 'total':260}
{ "_id" :2, 'total':30 }
(not what i wanted, as it has added 120 and 5 again to total due to duplication during unwinding)
So i tried:
db.test2.aggregate([
{'$unwind':'$b'},
{'$group':{'_id':'$_id', 'c':{'$push': '$c'},'f1':{'$sum':'$b.f1'}}},
{'$project':{'total':{'$add':[{'$arrayElemAt':['$c',0]},'$f1']}}}
])
outputs:
{ "_id" : 1, 'total':140}
{ "_id" :2, 'total':25 }
( what i wanted)
Is there any other way to achieve this?
You can try below query. Sum operator to first calculate sum in array followed by add to calculate total with other field.
db.test2.aggregate([{
$project: {
total: {"$add":["$c", {"$sum":"$b.f1"}]}
}
}]
An alternative:
db.test2.aggregate([{
$project: {
_id: 0,
c: "$c",
b: {
$reduce: {
input: "$b.f1",
initialValue: 0,
in: {
$add: ["$$value", "$$this"]
}
}
}
}
},
{
$project: {
_id: 0,
total: {
$sum: ["$c", "$b"]
}
}
}
])
That would create result:
{
"total" : 140
}
{
"total" : 25
}
If you need the field _id then replace the _id: 0 in both $project to _id: 1
That would create this result:
{
"_id" : 1,
"total" : 140
}
{
"_id" : 2,
"total" : 25
}

Find By Array Elements Comparison

Let's say I have a collection called test:
> db.test.find()
{ "_id" : ObjectId("5887a9202d599801b7df2c3c"), "name" : "soccer66", "ratings" : [ 5, 3.2, 1 ] }
{ "_id" : ObjectId("5887a9232d599801b7df2c3d"), "name" : "user1", "ratings" : [ 2, 3.2, 1 ] }
{ "_id" : ObjectId("5887a9262d599801b7df2c3e"), "name" : "user2", "ratings" : [ 2.5, 4.9, 1 ] }
What's the optimal way to select the documents where the first rating is greater than the second? I've tried lots of queries. For the sake of this post I ran the following again:
> db.test.aggregate([
{$project:{"isGreater":{$cond:[{$gt:["$ratings.0","$ratings.1"]},true,false]}}},
{$match:{"isGreater": true}}]);
>
> db.test.find({$where: "this.ratings.0" < "this.ratings.1"});
Error: error: {
"ok" : 0,
"errmsg" : "$where got bad type",
"code" : 2,
"codeName" : "BadValue"
}
db.test.find({"ratings.0": {"$gt": "ratings.1"}});
Proof I'm using the correct array index with ratings.#:
> db.test.find({"ratings.0":5});
{ "_id" : ObjectId("5887a9202d599801b7df2c3c"), "name" : "soccer66", "ratings" : [ 5, 3.2, 1 ] }
>
In your $where statement whole "where" predicate should be in quotes:
db.test.find({$where: "this.ratings[0] < this.ratings[1]"});
Do not use the $where operator.
You should use the Aggregation Framework for this.
db.test.aggregate([
{ "$match": { "ratings.1": { "$exists": true } } },
{ "$redact": {
"$cond": [
{ "$gt": [
{ "$arrayElemAt": [ "$ratings", 0 ] },
{ "$arrayElemAt": [ "$ratings", 1 ] }
]},
"$$KEEP",
"$$PRUNE"
]
}}
])

calculate the difference betwen Two values on the same document

I am new to Mongo aggregation.I want to calculate the difference betwen Two values (The last collection for each day -The first collection for each day).the data base record data every 5 mn for many ressource name.The structucture of the document is :
{
_id : ObjectId("5820511a95d447ed648b45d6"),
DeviceName : "OLT01FTV",
ResourceName : "CM MAC:00-07-11-11-39-20",
CollectionTime : ISODate("2016-11-07T09:30:00.000+01:00"),
GranularityPeriod : 5,
A : 0,
B: 17,
C: 4,
D: 21,
E: 3,
F: 0
}
A,B...F are the differrent counters.
Below, the illustration of that I'm trying to have :
result
([
{ "$match": {
"CollectionTime": {
$gte: ISODate("2016-09-05T00:00:00.000Z"),
$lt: ISODate("2016-10-07T00:00:00.000Z")
}
}},
{ "$unwind": "$u2000" },
{ "$group": {
"_id": null,
"firstUC": { "$first": "$UC" },
"lastUC": { "$last": "$UC" },
"firstSM-MISS": { "$first": "$SM-MISS" },
"lastSM-MISS": { "$last": "$SM-MISS" }
}},
{ "$project": {
"diff": {
"$divide": [
{ "$subtract": [ "$firstUC", "$lastUC" ] },
{ "$subtract": [ "$firstSM-MISS", "$lastSM-MISS" ] }
]
}
}}
])
This will get you the difference between the 'A' values for your above scenario. You can add the other fields if you want to get the difference for them also.
db.collection.aggregate([
{ "$match": {
"CollectionTime": {
$gte: ISODate("2016-11-01T00:00:00.000Z"),
$lt: ISODate("2016-11-30T00:00:00.000Z")
}
}},
{ "$sort": { "CollectionTime": 1 } },
{ "$group": {
"_id": null,
"firstA": { "$first": "$A" },
"lastA": { "$last": "$A" }
}},
{ "$project": {
_id: 0,
diffA: {
$subtract: [ "$lastA", "$firstA"]
}
}}
])
* EDIT *
So I'm using the following sample documents I created with the following to match your schema:
// Create 3 Documents 1 second apart
for (var i = 1; i < 4; i++) {
db.foo.insert({
DeviceName : "OLT01FTV",
ResourceName : "CM MAC:00-07-11-11-39-20",
CollectionTime : new Date(),
GranularityPeriod : 5,
A : 1*i,
B: 2*i,
C: 3*i,
D: 4*i,
E: 5*i,
F: 6*i
})
sleep(1000); // To add a delay between insertions so we can visibly see the date difference
}
This results in the following 3 documents being created:
> db.foo.find().pretty()
{
"_id" : ObjectId("582b1a6ced19a7334a5dee31"),
"DeviceName" : "OLT01FTV",
"ResourceName" : "CM MAC:00-07-11-11-39-20",
"CollectionTime" : ISODate("2016-11-15T14:23:40.934Z"),
"GranularityPeriod" : 5,
"A" : 1,
"B" : 2,
"C" : 3,
"D" : 4,
"E" : 5,
"F" : 6
}
{
"_id" : ObjectId("582b1a6ded19a7334a5dee32"),
"DeviceName" : "OLT01FTV",
"ResourceName" : "CM MAC:00-07-11-11-39-20",
"CollectionTime" : ISODate("2016-11-15T14:23:41.936Z"),
"GranularityPeriod" : 5,
"A" : 2,
"B" : 4,
"C" : 6,
"D" : 8,
"E" : 10,
"F" : 12
}
{
"_id" : ObjectId("582b1a6eed19a7334a5dee33"),
"DeviceName" : "OLT01FTV",
"ResourceName" : "CM MAC:00-07-11-11-39-20",
"CollectionTime" : ISODate("2016-11-15T14:23:42.939Z"),
"GranularityPeriod" : 5,
"A" : 3,
"B" : 6,
"C" : 9,
"D" : 12,
"E" : 15,
"F" : 18
}
The first step of the aggregation pipeline will match on all documents between the date range - which I set to beginning of November... so no worry there, then the sorting will sort by collection time:
After the grouping we have one document with the firstA and lastA value:
{ "_id" : null, "firstA" : 1, "lastA" : 3 }
And finally - perform the subtract in the projection and hide the ID field:
{ "diffA" : 2 }

How to group documents on index of array elements?

I'm looking for a way to take data such as this
{ "_id" : 5, "count" : 1, "arr" : [ "aga", "dd", "a" ] },
{ "_id" : 6, "count" : 4, "arr" : [ "aga", "ysdf" ] },
{ "_id" : 7, "count" : 4, "arr" : [ "sad", "aga" ] }
I would like to sum the count based on the 1st item(index) of arr. In another aggregation I would like to do the same with the 1st and the 2nd item in the arr array.
I've tried using unwind, but that breaks up the data and the hierarchy is then lost.
I've also tried using
$group: {
_id: {
arr_0:'$arr.0'
},
total:{
$sum: '$count'
}
}
but the result is blank arrays
Actually you can't use the dot notation to group your documents by element at a specified index. To two that you have two options:
First the optimal way using the $arrayElemAt operator new in MongoDB 3.2. which return the element at a specified index in the array.
db.collection.aggregate([
{ "$group": {
"_id": { "$arrayElemAt": [ "$arr", 0 ] },
"count": { "$sum": 1 }
}}
])
From MongoDB version 3.0 backward you will need to de-normalise your array then in the first time $group by _id and use the $first operator to return the first item in the array. From there you will need to regroup your document using that value and use the $sum to get the sum. But this will only work for the first and last index because MongoDB also provides the $last operator.
db.collection.aggregate([
{ "$unwind": "$arr" },
{ "$group": {
"_id": "$_id",
"arr": { "$first": "$arr" }
}},
{ "$group": {
"_id": "$arr",
"count": { "$sum": 1 }
}}
])
which yields something like this:
{ "_id" : "sad", "count" : 1 }
{ "_id" : "aga", "count" : 2 }
To group using element at position p in your array you will get a better chance using the mapReduce function.
var mapFunction = function(){ emit(this.arr[0], 1); };
var reduceFunction = function(key, value) { return Array.sum(value); };
db.collection.mapReduce(mapFunction, reduceFunction, { "out": { "inline": 1 } } )
Which returns:
{
"results" : [
{
"_id" : "aga",
"value" : 2
},
{
"_id" : "sad",
"value" : 1
}
],
"timeMillis" : 27,
"counts" : {
"input" : 3,
"emit" : 3,
"reduce" : 1,
"output" : 2
},
"ok" : 1
}

MongoDB Aggregate Variable to create Multiple Fields?

Given the following query, what is the best method to use $$priceToInflationRatio to help create multiple calculated fields? From what I have read on $let, it appears to only work for creating a single field -- I would like to use the variables across my entire $project section. Is that possible?
db.books.aggregate([
{$project: {
'priceInflationresult': {
$let: {
vars: {
'priceToInflationRatio': {
$multiply: [{'$divide': [{'$subtract': ['$price', 1]}, 5]}, 10]
}
},
in: {
'$cond': [
{'$gt': ['$price', 5]},
{'$mod': ['$$priceToInflationRatio', 1]},
1
]
},
}
}
}}
])
The in part of a $let expression is an object, so it can accept multiple keys, each of which can be an expression that is evaluated with the variables in scope:
> db.test.insert({ "_id" : 0, "a" : 1, "b" : 1 })
> db.test.aggregate([{
"$project" : {
"test" : {
"$let" : {
"vars" : {
"c" : 2,
"d" : 3
},
"in": {
"a" : { "$add" : ["$a", "$$c"] },
"b" : { "$add" : ["$b", "$$d"] }
}
}
}
}
}]);
{ "_id" : 0, "test" : { "a" : 3, "b" : 4 } }
Note that this will necessarily create subdocuments as top-level $let expressions are not allowed. You can change this with another $project stage.