I'm evaluating how well the MongoDB aggregation framework suits our needs as we are currently running on top of SQL Server. I'm having a hard time performing a specific query:
Say I have the following pseudo records (modeled as columns in a sql table and as a full document in a mongodb collection)
{
name: 'A',
timespent: 100,
},
{
name: 'B',
timespent: 200,
},
{
name: 'C',
timespent: 300,
},
{
name: 'D',
timespent: 400,
},
{
name: 'E',
timespent: 500,
}
I want to group the timespent field in to ranges and count the occurrences so I will get e.g. the following pseudo-records:
results{
0-250: 2,
250-450: 2,
450-650: 1
}
Note that these ranges (250, 450 and 650) are dynamic and will likely be altered over time by the user. In SQL we extracted the results with something like this:
select range, COUNT(*) as total from (
select case when Timespent <= 250 then '0-250'
when Timespent <= 450 then '200-450'
else '450-600' end as range
from TestTable) as r
group by r.range
Again, note that this sql is constructed dynamically by our app to fit the specific ranges available at any one time.
I'm struggling to find the appropriate constructs in the mongodb aggregation framework to perform such queries. I can query for the results of a single range by inserting a $match into the pipeline(i.e. getting the result of a single range) but I cannot grok how to extract all the ranges and their counts in a single pipeline query.
what corresponds to the "case" SQL statement in the aggregation framework, is the $cond operator (see manual). $cond statements can be nested to simulate "when-then" and "else", but I have chosen another approach, because it is easier to read (and to generate, see below): I'll use the $concat operator to write the range string, which then serves as grouping key.
So for the given collection:
db.xx.find()
{ "_id" : ObjectId("514919fb23700b41723f94dc"), "name" : "A", "timespent" : 100 }
{ "_id" : ObjectId("514919fb23700b41723f94dd"), "name" : "B", "timespent" : 200 }
{ "_id" : ObjectId("514919fb23700b41723f94de"), "name" : "C", "timespent" : 300 }
{ "_id" : ObjectId("514919fb23700b41723f94df"), "name" : "D", "timespent" : 400 }
{ "_id" : ObjectId("514919fb23700b41723f94e0"), "name" : "E", "timespent" : 500 }
the aggregate (hardcoded) looks like this:
db.xx.aggregate([
{ $project: {
"_id": 0,
"range": {
$concat: [{
$cond: [ { $lte: ["$timespent", 250] }, "range 0-250", "" ]
}, {
$cond: [ { $and: [
{ $gte: ["$timespent", 251] },
{ $lt: ["$timespent", 450] }
] }, "range 251-450", "" ]
}, {
$cond: [ { $and: [
{ $gte: ["$timespent", 451] },
{ $lt: ["$timespent", 650] }
] }, "range 450-650", "" ]
}]
}
}},
{ $group: { _id: "$range", count: { $sum: 1 } } },
{ $sort: { "_id": 1 } },
]);
and the result is:
{
"result" : [
{
"_id" : "range 0-250",
"count" : 2
},
{
"_id" : "range 251-450",
"count" : 2
},
{
"_id" : "range 450-650",
"count" : 1
}
],
"ok" : 1
}
In order to generate the aggregate command, you have to build the "range" projection as a JSON object ( or you could generate a string and then use JSON.parse(string) )
The generator looks like this:
var ranges = [ 0, 250, 450, 650 ];
var rangeProj = {
"$concat": []
};
for (i = 1; i < ranges.length; i++) {
rangeProj.$concat.push({
$cond: {
if: {
$and: [{
$gte: [ "$timespent", ranges[i-1] ]
}, {
$lt: [ "$timespent", ranges[i] ]
}]
},
then: "range " + ranges[i-1] + "-" + ranges[i],
else: ""
}
})
}
db.xx.aggregate([{
$project: { "_id": 0, "range": rangeProj }
}, {
$group: { _id: "$range", count: { $sum: 1 } }
}, {
$sort: { "_id": 1 }
}]);
which will return the same result as above.
Starting from MongoDB 3.4 we can use the $switch operator to perform a multi-switch statement in the $project stage.
The $group pipeline operator group the documents by "range" and return the "count" for each group using the $sum accumulator operator.
db.collection.aggregate(
[
{ "$project": {
"range": {
"$switch": {
"branches": [
{
"case": { "$lte": [ "$timespent", 250 ] },
"then": "0-250"
},
{
"case": {
"$and": [
{ "$gt": [ "$timespent", 250 ] },
{ "$lte": [ "$timespent", 450 ] }
]
},
"then": "251-450"
},
{
"case": {
"$and": [
{ "$gt": [ "$timespent", 450 ] },
{ "$lte": [ "$timespent", 650 ] }
]
},
"then": "451-650"
}
],
"default": "650+"
}
}
}},
{ "$group": {
"_id": "$range",
"count": { "$sum": 1 }
}}
]
)
With the following documents in our collection,
{ "_id" : ObjectId("514919fb23700b41723f94dc"), "name" : "A", "timespent" : 100 },
{ "_id" : ObjectId("514919fb23700b41723f94dd"), "name" : "B", "timespent" : 200 },
{ "_id" : ObjectId("514919fb23700b41723f94de"), "name" : "C", "timespent" : 300 },
{ "_id" : ObjectId("514919fb23700b41723f94df"), "name" : "D", "timespent" : 400 },
{ "_id" : ObjectId("514919fb23700b41723f94e0"), "name" : "E", "timespent" : 500 }
our query yields:
{ "_id" : "451-650", "count" : 1 }
{ "_id" : "251-450", "count" : 2 }
{ "_id" : "0-250", "count" : 2 }
We may want to add a $sort stage to the pipeline sort our document by range but this will only sort the documents in lexicographic order because of the type of "range".
Related
I am trying to create an aggregation MongoDB query.
Structure of data:
{
"object_name": Example,
"values": [ {"name":"value1", "value":1},
{"name":"value2", "value":10},
{"name":"total", "value":105}
}
Goal: Find object names where value1/total > 0.5 and value2/total > 0.25 and total > 100.
The data is structured in this way to provide indexes on the value_name and value fields.
What I tried - aggregate with the following pipelines:
$match: filter documents with total > 100:
$match: { values: { $elemMatch: { value_name: "total", value: {$gte: 100 }
$project: grab only the value_names that we need (there are close to 200 different names)
$project: {
values: {
$filter: {
input: "$values",
as: "value",
cond: { $or: [
{ $eq: [ "$$value.name", "name1"] },
{ $eq: [ "$$value.name", "name2"] },
{ $eq: [ "$$value.name", "total"] },
] }
}
},
name: 1
}
then, { $unwind: "$values" }
And here, I could $group to $divide: name1/total, name2/total however I'm stuck on how to get those values.
I can't simply do stats.value: because it does not know which value I'm referring to. I believe $group can't do $elemMatch to also match the name.
If there are simpler solutions that this, I'd greatly appreciate your input.
Please try this :
We're filtering documents where values array has an object with
name : total & value > 100.
Adding object with name : total
to document.
Leaving only objects that match with criteria
value1/total > 0.5 and value2/total > 0.25 in values array.
If
size of that array is greater than 1, then those two conditions are
met.
Finally projecting only object_name
Query :
db.yourCollectionName.aggregate([{ $match: { values: { $elemMatch: { name: "total", value: { $gte: 100 } } } } },
{
$addFields: {
totalValue: {
$arrayElemAt: [{
$filter: {
input: "$values",
as: "item",
cond: { $eq: ["$$item.name", 'total'] }
}
}, 0]
}
}
},
{
$project: {
values: {
$filter: {
input: "$values",
as: "value",
cond: {
$or: [
{ $cond: [{ $eq: ["$$value.name", "value1"] }, { $gt: [{ $divide: ["$$value.value", '$totalValue.value'] }, 0.5] }, false] },
{ $cond: [{ $eq: ["$$value.name", "value2"] }, { $gt: [{ $divide: ["$$value.value", '$totalValue.value'] }, 0.25] }, false] }
]
}
}
}, object_name: 1
}
}, {
$match: {
$expr: { $gt: [{ $size: "$values" }, 1] }
}
}, { $project: { object_name: 1, _id: 0 } }])
Collection Data :
/* 1 */
{
"_id" : ObjectId("5e20bd94d02e05b694d55fa5"),
"object_name" : "Example",
"values" : [
{
"name" : "value1",
"value" : 1
},
{
"name" : "value2",
"value" : 10
},
{
"name" : "total",
"value" : 105
},
{
"name" : "total1",
"value" : 105
}
]
}
/* 2 */
{
"_id" : ObjectId("5e20bdb1d02e05b694d56490"),
"object_name" : "Example2",
"values" : [
{
"name" : "value1",
"value" : 1
},
{
"name" : "value2",
"value" : 10
},
{
"name" : "total",
"value" : 5
},
{
"name" : "total1",
"value" : 5
}
]
}
/* 3 */
{
"_id" : ObjectId("5e20d1b7d02e05b694d7c57a"),
"object_name" : "Example3",
"values" : [
{
"name" : "value1",
"value" : 100
},
{
"name" : "value2",
"value" : 100
},
{
"name" : "total",
"value" : 200
},
{
"name" : "total1",
"value" : 205
}
]
}
/* 4 */
{
"_id" : ObjectId("5e20d1cad02e05b694d7c71c"),
"object_name" : "Example4",
"values" : [
{
"name" : "value1",
"value" : 200
},
{
"name" : "value2",
"value" : 40
},
{
"name" : "total",
"value" : 200
},
{
"name" : "total1",
"value" : 205
}
]
}
/* 5 */
{
"_id" : ObjectId("5e20d1e2d02e05b694d7c933"),
"object_name" : "Example5",
"values" : [
{
"name" : "value1",
"value" : 150
},
{
"name" : "value2",
"value" : 100
},
{
"name" : "total",
"value" : 200
},
{
"name" : "total1",
"value" : 205
}
]
}
Result :
/* 1 */
{
"object_name" : "Example5"
}
You may convert your array into object with $arrayToObject operator and add tmp field to have easy access to value1, value2, total values
db.collection.aggregate([
{
$addFields: {
tmp: {
$arrayToObject: {
$map: {
input: "$values",
as: "value",
in: {
k: "$$value.name",
v: "$$value.value"
}
}
}
},
name: 1
}
},
{
$match: {
$expr: {
$and: [
{
$gt: [
{
$divide: [
"$tmp.value1",
"$tmp.total"
]
},
0.5
]
},
{
$gt: [
{
$divide: [
"$tmp.value2",
"$tmp.total"
]
},
0.25
]
},
{
$gt: [
"$tmp.total",
100
]
}
]
}
}
},
{
$project: {
tmp: 0
}
}
])
MongoPlayground
If I have data in the following format:
[
{
_id: 1,
startDate: ISODate("2017-01-1T00:00:00.000Z"),
endDate: ISODate("2017-02-25T00:00:00.000Z"),
type: 'CAR'
},
{
_id: 2,
startDate: ISODate("2017-02-17T00:00:00.000Z"),
endDate: ISODate("2017-03-22T00:00:00.000Z"),
type: 'HGV'
}
]
Is it possible to retrieve data grouped by 'type', but also with a count of the type for each of month in a given date range e.g. between 2017/1/1 to 2017/4/1 would return:
[
{
_id: 'CAR',
monthCounts: [
/*January*/
{
from: ISODate("2017-01-1T00:00:00.000Z"),
to: ISODate("2017-01-31T23:59:59.999Z"),
count: 1
},
/*February*/
{
from: ISODate("2017-02-1T00:00:00.000Z"),
to: ISODate("2017-02-28T23:59:59.999Z"),
count: 1
},
/*March*/
{
from: ISODate("2017-03-1T00:00:00.000Z"),
to: ISODate("2017-03-31T23:59:59.999Z"),
count: 0
},
]
},
{
_id: 'HGV',
monthCounts: [
{
from: ISODate("2017-01-1T00:00:00.000Z"),
to: ISODate("2017-01-31T23:59:59.999Z"),
count: 0
},
{
from: ISODate("2017-02-1T00:00:00.000Z"),
to: ISODate("2017-02-28T23:59:59.999Z"),
count: 1
},
{
from: ISODate("2017-03-1T00:00:00.000Z"),
to: ISODate("2017-03-31T23:59:59.999Z"),
count: 1
},
]
}
]
The returned format is not really important, but what I am trying to achieve is in a single query to retrieve a number of counts for the same grouping (one per month). The input could be simply a start and end date to report from or more likely it could be an array of the date ranges to group by.
The algorithm for this is to basically "iterate" values between the interval of the two values. MongoDB has a couple of ways to deal with this, being what has always been present with mapReduce() and with new features available to the aggregate() method.
I'm going expand on your selection to deliberately show an overlapping month since your examples did not have one. This will result in the "HGV" values appearing in "three" months of output.
{
"_id" : 1,
"startDate" : ISODate("2017-01-01T00:00:00Z"),
"endDate" : ISODate("2017-02-25T00:00:00Z"),
"type" : "CAR"
}
{
"_id" : 2,
"startDate" : ISODate("2017-02-17T00:00:00Z"),
"endDate" : ISODate("2017-03-22T00:00:00Z"),
"type" : "HGV"
}
{
"_id" : 3,
"startDate" : ISODate("2017-02-17T00:00:00Z"),
"endDate" : ISODate("2017-04-22T00:00:00Z"),
"type" : "HGV"
}
Aggregate - Requires MongoDB 3.4
db.cars.aggregate([
{ "$addFields": {
"range": {
"$reduce": {
"input": { "$map": {
"input": { "$range": [
{ "$trunc": {
"$divide": [
{ "$subtract": [ "$startDate", new Date(0) ] },
1000
]
}},
{ "$trunc": {
"$divide": [
{ "$subtract": [ "$endDate", new Date(0) ] },
1000
]
}},
60 * 60 * 24
]},
"as": "el",
"in": {
"$let": {
"vars": {
"date": {
"$add": [
{ "$multiply": [ "$$el", 1000 ] },
new Date(0)
]
},
"month": {
}
},
"in": {
"$add": [
{ "$multiply": [ { "$year": "$$date" }, 100 ] },
{ "$month": "$$date" }
]
}
}
}
}},
"initialValue": [],
"in": {
"$cond": {
"if": { "$in": [ "$$this", "$$value" ] },
"then": "$$value",
"else": { "$concatArrays": [ "$$value", ["$$this"] ] }
}
}
}
}
}},
{ "$unwind": "$range" },
{ "$group": {
"_id": {
"type": "$type",
"month": "$range"
},
"count": { "$sum": 1 }
}},
{ "$sort": { "_id": 1 } },
{ "$group": {
"_id": "$_id.type",
"monthCounts": {
"$push": { "month": "$_id.month", "count": "$count" }
}
}}
])
The key to making this work is the $range operator which takes values for a "start" and and "end" as well as an "interval" to apply. The result is an array of values taken from the "start" and incremented until the "end" is reached.
We use this with startDate and endDate to generate the possible dates in between those values. You will note that we need to do some math here since the $range only takes a 32-bit integer, but we can take the milliseconds away from the timestamp values so that is okay.
Because we want "months", the operations applied extract the month and year values from the generated range. We actually generate the range as the "days" in between since "months" are difficult to deal with in math. The subsequent $reduce operation takes only the "distinct months" from the date range.
The result therefore of the first aggregation pipeline stage is a new field in the document which is an "array" of all the distinct months covered between startDate and endDate. This gives an "iterator" for the rest of the operation.
By "iterator" I mean than when we apply $unwind we get a copy of the original document for every distinct month covered in the interval. This then allows the following two $group stages to first apply a grouping to the common key of "month" and "type" in order to "total" the counts via $sum, and next $group makes the key just the "type" and puts the results in an array via $push.
This gives the result on the above data:
{
"_id" : "HGV",
"monthCounts" : [
{
"month" : 201702,
"count" : 2
},
{
"month" : 201703,
"count" : 2
},
{
"month" : 201704,
"count" : 1
}
]
}
{
"_id" : "CAR",
"monthCounts" : [
{
"month" : 201701,
"count" : 1
},
{
"month" : 201702,
"count" : 1
}
]
}
Note that the coverage of "months" is only present where there is actual data. Whilst possible to produce zero values over a range, it requires quite a bit of wrangling to do so and is not very practical. If you want zero values then it is better to add that in post processing in the client once the results have been retrieved.
If you really have your heart set on the zero values, then you should separately query for $min and $max values, and pass these in to "brute force" the pipeline into generating the copies for each supplied possible range value.
So this time the "range" is made externally to all documents, and you then use a $cond statement into the accumulator to see if the current data is within the grouped range produced. Also since the generation is "external", we really don't need the MongoDB 3.4 operator of $range, so this can be applied to earlier versions as well:
// Get min and max separately
var ranges = db.cars.aggregate(
{ "$group": {
"_id": null,
"startRange": { "$min": "$startDate" },
"endRange": { "$max": "$endDate" }
}}
).toArray()[0]
// Make the range array externally from all possible values
var range = [];
for ( var d = new Date(ranges.startRange.valueOf()); d <= ranges.endRange; d.setUTCMonth(d.getUTCMonth()+1)) {
var v = ( d.getUTCFullYear() * 100 ) + d.getUTCMonth()+1;
range.push(v);
}
// Run conditional aggregation
db.cars.aggregate([
{ "$addFields": { "range": range } },
{ "$unwind": "$range" },
{ "$group": {
"_id": {
"type": "$type",
"month": "$range"
},
"count": {
"$sum": {
"$cond": {
"if": {
"$and": [
{ "$gte": [
"$range",
{ "$add": [
{ "$multiply": [ { "$year": "$startDate" }, 100 ] },
{ "$month": "$startDate" }
]}
]},
{ "$lte": [
"$range",
{ "$add": [
{ "$multiply": [ { "$year": "$endDate" }, 100 ] },
{ "$month": "$endDate" }
]}
]}
]
},
"then": 1,
"else": 0
}
}
}
}},
{ "$sort": { "_id": 1 } },
{ "$group": {
"_id": "$_id.type",
"monthCounts": {
"$push": { "month": "$_id.month", "count": "$count" }
}
}}
])
Which produces the consistent zero fills for all possible months on all groupings:
{
"_id" : "HGV",
"monthCounts" : [
{
"month" : 201701,
"count" : 0
},
{
"month" : 201702,
"count" : 2
},
{
"month" : 201703,
"count" : 2
},
{
"month" : 201704,
"count" : 1
}
]
}
{
"_id" : "CAR",
"monthCounts" : [
{
"month" : 201701,
"count" : 1
},
{
"month" : 201702,
"count" : 1
},
{
"month" : 201703,
"count" : 0
},
{
"month" : 201704,
"count" : 0
}
]
}
MapReduce
All versions of MongoDB support mapReduce, and the simple case of the "iterator" as mentioned above is handled by a for loop in the mapper. We can get output as generated up to the first $group from above by simply doing:
db.cars.mapReduce(
function () {
for ( var d = this.startDate; d <= this.endDate;
d.setUTCMonth(d.getUTCMonth()+1) )
{
var m = new Date(0);
m.setUTCFullYear(d.getUTCFullYear());
m.setUTCMonth(d.getUTCMonth());
emit({ id: this.type, date: m},1);
}
},
function(key,values) {
return Array.sum(values);
},
{ "out": { "inline": 1 } }
)
Which produces:
{
"_id" : {
"id" : "CAR",
"date" : ISODate("2017-01-01T00:00:00Z")
},
"value" : 1
},
{
"_id" : {
"id" : "CAR",
"date" : ISODate("2017-02-01T00:00:00Z")
},
"value" : 1
},
{
"_id" : {
"id" : "HGV",
"date" : ISODate("2017-02-01T00:00:00Z")
},
"value" : 2
},
{
"_id" : {
"id" : "HGV",
"date" : ISODate("2017-03-01T00:00:00Z")
},
"value" : 2
},
{
"_id" : {
"id" : "HGV",
"date" : ISODate("2017-04-01T00:00:00Z")
},
"value" : 1
}
So it does not have the second grouping to compound to arrays, but we did produce the same basic aggregated output.
I have a mongo collection whose schema looks like this:
_id: ObjectId(),
segments: {
activity: 'value1',
activation: 'value2',
plan: 'value3'
}
I'm trying to use the aggregation framework to find out how many of my documents have the value1 for the segment activity for instance.
The problem is that I want to do that for every segment in the same request if possible, and that I don't know how many segments I'll have or even their name.
Basically here's what I'd like to do:
If I have these two documents:
{ _id: 1, segments: { activity: 'active', activation: 'inactive', plan: 'free' }
{ _id: 2, segments: { activity: 'inactive', activation: 'inactive', plan: 'free' }
I want to be able to see that two of them have the activation segment to inactive and the free plan, and that activity have 1 inactive and 1 active values. Here is what I want to get:
{
activity: {
active: 1,
inactive: 1
},
activation: {
inactive: 2
},
plan: {
free: 2
}
}
So basically, if you could just $group by key it would be great! Something like this:
{
$group: {
_id: { $concat: [ '$segments.$key', '-', '$segments.$key.$value' ],
count: { $sum: 1 }
}
}
Or if I could unwind on each key...
To get the counts, take advantage of the $cond operator in the $group pipeline step to evaluate the counts based on the subdocuments value, something like the following:
db.collection.aggregate([
{
"$group": {
"_id": "$_id",
"activity_active": {
"$sum": {
"$cond": [ { "$eq": [ "$segment.activity", "active" ] }, 1, 0 ]
}
},
"activity_inactive": {
"$sum": {
"$cond": [ { "$eq": [ "$segment.activity", "inactive" ] }, 1, 0 ]
}
},
"activation_active": {
"$sum": {
"$cond": [ { "$eq": [ "$segment.activation", "active" ] }, 1, 0 ]
}
},
"activation_inactive": {
"$sum": {
"$cond": [ { "$eq": [ "$segment.activity", "inactive" ] }, 1, 0 ]
}
},
"plan_free": {
"$sum": {
"$cond": [ { "$eq": [ "$segment.plan", "free" ] }, 1, 0 ]
}
}
}
},
{
"$project": {
"_id": 0,
"activity": {
"active": "$activity_active",
"inactive": "$activity_inactive"
},
"activation": {
"active": "$activation_active",
"inactive": "$activation_inactive"
},
"plan": {
"free": "$plan_free"
}
}
}
])
there could be a generic solution to this problem, but might need a bit post processing:
to get output similat to this:
{
"_id" : {
"activity" : "active",
"activation" : "inactive"
},
"plan" : [{
"type" : "free",
"total" : 1
}, {
"type" : "paid",
"total" : 1
}
]
}, {
"_id" : {
"activity" : "inactive",
"activation" : "inactive"
},
"plan" : [{
"type" : "free",
"total" : 1
}
]
}, {
"_id" : {
"activity" : "inactive",
"activation" : "active"
},
"plan" : [{
"type" : "paid",
"total" : 3
}, {
"type" : "free",
"total" : 6
}
]
}
use query like that:
db.collection.aggregate([{
$group : {
_id : {
activity : "$segments.activity",
activation : "$segments.activation",
plan : "$segments.plan"
},
total : {
$sum : 1
}
}
}, {
$group : {
_id : {
activity : "$_id.activity",
activation : "$_id.activation"
},
plan : {
$push : {
type : "$_id.plan",
total : "$total"
}
}
}
},
])
My collection will look this,
{
"_id" : ObjectId("55c8bd1d85b83e06dc54c0eb"),
"name" : "xxx",
"salary" : 10000,
"type" : "type1"
}
{
"_id" : ObjectId("55c8bd1d85b83e06dc54c0eb"),
"name" : "aaa",
"salary" : 10000,
"type" : "type2"
}
{
"_id" : ObjectId("55c8bd1d85b83e06dc54c0eb"),
"name" : "ccc",
"salary" : 10000,
"type" : "type2"
}
My query params will be coming as,
{salary=10000, type=type2}
so based on the query I need to fetch the count of above query params
The result should be something like this,
{ category: 'type1', count: 500 } { category: 'type2', count: 200 } { category: 'name', count: 100 }
Now I am getting count by hitting three different queries and constructing the result (or) server side iteration I can get the result.
Can anyone suggest or provide me good way to get above result
Your quesstion is not very clearly presented, but what it seems you wanted to do here was count the occurances of the data in the fields, optionally filtering those fields by the values that matches the criteria.
Here the $cond operator allows you to tranform a logical condition into a value:
db.collection.aggregate([
{ "$group": {
"_id": null,
"name": { "$sum": 1 },
"salary": {
"$sum": {
"$cond": [
{ "$gte": [ "$salary", 1000 ] },
1,
0
]
}
},
"type": {
"$sum": {
"$cond": [
{ "$eq": [ "$type", "type2" ] },
1,
0
]
}
}
}}
])
All values are in the same document, and it does not really make any sense to split them up here as this is additional work in the pipeline.
{ "_id" : null, "name" : 3, "salary" : 3, "type" : 2 }
Otherwise in the long form, which is not very performant due to needing to make a copy of each document for every key looks like this:
db.collection.aggregate([
{ "$project": {
"name": 1,
"salary": 1,
"type": 1,
"category": { "$literal": ["name","salary","type"] }
}},
{ "$unwind": "$category" },
{ "$group": {
"_id": "$category",
"count": {
"$sum": {
"$cond": [
{ "$and": [
{ "$eq": [ "$category", "name"] },
{ "$ifNull": [ "$name", false ] }
]},
1,
{ "$cond": [
{ "$and": [
{ "$eq": [ "$category", "salary" ] },
{ "$gte": [ "$salary", 1000 ] }
]},
1,
{ "$cond": [
{ "$and": [
{ "$eq": [ "$category", "type" ] },
{ "$eq": [ "$type", "type2" ] }
]},
1,
0
]}
]}
]
}
}
}}
])
And it's output:
{ "_id" : "type", "count" : 2 }
{ "_id" : "salary", "count" : 3 }
{ "_id" : "name", "count" : 3 }
If your documents do not have uniform key names or otherwise cannot specify each key in your pipeline condition, then apply with mapReduce instead:
db.collection.mapReduce(
function() {
var doc = this;
delete doc._id;
Object.keys(this).forEach(function(key) {
var value = (( key == "salary") && ( doc[key] < 1000 ))
? 0
: (( key == "type" ) && ( doc[key] != "type2" ))
? 0
: 1;
emit(key,value);
});
},
function(key,values) {
return Array.sum(values);
},
{
"out": { "inline": 1 }
}
);
And it's output:
"results" : [
{
"_id" : "name",
"value" : 3
},
{
"_id" : "salary",
"value" : 3
},
{
"_id" : "type",
"value" : 2
}
]
Which is basically the same thing with a conditional count, except that you only specify the "reverse" of the conditions you want and only for the fields you want to filter conditions on. And of course this output format is simple to emit as separate documents.
The same approach applies where to test the condition is met on the fields you want conditions for and return 1 where the condition is met or 0 where it is not for the summing the count.
You can use aggregation as following query:
db.collection.aggregate({
$match: {
salary: 10000,
//add any other condition here
}
}, {
$group: {
_id: "$type",
"count": {
$sum: 1
}
}
}, {
$project: {
"category": "$_id",
"count": 1,
_id: 0
}
}
I have a Json file imported into MongoDB. Every line on it is a user, and I have a field product, with the name of it. I know the value of every product, they are just few.
But this information is not stored on the Json.
I was able to do aggregation to retrieve the number of time that a user bought a product, but I would like to do a query to get directly the amount of money that each user spent.
This is my query:
db.source.aggregate([
{"$match": {
"$and":[
{"productName":{
"$in":[
"product2","product2","product3",
"product4","product5","product6"
]
}},
{ "$or": [
{"appID" : "nameOfAPP"},
{"appID": "NameOfAPP2"}
]}
]
}},
{ "$group": {
"_id": {
"id_user": "$id_user",
"productName": "$productName"
},
"count": { "$sum": 1}
}},
{ "$sort" : { "count": -1 } }
])
so the output is like that:
{ "_id" : { "id_user" : "user1", "productID" : "product2" }, "count" : 433 }
{ "_id" : { "id_user" : "user2", "productID" : "product1" }, "count" : 370 }
{ "_id" : { "id_user" : "user1", "productID" : "product3" }, "count" : 300 }
{ "_id" : { "id_user" : "user3", "productID" : "product6" }, "count" : 250 }
{ "_id" : { "id_user" : "user2", "productID" : "product5" }, "count" : 140 }
{ "_id" : { "id_user" : "user3", "productID" : "product4" }, "count" : 90 }
I know that product 1 costs 20$, product 2 costs 40$, product 3 costs 55$, product 4 costs -90$, product 5 costs 110$, product 6 costs 200$.
I would like to have an output like that:
{ "_id" : { "id_user" : "user1"}, "money_spent" : 600$ }
{ "_id" : { "id_user" : "user2"}, "money_spent" : 400$ }
etc
Can you help to get that result, I am new with MongoDB.
Thanks in advance.
If you cannot go to the original source data an are only working with an import then do this:
db.source.aggregate([
{"$match": {
"$and":[
{ "productName": {
"$in":[
"product1","product2","product3",
"product4","product5","product6"
]
}},
{ "$or": [
{"appID" : "nameOfAPP"},
{"appID": "NameOfAPP2"}
]}
]
}},
{ "$group": {
"_id": "$id_user",
"cost": {
"$sum": {
"$cond": [
{ "$eq": ["$_id.productId", "product1"] },
20,
{ "$cond": [
{ "$eq": ["$productName", "product2"] },
40,
{ "$cond": [
{ "$eq": [ "$productName", "product3"] },
55,
{ "$cond": [
{ "$eq": [ "$productName", "product4" ] },
-90,
{ "$cond": [
{ "$eq": [ "$productName", "product5" ] },
110,
200
]}
]}
]}
]}
}
}
}
}}
])
The $cond operator evaluates whether your field value matches the condition and places the appropriate value simply just $sum to get your result.
$cond provides a "ternary" operator or "if .. then .. else" that is used to evaluate the condition you provide in the first argument. You construct this to "cascade" where the condition evaluates to false in order to move on to the next condition to evaluate, otherwise return the value that matches your condition.
In this way your "known" values are applied as you aggregate for your expected total.