Mongo DB - Group By and Having Implementation - mongodb

I'm trying to write a Mongo DB query where I do the following,
JSON list - Representing documents in my collection,
Industry[]:
{
"_id" : ObjectId("57aa6058be0c853c8cee34cd"),
"options": {
"paramList" : [
{
"name" : "industryCategory",
"value" : "Travel",
"mandatory" : true
},
{
"name" : "someOtherThing",
"value" : "dontMind",
"mandatory" : true
}
]
}
"mostRecent" : true
},
{
"_id" : ObjectId("57aa6058be0c853c8cee34cd"),
"options": {
"paramList" : [
{
"name" : "industryCategory",
"value" : "Dining",
"mandatory" : true
},
{
"name" : "someOtherThing",
"value" : "dontMind",
"mandatory" : true
}
]
}
"mostRecent" : true
},
{
"_id" : ObjectId("57aa6058be0c853c8cee34cd"),
"options": {
"paramList" : [
{
"name" : "industryCategory",
"value" : "Travel",
"mandatory" : true
},
{
"name" : "someOtherThing",
"value" : "dontMind",
"mandatory" : true
}
]
}
"mostRecent" : true
}
I'm trying to group and get a count of values for those paramList - values where name is industryCategory. Essentially, the output I am looking for is something like this,
Travel: 2
Dining: 1
I'm trying to do the following,
Industry is the name of the collection,
db.Industry.aggregate (
[
{$match: {mostRecent: true}},
{$group: {
_id: '$options.paramList.value',
count: {$sum: 1}
}},
{$match: {'options.paramList.name': 'industryCategory'}}
])
I'm getting an empty result. Please suggest what I can do

Well, I didn't have anyone answering. But, in the meanwhile I managed to figure it out myself. Posting the answer below,
aggregate (
[
{"$match": {mostRecent: true}},
{"$unwind" : "$options.paramList"},
{"$group" :
{
_id: "$options.paramList.value",
count: {$sum : 1}
}
}
]
Essentially, use unwind wherever you have to iterate over arrays (a list of sub-documents). This has been a productive learning, at least for me.

Related

Mongodb Query to get the nth document

I need to create a query in mongodb that needs to return the SECOND TO THE LAST document. I am planning to use $group for this query but i dont know what aggregation function to use. I only know $first and $last.
I have an example collection below and also include the expected output. Thank you!
"_id" : ObjectId("60dc27ac54b7c46bfa1b84b4"),
"auditlogs" : [
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84be"),
"userid" : ObjectId("5ffe702d59a9205db81fcb69"),
"action" : "ADDTRANSACTION"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84bd"),
"userid" : ObjectId("5ffe644f9493e05db9245192"),
"action" : "EDITPROFILE"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84bc"),
"userid" : ObjectId("5ffe64949493e05db9245197"),
"action" : "DELETETRANSACTION"
} ]
"_id" : ObjectId("60dc27ac54b7c46bfa1b75ge2"),
"auditlogs" : [
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84bb"),
"userid" : ObjectId("5ffe64b69493e05db924519b"),
"action" : "ADDTRANSACTION"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84ba"),
"userid" : ObjectId("5ffe65419493e05db92451d4"),
"action" : "ADDTRANSACTION"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84b9"),
"userid" : ObjectId("5ffe65689493e05db92451d9"),
"action" : "CHANGEACCESS"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84b8"),
"userid" : ObjectId("5ffe65819493e05db92451dd"),
"action" : "DELETETRANSACTION"
},
{
"_id" : ObjectId("60dc27ac54b7c46bfa1b84b7"),
"userid" : ObjectId("5ffe65df9493e05db92451f3"),
"action" : "EDITPROFILE",
]
OUTPUT:
{"_id" : ObjectId("60dc27ac54b7c46bfa1b84b4"),"_id" : ObjectId("60dc27ac54b7c46bfa1b84bd"),"userid" : ObjectId("5ffe644f9493e05db9245192"),"action" : "EDITPROFILE"},
{"_id" : ObjectId("60dc27ac54b7c46bfa1b75ge2"),"_id" : ObjectId("60dc27ac54b7c46bfa1b84b8"),"userid" : ObjectId("5ffe65819493e05db92451dd"),"action" : "DELETETRANSACTION"}
You can't have two _id keys in one single object.
I've made the parent object's id to _parentId you can give it's a name anything you want except _id
Aggregation:
db.collection.aggregate([
{
$unwind: "$auditlogs"
},
{
"$project": {
"_parentId": "$_id",
"_id": "$auditlogs._id",
"action": "$auditlogs.action",
"userid": "$auditlogs.userid",
}
}
])
Playground
You can slice the array by -2 to get the last two item, then by 1 to get first one. Therefore, the array will be left the second to the last. Finally, unwind auditlogs so it can be changed from array to object which is structure that you want.
db.collection.aggregate([
{
$project: { auditlogs : { $slice: [ "$auditlogs", -2 ] } }
},
{
$project: { auditlogs : { $slice: [ "$auditlogs", 1 ] } }
},
{
$unwind: "$auditlogs"
}
])

How can I do match after second level unwind in mongodb?

I am working on a software that uses MongoDB as a database. I have a collection like this (this is just one document)
{
"_id" : ObjectId("5aef51e0af42ea1b70d0c4dc"),
"EndpointId" : "89799bcc-e86f-4c8a-b340-8b5ed53caf83",
"DateTime" : ISODate("2018-05-06T19:05:04.574Z"),
"Url" : "test",
"Tags" : [
{
"Uid" : "E2:02:00:18:DA:40",
"Type" : 1,
"DateTime" : ISODate("2018-05-06T19:05:04.574Z"),
"Sensors" : [
{
"Type" : 1,
"Value" : NumberDecimal("-98")
},
{
"Type" : 2,
"Value" : NumberDecimal("-65")
}
]
},
{
"Uid" : "12:3B:6A:1A:B7:F9",
"Type" : 1,
"DateTime" : ISODate("2018-05-06T19:05:04.574Z"),
"Sensors" : [
{
"Type" : 1,
"Value" : NumberDecimal("-95")
},
{
"Type" : 2,
"Value" : NumberDecimal("-59")
},
{
"Type" : 3,
"Value" : NumberDecimal("12.939770381907275")
}
]
}
]
}
and I want to run this query on it.
db.myCollection.aggregate([
{ $unwind: "$Tags" },
{
$match: {
$and: [
{
"Tags.DateTime": {
$gte: ISODate("2018-05-06T19:05:02Z"),
$lte: ISODate("2018-05-06T19:05:09Z"),
},
},
{ "Tags.Uid": { $in: ["C1:3D:CA:D4:45:11"] } },
],
},
},
{ $unwind: "$Tags.Sensors" },
{ $match: { "$Tags.Sensors.Type": { $in: [1, 2] } } },
{
$project: {
_id: 0,
EndpointId: "$EndpointId",
TagId: "$Tags.Uid",
Url: "$Url",
TagType: "$Tags.Type",
Date: "$Tags.DateTime",
SensorType: "$Tags.Sensors.Type",
Value: "$Tags.Sensors.Value",
},
},
])
the problem is, the second match (that checks $Tags.Sensors.Type) doesn't work and doesn't affect the result of the query.
How can I solve that?
If this is not the right way, what is the right way to run these conditions?
The $match stage accepts field names without a leading $ sign. You've done that correctly in your first $match stage but in the second one you write $Tags.Sensors.Type. Simply removing the leading $ sign should make your query work.
Mind you, the whole thing can be a bit simplified (and some beautification doesn't hurt, either):
You don't need to use $and in your example since it's assumed by default if you specify more than one criterion in a filter.
The $in that you use for the Tags.Sensors.Type filter can be a simple : kind of equality operator unless you have more than one element in the list of acceptable values.
In the $project stage, instead of (kind of) duplicating identical field names you can use the <field>: 1 syntax unless the order of the fields matters.
So the final query would be something like this.
db.myCollection.aggregate([
{
"$unwind" : "$Tags"
},
{
"$match" : {
"Tags.DateTime" : { "$gte" : ISODate("2018-05-06T19:05:02Z"), "$lte" : ISODate("2018-05-06T19:05:09Z") },
"Tags.Uid" : { "$in" : ["C1:3D:CA:D4:45:11"] }
}
}, {
"$unwind" : "$Tags.Sensors"
}, {
"$match" : {
"Tags.Sensors.Type" : { "$in" : [1,2] }
}
},
{
"$project" : {
"_id" : 0,
"EndpointId" : 1,
"TagId" : "$Tags.Uid",
"Url" : 1,
"TagType" : "$Tags.Type",
"Date" : "$Tags.DateTime",
"SensorType" : "$Tags.Sensors.Type",
"Value" : "$Tags.Sensors.Value"
}
}])

MongoDB filtering out subdocuments with lookup aggregation

Our project database has a capped collection called values which gets updated every few minutes with new data from sensors. These sensors all belong to a single sensor node, and I would like to query the last data from these nodes in a single aggregation. The problem I am having is filtering out just the last of ALL the types of sensors while still having only one (efficient) query. I looked around and found the $group argument, but I can't seem to figure out how to use it correctly in this case.
The database is structured as follows:
nodes:
{
"_id": 681
"sensors": [
{
"type": "foo"
},
{
"type": "bar"
}
]
}
values:
{
"_id" : ObjectId("570cc8b6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 10
}
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 681,
"value" : 20
}
{
"_id" : ObjectId("167bc997bb66750d5740665e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 200,
"value" : 20
}
{
"_id" : ObjectId("110cc9c6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-09T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 12
}
so let's imagine I want the data from node 681, I would want a structure like this:
nodes:
{
"_id": 681
"sensors": [
{
"_id" : ObjectId("570cc8b6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 10
},
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 681,
"value" : 20
}
]
}
Notice how one value of foo is not queried, because I want to only get the latest value possible if there are more than one value (which is always going to be the case). The ordering of the collection is already according to the timestamp because the collection is capped.
I have this query, but it just gets all the values from the database (which is waaay too much to do in a lifetime, let alone one request of the web app), so I was wondering how I would filter it before it gets aggregated.
query:
db.nodes.aggregate(
[
{
$unwind: "$sensors"
},
{
$match:{
nodeid: 681
}
},
{
$lookup:{
from: "values", localField: "sensors.type", foreignField: "type", as: "sensors"
}
}
}
]
)
Try this
// Pipeline
[
// Stage 1 - sort the data collection if not already done (optional)
{
$sort: {
"timestamp":1
}
},
// Stage 2 - group by type & nodeid then get first item found in each group
{
$group: {
"_id":{type:"$type",nodeid:"$nodeid"},
"sensors": {"$first":"$$CURRENT"} //consider using $last if your collection is on reverse
}
},
// Stage 3 - project the fields in desired
{
$project: {
"_id":"$sensors._id",
"timestamp":"$sensors.timestamp",
"type":"$sensors.type",
"nodeid":"$sensors.nodeid",
"value":"$sensors.value"
}
},
// Stage 4 - group and push it to array sensors
{
$group: {
"_id":{nodeid:"$nodeid"},
"sensors": {"$addToSet":"$$CURRENT"}
}
}
]
as far as I got document structure, there is no need to use $lookup as all data is in readings(values) collection.
Please see proposed solution:
db.readings.aggregate([{
$match : {
nodeid : 681
}
},
{
$group : {
_id : {
type : "$type",
nodeid : "$nodeid"
},
readings : {
$push : {
timestamp : "$timestamp",
value : "$value",
id : "$_id"
}
}
}
}, {
$project : {
_id : "$_id",
readings : {
$slice : ["$readings", -1]
}
}
}, {
$unwind : "$readings"
}, {
$project : {
_id : "$readings.id",
type : "$_id.type",
nodeid : "$_id.nodeid",
timestamp : "$readings.timestamp",
value : "$readings.value",
}
}, {
$group : {
_id : "$nodeid",
sensors : {
$push : {
_id : "$_id",
timestamp : "$timestamp",
value : "$value",
type:"$type"
}
}
}
}
])
and output:
{
"_id" : 681,
"sensors" : [
{
"_id" : ObjectId("110cc9c6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-09T12:06:46.344Z"),
"value" : 12,
"type" : "foo"
},
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"value" : 20,
"type" : "bar"
}
]
}
Any comments welcome!

MongoDB Aggregation Projection

If I have a collection as follows:
db.cafe.insert({name: "Cafe1", customers: [{name: "David", foods: [{name : "cheese"}, {name: "beef"}]}, {name: "Bill", foods: [{name: "fish"}]} ]})
db.cafe.find().pretty()
{
"_id" : ObjectId("54f5ae58baed23b7a34fccb6"),
"name" : "Cafe1",
"customers" : [
{
"name" : "David",
"foods" : [
{
"name" : "cheese"
},
{
"name" : "beef"
}
]
},
{
"name" : "Bill",
"foods" : [
{
"name" : "fish"
}
]
}
]
}
How can I extract an array containing just the food objects for people called "David".
Desired output is just the array of foods, i.e:
[{name: "cheese"}, {name: "beef"}]
I have tried an aggregation pipeline that unwinds the cafes customers, then matches on name then projects the food, e.g:
db.cafe.aggregate( [{$unwind : "$customers"}, {$match : {"customers.name": "David"}}, {$project : {"customers.foods": 1, _id : 0}
}] ).pretty()
{
"customers" : {
"foods" : [
{
"name" : "cheese"
},
{
"name" : "beef"
}
]
}
}
This seems close to the desired result, however, I'm left with the issue that the foods I want are referenced as an array under the property customers.foods. I would like the result to directly be:
[
{
"name" : "cheese"
},
{
"name" : "beef"
}
]
is there a way I can achieve the desired output?
You are doing your projection wrong.
db.cafe.aggregate( [
{ "$match" : { "customers.name": "David" }},
{ "$unwind" : "$customers" },
{ "$project" : { "foods": "$customers.foods", "_id": 0 }}
])
Output
{ "foods" : [ { "name" : "cheese" }, { "name" : "beef" } ] }
You can also get (something very, very close to) your desired output with a regular query:
> db.cafe.find({ "customers.name" : "David" }, { "customers.$.foods" : 1, "_id" : 0 })
{ "customers" : [ { "name" : "David", "foods" : [ { "name" : "cheese" }, { "name" : "beef" } ] } ] }
Customers will be an array containing just the first object with name : "David". You should prefer this approach to the aggregation as it's vastly more performant. You can extract the foods array in client code.

MongoDB $ne in sub documents

{
"_id" : ObjectId("53692eb238ed04c824679f18"),
"firstUserId" : 1,
"secondUserId" : 17,
"messages" : [
{
"_id" : ObjectId("5369338997b964b81d579fc6"),
"read" : true,
"dateTime" : 1399403401,
"message" : "d",
"userId" : 1
},
{
"_id" : ObjectId("536933c797b964b81d579fc7"),
"read" : false,
"dateTime" : 1399403463,
"message" : "asdf",
"userId" : 17
}
]
}
I'm trying to select all documents that have firstUserId = 1 and also have sub documents
that have userId differnet ($ne) to 1 and read = false.
I tried:
db.usermessages.find({firstUserId: 1, "messages.userId": {$ne: 1}, "messages.read": false})
But it returns empty cause messages have both 1 and 17.
And also how to count subdocuments that have given case?
Are you trying to get the count of all the documents which are returned after your match criteria? If Yes, then you might consider using aggregation framework. http://docs.mongodb.org/manual/aggregation/
Something like below could be done to get you the counts:
db.usermessages.aggregate(
{ "$unwind": "$messages" },
{ "$match":
{ "firstUserId": 1,
"messages.userId": { "$ne" : 1},
"messages.read": false
}
},
{ "$group": { "_id" :null, "count" : { "$sum": 1 } } }
)
Hope this helps.
PS: I have not tried this on my system.