MongoDB - Query to count the nested documents in an array - mongodb

MongoDB - Layout
I'm a newbie to MongoDB, so pardon me if I'm using wrong terminologies here.
Given the above layout (attached image), I need to come up with two queries:
Use case 1 : A query that would output the count of total number of elements under someArrayField for a given document identifier.
For example, for the above layout, I expect the query to return 2 for someArrayField, because it has 2 elements under it as nested documents.
Use case 2 : A query that would output the count of total number of elements under someArrayField for every document under Collection_1 (Collection_1 can contain multiple documents with the similar layout).
I know, I need to somehow use "Aggregation Pipeline" to get the desired results, but due to the lack of experience with NoSQL, I'm struggling a bit here.
So, any help is greatly appreciated!

Given the following test set :
{
"_id" : ObjectId("5b453cc7799fb211dc44a1e8"),
"id" : 1.0,
"nestedDoc" : {
"nestedArray" : [
{
"field1" : "value1",
"field2" : "value2",
"field3" : "value3"
},
{
"field1" : "value11",
"field2" : "value21",
"field3" : "value31"
},
{
"field1" : "value12",
"field2" : "value22",
"field3" : "value32"
}
]
}
}
{
"_id" : ObjectId("5b453d23799fb211dc44a1e9"),
"id" : 2.0,
"nestedDoc" : {
"nestedArray" : [
{
"field1" : "value1",
"field2" : "value29",
"field3" : "value39"
},
{
"field1" : "value118",
"field2" : "value281",
"field3" : "value381",
"field4" : "value281"
},
{
"field1" : "value172",
"field2" : "value272",
"field3" : "value372"
}
]
}
}
Use case 1 :
Run the following aggregation query :
db.test1.aggregate(
[
// Stage 1 : filter documents
{
$match: {
id:1
}
},
// Stage 2 : count array size and return only it.
{
$project: {
arraySize:{$size:"$nestedDoc.nestedArray"},
_id:0
}
},
]
);
**Use case 2 ** Run the following :
db.test1.aggregate(
[
// Stage 1 count array size for each document
{
$project: {
_id:0,
arraySize:{$size:"$nestedDoc.nestedArray"}
}
},
// Stage 2 group all documents and sum arraySize
{
$group: {
_id:null,
totalArraysSize:{$sum:"$arraySize"}
}
},
]
);

Related

MongoDB - find document whose array length is less than or equal to 5

Can't we pass an object to $size operator in mongoose? Is there any ways to query on array for length so we can fetch document which contains an array of a particular length.
Hers is Sample Document
"_id" : ObjectId("5e8c9becd1257f66c4b8cd63"),
"index" : 0,
"name" : "Aurelia Gonzales",
"isActive" : false,
"registered" : ISODate("2015-02-11T09:52:39.000+05:30"),
"age" : 20,
"gender" : "female",
"eyeColor" : "green",
"favoriteFruit" : "banana",
"company" : {
"title" : "YURTURE",
"email" : "aureliagonzales#yurture.com",
"phone" : "+1 (940) 501-3963",
"location" : {
"country" : "USA",
"address" : "694 Hewes Street"
}
},
"tags" : [
"enim",
"id",
"velit",
"ad",
"consequat"
]
}
Here is query
db.admin.aggregate([
{
$match : {tags : {$size : {$lte : 5}}}
}
])
Here is Output
{
"message" : "$size needs a number",
"ok" : 0,
"code" : 2,
"codeName" : "BadValue",
"name" : "MongoError"
}
You can't use $size like that & needed to use aggregation $size operator to do this.
Query :
db.collection.find({
$expr: { /** Allows the use of aggregation expressions within the query language */
$lte: [
{
$size: "$tags"
},
5
]
}
})
Test : MongoDB-Playground
Although if the size of the array is important enough, it could be stored in the documents and indexed to fetch much faster results.
Following a similar logic a solution could be, two stage aggregation using $addFields and $size, $lte.
db.collection.aggregate([
{
$addFields: {
sizeOfTags: {
$size: "$tags"
}
}
},
{
$match: {
sizeOfTags: {
$lte: 5
}
}
}
])

How to filter and aggregate data from one collection into another format using MongoDB

I have one collection named 'ctrlcharts'.
e.g.
{
"_id" : ObjectId("57fc695492af567031246736"),
"deviceId" : "A001",
"sensorId" : "S003",
"time" : "2016/10/11 12:23:50",
"charts" : [
{
"sensor" : "ch_11",
"value" : 120
},
{
"sensor" : "ch_12",
"value" : 150
}
]
}
How to filter "sensor" : "ch_11" and aggregate data from one collection into another format using MongoDB
e.g.
{
"time" : "2016/10/11 12:23:50",
"sensor" : "ch_12",
"value" : 150
}
I tried below code
db.ctrlcharts.aggregate([
{ $match: {"deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50"}},
{ $project: {
_id: 0,
time : 1 ,
sensor : "$charts.sensor"
value : "$charts.value"
}
}
])
But I got the result as
{
"time" : "2016/10/11 12:23:50",
"sensor" : ["ch_11","ch_12"],
"value" : [120,150]
}
Thanks
You tried best....just use $unwind
db.ctrlcharts.aggregate(
{$unwind:"$charts"},
{$match: {"deviceId" :"A001", "charts.sensor":"ch_12", "time" : "2016/10/11 12:23:50"}},
{$project:{_id:0,time:1, sensor : "$charts.sensor", value :"$charts.value"}}).pretty()
You can use $unwind (aggregation) to separate charts array.
db.ctrlcharts.aggregate( [ { $unwind : "$charts" } ] )
This will produce result like -
{ "_id" : ObjectId("57ff397a007c43ecacf10512"), "deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50", "charts" : { "sensor" : "ch_11", "value" : 120 } }
{ "_id" : ObjectId("57ff397a007c43ecacf10512"), "deviceId" : "A001", "sensorId" : "S003", "time" : "2016/10/11 12:23:50", "charts" : { "sensor" : "ch_12", "value" : 150 } }
and then use your match query
Use the $arrayElemAt and $filter operators to query the array more efficiently without the need to $unwind. The reason why $unwind is not as efficient is that it produces a cartesian product of the documents i.e. a copy of each document per array entry, which uses more memory (possible memory cap on aggregation pipelines of 10% total memory) and therefore takes time to produce as well processing the documents during the flattening process.
The $filter will return a subset of the array that only contains the elements that match the filter condition. The $arrayElemAt operator then returns the element from the filtered array at the specified array index to give you the subdocument you need.
A further $project is necessary to flatten the fields to give you the desired result:
db.ctrlcharts.aggregate([
{ "$match": {
"deviceId": "A001",
"sensorId": "S003",
"time": "2016/10/11 12:23:50",
"charts.sensor": "ch_11"
} },
{
"$project": {
"time": 1,
"chart": {
"$arrayElemAt": [
"$filter": {
"input": "$charts",
"as": "item",
"cond": { "$eq": ["$$item.sensor", "ch_11"] }
}, 0
]
}
}
},
{
"$project": {
"_id": 0,
"time": 1,
"sensor": "$chart.sensor"
"value": "$chart.value"
}
}
])

MongoDB filtering out subdocuments with lookup aggregation

Our project database has a capped collection called values which gets updated every few minutes with new data from sensors. These sensors all belong to a single sensor node, and I would like to query the last data from these nodes in a single aggregation. The problem I am having is filtering out just the last of ALL the types of sensors while still having only one (efficient) query. I looked around and found the $group argument, but I can't seem to figure out how to use it correctly in this case.
The database is structured as follows:
nodes:
{
"_id": 681
"sensors": [
{
"type": "foo"
},
{
"type": "bar"
}
]
}
values:
{
"_id" : ObjectId("570cc8b6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 10
}
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 681,
"value" : 20
}
{
"_id" : ObjectId("167bc997bb66750d5740665e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 200,
"value" : 20
}
{
"_id" : ObjectId("110cc9c6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-09T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 12
}
so let's imagine I want the data from node 681, I would want a structure like this:
nodes:
{
"_id": 681
"sensors": [
{
"_id" : ObjectId("570cc8b6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "foo",
"nodeid" : 681,
"value" : 10
},
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"type" : "bar",
"nodeid" : 681,
"value" : 20
}
]
}
Notice how one value of foo is not queried, because I want to only get the latest value possible if there are more than one value (which is always going to be the case). The ordering of the collection is already according to the timestamp because the collection is capped.
I have this query, but it just gets all the values from the database (which is waaay too much to do in a lifetime, let alone one request of the web app), so I was wondering how I would filter it before it gets aggregated.
query:
db.nodes.aggregate(
[
{
$unwind: "$sensors"
},
{
$match:{
nodeid: 681
}
},
{
$lookup:{
from: "values", localField: "sensors.type", foreignField: "type", as: "sensors"
}
}
}
]
)
Try this
// Pipeline
[
// Stage 1 - sort the data collection if not already done (optional)
{
$sort: {
"timestamp":1
}
},
// Stage 2 - group by type & nodeid then get first item found in each group
{
$group: {
"_id":{type:"$type",nodeid:"$nodeid"},
"sensors": {"$first":"$$CURRENT"} //consider using $last if your collection is on reverse
}
},
// Stage 3 - project the fields in desired
{
$project: {
"_id":"$sensors._id",
"timestamp":"$sensors.timestamp",
"type":"$sensors.type",
"nodeid":"$sensors.nodeid",
"value":"$sensors.value"
}
},
// Stage 4 - group and push it to array sensors
{
$group: {
"_id":{nodeid:"$nodeid"},
"sensors": {"$addToSet":"$$CURRENT"}
}
}
]
as far as I got document structure, there is no need to use $lookup as all data is in readings(values) collection.
Please see proposed solution:
db.readings.aggregate([{
$match : {
nodeid : 681
}
},
{
$group : {
_id : {
type : "$type",
nodeid : "$nodeid"
},
readings : {
$push : {
timestamp : "$timestamp",
value : "$value",
id : "$_id"
}
}
}
}, {
$project : {
_id : "$_id",
readings : {
$slice : ["$readings", -1]
}
}
}, {
$unwind : "$readings"
}, {
$project : {
_id : "$readings.id",
type : "$_id.type",
nodeid : "$_id.nodeid",
timestamp : "$readings.timestamp",
value : "$readings.value",
}
}, {
$group : {
_id : "$nodeid",
sensors : {
$push : {
_id : "$_id",
timestamp : "$timestamp",
value : "$value",
type:"$type"
}
}
}
}
])
and output:
{
"_id" : 681,
"sensors" : [
{
"_id" : ObjectId("110cc9c6ac55850d5740784e"),
"timestamp" : ISODate("2016-04-09T12:06:46.344Z"),
"value" : 12,
"type" : "foo"
},
{
"_id" : ObjectId("190ac8b6ac55850d5740776e"),
"timestamp" : ISODate("2016-04-12T12:06:46.344Z"),
"value" : 20,
"type" : "bar"
}
]
}
Any comments welcome!

MongoDB $ne in sub documents

{
"_id" : ObjectId("53692eb238ed04c824679f18"),
"firstUserId" : 1,
"secondUserId" : 17,
"messages" : [
{
"_id" : ObjectId("5369338997b964b81d579fc6"),
"read" : true,
"dateTime" : 1399403401,
"message" : "d",
"userId" : 1
},
{
"_id" : ObjectId("536933c797b964b81d579fc7"),
"read" : false,
"dateTime" : 1399403463,
"message" : "asdf",
"userId" : 17
}
]
}
I'm trying to select all documents that have firstUserId = 1 and also have sub documents
that have userId differnet ($ne) to 1 and read = false.
I tried:
db.usermessages.find({firstUserId: 1, "messages.userId": {$ne: 1}, "messages.read": false})
But it returns empty cause messages have both 1 and 17.
And also how to count subdocuments that have given case?
Are you trying to get the count of all the documents which are returned after your match criteria? If Yes, then you might consider using aggregation framework. http://docs.mongodb.org/manual/aggregation/
Something like below could be done to get you the counts:
db.usermessages.aggregate(
{ "$unwind": "$messages" },
{ "$match":
{ "firstUserId": 1,
"messages.userId": { "$ne" : 1},
"messages.read": false
}
},
{ "$group": { "_id" :null, "count" : { "$sum": 1 } } }
)
Hope this helps.
PS: I have not tried this on my system.

aggregate request MongoDB

I'd like get a multiple fields in a collection list with a condition. I tried a aggregate request but i have an error.
My request
db.people.aggregate({$match:{createdDate:{$exists:true},"ad":"noc2"}},{$group:{value2:$value2}});
My Json :
db.test.findOne();
{
"_id" : ObjectId("51e7dd16d2f8db27b56ea282"),
"ad" : "noc2",
"list" : {
"p45" : {
"id" : "p45",
"date" : ISODate("2014-01-01T12:18:30.568Z"),
"value3" : 21,
"value1" : 100,
"value2" : 489
},
"p6" : {
"id" : "p6"
"date" : ISODate("2013-07-18T12:18:30.568Z"),
"value3" : 21,
"value1" : 100,
"value2" : 489
},
"p4578" : {
"id" : "4578"
"date" : ISODate("2013-07-18T12:18:30.568Z"),
"value3" : 21,
"value1" : 100,
"value2" : 489
}
}
}
I want to get this json, for example, in result :
{id:p45,value:587},{id:p4578,value:47},{id:p6,value:2}
There are several issues with your sample document and aggregation:
the sample doc will not match your aggregation query because you are matching on createdDate field existing
the $group() operator works on a document level and needs an _id field to group by
your list field is an embedded document, not an array
aside from formatting, there is no obvious way to relate the sample values to the calculated result you are looking for
Here is an adjusted example document with the list as an array as well as some unique values for each item that happen to add up to the value numbers you mentioned:
db.people.insert({
"ad" : "noc2",
"createdDate" : ISODate(),
"list" : [
{
"id" : "p45",
"date" : ISODate("2014-01-01T12:18:30.568Z"),
"value3" : 21,
"value1" : 77,
"value2" : 489
},
{
"id" : "p6",
"date" : ISODate("2013-07-18T12:18:30.568Z"),
"value3" : 20,
"value1" : 20,
"value2" : 7
},
{
"id" : "4578",
"date" : ISODate("2013-07-18T12:18:30.568Z"),
"value3" : 21,
"value1" : 300,
"value2" : -319
}
]
})
You can now use the $unwind operator to extract the matching subdocuments.
It is unclear from the current question description what $group operation you are trying to achieve and whether you actually need $group.
Here is an example using the Aggregation Framework to $add (total) the three values for each list item:
db.people.aggregate(
// Find relevant docs (can use index if available)
{ $match: {
createdDate: { $exists:true },
"ad":"noc2"
}},
// Convert list array to stream of documents
{ $unwind: "$list" },
// Add (value1, value2, value3) for each list item
{ $project: {
_id: "$list.id",
value: { $add: [ "$list.value1", "$list.value2", "$list.value3"] }
}}
);
Sample output:
{
"result" : [
{
"_id" : "p45",
"value" : 587
},
{
"_id" : "p6",
"value" : 47
},
{
"_id" : "4578",
"value" : 2
}
],
"ok" : 1
}
Note that I've only aggregated using a single document for this example, and the output will be one result document for every item in the list array.
In real usage you may want to add an additional aggregation step to $group by document _id or some other criteria, depending on the outcome you are trying to achieve.