I have json with the following structure
db.testCollection.insert(
{
"m_id": 2,
"sys_data":[
{"sattr":
{
"size": 2,
"d_data":
[
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"03/01/01"}
]
}
}
]
}
db.testCollection.insert(
{
"m_id": 2,
"sys_data":[
{"sattr":
{
"size": 2,
"d_data":
[
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
]
}
}
]
}
I want to get the count where d_date ='02/01/01', So the output for above json is 3. (two from first json and one from the second)
Use the aggregation framework to get the count. This allows you to $unwind the deeply nested embedded documents (with the help of the dot notation), filter out the unwanted documents using the $match operator which is similar to the find() query and use the $sum accumulator operator in the $group pipeline to determine the count of the matching documents.
Something like this:
Populate test collection:
db.testCollection.insert([
{
"m_id" : 2,
"sys_data" : [
{
"sattr" : {
"size" : 2,
"d_data" : [
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
},
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
},
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "03/01/01"
}
]
}
}
]
},
{
"m_id" : 2,
"sys_data" : [
{
"sattr" : {
"size" : 2,
"d_data" : [
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
}
]
}
}
]
}])
Run the aggregation pipeline:
var pipeline = [
{
"$match": {
"sys_data.sattr.d_data.d_date" : "02/01/01"
}
},
{
"$unwind": "$sys_data"
},
{
"$unwind": "$sys_data.sattr.d_data"
},
{
"$match": {
"sys_data.sattr.d_data.d_date" : "02/01/01"
}
},
{
"$group": {
"_id": null,
"count": { "$sum": 1 }
}
}
];
db.testCollection.aggregate(pipeline);
Sample output:
/* 0 */
{
"result" : [
{
"_id" : null,
"count" : 3
}
],
"ok" : 1
}
Check out this docs from mongodb
db.testCollection.count( { d_date : "02/01/01" } )
Also, the above is equivalent to:
db.testCollection.find( { d_date: "02/01/01" } ).count()
If you feel slow performance check this question.
Related
I want to group by and count follow_user.tags.tag_id per record, so no matter how many times the same tag_id show up on the same record, it only counts as 1.
My database structure looks like this:
{
"external_userid" : "EXID1",
"follow_user" : [
{
"userid" : "USERID1",
"tags" : [
{
"tag_id" : "TAG1"
}
]
},
{
"userid" : "USERID2",
"tags" : [
{
"tag_id" : "TAG1"
},
{
"tag_id" : "TAG2"
}
]
}
]
},
{
"external_userid" : "EXID2",
"follow_user" : [
{
"userid" : "USERID1",
"tags" : [
{
"tag_id" : "TAG2"
}
]
}
]
}
Here's my query:
[
{ "$unwind": "$follow_user" }, { "$unwind": "$follow_user.tags" },
{ "$group" : { "_id" : { "follow_user᎐tags᎐tag_id" : "$follow_user.tags.tag_id" }, "COUNT(_id)" : { "$sum" : 1 } } },
{ "$project" : { "total" : "$COUNT(_id)", "tagId" : "$_id.follow_user᎐tags᎐tag_id", "_id" : 0 } }
]
What I expected:
{
"total" : 1,
"tagId" : "TAG1"
},
{
"total" : 2,
"tagId" : "TAG2"
}
What I get:
{
"total" : 2,
"tagId" : "TAG1"
},
{
"total" : 2,
"tagId" : "TAG2"
}
$set - Create a new field follow_user_tags.
1.1. $setUnion - To distinct the value from the Result 1.1.1.
1.1.1. $reduce - Add the value of follow_user.tags.tag_id into array.
$unwind - Deconstruct follow_user_tags array field to multiple documents.
$group - Group by follow_user_tags and perform total count via $sum.
$project - Decorate output document.
db.collection.aggregate([
{
$set: {
follow_user_tags: {
$setUnion: {
"$reduce": {
"input": "$follow_user.tags",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this.tag_id"
]
}
}
}
}
}
},
{
$unwind: "$follow_user_tags"
},
{
$group: {
_id: "$follow_user_tags",
total: {
$sum: 1
}
}
},
{
$project: {
_id: 0,
tagId: "$_id",
total: 1
}
}
])
Sample Mongo Playground
please, help , I have following type of documents:
db.g.find({_id:ObjectId("605929e0122984ad3c4c537a") }).pretty()
{
"_id" : ObjectId("605929e0122984ad3c4c537a"),
"a" : [
{
"p" : [
{
"pid" : 1,
"c" : {
"t" : [
{
"x" : 1,
"y" : 2
},
{
"z" : 1,
"x" : 5
},
{
"h" : 1
}
]
}
},
{
"d" : 1
}
]
},
{
"p" : [
{
"pid" : 2,
"c" : {
"t" : [
{
"x" : 4
}
]
}
},
{
"pid" : 3,
"c" : {
"t" : [
{
"y" : 4
}
]
}
}
]
}
]
}
And I need to fetch only the values from the fields:"a.p.pid" , and all "x" or "y" if they exist , so the final result to look like:
{pid:1,x:1,y:2}
{pid:1,x:5}
{pid:2,x:4}
{pid:3,y:4}
Collection is pretty big and doing 3x$unwind take alot of time ...
Attempting with $redact, $map / $filter but no success ... , any help will be highly appreciated ...
Demo - https://mongoplayground.net/p/w0wWJRdBP-J
Use $unwind twice to seperate each pid element
Use $filter to get an array of objects where x or y is present.
db.collection.aggregate([
{ "$unwind": "$a" },
{ "$unwind": "$a.p" },
{ "$project": {
_id: 0, pid: "$a.p.pid",
t: {
$filter: { input: "$a.p.c.t", as: "item",
cond: { $or: [
{ $ne: [ { $type: "$$item.x" }, "missing" ] },
{ $ne: [ { $type: "$$item.y" }, "missing" ]}
]}
}
}}
},
{ "$unwind": "$t" },
{ "$project": { pid: 1, x: "$t.x", y: "$t.y" } }
])
I'm trying to figure out what I'm doing wrong, I have collected the following, "Subset of data", "Desired output"
This is how my data objects look
[{
"survey_answers": [
{
"id": "9ca01568e8dbb247", // As they are, this is the key to groupBy
"option_answer": 5, // Represent the index of the choosen option
"type": "OPINION_SCALE" // Opinion scales are 0-10 (meaning elleven options)
},
{
"id": "ba37125ec32b2a99",
"option_answer": 3,
"type": "LABELED_QUESTIONS" // Labeled questions are 0-x (they can change it from survey to survey)
}
],
"survey_id": "test"
},
{
"survey_answers": [
{
"id": "9ca01568e8dbb247",
"option_answer": 0,
"type": "OPINION_SCALE"
},
{
"id": "ba37125ec32b2a99",
"option_answer": 3,
"type": "LABELED_QUESTIONS"
}
],
"survey_id": "test"
}]
My desired output is:
[
{
id: '9ca01568e8dbb247'
results: [
{ _id: 5, count: 1 },
{ _id: 0, count: 1 }
]
},
{
id: 'ba37125ec32b2a99'
results: [
{ _id: 3, count: 2 }
]
}
]
Active query
Model.aggregate([
{
$match: {
'survey_id': survey_id
}
},
{
$unwind: "$survey_answers"
},
{
$group: {
_id: "$survey_answers.option_answer",
count: {
$sum: 1
}
}
}
])
Current output
[
{
"_id": 0,
"count": 1
},
{
"_id": 3,
"count": 2
},
{
"_id": 5,
"count": 1
}
]
I added your records to my db. Post that I tried your commands one by one.
$unwind results you similar to -
> db.survey.aggregate({$unwind: "$survey_answers"})
{ "_id" : ObjectId("5c3859e459875873b5e6ee3c"), "survey_answers" : { "id" : "9ca01568e8dbb247", "option_answer" : 5, "type" : "OPINION_SCALE" }, "survey_id" : "test" }
{ "_id" : ObjectId("5c3859e459875873b5e6ee3c"), "survey_answers" : { "id" : "ba37125ec32b2a99", "option_answer" : 3, "type" : "LABELED_QUESTIONS" }, "survey_id" : "test" }
{ "_id" : ObjectId("5c3859e459875873b5e6ee3d"), "survey_answers" : { "id" : "9ca01568e8dbb247", "option_answer" : 0, "type" : "OPINION_SCALE" }, "survey_id" : "test" }
{ "_id" : ObjectId("5c3859e459875873b5e6ee3d"), "survey_answers" : { "id" : "ba37125ec32b2a99", "option_answer" : 3, "type" : "LABELED_QUESTIONS" }, "survey_id" : "test" }
I am not adding code for match since that is okay in your query as well
The grouping would be -
> db.survey.aggregate({$unwind: "$survey_answers"},{$group: { _id: { 'optionAnswer': "$survey_answers.option_answer", 'id':"$survey_answers.id"}, count: { $sum: 1}}})
{ "_id" : { "optionAnswer" : 0, "id" : "9ca01568e8dbb247" }, "count" : 1 }
{ "_id" : { "optionAnswer" : 3, "id" : "ba37125ec32b2a99" }, "count" : 2 }
{ "_id" : { "optionAnswer" : 5, "id" : "9ca01568e8dbb247" }, "count" : 1 }
You can group on $survey_answers.id to bring it into projection.
The projection is what you're missing in your query -
> db.survey.aggregate({$unwind: "$survey_answers"},{$group: { _id: { 'optionAnswer': "$survey_answers.option_answer", 'id':'$survey_answers.id'}, count: { $sum: 1}}}, {$project : {answer: '$_id.optionAnswer', id: '$_id.id', count: '$count', _id:0}})
{ "answer" : 0, "id" : "9ca01568e8dbb247", "count" : 1 }
{ "answer" : 3, "id" : "ba37125ec32b2a99", "count" : 2 }
{ "answer" : 5, "id" : "9ca01568e8dbb247", "count" : 1 }
Further you can add a group on id and add results to a set. And your final query would be -
db.survey.aggregate(
{$unwind: "$survey_answers"},
{$group: {
_id: { 'optionAnswer': "$survey_answers.option_answer", 'id':'$survey_answers.id'},
count: { $sum: 1}
}},
{$project : {
answer: '$_id.optionAnswer',
id: '$_id.id',
count: '$count',
_id:0
}},
{$group: {
_id:{id:"$id"},
results: { $addToSet: {answer: "$answer", count: '$count'} }
}},
{$project : {
id: '$_id.id',
answer: '$results',
_id:0
}})
Hope this helps.
I have the following collection for messages:
{
"_id" : ObjectId("56214d5632001bae07a6e6b3"),
"sender_id" : 8,
"receiver_id" : 2,
"content" : "fdgfd",
"state" : 1,
"timestamp" : 1445023062899.0000000000000000
},
{
"_id" : ObjectId("56214d5c32001bae07a6e6b4"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "fasfa",
"state" : 1,
"timestamp" : 1445023068443.0000000000000000
},
{
"_id" : ObjectId("56214d8032001bae07a6e6b5"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "dfdsfds",
"state" : 1,
"timestamp" : 1445023104363.0000000000000000
},
{
"_id" : ObjectId("56214d8032001bae07a6e6b6"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "fdsf",
"state" : 1,
"timestamp" : 1445023104825.0000000000000000
},
{
"_id" : ObjectId("56214d8132001bae07a6e6b7"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "sfsdfs",
"state" : 1,
"timestamp" : 1445023105436.0000000000000000
},
{
"_id" : ObjectId("56214d8132001bae07a6e6b8"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "f",
"state" : 1,
"timestamp" : 1445023105963.0000000000000000
},
{
"_id" : ObjectId("56214d8432001bae07a6e6b9"),
"sender_id" : 2,
"receiver_id" : 8,
"content" : "qwqwqwq",
"state" : 1,
"timestamp" : 1445023108202.0000000000000000
},
{
"_id" : ObjectId("56214db032001bae07a6e6ba"),
"sender_id" : 9902,
"receiver_id" : 2,
"content" : "fsafa",
"state" : 1,
"timestamp" : 1445023152297.0000000000000000
}
I'm trying to get all unique users ids that had been messaging with user 2, along with the last content message. So the result should be:
[ { user: 8, lastContent: "qwqwqwq" }, { user: 9902, lastContent: "fsafa" } ]
By now, I have the following code:
db.getCollection('messenger').group({
keyf: function(doc) {
return { user: doc.user };
},
cond: {
$or : [
{ sender_id : 2 },
{ receiver_id : 2 }
]
},
reduce: function( curr, result ) {
result.user = (curr.sender_id == 2 ? curr.receiver_id : curr.sender_id);
result.content = curr.content;
},
initial: { } })
But I only get the last id. The result:
{
"0" : {
"user" : 9902.0000000000000000,
"content" : "fsafa"
} }
Can anyone help me with this?
You need to use the .aggregate() method. You need to reduce the size of documents in the pipeline using the $match operator which filter out all documents where the receiver_id is not equal to 2. After that you need to $sort your document by timestamp in descending order this will help us get the content of last message sent. Now comes the $group stage where you group your documents and use the $addToSet operator which returns array of distinct sender_id and distinct receiver_id and the $last operator to get the last message content. Now to get the user_ids we need union of distinct sender_id and receiver_id which we can get after $projection using the $setUnion operator.
db.messenger.aggregate([
{ "$match": {
"$or": [
{ "sender_id": 2 },
{ "receiver_id": 2 }
]
}},
{ "$sort": { "timestamp": 1 } },
{ "$group": {
"_id": null,
"receiver_id": {
"$addToSet": { "$receiver_id" }
},
"sender_id": {
"$addToSet": { "$sender_id" }
},
"lastContent": { "$last": "$content" }
}},
{ "$project": {
"_id": 0,
"lastContent": 1,
"user_ids": {
"$setUnion": [
"$sender_id",
"$receiver_id"
]
}
}}
])
Which returns:
{ "lastContent" : "fsafa", "user_ids" : [ 9902, 2, 8 ] }
Now if what you want is distinct user alongside their last content message with user 2 then here it is:
db.messenger.aggregate([
{ "$match": {
"$or": [
{ "sender_id": 2 },
{ "receiver_id": 2 }
]
}},
{ "$sort": { "timestamp": 1 } },
{ "$group": {
"_id": {
"sender": "$sender_id",
"receiver": "$receiver_id"
},
"lastContent": {
"$last": "$content"
},
"timestamp": { "$last": "$timestamp" },
"sender": { "$addToSet": "$sender_id" },
"receiver": { "$addToSet": "$receiver_id" }
}},
{ "$project": {
"_id": 0,
"user": {
"$setDifference": [
{ "$setUnion": [ "$sender", "$receiver" ] },
[ 2 ]
]
},
"lastContent": 1,
"timestamp": 1
}},
{ "$unwind": "$user" },
{ "$sort": { "timestamp": 1 } },
{ "$group": {
"_id": "$user",
"lastContent": { "$last": "$lastContent" }
} }
])
Which yields:
{ "_id" : 9902, "lastContent" : "fsafa" }
{ "_id" : 8, "lastContent" : "qwqwqwq" }
I have collection entry like that
[
{
shape : [{id:1,status:true},{id:2,status:false}]
},
{
shape : [{id:1,status:true}]
}
]
I want to fetch data which exactly match array , means contain all ele. of array.
Ex. where shape.id = [1,2] / [ {id: [1,2] } ] (any one is prefer)
then it should return only
[
{
shape : [{id:1,status:true},{id:2,status:false}]
}
]
So help me if is there any native mongodb query .
Thanks
--ND
Here is much simpler query;
db.shapes.find({'shape.id':{$all:[1,2]},shape:{$size:2}});
If mongo documents as below
{
"_id" : ObjectId("54eeb68c8716ec70106ee33b"),
"shapeSize" : [
{
"shape" : [
{
"id" : 1,
"status" : true
},
{
"id" : 2,
"status" : false
}
]
},
{
"shape" : [
{
"id" : 1,
"status" : true
}
]
}
]
}
Then used below aggregation to match the criteria
db.collectionName.aggregate({
"$unwind": "$shapeSize"
}, {
"$match": {
"$and": [{
"shapeSize.shape.id": 2
}, {
"shapeSize.shape.id": 1
}]
}
}, {
"$project": {
"_id": 0,
"shape": "$shapeSize.shape"
}
})