please, help , I have following type of documents:
db.g.find({_id:ObjectId("605929e0122984ad3c4c537a") }).pretty()
{
"_id" : ObjectId("605929e0122984ad3c4c537a"),
"a" : [
{
"p" : [
{
"pid" : 1,
"c" : {
"t" : [
{
"x" : 1,
"y" : 2
},
{
"z" : 1,
"x" : 5
},
{
"h" : 1
}
]
}
},
{
"d" : 1
}
]
},
{
"p" : [
{
"pid" : 2,
"c" : {
"t" : [
{
"x" : 4
}
]
}
},
{
"pid" : 3,
"c" : {
"t" : [
{
"y" : 4
}
]
}
}
]
}
]
}
And I need to fetch only the values from the fields:"a.p.pid" , and all "x" or "y" if they exist , so the final result to look like:
{pid:1,x:1,y:2}
{pid:1,x:5}
{pid:2,x:4}
{pid:3,y:4}
Collection is pretty big and doing 3x$unwind take alot of time ...
Attempting with $redact, $map / $filter but no success ... , any help will be highly appreciated ...
Demo - https://mongoplayground.net/p/w0wWJRdBP-J
Use $unwind twice to seperate each pid element
Use $filter to get an array of objects where x or y is present.
db.collection.aggregate([
{ "$unwind": "$a" },
{ "$unwind": "$a.p" },
{ "$project": {
_id: 0, pid: "$a.p.pid",
t: {
$filter: { input: "$a.p.c.t", as: "item",
cond: { $or: [
{ $ne: [ { $type: "$$item.x" }, "missing" ] },
{ $ne: [ { $type: "$$item.y" }, "missing" ]}
]}
}
}}
},
{ "$unwind": "$t" },
{ "$project": { pid: 1, x: "$t.x", y: "$t.y" } }
])
Related
I want to group by and count follow_user.tags.tag_id per record, so no matter how many times the same tag_id show up on the same record, it only counts as 1.
My database structure looks like this:
{
"external_userid" : "EXID1",
"follow_user" : [
{
"userid" : "USERID1",
"tags" : [
{
"tag_id" : "TAG1"
}
]
},
{
"userid" : "USERID2",
"tags" : [
{
"tag_id" : "TAG1"
},
{
"tag_id" : "TAG2"
}
]
}
]
},
{
"external_userid" : "EXID2",
"follow_user" : [
{
"userid" : "USERID1",
"tags" : [
{
"tag_id" : "TAG2"
}
]
}
]
}
Here's my query:
[
{ "$unwind": "$follow_user" }, { "$unwind": "$follow_user.tags" },
{ "$group" : { "_id" : { "follow_user᎐tags᎐tag_id" : "$follow_user.tags.tag_id" }, "COUNT(_id)" : { "$sum" : 1 } } },
{ "$project" : { "total" : "$COUNT(_id)", "tagId" : "$_id.follow_user᎐tags᎐tag_id", "_id" : 0 } }
]
What I expected:
{
"total" : 1,
"tagId" : "TAG1"
},
{
"total" : 2,
"tagId" : "TAG2"
}
What I get:
{
"total" : 2,
"tagId" : "TAG1"
},
{
"total" : 2,
"tagId" : "TAG2"
}
$set - Create a new field follow_user_tags.
1.1. $setUnion - To distinct the value from the Result 1.1.1.
1.1.1. $reduce - Add the value of follow_user.tags.tag_id into array.
$unwind - Deconstruct follow_user_tags array field to multiple documents.
$group - Group by follow_user_tags and perform total count via $sum.
$project - Decorate output document.
db.collection.aggregate([
{
$set: {
follow_user_tags: {
$setUnion: {
"$reduce": {
"input": "$follow_user.tags",
"initialValue": [],
"in": {
"$concatArrays": [
"$$value",
"$$this.tag_id"
]
}
}
}
}
}
},
{
$unwind: "$follow_user_tags"
},
{
$group: {
_id: "$follow_user_tags",
total: {
$sum: 1
}
}
},
{
$project: {
_id: 0,
tagId: "$_id",
total: 1
}
}
])
Sample Mongo Playground
I have a MongoDb collection as below :
Each document has a "bricks" array inside "currentVersion".
Now for few of Bricks array , there are some redundant brick ids.
I need to retrieve all the documents which are having redundant brick Ids.
{
name: "page1",
"currentVersion" : {
"bricks" : [
{
"id" : 1.0
},
{
"id" : 2.0
}
]
}
}
{
name: "page2",
"currentVersion" : {
"bricks" : [
{
"id" : 13.0
},
{
"id" : 13.0
}
]
}
}
{
name: "page3",
"currentVersion" : {
"bricks" : [
{
"id" : 20.0
},
{
"id" : 30.0
}
]
}
}
{
"name" : "page4",
"currentVersion" : {
"bricks" : [
{
"id" : 50.0
},
{
"id" : 50.0
},
{
"id" : 70.0
}
]
}
}
My approach is below:
I'm creating two arrays - 'origValuesSize' and 'allValuesSize'
If these two values are not equal , then that is the document with redundant values
Can you please suggest how I could filter the documents based on above condition.
Need help in the below piece of code
db.pages.find({},db.aggregate(
[
{ $project: {
origValuesSize: { $size: {$concatArrays : [ "$currentVersion.bricks.id" ]} },
allValuesSize: {$size: {$setUnion : [ "$currentVersion.bricks.id" ]}},
$filter: --> Need Help Here to compare the above two sizes and filter accordingly
}}
]
))
Edit : 1
My Expected Result would be as below:
I need to display only those Documents which are having redundant brick Ids in "bricks" array .
{
name: "page2",
"currentVersion" : {
"bricks" : [
{
"id" : 13.0
},
{
"id" : 13.0
}
]
}
}
{
"name" : "page4",
"currentVersion" : {
"bricks" : [
{
"id" : 50.0
},
{
"id" : 50.0
},
{
"id" : 70.0
}
]
}
}
Thanks in advance,
You can use expression with aggregation operator in match stage,
$size to get total elements in currentVersion.bricks
$setUnion to get total uniue ids in currentVersion.bricks.id, and $size tot get total elements
$ne to match both size should not same
db.pages.aggregate([
{
$match: {
$expr: {
$ne: [
{ $size: "$currentVersion.bricks" },
{ $size: { $setUnion: "$currentVersion.bricks.id" } }
]
}
}
}
])
Playground
MongoDB +3.2,
$project to show required fields
get size of both and match $eq conditions, it will return true when both same otherwise false
$match isEqual is false
db.pages.aggregate([
{
$project: {
name: 1,
currentVersion: 1,
isEqual: {
$eq: [
{ $size: "$currentVersion.bricks" },
{ $size: { $setUnion: "$currentVersion.bricks.id" } }
]
}
}
},
{ $match: { isEqual: false } }
])
Playground
I have a huge dataset that needs to be updated quickly to reduce downtime for an application
I would like for documents that have example structure
{
"_id" : "5ed4a65efdb46e0001dc37ab",
"arr" : [
{
"oldProp" : "x"
},
{
"oldProp" : "y"
}
]
}
to be updated to
{
"_id" : "5ed4a65efdb46e0001dc37ab",
"arr" : [
{
"oldProp" : "x",
"newProp" : "x"
},
{
"oldProp" : "y",
"newProp" : "y"
}
]
}
Without using cursor!
db.getCollection('someCollection').update({x: { $type:"array"}}, [
{
$set: {
x: {
$map: {
input: "$x",
as: "entry",
in: {
$mergeObjects: [
"$$entry",
{
newProp: "$$entry.oldProp"
}
]
}
}
}
}
}],false, true)
I have json with the following structure
db.testCollection.insert(
{
"m_id": 2,
"sys_data":[
{"sattr":
{
"size": 2,
"d_data":
[
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"03/01/01"}
]
}
}
]
}
db.testCollection.insert(
{
"m_id": 2,
"sys_data":[
{"sattr":
{
"size": 2,
"d_data":
[
{"d_counter": 2,
"client_ip":"1.1.1.1",
"d_date":"02/01/01"}
]
}
}
]
}
I want to get the count where d_date ='02/01/01', So the output for above json is 3. (two from first json and one from the second)
Use the aggregation framework to get the count. This allows you to $unwind the deeply nested embedded documents (with the help of the dot notation), filter out the unwanted documents using the $match operator which is similar to the find() query and use the $sum accumulator operator in the $group pipeline to determine the count of the matching documents.
Something like this:
Populate test collection:
db.testCollection.insert([
{
"m_id" : 2,
"sys_data" : [
{
"sattr" : {
"size" : 2,
"d_data" : [
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
},
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
},
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "03/01/01"
}
]
}
}
]
},
{
"m_id" : 2,
"sys_data" : [
{
"sattr" : {
"size" : 2,
"d_data" : [
{
"d_counter" : 2,
"client_ip" : "1.1.1.1",
"d_date" : "02/01/01"
}
]
}
}
]
}])
Run the aggregation pipeline:
var pipeline = [
{
"$match": {
"sys_data.sattr.d_data.d_date" : "02/01/01"
}
},
{
"$unwind": "$sys_data"
},
{
"$unwind": "$sys_data.sattr.d_data"
},
{
"$match": {
"sys_data.sattr.d_data.d_date" : "02/01/01"
}
},
{
"$group": {
"_id": null,
"count": { "$sum": 1 }
}
}
];
db.testCollection.aggregate(pipeline);
Sample output:
/* 0 */
{
"result" : [
{
"_id" : null,
"count" : 3
}
],
"ok" : 1
}
Check out this docs from mongodb
db.testCollection.count( { d_date : "02/01/01" } )
Also, the above is equivalent to:
db.testCollection.find( { d_date: "02/01/01" } ).count()
If you feel slow performance check this question.
I have structure shown below:
{
field1: "somevalue",
name:"xtz",
nested_documents: [
{
x:1,
y:2,
info:[
{name:"sachin",value:"test"},
{name:"sachin", value:"test"}
]
},
{
x:1,
y:3,
info:[
{name:"sachin1",value:"test"},
{name:"sachin2", value:"test"}
]
},
{
x:4,
y:3,
info:[
{name:"sachin",value:"test"},
{name:"sachin", value:"test"}
]
}
]
}
I know that I can retrieve element present inside 1st array using below code:
db.test.find({"nested_documents.x": 1},{_id: 0, nested_documents: {$elemMatch: {x: 1}}}
But, I want to apply same logic for name attribute.
I want to retrieve only document that has name `sachin'.
What I have tries is shown below:
db.test.find({"nested_documents.info.name": "sachin"},
{_id: 0, 'nested_documents.info': {$elemMatch: {name: "sachin"}}});
But Mongo db says it does not support '.' operator inside projection :(. Is there any other way to do this?(Using command prompt or code)
Command to insert document is shown below:
db.test.insert( {
field1: "somevalue",
name:"xtz",
nested_documents: [
{
x:1,
y:2,
info:[
{name:"sachin",value:"test"},
{name:"sachin", value:"test"}
]
},
{
x:1,
y:3,
info:[
{name:"sachin1",value:"test"},
{name:"sachin2", value:"test"}
]
},
{
x:4,
y:3,
info:[
{name:"sachin",value:"test"},
{name:"sachin", value:"test"}
]
}
]
}
)
I am expecting output as:
{ "_id" : ObjectId("5142e0f153cd2aab3a3bae5b"),
"nested_documents" : [
{ "x" : 1, "y" : 2,
"info" : [
{ "name" : "sachin", "value" : "test" },
{ "name" : "sachin", "value" : "test" }
]
},
{ "x" : 4, "y" : 3,
"info" : [ { "name" : "sachin", "value" : "test" },
{ "name" : "sachin", "value" : "test" }
]
}
]
}
You would have to use aggregate() with a double $unwind, like this:
db.test.aggregate([
// filter for documents with x=1
// note: this will use an index, if defined on "nested_documents.x"
//{ $match: { "nested_documents.x": 1 } },
// reduce data to nested_documents, as other fields are not considered
{ $project: { nested_documents: 1 } },
// flatten the outer array
{ $unwind: "$nested_documents" },
// filter for nested_documents with x=1
// note that at this point nested_documents is no longer an array
//{ $match: { "nested_documents.x": 1 } },
// flatten the inner array
{ $unwind: "$nested_documents.info" },
// filter for nested_documents.info.name = "sachin"
// note that at this point nested_documents.info is no longer an array
{ $match: { "nested_documents.info.name": "sachin" } },
// format output: re-create inner array
{ $group: { _id: { id: "$_id",
nested_documents: {
x: "$nested_documents.x",
y: "$nested_documents.y"
}
},
info: { $push: "$nested_documents.info" } } },
{ $project: { "nested_documents.x": "$_id.nested_documents.x",
"nested_documents.y": "$_id.nested_documents.y",
"nested_documents.info": "$info" } },
// format output: re-create outer array
{ $group: { _id: "$_id.id", nested_documents: { $push: "$nested_documents" } } },
])
note: I put in as //comments the logic to filter for x=1 as you had in a previous example
and the result is:
{
"result" : [
{
"_id" : ObjectId("515d873457a0887a97cc8d19"),
"nested_documents" : [
{
"x" : 4,
"y" : 3,
"info" : [
{
"name" : "sachin",
"value" : "test"
},
{
"name" : "sachin",
"value" : "test"
}
]
},
{
"x" : 1,
"y" : 2,
"info" : [
{
"name" : "sachin",
"value" : "test"
},
{
"name" : "sachin",
"value" : "test"
}
]
}
]
}
],
"ok" : 1
}
For more information on aggregate, refer to http://docs.mongodb.org/manual/applications/aggregation/