I need to find collections based nested on values.But my collection having dynamic values . See below code. In which image name keys are dynamic ( _DSC9691.jpg , _DSC9514.JPG ) and " key1 " is dynamic items. Now I need to find collection based on component, material, Subtype
{
"_id" : ObjectId("5ce2df8498f10b276cb466c4"),
"num" : "1",
"lat" : "39.941436099965",
"lon" : "-86.0691700063581",
"images" : {
"_DSC9691.jpg" : {
"key1" : {
"component" : "Sleeve",
"condition" : "",
"sub_type" : {
"Auto Sleeve" : true
},
"material" : "",
"misc" : ""
}
}
}}
{
"_id" : ObjectId("5ce2df8498f10b276cb466c7"),
"num" : "4",
"lat" : "39.9413828961847",
"lon" : "-86.0715084495015",
"images" : {
"_DSC9554.JPG" : {
},
"_DSC9514.JPG" : {
},
"_DSC9622.JPG" : {
}
}}
#Nagendran you won't be able to perform those operations because the nested document that you want to watch isn't named properly. I suggest you to rename that field using a common name and try to use the code bellow. Also remember to not use blank spaces on field names like "Auto Sleeve".
Object:
{
"_id" : ObjectId("5ce2df8498f10b276cb466c7"),
"num" : "4",
"lat" : "39.9413828961847",
"lon" : "-86.0715084495015",
"images" : [
{
"name" : "some name",
"key":
{
"component" : "Sleeve",
"condition" : "",
"sub_type" : {
"Auto_Sleeve" : true
},
"material" : "",
"misc" : ""
},
},
{
"name" : "some name 2",
"key":
{
"component" : "Sleeve 2",
"condition" : "",
"sub_type" : {
"Auto_Sleeve" : true
},
"material" : "",
"misc" : ""
},
},
]
}
Query:
db.collection.find({
"images.key.sub_type.Auto_Sleeve": true
})
If you want you can use aggregation framework to filter inside the "images" nested document.
To get a litle bit more information you can access:
https://docs.mongodb.com/manual/aggregation/
https://docs.mongodb.com/manual/tutorial/query-documents/
https://www.mongodb.com/blog/post/6-rules-of-thumb-for-mongodb-schema-design-part-1
https://university.mongodb.com/
I am trying to aggregate documents in a collection to a nested graph. Below is the sample data from comments store application I am working with -
Posts Collection:
{
"_id" : "1",
"text" : "hey",
}
{
"_id" : "2",
"text" : "hello",
"replyTo" : "1"
}
{
"_id" : "3",
"text" : "What's up?",
"replyTo" : "2"
}
{
"_id" : "4",
"text" : "How are you",
"replyTo" : "1"
}
{
"_id" : "5",
"text" : "Knock, knock!",
}
The result I am expecting should look like :
{
"_id": "1",
"text": "hey",
"replies": [
{
"_id": "2",
"text": "hello",
"replyTo": "1",
"replies": [
{
"_id": "3",
"text": "What's up?",
"replyTo": "2"
}
]
},
{
"_id": "4",
"text": "How are you",
"replyTo": "1"
}
]
}
{
"_id" : "5",
"text" : "Knock, knock!",
}
I tried $graphLookup which is processing documents recursively but the results is a flat array which is not what I expect -
db.posts.aggregate(
[
{
"$graphLookup" : {
"from" : "posts",
"startWith" : "$_id",
"connectFromField" : "_id",
"connectToField" : "replyTo",
"as" : "replies"
}
}
],
{
"allowDiskUse" : false
}
);
Result:
{
"_id" : "1",
"text" : "hey",
"slug_path" : "1",
"replies" : [
{
"_id" : "2",
"text" : "hello",
"slug_path" : "1/2",
"replyTo" : "1"
},
{
"_id" : "4",
"text" : "How are you",
"slug_path" : "1/4",
"replyTo" : "1"
},
{
"_id" : "3",
"text" : "what's up?",
"slug_path" : "1/2/3",
"replyTo" : "2"
}
]
}
{
"_id" : "2",
"text" : "hello",
"slug_path" : "1/2",
"replyTo" : "1",
"replies" : [
{
"_id" : "3",
"text" : "what's up?",
"slug_path" : "1/2/3",
"replyTo" : "2"
}
]
}
{
"_id" : "4",
"text" : "How are you",
"slug_path" : "1/4",
"replyTo" : "1",
"replies" : [
]
}
{
"_id" : "3",
"text" : "what's up?",
"slug_path" : "1/2/3",
"replyTo" : "2",
"replies" : [
]
}
{
"_id" : "5",
"text" : "Knock, knock!",
"replies" : [
]`
}
Is there a way I can achieve the type of aggregation I am expecting?
Thanks
Davinder
I ended up ditching $graphLookup as it can only give 1-level parent-child relationships and can't recursively create a tree like structure as results. I was able to create tree like structure completely in Java and program is running fast from performance perspective.
I have a situation where I have got one result from aggregation where I am getting data in this format.
{
"_id" : ObjectId("5a42432d69cbfed9a410e8ad"),
"bacId" : "BAC0023444",
"cardId" : "2",
"defaultCardOrder" : "2",
"alias" : "Finance",
"label" : "Finance",
"for" : "",
"cardTooltip" : {
"enable" : true,
"text" : ""
},
"dataBlocks" : [
{
"defaultBlockOrder" : "1",
"blockId" : "1",
"data" : "0"
},
{
"defaultBlockOrder" : "2",
"blockId" : "2",
"data" : "0"
},
{
"defaultBlockOrder" : "3",
"blockId" : "3",
"data" : "0"
}
],
"templateBlocks" : [
{
"blockId" : "1",
"label" : "Gross Profit",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"blockId" : "2",
"label" : "Profit Forecast",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"blockId" : "3",
"label" : "Resource Billing",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
}
]
},
{
"_id" : ObjectId("5a42432d69cbfed9a410e8ad"),
"bacId" : "BAC0023444",
"cardId" : "3",
"defaultCardOrder" : "3",
"alias" : "Staffing",
"label" : "Staffing",
"for" : "",
"cardTooltip" : {
"enable" : true,
"text" : ""
},
"dataBlocks" : [
{
"defaultBlockOrder" : "1",
"blockId" : "1",
"data" : "1212"
},
{
"defaultBlockOrder" : "2",
"blockId" : "2",
"data" : "1120"
},
{
"defaultBlockOrder" : "3",
"blockId" : "3",
"data" : "1200"
}
],
"templateBlocks" : [
{
"blockId" : "1",
"label" : "Staffing Planner",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"blockId" : "2",
"label" : "Baseline",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"blockId" : "3",
"label" : "Projected",
"quarter" : "",
"data" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
}
]
}
Now I want to compare the two array of objects for each row, here in this case its "dataBlocks" and "templateBlocks" based on "blockId" s and I want to get the result in the following format.
{
"_id" : ObjectId("5a42432d69cbfed9a410e8ad"),
"bacId" : "BAC0023444",
"cardId" : "2",
"defaultCardOrder" : "2",
"alias" : "Finance",
"label" : "Finance",
"for" : "",
"cardTooltip" : {
"enable" : true,
"text" : ""
},
"blocks" : [
{
"defaultBlockOrder" : "1",
"blockId" : "1",
"data" : "0",
"label" : "Gross Profit",
"quarter" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"defaultBlockOrder" : "2",
"blockId" : "2",
"data" : "0",
"label" : "Profit Forecast",
"quarter" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"defaultBlockOrder" : "3",
"blockId" : "3",
"data" : "0",
"label" : "Resource Billing",
"quarter" : "",
"dataType" : {
"typeId" : "2"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
}
]
},
{
"_id" : ObjectId("5a42432d69cbfed9a410e8ad"),
"bacId" : "BAC0023444",
"cardId" : "3",
"defaultCardOrder" : "3",
"alias" : "Staffing",
"label" : "Staffing",
"for" : "",
"cardTooltip" : {
"enable" : true,
"text" : ""
},
"dataBlocks" : [
{
"defaultBlockOrder" : "1",
"blockId" : "1",
"data" : "1212",
"label" : "Staffing Planner",
"quarter" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"defaultBlockOrder" : "2",
"blockId" : "2",
"data" : "1120",
"label" : "Baseline",
"quarter" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
},
{
"defaultBlockOrder" : "3",
"blockId" : "3",
"data" : "1200",
"label" : "Projected",
"quarter" : "",
"dataType" : {
"typeId" : "1"
},
"tooltip" : {
"enable" : true,
"text" : ""
}
}
]
}
Is it possible to get it done with mongodb ? I am using 3.4 and trying to achieve this using aggregation.
Thanks in advance.
You can try below aggregation in 3.6.
The query below iterates the dataBlocks array and merges the data block element with template block element. The template block is looked up using $indexofArray which locates the array index with matching block id and $arrayElemAt to access the element at the found index.
db.collection_name.aggregate([{"$addFields":{
"blocks":{
"$map":{
"input":"$dataBlocks",
"in":{
"$mergeObjects":[
"$$this",
{"$arrayElemAt":[
"$templateBlocks",
{"$indexOfArray":["$templateBlocks.blockId","$$this.blockId"]}
]
}
]
}
}
}
}}])
For 3.4, replace $mergeObjects with combination of $arrayToObject, $objectToArray and $concatArrays to merge the each array element from both arrays.
db.collection_name.aggregate([{"$addFields":{
"blocks":{
"$map":{
"input":"$dataBlocks",
"in":{
"$arrayToObject":{
"$concatArrays":[
{"$objectToArray":"$$this"},
{"$objectToArray":{
"$arrayElemAt":[
"$templateBlocks",
{"$indexOfArray":["$templateBlocks.blockId","$$this.blockId"]
}
]
}}
]
}
}
}
}
}}])
You can use project with exclusion as last stage to remove array fields from output.
{"$project":{"templateBlocks":0,"dataBlocks":0}}
The following query does the job:
db.merge.aggregate([
// unwind twice
{$unwind: "$templateBlocks"},
{$unwind: "$dataBlocks"},
// get rid of documents where dataBlocks.blockId and
// templateBlocks.blockId are not equal
{$redact: {$cond: [{
$eq: [
"$dataBlocks.blockId",
"$templateBlocks.blockId"
]
},
"$$KEEP",
"$$PRUNE"
]
}
},
// merge dataBlocks and templateBlocks into a single document
{$project: {
bacId: 1,
cardId: 1,
defaultCardOrder: 1,
alias: 1,
label: 1,
for: 1,
cardTooltip: 1,
dataBlocks: {
defaultBlockOrder: "$dataBlocks.defaultBlockOrder",
blockId: "$dataBlocks.blockId",
data: "$dataBlocks.data",
label: "$templateBlocks.label",
quarter: "$templateBlocks.quarter",
data: "$templateBlocks.data",
dataType: "$templateBlocks.dataType",
tooltip: "$templateBlocks.tooltip"
}
}
},
// group to put correspondent dataBlocks to an array
{$group: {
_id: {
_id: "$_id",
bacId: "$bacId",
cardId: "$cardId",
defaultCardOrder: "$defaultCardOrder",
alias: "$alias",
label: "$label",
for: "$for",
cardTooltip: "$cardTooltip"
},
dataBlocks: {$push: "$dataBlocks" }
}
},
// remove the unnecessary _id object
{$project: {
_id: "$_id._id",
bacId: "$_id.bacId",
cardId: "$_id.cardId",
defaultCardOrder: "$_id.defaultCardOrder",
alias: "$_id.alias",
label: "$_id.label",
for: "$_id.for",
cardTooltip: "$_id.cardTooltip",
dataBlocks: "$dataBlocks"
}
}
])
Take into account that performance depends of size of your data set as the query unwinds twice and it may produce significant amount of intermediate documents.
I'm using meteor(mongodb) as a backend and I'm trying to query a collection that holds the data this way:
{
"name" : "Some name",
"data" : [
{
"0" : { "type" : "textInput", "value" : "Text", },
"1" : { "type" : "textInput", "value" : "Text", },
"2" : { "type" : "userInput", "value" : {
"userIds" : [ ... "Some mongo objectIds" ... ],
},
},
"3" : { "type" : "textInput", "value" : "Some text", }
},
{
"0" : { "type" : "textInput", "value" : "some text", },
"1" : { "type" : "textInput", "value" : "some text", },
}
],
}
data field can hold any number of objects, each object is a map from number to object with type and value fields.
Specifically, I would like to find all the documents that has a userInput and hold a specific userId (mongo objectId). How can I do it with this data structure?
In this example I can look for "Some mongo objectIds" to find this document (first object of data, index 2).
In my application I need to edit a document... and while editing the document the current [unchanged] version should be still available to other users. For instance, when I start editing a document, I create a new copy of it in a separate collection, and when done I replace the current version with the new version of the temporary collection.
Let's assume the original document stored in collection users looks like this:
{
"_id" : ObjectId("53b986e2fe000000019a5a13"),
"name" : "Joe",
"birthDate" : "2080-12-11",
"publications" : [
{ "title" : "title 1", "description" : "description 1" },
{ "title" : "title 2", "description" : "description 2" }
]
}
Then, let's assume the document above is copied to another collection (e.g. users.wip) and modified like this:
{
"_id" : ObjectId("53b986e2fe000000019a5a13"),
"name" : "Joe",
"birthDate" : "1980-12-11",
"publications" : [
{ "title" : "bye bye", "description" : "Blah blah" },
{ "title" : "title 2", "description" : "description 2" },
{ "title" : "title 3", "description" : "description 3" }
]
}
How do I replace the whoe document? The problem is that when I try to update the original document with
{ "$set" : {
"name" : "Joe",
"birthDate" : "1980-12-11",
"publications" : [
{ "title" : "bye bye", "description" : "Blah blah" },
{ "title" : "title 2", "description" : "description 2" },
{ "title" : "title 3", "description" : "description 3" }
]
}}
I always get the following error message:
10150 Field name duplication not allowed with modifiers
That said, what's the correct way to update an entire document?
To replace the whole document, you don't use $set, but just provide the new doc to the update call:
db.test.update({"_id": ObjectId("53b986e2fe000000019a5a13")}, {
"_id" : ObjectId("53b986e2fe000000019a5a13"),
"name" : "Joe",
"birthDate" : "1980-12-11",
"publications" : [
{ "title" : "bye bye", "description" : "Blah blah" },
{ "title" : "title 2", "description" : "description 2" },
{ "title" : "title 3", "description" : "description 3" }
]
})
However, with the current 3.0 driver, it would be best to use replaceOne instead:
db.test.replaceOne({"_id": ObjectId("53b986e2fe000000019a5a13")}, {
"name" : "Joe",
"birthDate" : "1980-12-11",
"publications" : [
{ "title" : "bye bye", "description" : "Blah blah" },
{ "title" : "title 2", "description" : "description 2" },
{ "title" : "title 3", "description" : "description 3" }
]
})