Here's a sample document of my Mongo DB structure:
{ _id: 1
records: [{n: "Name", v: "Kevin"},
{n: "Age", v: "100"},
...,
{n: "Field25", v: "Value25"} ]
}
To search on all documents having Name of "Kevin" and an Age of "100", I'm using $all with $elemMatch. I need to use $elemMatch's for an exact sub-document match of n: "Name" and v: "Kevin", as well as $all since I'm querying on an array.
db.collection.find({"records" : { $all: [
{$elemMatch: {n: "Name", v: "Kevin"},
{$elemMatch: {n: "Age", v: "100"}
]}})
However, the $all operator is inefficient when the first $elemMatch argument is non-selective, i.e. there are many documents that match this field.
The Mongo Docs elaborate:
In the current release, queries that use the $all operator must scan
all the documents that match the first element in the query array. As
a result, even with an index to support the query, the operation may
be long running, particularly when the first element in the array is
not very selective.
Is there a better alternative for my queries?
I would suggest an radical change to your structure, and this would simplify the queries. Sorry if this change is not possible, but without more data I do not see any problem with that:
{ _id: 1
records: [{"Name":"Kevin",
"Age":"100",
...,
"Field25":"Value25"} ]
}
And the query:
db.collection.find("records":{$elemMatch:{Name:"Kevin","Age":100}})
This will return all the objects that have a record (assuming there are more records, for example if they are students of a class) matching all of the conditions mentioned.
But in case you want to have just a document per _id, forget about the records array:
{ _id: 1,
"Name":"Kevin",
"Age":"100",
...,
"Field25":"Value25"} ]
}
and the query:
db.collection.find({Name:"Kevin","Age":100})
Hope this helps.
I think this should be the best solution.
db.collection.find({},
{
records: {
$filter: {
input: "$records",
as: "record",
cond: {
$or: [
{
$eq: [
"$$record.v",
"Kevin"
]
},
{
$eq: [
"$$record.v",
"100"
]
}
]
}
}
}
})
solution link: https://mongoplayground.net/p/aQAw0cG9Ipm
Related
This seems like a simple question but I couldn't find an answer. This is very easy to do in SQL.
I want to query mongodb by searching for a value that is a combination of two fields. For example I want to search for documents where fieldA + fieldB == ID01
I tried the following but it doesn't work.
collection.find({{$concat:[$fieldA, $fieldB]}: 'ID01'})
You can try using aggregate framework -
db.collection.aggregate(
[
{ $project: { newField: { $concat: [ "$fieldA", "$fieldB" ] }, fieldA: 1, fieldB: 1 } },
{ $match: { newField: 'ID01' } }
]
)
So, this question stems from another question I asked ( MongoDB update with conditional addToSet/pull )
Essentially in that SO question, I found that I couldn't use the typical $in syntax if my array is an array of objects.
Typical syntax for conditionally updating a field:
$cond: {
$in: [{element}, $arr], // Need to provide the element Exactly, including _id if one exists
$setDifference: [$arr, [{element}]]
$concactArrays: [$arr, [{element}]]
}
The reason $in doesn't work is that if _id is auto-generated, you won't be able to match it for the $in or the $setDifference. The solution I arrived at uses $filter, but is fragile in that the array needs to at least be initialed a default [] (otherwise $size will error out on a null array)
On to my question.
Is there a way to make $in or $elemMatch work in this scenario?
I couldn't get the following to work (says $elemMatch is an unrecognized expression:
$cond: {
$in: [arr: {$elemMatch: {element}}, $arr],
Full Example
Mongo Object
{_id: 1, message: 'text1'}
{_id: 2, message: 'text2', reactions: [{_id: autoGen1, user: 'bob', reaction:'👍'},
{_id: autoGen2, user: 'bob', reaction:'👎'}
{_id: autoGen3, user: 'meg', reaction:'😵'}]}
Update Query
db.collection.update(
{},
[
{
$set: {
reactions: {
$cond: [
{
$in: [
{
reactions: { $elemMatch: {user: "bob", reaction: "good"}}
},
"$reactions"
]
},
"1",
"2"
]
}
}
}
])
https://mongoplayground.net/p/Ig0TKMAMNkI
I am using Azure Cosmos DB's API for MongoDB with Pymongo. My goal is to filter array inside array and return only filtered results. Aggregation query works for the first array, but returns full inside array after using map, filter operations. Please find Reproducible Example in Mongo Playground: https://mongoplayground.net/p/zS8A7zDMrmK
Current query use $project to filter and return result by selected Options but still returns every object in Discount_Price although query has additional filter to check if it has specific Sales_Week value.
Let me know in comments if my question is clear, many thanks for all possible help and suggestions.
It seemed you troubled in filtering nested array.
options:{
$filter: {
input: {
$map: {
input: "$Sales_Options",
as: 's',
in: {
City: "$$s.City",
Country: "$$s.Country",
Discount_Price: {
$filter: {
input: "$$s.Discount_Price",
as: "d",
cond: {
$in: ["$$d.Sales_Week", [2, 7]]
}
}
}
}
}
},
as: 'pair',
cond: {
$and: [{
$in: [
'$$pair.Country',
[
'UK'
]
]
},
{
$in: [
'$$pair.City',
[
'London'
]
]
}
]
}
}
}
Working Mongo plaground. If you need price1, you can use $project in next stage.
Note : If you follow the projection form upper stage use 1 or 0 which is good practice.
I'd steer you towards the $unwind operator and everything becomes a lot simpler:
db.collection.aggregate([
{$match: {"Store": "AB"}},
{$unwind: "$Sales_Options"},
{$unwind: "$Sales_Options.Discount_Price"},
{$match: {"Sales_Options.Country": {$in: [ "UK" ]},
"Sales_Options.City": {$in: [ "London" ]},
"Sales_Options.Discount_Price.Sales_Week": {$in: [ 2, 7 ]}
}
}
])
Now just $project the fields as appropriate for your output.
I have a below mongo document stored
{
"Field1": "ABC",
"Field2": [
{ "Field3": "ABC1","Field4": [ {"id": "123" }, { "id" : "234" }, { "id":"345" }] },
{ "Field3": "ABC2","Field4": [ {"id": "123" }, { "id" : "234" }, { "id":"345" }] },
{ "Field3": "ABC3","Field4": [{ "id":"345" }] },
]
}
from the above, I want to fetch the subdocuments which is having id "123"
ie.
{
"Field3" : "ABC1",
"Field4" : [ { "id": "123"} ]
} ,
{
"Field3" : "ABC2",
"Field4" : [ { "id": "123"} ]
}
1. Java way
A. use Mongo find method to get the ABC document from Mongo DB
B. for Loop to Iterate the Field2 Json Array
C. Again for Loop to Iterate over Field4 Json Array
D. Inside the nested for loop I've if condition to Match id value to "123"
E. Store the Matching subdocument into List
2. Mongo Way
A. Use Aggregation query to get the desired output from DB.No Loops and conditions in the Java side.
B. Aggregation Query below stages
I) $Match - match the ABC document
II) $unwind - Field2
III) $unwind - Field4
IV) $match - Match the with id ( value is "123")
V) $group - group the document based on Field3 (based on "ABC1" or "ABC2")
VI) execute aggregation and return results
Both are working good and returning proper results.
Question is which one is the better to follow and why ? I used the aggregation in restful service get method, So executing aggregation queries 1000 or more times in parallel will cause any performance problems?
With Aggregation, the whole query is executed as a single process on the MongoDB server - the application program will get the results cursor from the server.
With Java program also you are getting a cursor from the database server as input to the processing in the application. The response cursor from the server is going to be larger set of data and will use more network bandwidth. And then there is processing in the application program, and this adds more steps to complete the query.
I think the aggregation option is a better choice - as all the processing (the initial match and filtering the array) happens on the database server as a single process.
Also, note the aggregation query steps you had posted can be done in an efficient way. Instead of multiple stages (2, 3, 4 and 5) you can do those operations in a two stages - use a $project with $map on the outer array and then $filter on the inner array and then $filter the outer array.
The aggregation:
db.test.aggregate( [
{
$addFields: {
Field2: {
$map: {
input: "$Field2",
as: "fld2",
in: {
Field3: "$$fld2.Field3",
Field4: {
$filter: {
input: "$$fld2.Field4",
as: "fld4",
cond: { $eq: [ "$$fld4.id", "123" ] }
}
}
}
}
}
}
},
{
$addFields: {
Field2: {
$filter: {
input: "$Field2",
as: "f2",
cond: { $gt: [ { $size: "$$f2.Field4" }, 0 ] }
}
}
}
},
] )
The second way is probably better because it returns a smaller result from the datastore; shlepping bits over the wire is expensive.
I would like to use mongo projections in order to return less data to my application. I would like to know if it's possible.
Example:
user: {
id: 123,
some_list: [{x:1, y:2}, {x:3, y:4}],
other_list: [{x:5, y:2}, {x:3, y:4}]
}
Given a query for user_id = 123 and some 'projection filter' like user.some_list.x = 1 and user.other_list.x = 1 is it possible to achieve the given result?
user: {
id: 123,
some_list: [{x:1, y:2}],
other_list: []
}
The ideia is to make mongo work a little more and retrieve less data to the application. In some cases, we are discarding 80% of the elements of the collections at the application's side. So, it would be better not returning then at all.
Questions:
Is it possible?
How can I achieve this. $elemMatch doesn't seem to help me. I'm trying something with unwind, but not getting there
If it's possible, can this projection filtering benefit from a index on user.some_list.x for example? Or not at all once the user was already found by its id?
Thank you.
What you can do in MongoDB v3.0 is this:
db.collection.aggregate({
$match: {
"user.id": 123
}
}, {
$redact: {
$cond: {
if: {
$or: [ // those are the conditions for when to include a (sub-)document
"$user", // if it contains a "user" field (as is the case when we're on the top level
"$some_list", // if it contains a "some_list" field (would be the case for the "user" sub-document)
"$other_list", // the same here for the "other_list" field
{ $eq: [ "$x", 1 ] } // and lastly, when we're looking at the innermost sub-documents, we only want to include items where "x" is equal to 1
]
},
then: "$$DESCEND", // descend into sub-document
else: "$$PRUNE" // drop sub-document
}
}
})
Depending on your data setup what you could also do to simplify this query a little is to say: Include everything that does not have a "x" field or if it is present that it needs to be equal to 1 like so:
$redact: {
$cond: {
if: {
$eq: [ { "$ifNull": [ "$x", 1 ] }, 1 ] // we only want to include items where "x" is equal to 1 or where "x" does not exist
},
then: "$$DESCEND", // descend into sub-document
else: "$$PRUNE" // drop sub-document
}
}
The index you suggested won't do anything for the $redact stage. You can benefit from it, however, if you change the $match stage at the start to get rid of all documents which don't match anyway like so:
$match: {
"user.id": 123,
"user.some_list.x": 1 // this will use your index
}
Very possible.
With findOne, the query is the first argument and the projection is the second. In Node/Javascript (similar to bash):
db.collections('users').findOne( {
id = 123
}, {
other_list: 0
} )
Will return the who'll object without the other_list field. OR you could specify { some_list: 1 } as the projection and returned will be ONLY the _id and some_list
$filter is your friend here. Below produces the output you seek. Experiment with changing the $eq fields and target values to see more or less items in the array get picked up. Note how we $project the new fields (some_list and other_list) "on top of" the old ones, essentially replacing them with the filtered versions.
db.foo.aggregate([
{$match: {"user.id": 123}}
,{$project: { "user.some_list": { $filter: {
input: "$user.some_list",
as: "z",
cond: {$eq: [ "$$z.x", 1 ]}
}},
"user.other_list": { $filter: {
input: "$user.other_list",
as: "z",
cond: {$eq: [ "$$z.x", 1 ]}
}}
}}
]);