Here's a snapshot of my document:
I would like to know how can i retrieve only the documents where ANY of the breakout items has a property named source.
I tried the following: db.getCollection('receipts').find({"sizeBreakout.packBreackout.breakout.source":{"$exists":true}})
but a empty result is being returned always... why?!? what is the correct syntax for this query?!?
Edit1:
Attached File: https://drive.google.com/file/d/0B2zKseaQl2gnVlFWZFlaRzloMDg/view?usp=sharing
Well, if you have MongoDB 3.2, you can use this simple query.
MongoDB version 3.2+
db.docs.find({$filter: {input: "$sizeBreakout.packBreackout.breakout",
as: breakout,
cond:{$exists:{"$$breakout.source": true}}}})
MongoDB version < 3.2
db.docs.aggregate([
{$unwind: "$sizeBreakout.packBreakout.breakout"},
{$match: {"sizeBreakout.packBreakout.breakout.source":{$exists: true}}}
])
This query will find all documents where sizeBreakout.packBreackout.breakout.source exists.
Related
I am trying to use MongoDB aggregation in a Meteor application. I am using Meteor 1.11, which uses MongoDB 4.2.8. I built and tested my aggregation pipeline with Robo 3T and also tested it with the Mongo shell. It works fine in those contexts, but I can't get it working in a Meteor method.
I narrowed the problem down to the match stage, which is first in the pipeline. When running in a Meteor method, this stage appears to match all documents in the collection. When run in Robo 3T, it returns 4 documents. Here's how I run it in a Meteor method:
MyCollection
.rawCollection()
.aggregate([{$match: {vocabularyName: "IPF", type: "T"}}])
.toArray()
.then(docs => {
console.log(`got ${docs.length} docs`);
console.log(docs[0]);
console.log(docs[1]);
})
.catch(error => console.log(`error ${error} in db_methods`));
Every document in MyCollection contains a "type" field with varying values; most (but not all) contain a "vocabularyName" field. When I run this in a Meteor method, the returned array size matches the total document count and the first two results documents contain no vocabularyName field at all, and "type: 'S'". Which does not make any sense.
On a guess, I also tried this pipeline, but it made no difference:
[ {"$match": {"$and": [{ "vocabularyName": "IPF"}, {"type": "T"}]}} ]
(* MongoDB shell version: 2.6.12 *)
I have a collection logs and a collection users. logs has a field userId which represents the user who sent the log. users has a field _id.
I use Robo 3T to undertake queries to data in remote server.
Now, I'm interested in the logs whose url is /subscribe, and I want to see those user information behind. So I write the following query:
db.getCollection('logs').find({ "url" : "/subscribe" }).aggregate({
$lookup:{
from:"users",
localField:"userId",
foreignField:"_id",
as:"logs_users"
}
})
But I get an error:
Does anyone know how to solve this?
Edit 1: I got a new error:
You should not but use aggregate after the find, instead, use aggregate directly and use $match operator to find the documents and then $lookup
db.getCollection('logs').aggregate([
{
$match:{"url" : "/subscribe"}
},
{
$lookup:{
from:"users",
localField:"userId",
foreignField:"_id",
as:"logs_users"
}
}
])
Update: To use $lookup your MongoDB version should be equal or greater than 3.2, as this operator(joins) don't work in older versions of MongoDB
I am trying a fairly complex aggregate command on two collections involving $lookup pipeline. This normally works just fine on simple aggregation as long as index is set on foreignField.
But my $lookup is more complex as the indexed field is not just a normal Int64 field but actually an array of Int64. When doing a simple find(), it is easy to verify using explain() that the index is being used. But explaining the aggregate pipeline does not explain whether index is being used in the $lookup pipeline. All my timing tests seem to indicate that the index is not being used. MongoDB version is 3.6.2. Db compatibility is set to 3.6.
As I said earlier, I am not using simple foreignField lookup but the 3.6-specific pipeline + $match + $expr...
Could using pipeline be showstopper for the index? Does anyone have any deep experience with the new $lookup pipeline syntax and/or the index on an array field?
Examples
Either of the following works fine and if explained, shows that index on followers is being used.
db.col1.find({followers: {$eq : 823778}})
db.col1.find({followers: {$in : [823778]}})
But the following one does not seem to make use of the index on followers [there are more steps in the pipeline, stripped for readability].
db.col2.aggregate([
{$match:{field: "123"}},
{$lookup:{
from: "col1",
let : {follower : "$follower"},
pipeline: [{
$match: {
$expr: {
$or: [
{ $eq : ["$follower", "$$follower"] },
{ $in : ["$$follower", "$followers"]}
]
}
}
}],
as: "followers_all"
}
}])
This is a missing feature which is going to part of 3.8 version.
Currently eq matches in lookup sub pipeline are optimised to use indexes.
Refer jira fixed in 3.7.1 ( dev version).
Also, this may be relevant as well for non-multi key indexes.
In find operation fields can be excluded, but what if I want to do a find then a sort and just after then the projection. Do you know any trick, operation for it?
doc: fields {Object}, the fields to return in the query. Object of fields to include or exclude (not both), {‘a’:1}
You can run a usual find query with conditions, projections, and sort. I think you want to sort on a field that you don't want to project. But don't worry about that, you can sort on that field even after not projecting it.
If you explicitly select projection of sorting field as "0", then you won't be able to perform that find query.
//This query will work
db.collection.find(
{_id:'someId'},
{'someField':1})
.sort('someOtherField':1)
//This query won't work
db.collection.find(
{_id:'someId'},
{'someField':1,'someOtherField':0})
.sort('someOtherField':1)
However, if you still don't get required results, look into the MongoDB Aggregation Framework!
Here is the sample query for aggregation according to your requirement
db.collection.aggregate([
{$match: {_id:'someId'}},
{$sort: {someField:1}},
{$project: {_id:1,someOtherField:1}},
])
I'm unable to use maxTimeMS parameter with Mongo 2.6 and Pymongo 2.7.1
As per the documentation on this page Official Mongodb Aggregation Page the aggregation method should return a Cursor object. However, when I run the query locally on a mongod instance (2.6+) with pymongo 2.7.1, I get a dict object!
In [14]: obj = coll.aggregate({'$group': {'_id': '$l', 'n': {'$sum':
1}}})
In [15]: type(obj) Out[15]: dict
Can anyone help me understand what is happening here?
Yes, you can use maxTimeMS with pymongo aggregation.
c.foo.bar.aggregate([], maxTimeMS=1000)
{u'ok': 1.0, u'result': []}
If you want a cursor:
for result in c.foo.bar.aggregate([], cursor={}, maxTimeMS=1000):
... print result
The aggregate command didn't support cursors before MongoDB 2.6 so it had to be added as an option to avoid breaking existing applications.
This is covered in the driver documentation where it is described that in order to return a cursor, you need to specify the arguments in addition to the pipeline in your .aggregate() method:
cursor = coll.aggregate([{'$group': { '_id': '$l', 'n': {'$sum': 1} }}],cursor={})
Note that the returned object here is a CommandCursor and not a cursor.
This is because various modifiers such as .limit() and .skip() and other options do not apply in the context of an aggregation result. As such $maxTimeMS is not a valid option for this type of cursor.
In addition, it would not do what you think it will even where valid. The reason being that the "cursor" execution is only counted "after" the "aggregation pipeline" execution is complete, so in this case, just fetching the results.
Look at the .currentOp() and .killOp() implementations for other ways to control long running aggregation tasks.