(* MongoDB shell version: 2.6.12 *)
I have a collection logs and a collection users. logs has a field userId which represents the user who sent the log. users has a field _id.
I use Robo 3T to undertake queries to data in remote server.
Now, I'm interested in the logs whose url is /subscribe, and I want to see those user information behind. So I write the following query:
db.getCollection('logs').find({ "url" : "/subscribe" }).aggregate({
$lookup:{
from:"users",
localField:"userId",
foreignField:"_id",
as:"logs_users"
}
})
But I get an error:
Does anyone know how to solve this?
Edit 1: I got a new error:
You should not but use aggregate after the find, instead, use aggregate directly and use $match operator to find the documents and then $lookup
db.getCollection('logs').aggregate([
{
$match:{"url" : "/subscribe"}
},
{
$lookup:{
from:"users",
localField:"userId",
foreignField:"_id",
as:"logs_users"
}
}
])
Update: To use $lookup your MongoDB version should be equal or greater than 3.2, as this operator(joins) don't work in older versions of MongoDB
Related
I am trying to use MongoDB aggregation in a Meteor application. I am using Meteor 1.11, which uses MongoDB 4.2.8. I built and tested my aggregation pipeline with Robo 3T and also tested it with the Mongo shell. It works fine in those contexts, but I can't get it working in a Meteor method.
I narrowed the problem down to the match stage, which is first in the pipeline. When running in a Meteor method, this stage appears to match all documents in the collection. When run in Robo 3T, it returns 4 documents. Here's how I run it in a Meteor method:
MyCollection
.rawCollection()
.aggregate([{$match: {vocabularyName: "IPF", type: "T"}}])
.toArray()
.then(docs => {
console.log(`got ${docs.length} docs`);
console.log(docs[0]);
console.log(docs[1]);
})
.catch(error => console.log(`error ${error} in db_methods`));
Every document in MyCollection contains a "type" field with varying values; most (but not all) contain a "vocabularyName" field. When I run this in a Meteor method, the returned array size matches the total document count and the first two results documents contain no vocabularyName field at all, and "type: 'S'". Which does not make any sense.
On a guess, I also tried this pipeline, but it made no difference:
[ {"$match": {"$and": [{ "vocabularyName": "IPF"}, {"type": "T"}]}} ]
I am trying a fairly complex aggregate command on two collections involving $lookup pipeline. This normally works just fine on simple aggregation as long as index is set on foreignField.
But my $lookup is more complex as the indexed field is not just a normal Int64 field but actually an array of Int64. When doing a simple find(), it is easy to verify using explain() that the index is being used. But explaining the aggregate pipeline does not explain whether index is being used in the $lookup pipeline. All my timing tests seem to indicate that the index is not being used. MongoDB version is 3.6.2. Db compatibility is set to 3.6.
As I said earlier, I am not using simple foreignField lookup but the 3.6-specific pipeline + $match + $expr...
Could using pipeline be showstopper for the index? Does anyone have any deep experience with the new $lookup pipeline syntax and/or the index on an array field?
Examples
Either of the following works fine and if explained, shows that index on followers is being used.
db.col1.find({followers: {$eq : 823778}})
db.col1.find({followers: {$in : [823778]}})
But the following one does not seem to make use of the index on followers [there are more steps in the pipeline, stripped for readability].
db.col2.aggregate([
{$match:{field: "123"}},
{$lookup:{
from: "col1",
let : {follower : "$follower"},
pipeline: [{
$match: {
$expr: {
$or: [
{ $eq : ["$follower", "$$follower"] },
{ $in : ["$$follower", "$followers"]}
]
}
}
}],
as: "followers_all"
}
}])
This is a missing feature which is going to part of 3.8 version.
Currently eq matches in lookup sub pipeline are optimised to use indexes.
Refer jira fixed in 3.7.1 ( dev version).
Also, this may be relevant as well for non-multi key indexes.
I am implementing search functionality using Elasticsearch.
I receive "username" set returned by Elasticsearch after which I need to query a collection in MongoDB for latest comment of each user in the "username" set.
Question: Lets say I receive ~100 usernames everytime I query Elasticsearch what would be the fastest way to query MongoDB to get the latest comment of each user. Is querying MongoDB 100 times in a for loop using .findOne() the only option?
(Note - Because latest comment of a user changes very often, I dont want to store it in Elasticsearch as that will trigger retrieve-change-reindex process for the entire document far too frequently)
This answer assumes following schema for your mongo db stored in comments db.
{
"_id" : ObjectId("5788b71180036a1613ac0e34"),
"username": "abc",
"comment": "Best"
}
assuming usernames is the list of users you get from elasticsearch, you can perform following aggregate:
a =[
{$match: {"username":{'$in':usernames}}},
{$sort:{_id:-1}},
{
$group:
{
_id: "$username",
latestcomment: { $first: "$comment" }
}
}
]
db.comments.aggregate(a)
You can try this..
db.foo.find().sort({_id:1}).limit(100);
The 1 will sort ascending (old to new) and -1 will sort descending (new to old.)
Here's a snapshot of my document:
I would like to know how can i retrieve only the documents where ANY of the breakout items has a property named source.
I tried the following: db.getCollection('receipts').find({"sizeBreakout.packBreackout.breakout.source":{"$exists":true}})
but a empty result is being returned always... why?!? what is the correct syntax for this query?!?
Edit1:
Attached File: https://drive.google.com/file/d/0B2zKseaQl2gnVlFWZFlaRzloMDg/view?usp=sharing
Well, if you have MongoDB 3.2, you can use this simple query.
MongoDB version 3.2+
db.docs.find({$filter: {input: "$sizeBreakout.packBreackout.breakout",
as: breakout,
cond:{$exists:{"$$breakout.source": true}}}})
MongoDB version < 3.2
db.docs.aggregate([
{$unwind: "$sizeBreakout.packBreakout.breakout"},
{$match: {"sizeBreakout.packBreakout.breakout.source":{$exists: true}}}
])
This query will find all documents where sizeBreakout.packBreackout.breakout.source exists.
I need to make a query using aggregate in the MongoDB, but as the meteor does not support'm trying to use the extension server-aggregation.
I'm using the code on the server like this:
var result = aggregates('publicadores',[{$match: {_id : _id}},{ $unwind: '$relatorios' },{ $sort: {'relatorios.mes': -1 }}]);
The consultation is being done correctly, but when passing data by Publish Meteor is returning the following error:
Exception from sub MvPSGj5bf2jHFsRng Error: Publish function returned an array of non-Cursors
Does anyone know how to resolve this error, or some other way to apply the aggregate in Meteor?
You can try https://atmospherejs.com/meteorhacks/aggregate packages for MongoDB