I have the following model in mongo db:
User collection
{
_id:12345,
name:"Joe",
age:15,
}
Addresses collection
{
_id:7663,
userId:12345,
Street:"xyz",
number:"1235",
city:"New York",
state:"NY"
}
Now I want to get all the addresses of users above the age of 20. What I thought was to query all the ids of users above 20 and with the result of this query use the $in operator to find the addresses.
My question is, is there a way to turn this into one query? Is there a better way to query this?
(obs: this is just an example, with my problem I cannot embed addresses into users)
In Mongo shell you can use the result of one query in another. For example:
use database // the name of your database
db.coll1.find({_id:{$nin:db.coll2.distinct("coll1_id")}})
Here collection coll1 contains an _id field. Then you can check for any ids that are not in collection coll2's list of the coll1_id field. So this is a way to clean up two tables if you have records in coll1 which have no reference via the coll1_id field in coll2.
Another approach doing the same thing:
use database // the name of your database
temp = db.coll2.distinct("coll1_id");
db.coll1.find({_id:{$nin:temp}})
The first example does it in one command, the second does it in two, but the concept is the same. Using results from one query in another. Many different ways to do this. Also, the .toArray() method can be useful to create arrays if you're doing more than just using distinct().
Use the aggregation framework where the $lookup pipeline stage provides the functionality to join the two collections:
db.user.aggregate([
{ "$match": { "age": { "$gt": 20 } } },
{
"$lookup": {
"from": "addresses",
"localField": "_id",
"foreignField": "userId",
"as": "address"
}
}
])
The above will create a new array field called address (as specified in the $lookup as option) and this contains the matching documents from the from collection addresses. If the specified name already exists in the input document, the existing field is overwritten.
Related
I'm trying to perform an aggregation using $lookup between two fields of type ObjectId. I've already checked my data and it should be returning something. Also, the schemas indeed define the fields as ObjectId, so if I am not mistaken I should not need to convert the data type as I've seen in some posts. Here's my query:
db.workerlocationcontexts.aggregate([
{
$lookup: {
from: "LocationSensors",
localField: "sensor",
foreignField: "_id",
as: "test"
}
}
])
Here are my collections/sample data:
WorkerLocationContexts
{
"_id":{"$oid":"615676c885ccad55a493503b"},
"updatedAt":{"$date":"2021-10-01T02:47:36.207Z"},
"createdAt":{"$date":"2021-10-01T02:47:36.207Z"},
"sensor":{"$oid":"6181e5f83fca98374cf901fd"},
"worker":{"$oid":"6153dcfb58ad722c747eb42d"},
"__v":0
}
LocationSensor
{
"_id":{"$oid":"6181e5f83fca98374cf901fd"},
"name":"Location Sensor 1",
"description":"Location sensor for Location 2",
"location": "$oid":"6181df3b3fca98374cf901fb"},
"trackerType":"RFID",
"sensorType":"ENVIRONMENT",
"type":"LOCATION",
"__v":0
}
The result I'm getting are all my WorkerLocationContexts, but the field "test" returns as an empty array.
Can someone help me?
Thanks in advance!
from field into $lookup seems to be case sensitive so it is a common problem the collection name typed in that field.
One common problem is not add the final "s" while using mongoose. Os, as in this case, the use of capital letters, yo using from: locationsensors should works like this example
The from field has to be the same as the collection name, otherwise Mongo can't find the desired collection.
I am trying to get the email address from an patient table in mongoDB compass and I need to join it with an patientId in the orders collection that looks like patientId : "pat_XXXXXXX" to the ID in the Orders collection that looks like _id : ObjectId('XXXXXXXX'). The problem I have is when I do a $lookup function the array returns null.
{
"$lookup": {
"from": "patients",
"localField": "patientId",
"foreignField": "_id",
"as": "patients"
}}
Then when I tried to use the function $toObjectId to change the patientId to an Object like this { "addFields": { "userObjectId": { "$toObjectId": "patientId" }}}, I get the error Failed to optimize pipeline :: caused by :: Failed to parse objectId 'patientId' in $convert with no onError value: Invalid string length for parsing to OID, expected 24 but found 9. The patientId length is 24 characters after the pat_ part so I think that is what the problem is. Is there a way to join an Id with a prefix to an ObjectId?
I managed to fix the problem by doing the following,
Go to the Patients Collection and choose the aggregations section.
Click add stage and Choose the option $addFields, then create a variable patientId where you covert the Object Id to string and concatenate pat_ the front of the Id.
{patient_Id: {$concat : ["pat_", {$toString : "$_id"}]}}
Then add another stage and choose the option $lookup. This will connect the two collections of the patients and orders. This can be done using the following
{
from: 'orders',
localField: 'patient_Id',
foreignField: 'patientId',
as: 'orders'
}
Add another stage $unwind this will unpack the orders collection allowing you to use the variables there.
{
path: "$orders"
}
The main issue i'm facing is doing multi document joins. For example, when I try and run a query on orders by customer I can't find an easy way to join orders to customers on a customer id because in the orders object the customer id has _user$ appended to the beginning of the user ID and I don't know how to truncate that in an aggregation pipeline.
eg; In the order object, the customer is defined as _p_customer:"_User$8qXjk40eOd"
But in the user table the _id is just 8qXjk40eOd and therfore the default lookup function in charts cannot match the two.
Note: I'm using parse server and it stores pointers in the above mentioned way.
This is how the data is being stored in the mongoDB order collection.
customer field in the above image is the pointer to the _User collection
This is how the data is stored in the mongoDB _User collection.
My requirement is to write mongo query that find all users and their related orders count. I'm not able to do that because I don't how to remove _User$ from the order row and join it the _User via _id.
before lookup in aggregate substr the field you want to use in $lookup
db.collection.aggregate([
{
$addFields :{
nUser : { $substr: [ "$_p_customer", 6, -1 ] } // _User$ has 6 characters
}
},
{
$lookup :{
from:"usersCollection",
localField:"nUser",
foreignField : "_id",
as : "UserObject"
}
}
])
Kind of a complex one here and i'm pretty new to Mongo, so hopefully someone can help. I have a db of users. Each user has a state/province listed. I'm trying to create another collection of the total users in each state/province. Because users sign up pretty regularly, this will be an growing total i'm trying to generate and display on a map.
I'm able to query the database to find total number of users in a specific state, but i want to do this for all users and come out with a list of totals in all states/provinces and have a separate collection in the DB with all states/provinces listed and the field TOTAL to be dynamically populated with the count query of the other collection. But i'm not sure how to have a query be the result of a field in another collection.
used this to get users totals:
db.users.aggregate([
{"$group" : {_id:"$state", count:{$sum:1}}}
])
My main question is how to make the results of a query the value of a field in each corresponding record in another collection. Or if that's even possible.
Thanks for any help or guidance.
Looks like that On-Demand Materialized Views (just added on version 4.2 of MongoDB) should solve your problem!
You can create an On-Demand Materialized View using the $merge operator.
A possible definition of the Materialized View could be:
updateUsersLocationTotal = function() {
db.users.aggregate( [
{ $match: { <if you need to perform a match, like $state, otherwise remove it> } },
{ $group: { _id:"$state", users_quantity: { $sum: 1} } },
{ $merge: { into: "users_total", whenMatched: "replace" } }
] );
};
And then you perform updates just by calling updateUsersLocationTotal()
After that you can query the view just like a normal collection, using db.users_total.find() or db.users_total.aggregate().
This question already has an answer here:
How to perform a $text search on a 'joined' collection via $lookup?
(1 answer)
Closed 3 years ago.
Full text search in MongoDB seems to be a nice feature, especially when it comes to the need of high performance search and indexes.
However, I am wondering why a full text search is not allowed over more than one collection.
For example: I have a supplier and a products collection (every supplier has n products) I would really like to do a search over all products, but the name of the supplier should also be matched.
Right now I am doing this with a $lookup and then a match with a regular expression. This works, but its very slowly (500 - 800ms).
If I used a full text search with an index, this would increase performance significantly. But the $text-operator has to be the first stage in the aggregation pipeline, not allowing me to use a $lookup first.
(See restrictions-section: https://docs.mongodb.com/manual/reference/operator/query/text/#op._S_text)
Any ideas how I could increase the speed of a text search over multiple collections?
for someone who still looking for solution
db.getCollection('projects').aggregate([{
"$match": {
"$text": {"$search": query }
}}, {
"$lookup": {
"from": "users",
"localField": "uuid",
"foreignField": "uuid",
"as": "user"
}}, {
"$sort": {
"score": { "$meta": "textScore" }
}
}]);
where query is a text search string.
This query is for projects collection with key uuid, which refers to users collection with uuid key, all sorted by relevance.