I'm facing an issue while data fetching from database.
Here is my db collections structure:
Collection: customers
Document structure:
{
_id: ObjectId("5f4776ec5c5e60582a3cb3a4"),
name: "Karan",
phone: "98767867867"
}
Collection: merchants
Document structure:
{
_id: ObjectId("5f4777b45c5e60582a3cb3ab"),
business_name: "Myntra",
phone: "343434344"
}
Collection: transactions
Document structure:
{
"_id" : ObjectId("5f477cd6d6053560bf945280"),
"transact_from" : ObjectId("5f4776ec5c5e60582a3cb3a4"),
"transact_from_ref" : "users",
"transact_to" : ObjectId("5f4777b45c5e60582a3cb3ab"),
"transact_to_ref" : "merchants",
"transaction_amount" : 250,
},
{
"_id" : ObjectId("5f477cd6d6053560bf945280"),
"transact_from" : ObjectId("5f4777b45c5e60582a3cb3ab"),
"transact_from_ref" : "merchants",
"transact_to" : ObjectId("5f4776ec5c5e60582a3cb3a4"),
"transact_to_ref" : "users",
"transaction_amount" : 250,
}
You can see here that in transactions records, from and to user id can be of either merchants or customers collection id.
I have to fetch data by using concept of joins:
1. populate (in find query)
2. lookup (in aggregate query)
CASE 1: When I use .find() query with populate(), the result are getting as per my expectation because I defined "refPath" instead of "ref" in Model for populating
Result:
[
{
"_id" : ObjectId("5f477cd6d6053560bf945280"),
"transact_from" : "transact_to" : {
_id: ObjectId("5f4777b45c5e60582a3cb3ab"),
name: "Karan",
phone: "98767867867"
},
"transact_from_ref" : "users",
"transact_to" : {
_id: ObjectId("5f4777b45c5e60582a3cb3ab"),
business_name: "Myntra",
phone: "343434344"
},
"transact_to_ref" : "merchants",
"transaction_amount" : 250,
}
{
"_id" : ObjectId("5f477cd6d6053560bf945280"),
"transact_from" : {
_id: ObjectId("5f4777b45c5e60582a3cb3ab"),
business_name: "Myntra",
phone: "343434344"
},
"transact_from_ref" : "merchants",
"transact_to" : {
_id: ObjectId("5f4777b45c5e60582a3cb3ab"),
name: "Karan",
phone: "98767867867"
},
"transact_to_ref" : "users",
"transaction_amount" : 250,
}
]
Issue is Searching is not happening properly for joined fields: transact_from.name, transact_to.business_name.
CASE 2: When I use .aggregate() with lookup(), I'm unable to join collections dynamically. Is there any concept like "refPath" for lookup?
Hope you are understanding my issue. Please help me if anyone has a solution?
Related
Hello im learning MongoDB and my exercise is to update id in one of my collection to another id. I Have two collection
First collection "workers" :
{
"_id" : ObjectId("6224ec342c2d7202b9ad9af6"),
"id_worker" : 180,
"id_boss" : 100,
"hired" : ISODate("2005-02-20T00:00:00Z"),
"id_group" : 10
}
Second collection "groups":
{
"_id" : ObjectId("6224ebe12c2d7202b9ad9af1"),
"id_group" : 10,
"name" : "Administrators",
"addres" : "Example"
}
I would like to change an "_id" of my worker to its "identifiaction id" from collection "groups"
So my worker should look like:
{
"_id" : ObjectId("6224ec342c2d7202b9ad9af6"),
"id_worker" : 180,
"id_boss" : 100,
"hired" : ISODate("2005-02-20T00:00:00Z"),
"id_group" : ObjectId("6224ebe12c2d7202b9ad9af1")
}
I have query like this. When i use print it shows list of my workers with name of the group. How to change it to modify this collection? use update?
var workers = db.workers.find();
while (workers.hasNext()) {
worker = workers.next();
group = db.groups.findOne({"id_group": worker.id_group});
print(group.name);
}
Try this:
db.workers.aggregate([
{
$lookup: {
from: "groups",
localField: "id_group",
foreignField: "id_group",
as: "group"
}
},
{
$project: {
id_worker: 1,
id_boss: 1,
hired: 1,
id_group: { $first: "$group.name" }
}
}
])
Mongo Playground
Here's my Schema
var PositiveSchema = new mongoose.Schema({
schoolID: {
type: mongoose.Schema.Types.ObjectId, ref: 'School'
},
name: String,
restrictAwardTo: Object
})
Now restrictAwardTo saves the data in this format
"restrictAwardTo" : [
{
"_id" : "5c31907d908eb8404657cbf0",
"firstName" : "Admin 2a"
},
{
"_id" : "5c1a7677c98da061141475a8",
"firstName" : "Justice"
},
{
"_id" : "5c1a7677c98da061141475a9",
"firstName" : "Itik"
}
],
How can I search inside my document using one of the _id listed under restrictAwardTo? I tried the solutions given below
mongooseJS find docs with IDs in an array
mongoose query: find an object by id in an array but it returns empty.
in Robo3t db.getCollection('positives').find({ 'restrictAwardTo._id' : ObjectId('5c31907d908eb8404657cbf0') })
Update: In Robo3t, this query db.getCollection('positives').find({ 'restrictAwardTo._id' : {$in: ['5c1a7677c98da061141475a7']} }) works. Now I'm making it work for mongoose too.
Here's the mongoose that works for me:
Positive.find({ schoolID: mongoose.mongo.ObjectId(schoolID), "restrictAwardTo._id": { $in: [userID]} })
But I'm not entirely sure of the performance for large records.
You could go through this way.
Positive.findOne({'restrictAwardTo': {$elemMatch: {_id: userID}}},
(err,schoolInfo) => { });
and think you in advance for the help. I have recently started using mongoDB for some personal project and I'm interested in finding a better way to query my data.
My question is: I have the following collection:
{
"_id" : ObjectId("5dbd77f7a204d21119cfc758"),
"Toyota" : {
"Founder" : "Kiichiro Toyoda",
"Founded" : "28 August 1937",
"Subsidiaries" : [
"Lexus",
"Daihatsu",
"Subaru",
"Hino"
]
}
}
{
"_id" : ObjectId("5dbd78d3a204d21119cfc759"),
"Volkswagen" : {
"Founder" : "German Labour Front",
"Founded" : "28 May 1937",
"Subsidiaries" : [
"Audi",
"Volkswagen",
"Skoda",
"SEAT"
]
}
}
I want to get the object name for example here I want to return
[Toyota, Volkswagen]
I have use this method
var names = {}
db.cars.find().forEach(function(doc){Object.keys(doc).forEach(function(key){names[key]=1})});
names;
which gave me the following result:
{ "_id" : 1, "Toyota" : 1, "Volkswagen" : 1 }
however, is there a better way to get the same result and also to just return the names of the objects. Thank you.
I would suggest you to change the schema design to be something like:
{
_id: ...,
company: {
name: 'Volkswagen',
founder: ...,
subsidiaries: ...,
...<other fields>...
}
You can then use the aggregation framework to achieve a similar result:
> db.test.find()
{ "_id" : 0, "company" : { "name" : "Volkswagen", "founder" : "German Labour Front" } }
{ "_id" : 1, "company" : { "name" : "Toyota", "founder" : "Kiichiro Toyoda" } }
> db.test.aggregate([ {$group: {_id: null, companies: {$push: '$company.name'}}} ])
{ "_id" : null, "companies" : [ "Volkswagen", "Toyota" ] }
For more details, see:
Aggregation framework
$group
Accumulator operators
As a bonus, you can create an index on the company.name field, whereas you cannot create an index on varying field names like in your example.
I am new to MongoDB so this is probably a basic question (hopefully). I currently have 10 million records with 410 fields loaded in a mongodb collection like so:
{
"_id" : ObjectId("........"),
"AddressID" : 123455,
"IndividualId" : 1,
"personfirstname" : "FirstName",
"personmiddleinitial" : "M",
"personlastname" : "LastName",
"etc": "....."
}
I need to wrap all of this data into an embedded document like so:
{
"_id" : ObjectId("........"),
"data" : {
"AddressID" : 123455,
"IndividualId" : 1,
"personfirstname" : "FirstName",
"personmiddleinitial" : "M",
"personlastname" : "LastName",
"etc": "....."
}
I don't necessarily need to update this data in-place but that would be nice. If I need to export this data somehow specifying the new format and then re-import the new, updated data that is fine. Performing this via the MongoDB shell would be ideal.
As suggested by chridam within comments you can execute the following aggregation pipeline:
db.collectionName.aggregate([
{ $project: { _id: "$_id", data: "$$ROOT" } },
{ $out: "newCollectionName" }
]);
This way you have the _id field both at root level and in the data object. Thus, you can execute a massive update to unset the second one:
db.newCollectionName.updateMany(
{},
{ $unset: { "data._id": "" } }
);
Finally, you can drop the first collection and rename the second to restore the original name on the updated collection:
db.collectionName.drop();
db.newCollectionName.rename("collectionName");
This approach fully works within the database, avoiding fetching any of your 10 million documents.
You can simply do this in the shell with the following
db.test.find().forEach(function(doc){
doc = { _id: doc._id, data: doc };
delete doc.data._id;
db.test.save(doc);
});
For example, if we insert the following documents:
> db.test.insertMany([
... {
... _id: ObjectId("5a91af8908e17c5997e03b7e"),
... field1: false,
... field2: 0,
... field3: "No"
... },
... {
... _id: ObjectId("5a91afbc08e17c5997e03b7f"),
... field1: true,
... field2: 1,
... field3: "Yes"
... }])
{
"acknowledged" : true,
"insertedIds" : [
ObjectId("5a91af8908e17c5997e03b7e"),
ObjectId("5a91afbc08e17c5997e03b7f")
]
}
Then run:
db.test.find().forEach(function(doc){
doc = { _id: doc._id, data: doc };
delete doc.data._id;
db.test.save(doc);
});
Our documents now look like this:
> db.test.find().pretty()
{
"_id" : ObjectId("5a91af8908e17c5997e03b7e"),
"data" : {
"field1" : false,
"field2" : 0,
"field3" : "No"
}
}
{
"_id" : ObjectId("5a91afbc08e17c5997e03b7f"),
"data" : {
"field1" : true,
"field2" : 1,
"field3" : "Yes"
}
}
I have a collection (users) containing some documents like so :
{
_id: ObjectId("56d45406be05db4022be51f9"),
morecontent : ""
},
{
_id: ObjectId("56d45406be05db3021be32e3"),
morecontent : ""
}
I would like to create a new document for every entry in the user collection.
The documents would be created from a notification object like this :
{
type: 'alert',
msg: 'This is important'
}
The collection notifications should look something like this :
{
_id: ObjectId("56d45406be05db3021bf20a1"),
someoldcontent: "This was here before the request"
},
{
_id : ObjectId("56d45406be05db4022be20b1"),
user: ObjectId("56d45406be05db4022be51f9"),
type: 'alert',
msg: 'This is important'
},
{
_id : ObjectId("56d45406be05db3021be32e3"),
user: ObjectId("56d45406be05db3021be32e3"),
type: 'alert',
msg: 'This is important'
}
Is there any way to do this in a mongodb request?
Thanks to professor79 for helping me a lot with the question.
After lots of effort, the query was found. We used the aggregation framework in order to succeed.
The only 2 aggregations needed are $project and $out. The $out will automatically take care of the new _id in the notification document.
Given this collection named user :
{
_id: ObjectId("56d45406be05db4022be51f9"),
morecontent : ""
},
{
_id: ObjectId("56d45406be05db3021be32e3"),
morecontent : ""
}
We want to create a notification containing the same msg and type field for every document in the user collection.
Each notifications will be in the notifications collection an will have as a reference its corresponding userId.
Here is the query to achieve such a result :
db.user.aggregate([
{
$project: {
userId: '$_id',
type: { $literal: 'danger' },
msg: { $literal: 'This is a message' },
},
},
{ $out: 'notifications' },
])
The output in notifications collection:
{
"_id" : ObjectId("56d6112e197b4ea11a87de1a"),
"userId" : ObjectId("56d45406be05db4022be51f9"),
"type" : "danger",
"msg" : "This is a message"
},
{
"_id" : ObjectId("56d6112e197b4ea11a87de1b"),
"userId" : ObjectId("56d45406be05db3021be32e3"),
"type" : "danger",
"msg" : "This is a message"
}
As we have some chat to clarify problem:
the steps are needed to perform this operation server side:
first step - match > get users id
project ids to new documents as required
out -> store output in notification collection