So I have a collection that has over 1million documents clearly, I don't need to search 1000 or more documents as it should only be updating the latest document.
const nowplayingData = {"type":"S",
"station": req.params.stationname,
"song": data[1],
"artist": data[0],
"timeplay":npdate};
LNowPlaying.findOneAndUpdate( nowplayingData,
{ $addToSet: { history: [uuid] } },
{ upsert: true, sort: {_id:-1} },
function(err) {
if (err) {
console.log('ERROR when submitting round');
console.log(err);
}
});
I have added a sort on it, so that it is able to get the latest one first, but if the document is not in there and it's the first time the document is being added then the way the script is written will query all documents to find.
When really it only needs to look at the last say 100 documents or even better if timeplay is in the last 5 minutes.
You can do something like:
const nowplayingData = {
"type":"S","station": req.params.stationname, "song": data[1], "artist": data[0],
"timeplay":{$gte: beforeFiveMin}};
But this will create a new document almost every time...so you will need to maintain it...
Related
I'm Implementing Keyset pagination in a nestjs and mongodb project. And I'm trying to customize the original pagination solution. I want to make a condition if the startId document is the first document present in the collection.
This is the Code I'm trying. AND THE ISSUE WITH THIS CODE IS THAT IT RETURNS ALL THE DOCUMENTS WHATEVER THE ID YOU GIVE IN THE QUERY. I KNOW MY LOGIC COULD BE WRONG BUT ONE THING ABOUT I'M SURE AND THAT IS I'M WRITING THE SYNTAX, IS WRONG, AS THIS IS MY FIRST TIME EXPERIECE WITH MONGO QUERIES
async findAll( page?: number, documentsToSkip = 0, limitOfDocuments?: number,
startId?: string,
): Promise<userDocument[]> {
return await this.userModel
.find({
$cond: {
if: { $first: { _id: startId } },
then: { $gte: startId },
else: { $gt: startId },
},
})
.skip(documentsToSkip)
.limit(limitOfDocuments);
}
Explanation of the above code. For example I'm dealing with Users
1- If the startId(passed in query) is equal to the _id of the first document present in the user collection then the below then should be executed
then: { $gte: startId },
2- If the startId(passed in query) is not equal to the _id of the first document present in the user collection then the below else should be executed. Lets's say the pagination limit is set to 10 documents per page. And If I'm providing the id of the 11th document then else should be executed
else: { $gt: startId },
REASON FOR All THIS
The keyset solution present on the internet uses this
_id: { $gt: startId } and with this the document of the startedId is automatically skipped. SO I'M TRYING TO DO IF THE PAGINATION STARTS FROM THE FIRST DOCUMENT THEN THE FIRST DOCUMENT ITSELF SHOULD BE PRESENT AND VISIBLE. BUT THEN IF USER MOVES TO THE SECOND PAGE OF THE PAGINATION THE LAST DOCUMENT OF THE FIRST PAGE SHOULD NOT BE VISIBLE AT THE SECOND PAGE, AS THE LAST DOCUMENT ID BECOMES THE STARTING ID FOR THE SECOND PAGE DOCUMENTS
I've made the solution for my particular scenario. The issue was I hade to customize original pagination according to the first document present in the collection. But $first wasn't working for me. AS $first returns the first element of an Array and not the first document itself.
SOLUTION
1- To find the first document use findOne() method without any parameter. And it will simply return the first document of the collection.
let firstDocument: userDocument = await this.userModel.findOne();
2-For simple customization of the Pagination
if (startId == firstDocument._id) {
return await this.userModel
.find({
_id: { $gte: startId },
})
.skip(documentsToSkip)
.limit(limitOfDocuments);
} else {
return await this.userModel
.find({
_id: { $gt: startId },
})
.skip(documentsToSkip)
.limit(limitOfDocuments);
}
3- I had to make other changes as well so my original function now look different In case If someone want to know
async findAll(
page?: number,
limitOfDocuments?: number,
startId?: string,
): Promise<PaginatedUsersDto> {
let firstDocument: userDocument = await this.userModel.findOne();
let count = await this.userModel.count();
let totalPages = (count / limitOfDocuments).toFixed();
console.log('First Doucment in User Colelction', firstDocument);
if (startId == firstDocument._id) {
this.paginatedUsers = await this.userModel
.find({
_id: { $gte: startId },
})
.sort({ _id: 1 })
.skip((page - 1) * limitOfDocuments)
.limit(limitOfDocuments);
} else {
this.paginatedUsers = await this.userModel
.find({
_id: { $gt: startId },
})
.skip((page - 1) * limitOfDocuments)
.limit(limitOfDocuments);
}
let paginatedObj = {
paginatedUsers: this.paginatedUsers,
totalPages: totalPages,
count: count,
};
return paginatedObj;
}
This is my code which searches the whole collection and returns the documents that the value of their name fields are either Dexter or Prison Break or Breaking bad.
Why should $search be at the top of stages; otherwise, I will get an error. Plus, I read on MongoDB doc that "The $match stage that includes a $text must be the first stage in the pipeline."
Here
app.get('/', (req, res) => {
db.collection('subs')
.aggregate([
{ $match: { $text: { $search: 'honey' } } },
{ $match: { name: { $in: ['Dexter', 'Prison Break', 'Breaking Bad'] } } },
])
.toArray((err, result) => {
if (err) {
throw new err();
}
res.json({
length: result.length,
body: { result },
});
});
});
I suppose the second line which filters the documents based on their name should come first; in order to reduce time and get a quick result because in this case, MongoDB will not have to search the whole collection and just search a few documents and returns the result.
why is that? is there any way for optimization?
I think it's because text search is required a special "text" index. Moving the "$search" operator to the second place makes impossible to use this index.
I am reading all documents of a specific schema from Mongoose. Now in my program I am doing some modifications to the results I got from Mongoose over time. Something like this:
var model = mongoose.model("Doc", docSchema);
model.find(function(err, result){
// for each result do some modifications
});
How can I send all the results back to the database to be saved? Currently I am iterating the documents and doing a save() on every document. I think there must be a better way. But currently I only find information on updating documents IN the database without returning them. Or bulk updates which do the SAME to update to each document.
You can use update query with multi:true which update all documents in your db.
please find below reference code,
model.update({ "_id": id }, { $set: { "Key": "Value" } }, { multi: true }, function (err, records) {
if (err || !records) {
return res.json({ status: 500, message: "Unable to update documents." });
} else {
return res.json({ status: 200, message: "success" });
}
});
If you are trying to make the same change to each document in the results, you could do something like this:
model.update({ _id: { $in: results.map(doc=>doc._id) }}, { yourField: 'new value' }, { multi: true })
I have a DB for a forum with 3 collections: Threads, Posts, Comments.
I have a GET request to return an individual forum thread that populates each thread with user's posts, and each user post with any comments that were made on it which is working as shown below:
router.get('/:id', (req, res) => {
Threads
.findById(req.params.id)
.lean()
.populate({path: 'posts'})
.exec(function(err, docs){
var options = {
path: 'posts.comments',
model: 'comments'
};
if(err) return res.json(500);
Threads.populate(docs, options, function(err, thread){
res.json(thread);
})
})
})
When this GET request is made it will return a forum thread like so:
{
"_id": "5924ad549a08ed4e70a9c89f",
"title": "Testing Full Schemas",
"author": "Mongoose",
"content": "Schema Content",
"posts": [
{
"_id": "5924ad999a08ed4e70a9c8a0",
"content": "New Schema Post",
"user": "Mongodb",
"comments": [
{
"_id": "5924ae489a08ed4e70a9c8a1",
"comment": "New Schema Content",
"user": "Matt",
"likes": 0,
"created": "2017-05-25T12:41:58.319Z"
}
]
}
Now I need a GET request to return an array of ALL threads (router.get('/')) with each threads posts and comments to be populated. I tried to replace:
Threads
.findById(req.params.id)
with
Threads
.find(req.params.id)
but it is not working. Does anyone have an idea of how this could be accomplished?
To return all the threads, simply use find without any match condition in it.
Also, populate posts and 'posts.comment' in the find query itself, you don't need to do it in the callback of the find.
use population across multiple levels
**Try this:
Threads.find({})
.populate({
path:'posts',
populate :{
path : comments
}
})
.exec(function(err,docs){
//docs is the array of all the Threads, with posts and comments populated within it
})
Read Mongoose Documentation on Populate and Nested Population for detailed information. ( Search for Populating across multiple levels)
findById and findOne returns a single document, where find returns a cursor. Once you go through the cursor of find, you are at the end, and there are no more documents.
Using find query:-
ModelName.find({_id:req.params.id})
.populate({
path:'posts',
populate :{
path : comments
}
},(error,data)=>{
if(error){
res.json(error);
}
else{
res.json(data);
}
})
Using findById query:-
ModelName.findById(req.params.id)
.populate({
path:'posts',
populate :{
path : comments
}
},(error,data)=>{
if(error){
res.json(error);
}
else{
res.json(data);
}
})
Hi I'm trying to do a query using aggregation(mongodb version 2.2) to return a list of products in a certain store owned by a certain user. I think I have the first 2 steps correctly, but stuck on the next ones.
Data:
Here's the incomplete query I have:
collection.aggregate(
[ { '$match': { "username": test } },
{ '$unwind': '$stores'}
],
function(err, results) {
assert.equal(err, null);
console.log(results)
callback(results);
}
);
Can anyone advice on how to go about with the next few ones?