I work on debian 10(buster) with mongod ver. 4.2.17. I want to use TTL to remove older documents on collection. I do walk around and I'm sure that no typo on the query.
I followed dummy example offical mongo on this link, where create index on timestamp and expireAfterSeconds.
db.log_events.createIndex( { "createdAt": 1 }, { expireAfterSeconds: 30 } )
I Also added documents on collection with this
db.log_events.insert( {
"createdAt": new Date(),
"logEvent": 2,
"logMessage": "Success!"
} )
but still not work. I do read that the logs not immediately on this but after wait for some hours its still not deleted. Also on mongo logs, there's no TTL log appear. I do checking the indexes with db.log_events.getIndexes() and they do appear.
Any suggestion how to solve it?
I trying to find whats wrong with the mongo, then I tried to activate TTL thread with this reference
db.adminCommand({setParameter: 1, ttlMonitorEnabled: true});
Then I tried to using this suggestion command to check whether the thread are running or not with db.serverStatus().metrics.ttl and return as
{ "deletedDocuments": NumberLong(0), "passes": NumberLong(0) }
also using the db.adminCommand ({getParameter: 1, ttlMonitorEnabled: true}) and return was: true.
It's solved by re-install the mongo on the machine and when check the serverStatus() again it works like a charm. Still dont know whats the problem but reinstall fixing my issue.
Related
Since version 3.6 MongoDB requires the use of cursor or explain in aggregate queries. It's a breaking change so I have to modify my earlier code.
But when I add cursor or explain to my query, the request simply enters an endless loop and MongoDB never responds. It doesn't even seem to time out.
This simple aggregation just hangs the code and never responds:
db.collection('users').aggregate([{ $match: { username: 'admin' }}],
{ cursor: {} },
(err, docs) => {
console.log('Aggregation completed.');
});
I can replace { cursor: {} } with { explain: true } and the result is the same. It works perfectly under older MongoDB versions without this one parameter.
Without cursor or explain I get this error message:
The 'cursor' option is required, except for aggregate with the explain argument
I'm not the only one who ran into this:
https://github.com/nosqlclient/nosqlclient/issues/419
OK, this was a little tricky, but finally it works. Looks like there are some major breaking changes in MongoDB's Node.js driver which nobody bothered to tell me.
1. The Node.js MongoDB driver has to be upgraded. My current version is 3.0.7.
2. The way how MongoDB connects has been changed, breaking any old code. The client connection now returns a client object, not merely the db. It has to be handled differently. There is a SO answer explaining it perfectly:
db.collection is not a function when using MongoClient v3.0
3. Aggregations now return an AggregationCursor object, not an array of data. Instead of a callback now you have to iterate through it.
var cursor = collection.aggregate([ ... ],
{ cursor: { batchSize: 1 } });
cursor.each((err, docs) => {
...
});
So it seems you have to rewrite ALL your db operations after upgrading to MongoDB 3.6. Yay, thanks for all the extra work, MongoDB Team! Guess that's where I'm done with your product.
I am facing this issue with mongodb.
My code is something like this
for(loop) {
var cursorQuery = db.beacon_0000.aggregate([
{
$match: {
...
}
},
{
$project: {
...
}
},
{
$group: {
...
}
},
{
$sort: {
...
}
}
], {allowDiskUse: true} );
...
while(cursorQuery.hasNext()) {
var cursor = cursorQuery.next();
...
}
}
I run the above query via command and mongo shell as
$ mongo dbName file.js
After a while I get the cursor didn't exist on server error at line cursorQuery.hasNext().
In find query if I get this error, I can resolve by adding addOption(DBQuery.Option.noTimeout)
However this option does not seem to be available with aggregate
Please let me know how can I resolve or workaround this issue.
Just to provide additional update:
When say I use
var cursor = db.collection..aggregate([ ...], {allowDiskUse: true} ).addOption(DBQuery.Option.noTimeout)
I get this error
E QUERY TypeError: Object # has no method 'addOption'
However when say I use
var cursor = db.collection..find({...}, {...}).addOption(DBQuery.Option.noTimeout)
It works fine.
Checking the aggregate doc
https://docs.mongodb.com/v3.0/reference/method/db.collection.aggregate/
It says:
Returns:A cursor to the documents produced by the final stage of the aggregation pipeline operation
And then checking cursor doc
https://docs.mongodb.com/v3.0/reference/method/cursor.addOption/#cursor.addOption
There is no suggestion that aggregate cursor is different from find cursor and former does not support DBQuery.Option.noTimeout.
So is there a bug at mongodb for this. Any way to fix it or have a workaround.
Note mongodb version is 3.0
I had the same issue and solved it by setting the idle cursor timeout from default 10 minutes to 1 hour. This is configurable since mongodb 2.6.9. See:
https://jira.mongodb.org/browse/SERVER-8188
https://docs.mongodb.com/manual/reference/parameters/#param.cursorTimeoutMillis
The default cursor timeout is 600000 ms = 10 minutes. You can alter it in different ways:
on startup: mongod --setParameter cursorTimeoutMillis=<num>
or: mongos --setParameter cursorTimeoutMillis=<num>
or during operation, using the mongo shell: db.adminCommand({setParameter:1, cursorTimeoutMillis: <num>})
Mongos is not transferring the command to its mongod's of the cluster. Also the Primary does not replicate the command to its replicaSet members. Thus, you need to execute the command on every mongos and mongod where the query might run.
You have sort of answered this yourself.
Adding the option addOption(DBQuery.Option.noTimeout) will indeed fix the issue when using find because it stops the cursor from timing out and therefore it will exist when you try .hasNext()
However the cursor for aggregation does not have that option so you can't stop it from timing out unfortunately.
you can actually use : maxTimeMS
as it described in documentation:
Optional. Specifies a time limit in milliseconds for processing operations on a cursor. If you do not specify a value for maxTimeMS, operations will not time out. A value of 0 explicitly specifies the default unbounded behavior.
There is an option in mongodb documentation that you can set it no a non-negative number for the time that you want your cursor to be alive.
you can see the more detail on : documentation
There is extremely weird thing happening on my database. I have a query:
db.Tag.find({"word":"foo"})
this thing matches one object. it's nice.
Now, there's second query
db.Tag.find({$or: [{"word":"foo"}]})
and the second one does not give any results.
There's some kind of magic I obviously don't understand :( What is wrong in second query?
in theory, $or requires two or more parameters, so I can fake it with:
db.Tag.find({$or: [{"word":"foo"},{"word":"foo"}]})
but still, no results.
Your second query is perfectly fine, and it should work. Though the docs says, that $or performs logical operation on array of two or more expression, but it would work for single expression also.
Here's a sample that you can see, and try out, to get it to work: -
> db.col.insert({"foo": "Rohit"})
> db.col.insert({"foo": "Aman", "bar": "Rohit"})
>
> db.col.find({"foo": "Rohit"})
{ "_id" : ObjectId("50ed6bb1a401d9b4576417f7"), "foo" : "Rohit" }
> db.col.find({$or: [{"foo": "Rohit"}]})
{ "_id" : ObjectId("50ed6bb1a401d9b4576417f7"), "foo" : "Rohit" }
So, as you can see, both your query when used for my collection works fine. So, there is certainly something wrong somewhere else. Are you sure you have data in your collection?
Okaay, server admin installed mongodb from debian repo. Debian repo had 1.4.4 version of mongodb, aandd looks like $or is simply not yet supported out there :P
Ok, so I'm trying to roll out a small update to my site. One update includes querying upon a field that may or may not exist. This doesn't work as I want, so I decided to just make it so that the field always exists in my database. I used this line at the MongoDB shell:
> db.entries.update({Published: null},{$set: {Published: true}},false,true);
Now, I'm not fully understanding how this caused every entry object where Published is null to be deleted. I mean, it literally was deleted. I tried looking up some IDs, and .findOne will return null for them.
How does that line work? I thought it would take every entry where Published is null(doesn't exist), and set Published to true.
Reading about operator behavior is better than guessing operator behavior. Search for null is different from performing a check for existence.
MongoDB has a dedicated $exists operator:
http://www.mongodb.org/display/DOCS/Advanced+Queries#AdvancedQueries-%24exists
To be honest, I'm not sure why it now works with changes, or at least, why it deleted everything with that command.
My ending command looked like this:
db.entries.update({Published: {$exists: false},$atomic: true},{$set:{"Published":true}},false,true);
I thought it would take every entry where Published is null(doesn't exist), and set Published to true.
OK, so these are two different things.
Published is null:
{ Published : null, post : 'blah' }
Published does not exist:
{ post : 'blahblah' }
You may want to post this question over at the MongoDB user group (developers check it very often) at http://groups.google.com/group/mongodb-user
Updates do not delete documents. In fact, the update you ran does what you intended, for example, if you wanted y to always have a value:
> db.foo.insert({x:1})
> db.foo.insert({x:2})
> db.foo.insert({y:null})
> db.foo.insert({y:1})
> db.foo.update({y:null},{$set : {y:true}}, false, true)
> db.foo.find()
{ "_id" : ObjectId("4db02aabbe5a5418fb65d24c"), "y" : true }
{ "_id" : ObjectId("4db02aafbe5a5418fb65d24d"), "y" : 1 }
{ "_id" : ObjectId("4db02aa1be5a5418fb65d24a"), "x" : 1, "y" : true }
{ "_id" : ObjectId("4db02aa4be5a5418fb65d24b"), "x" : 2, "y" : true }
There must have been another operation that did the delete. There might be a record of it in the logs (or there might not... it depends how long it took). It's impossible to tell from the info here what caused the deletions, but the update isn't the culprit here.
I have an app using Mongoid on top of MongoDB and an update is failing silently.
The code looks like this:
# Are we getting a new attribute? Yes we are!
p "attr first name = #{#attributes['first_name']}"
if user.update_attributes!(#attributes)
u = User.find(params[:id]).to_json
end
No exception is thrown in this code. So I looked at my MongoDB log and constructed this query based on what mongo is trying to do:
db.users.update({ "_id": "4d5561276ce886c496000001" }, { $set: { "first_name": "Erinamodobo" } }, false);
Now this does not cause any exceptions but when I grab the record that was supposed to be updated with this query:
db.users.find({"email":"escharling#somecompany.com"})
I see that the "first_name" attribute has not been updated.
Any idea why this could be happening? Sounds like something stupid.
Thanks!
You need to find out what
user.update_attributes!(#attributes)
is actually doing. This could be an issue with Mongoid. When you configure Mongoid, you can set up a logger. There you should be able to see what the driver is writing to MongoDB, and that should help answer your question. The next best thing is to post your code to the Mongoid mailing list (http://groups.google.com/group/mongoid) where people who work with Mongoid all the time will probably know what's going on.
Updating Mongoid to the latest rc.7 fixed this issue
Enable "safe mode" on your operations.