Does Mongo automatically track a version which is incremented for each update and can be used for optimistic locking?
So something that would correspond to functionality described here in ElasticSearch http://www.elasticsearch.org/blog/elasticsearch-versioning-support/
No, MongoDB does not store versions of documents. There is only the "current" version of the document as far as the database API is concerned.
It would be necessary to create your own scheme if you wanted such a thing. You could use a version field in your document and use $inc for example to increment it as needed and also verify that the version value matched before applying an update.
Example:
db.myCollection.update(
{ _id: "abc123", _version: 5 },
{
$set: { 'fieldA' : 'some-value' },
$inc: { '_version' : 1 }
}
)
The above example would find a document with the specific _id and _version fields. If matched, a field called fieldA is set to a new value and the _version field is incremented by 1. If another update was attempted on the same document and version, it would fail as the _version would have been updated to 6.
Related
We need to cache records for a service with a terrible API.
This service provides us with API to query for data about our employees, but does not inform us whether employees are new or have been updated. Nor can we filter our queries to them for this information.
Our proposed solution to the problems this creates for us is to periodically (e.g. every 15 minutes) query all our employee data and upsert it into a Mongo database. Then, when we write to the MongoDb, we would like to include an additional property which indicates whether the record is new or whether the record has any changes since the last time it was upserted (obviously not including the field we are using for the timestamp).
The idea is, instead of querying the source directly, which we can't filter by such timestamps, we would instead query our cache which would include said timestamp and use it for a filter.
(Ideally, we'd like to write this in C# using the MongoDb driver, but more important right now is whether we can do this in an upsert call or whether we'd need to load all the records into memory, do comparisons, and then add the timestamps before upserting them....)
There might be a way of doing that, but how efficient that is, still needs to be seen. The update command in MongoDB can take an aggregation pipeline to perform an update operation. We can use the $addFields stage of MongoDB to add a new field denoting the update status, and we can use $function to compute its value. A short example is:
db.collection.update({
key: 1
},
[
{
"$addFields": {
changed: {
"$function": {
lang: "js",
"args": [
"$$ROOT",
{
"key": 1,
data: "somedata"
}
],
"body": "function(originalDoc, newDoc) { return JSON.stringify(originalDoc) !== JSON.stringify(newDoc) }"
}
}
}
}
],
{
upsert: true
})
Here's the playground link.
Some points to consider here, are:
If the order of fields in the old and new versions of the doc is not the same then JSON.stringify will fail.
The function specified in $function will run on the server-side, so ideally it needs to be lightweight. If there is a large number of users, that will get upserted, then it may or may not act as a bottleneck.
Unfortunately, I can't figure out how to turn the _id ObjectID, into a _id with a unique number generated by a number.
For example:
Right now the generation is like this
[
{
_id: 'adioj2ouro21jr9o3',
// ...
}
]
And we need to
[
{
id: 1,
// ...
}
]
The build-in mechanism in mongoDB to auto generate ObjectId() is a very good and easy way to have unique _id and also contain the insertion date which sometimes make troubleshooting easier.
You cannot replace having default _id key with id , but you can have both _id and id ...
However you can insert different document in _id instead of the default ObjectId().
if you want the _id to be number you can read max(_id) and insert new document with inc(max(_id)) but this is not scalalble solution since if your writes increase it can become a bottleneck at some point.
Finally it is recomended to leave the default ObjectId() as your auto generated _id ...
https://www.mongodb.com/basics/mongodb-auto-increment
I have not used mongoDB myself. It appears that mongoDB doesnt support auto increment like MySQL would.
Could you use javascript to add N+1 on the last id in the table and manually create the ID field as needed?
I have to add new value (or edit actual) company. I tried this code in console:
db.users.update({email: /debute/},{company: "test"})
But now, in database I have only _id and company. Why other values removed? Or how can I prevent of removing other values in future?
By default, the update function replaces the entire document with the document you pass. But you can also use it to only replace individual fields and leave the others alone by using the $set operator.
db.users.update({email: /debute/},{ $set: { company: "test"} } );
You might also want to specify a third argument { multi: true } to the update. Otherwise it will only update the first document it finds which matches the condition.
You need to use an update operator on order to update an existing document, your code replaces the document with the new document. Here is how to do an update:
db.users.update({email: /debute/},{$set: {company: "test"}})
Through the PHP problem when inserting stuff into MongoDB explained in the answer by Denver Matt in this question I created duplicate IDs in a dataset in MongoDB. Fixing the PHP code is easy, but to be able to still use my dataset I wonder:
Can I change the MongoId manually without breaking something? Or can I just reset this ID somehow into a new one?
The _id field of a document is immutable, as discussed in this documentation. Attempting to modify its value would result in an error/exception, as in:
> db.foo.drop()
> db.foo.insert({ _id: 1 })
> db.foo.update({ _id: 1 }, { $set: { _id: 3 }})
Mod on _id not allowed
> db.foo.find()
{ "_id" : 1 }
If you do need to alter the identifier of a document, you could fetch it, modify the _id value, and then re-persist the document using insert() or save(). insert() may be safer on the off chance that you new _id value conflicts and you're rather see a uniqueness error than overwrite the existing document (as save() would do). Afterwards, you'll need to go back and remove the original document.
Since you can't do all of this in a single atomic transaction, I would suggest the following order of operations:
findOne() existing document by its _id
Modify the returned document's _id property
insert() the modified document back into the collection
If the insert() succeeded, remove() the old document by the original _id
I want to know how to reference the returned document attributes
from find and use it within modify. E.x. :
var totalNoOfSubjects = 5;
db.people.findAndModify( {
query: { name: "Tom", state: "active", rating: { $gt: 10 } },
sort: { rating: 1 },
update: { $set: { average: <reference score value returned by find>/totalNoOfSubjects} }
} );
My understanding is that findAndModify locks the document, hence I want to perform the update in the modify using the attributes found in the find. This will make the operation
atomic.
I am wondering if this is supported by mongo.
No, you cannot refer to values in the found document during the update portion of a findAndModify. It's the same as update in this respect.
As such, you cannot do this atomically as you need to first fetch the document and then craft the update or findAndMondify to contain the value computed from your fetched doc.
See https://jira.mongodb.org/browse/SERVER-458 for one way this may be addressed in the future.
Atomicity is exactly the reason for findAndModify.
As stated in the docs, Mongo will find one or more documents (matching the query specified) modify one document (using the update specified). The whole process is atomic. Default implementation has Mongo returning the found document (in its unchanged state). This can be modified using the new option.