I'm using MongoEngine in Python to work with my data model.
I have a data model which essentially looks like this as represented in BSON:
{
'id': ...
'revisions': [
{
'id': ...
'revision': 1,
'derivatives': [
{
'id': ...
'name': 'Derivative 1'
}
]
}
]
}
We'll call the outermost document the owner, all subdocuments in owner.revisions will be called revision, and all subdocuments in revision.derivatives will be called derivative.
I'm looking to $addToSet to the derivatives set inside a specific revision inside of a specific owner. If I had to write this in Python, it'd look like this:
def add_to_set(owner_id, revision_id, new_derivative):
for owner in owner_collection:
if owner.id == owner_id:
# found the right owner
for revision in owner.revisions:
if revision.id == revision_id:
# we've found the right revision in the right owner
# now append and get out
revision.derivatives.append(new_derivative)
return
How can I run this kind of query, selecting the right revision inside of the right owner and atomically appending to the inner derivatives collection on that revision?
Having a hard time figuring out how to get started with an update query like this.
The problem is with using save based on an older version of the object - it's not thread-safe.
You want to use update operator which atomically manipulates an object based on matched condition, rather than unilaterally overwrite an object (by _id) with an entire new object.
Example of unsafe "save":
Thread 1: reads object with field {_id:1, a:1}, increments a, saves {_id:1,a:2}
Thread 2: reads object with field {_id:1, a:1}, increments a, saves {_id:1,a:2}
Contrast this with "safe" update:
Thread 1: updates object matching {_id:1}using operator {$inc:{a:1}}
Thread 2: updates object matching {_id:1}using operator {$inc:{a:1}}
No matter what order the two threads execute in in the second case the value of a will be 2 (since it will be incremented by 1 twice on the server.
I'm guessing the same thing is happening with your object - the analogous update operation you want is $push operator.
Related
Apologies if this is a re-post, but I wasn't able to quite get the query I want from the mongodb documentation examples.
Here's my issue. I am unable execute in a single query to either update an array_field of an existing document or add a new document and initialize the array_field with an initial value.
I can use findOne() with some conditional logic, and probably solve this, but I would think mongodb has an implementation of this use case
Here's the code so far:
#data_json = JSON document to be added to collection
collection.update_one({"json_id":data_json["json_id"], "_dashbd_id_":dashboard_id},{{"$addToSet": {"array_field":keyword}},{"$setOnInsert":data_json}}, upsert=True)
I'm querying by the json_id, and _dashbd_id_ from my collection. If it exists, then I intend to add the "keyword" to the array_field. If it doesn't exist, create a new document as data_json which include array_field = [keyword]
Any hints and suggestions are appreciated!
If I understood you correctly you want to update values in Database only if they do not exist as well as create new documents with arrays in them. Okay there is a way in mongodb which I will mention in this reply. I think you should know few commands first that will help you achieve similar result (again there is a simple way just read on)
Let me start with the first part:
to update an element in an array you use dot notation to the index example:
db.collection_name.update({"_id": id}, {'$set': {"array_name.indexNumber": value}})
say we have the following document in collection name cars
db.cars.findOne():
{
_id: 1
name: EvolutionX
brand: Mitsubish
year: 2012
mods: [ turbo, headlights ]
}
Say in the above example we want to update headlights with rearlights we do the following (using mongoshell you can drop quotes in key names, Not when using the array index though):
db.cars.update({id:1}, {$set:{"mods.1":"rearlights"}})
1 is the index to headlights.
Note and be careful here that if you did not use index inside of an array like
db.cars.update({id:1}, {$set:{"mods":"rearlights"}})
this will overwrite the existing document _id:1 and it will lose all other attributes or fields inside the document so it will result in the follow:
db.cars.findOne():
{
_id: 1
mods: [ rearlights ]
}
Now, say we want to add an element tires to mods array you can use $push as:
db.collection_name.update({"_id": id}, {'$push': {"array_name": value}})
so it will be
db.cars.update({"_id":1}, {"$push":{"mods":"tires"}})
now say instead of updating mods array you want to remove "headlights". In this case you use $pop
db.cars.update({"_id":1}, {"$pop":{"mods":"headlights"}})
Now with that in mind. The easy way: in mongodb to add to array only if element does not exist you can use $addToSet. I love this operator because it will only add to array if the element does not exist. Here is how to use it:
db.cars.update({"_id":1}, {"$addToSet":{"mods":"headlights"}})
Now if headlights is in the array it will not be added, else it will be added to the end of array.
Okay that is the first part of the question. The second part which is initializing a document with an array. Okay there are two thoughts here: the first is you do not have to. using the addToSet you can create the array if it does not exist as (assuming _id 2 exist but without mods array):
db.cars.update({"_id":2}, {"$addToSet":{"mods":"bonnet"}})
This will create the array if document _id:2 exist. Assuming _id:3 does not exist you will have plug in a third attribute called upsert
db.cars.update({"_id":3}, {"$addToSet":{"mods":"headlights"}}, {upsert:true})
this will create a third document with array mods with headlights inside of it and _id:3. Note though no other attributes will be added only the _id and mods array
the second thought is when you insert a new document you insert it with empty mods array as mod:[]
I hope that helps
suppose your data_json ,dashboard_id and keyword contain following detail.
dashboard_id = ObjectId("5423200e6694ce357ad2a1ac")
keyword = "testingKeyword"
data_json =
{
"json_id":ObjectId("5423200e6694ce357ad2a1ac"),
"item":"EFG222",
"reorder":false,
}
if you execute below query
db.collection_name.update({"json_id":data_json["json_id"], "_dashbd_id_":dashboard_id},{{"$addToSet": {"array_field":keyword}},{ upsert=True})
than it will push keyword to array_field if document exist or it will insert new document with following detail as below.
{
"_id":ObjectId("5sdvsdv6sdv694ce357ad2a1ac"),
"json_id":ObjectId("5423200e6694ce357ad2a1ac"),
"dashboard_id": ObjectId("sddfb6694ce357ad2a1ac")
"item":"EFG222",
"reorder":false,
"array_field":
[
"testingKeyword"
]
}
I am new to Sails and Mongo Db. Currently I am trying to implement a CRUD Function using Sails where I want to save user details in Mongo db.In the model I have the following attributes
"id":{
type:'Integer',
min:100,
autoincrement:true
},
attributes: {
name:{
type:'String',
required:true,
unique:true
},
email_id:{
type:'EMAIL',
required:false,
unique:false
},
age:{
type:'Integer',
required:false,
unique:false
}
}
I want to ensure that the _id is overridden with my values starting from 100 and is auto incremented with each new entry. I am using the waterline model and when I call the Api in DHC, I get the following output
"name": "abc"
"age": 30
"email_id": "abc#gmail.com"
"id": "5587bb76ce83508409db1e57"
Here the Id given is the object Id.Can somebody tell me how to override the object id with an Integer starting from 100 and is auto incremented with every new value.
Attention: Mongo id should be unique as possible in order to scale well. The default ObjectId is consist of a timestamp, machine ID, process ID and a random incrementing value. Leaving it with only the latter would make it collision prone.
However, sometimes you badly want to prettify the never-ending ObjectID value (i.e. to be shown in the URL after encoding). Then, you should consider using an appropriate atomic increment strategy.
Overriding the _id example:
db.testSOF.insert({_id:"myUniqueValue", a:1, b:1})
Making an Auto-Incrementing Sequence:
Use Counters Collection: Basically a separated collection which keeps track the last number of the sequence. Personally, I have found it more cohesive to store the findAndModify function in the system.js collection, although it lacks version control's capabilities.
Optimistic Loop
Edit:
I've found an issue in which the owner of sails-mongo said:
MongoDb doesn't have an auto incrementing attribute because it doesn't
support it without doing some kind of manual sequence increment on a
separate collection or document. We don't currently do this in the
adapter but it could be added in the future or if someone wants to
submit a PR. We do something similar for sails-disk and sails-redis to
get support for autoIncremeting fields.
He mentions the first technique I added in this answer:
Use Counters Collection. In the same issue, lewins shows a workaround.
Is it possible to update a single document field in the Level3 array using $update and $elemMatch? I realize I cannot use the positional operator multiple times given this case and historically I've modified the Level2 nested document with the required deeper changes since these documents aren't very large. I'm hoping there is some syntax that makes it possible to update Level3 array documents using $elemMatch without knowing the position of the target document in the Level3 array or containing document in Level2.
Example:
db.collection.update({_id:'123', level2:{$elemMatch:{'level3.id':'bbb','level3.e1':'hij'}},{'level2.level3.createdDate':new Date()})
{
_id:'123',
f1:'abc',
f2:'def',
level2:[
{_
id:'aaa',
e1:'hij',
e2:'lmo'
level3:[
{
name:'foo',
type:'bar',
createdDate:'2013-3-28T05:18:00'
}]
},
{_
id:'bbb',
e1:'hij',
e2:'lmo'
level3:[
{
name:'foo2',
type:'bar2',
createdDate:'2013-3-28T05:19:00'
}]
}
]
}
There is no way to do this currently using a regular update operation for reasons you noted.
The only work around you can use at the moment is to add versioning to your document and use optimistic locking by reading the document, finding the appropriate elements to modify in your application, changing their values and then using an update that includes the version in the read document (so that if other thread updated the document between your query and your update you would not overwrite the changes but would have to reload the document and try again.
The versioning strategy would not have to be based on the entire document, you can version the first level array elements and then you would be able to update just the sub-array you were concerned with (via an update with $set).
I am trying to use upsert in MongoDB to update a single field in a document if found OR insert a whole new document with lots of fields. The problem is that it appears to me that MongoDB either replaces every field or inserts a subset of fields in its upsert operation, i.e. it can not insert more fields than it actually wants to update.
What I want to do is the following:
I query for a single unique value
If a document already exists, only a timestamp value (lets call it 'lastseen') is updated to a new value
If a document does not exists, I will add it with a long list of different key/value pairs that should remain static for the remainder of its lifespan.
Lets illustrate:
This example would from my understanding update the 'lastseen' date if 'name' is found, but if 'name' is not found it would only insert 'name' + 'lastseen'.
db.somecollection.update({name: "some name"},{ $set: {"lastseen": "2012-12-28"}}, {upsert:true})
If I added more fields (key/value pairs) to the second argument and drop the $set, then every field would be replaced on update, but would have the desired effect on insert. Is there anything like $insert or similar to perform operations only when inserting?
So it seems to me that I can only get one of the following:
The correct update behavior, but would insert a document with only a subset of the desired fields if document does not exist
The correct insert behavior, but would then overwrite all existing fields if document already exists
Are my understanding correct? If so, is this possible to solve with a single operation?
MongoDB 2.4 has $setOnInsert
db.somecollection.update(
{name: "some name"},
{
$set: {
"lastseen": "2012-12-28"
},
$setOnInsert: {
"firstseen": <TIMESTAMP> # set on insert, not on update
}
},
{upsert:true}
)
There is a feature request for this ( https://jira.mongodb.org/browse/SERVER-340 ) which is resolved in 2.3. Odd releases are actually dev releases so this will be in the 2.4 stable.
So there is no real way in the current stable versions to do this yet. I am afraid the only method is to actually do 3 conditional queries atm: 1 to check the row, then a if to either insert or update.
I suppose if you had real problems with lock here you could do this function with sole JS but that's evil however it would lock this update to a single thread.
I am trying to use upsert in MongoDB to update a single field in a document if found OR insert a whole new document with lots of fields. The problem is that it appears to me that MongoDB either replaces every field or inserts a subset of fields in its upsert operation, i.e. it can not insert more fields than it actually wants to update.
What I want to do is the following:
I query for a single unique value
If a document already exists, only a timestamp value (lets call it 'lastseen') is updated to a new value
If a document does not exists, I will add it with a long list of different key/value pairs that should remain static for the remainder of its lifespan.
Lets illustrate:
This example would from my understanding update the 'lastseen' date if 'name' is found, but if 'name' is not found it would only insert 'name' + 'lastseen'.
db.somecollection.update({name: "some name"},{ $set: {"lastseen": "2012-12-28"}}, {upsert:true})
If I added more fields (key/value pairs) to the second argument and drop the $set, then every field would be replaced on update, but would have the desired effect on insert. Is there anything like $insert or similar to perform operations only when inserting?
So it seems to me that I can only get one of the following:
The correct update behavior, but would insert a document with only a subset of the desired fields if document does not exist
The correct insert behavior, but would then overwrite all existing fields if document already exists
Are my understanding correct? If so, is this possible to solve with a single operation?
MongoDB 2.4 has $setOnInsert
db.somecollection.update(
{name: "some name"},
{
$set: {
"lastseen": "2012-12-28"
},
$setOnInsert: {
"firstseen": <TIMESTAMP> # set on insert, not on update
}
},
{upsert:true}
)
There is a feature request for this ( https://jira.mongodb.org/browse/SERVER-340 ) which is resolved in 2.3. Odd releases are actually dev releases so this will be in the 2.4 stable.
So there is no real way in the current stable versions to do this yet. I am afraid the only method is to actually do 3 conditional queries atm: 1 to check the row, then a if to either insert or update.
I suppose if you had real problems with lock here you could do this function with sole JS but that's evil however it would lock this update to a single thread.