Concurrent update of array elements which are embedded documents in MongoDB - mongodb

I have documents like this one at collection x at MongoDB:
{
"_id" : ...
"attrs" : [
{
"key": "A1",
"type" : "T1",
"value" : "13"
},
{
"key": "A2",
"type" : "T2",
"value" : "14"
}
]
}
The A1 and A2 elements above are just examples: the attrs field may hold any number of array elements.
I'd need to access concurrently to the attrs array from several independent clients accessing to MongoDB. For example, considers two clients, one wanting to change the value of the element identified by key equal to "A1" to "80" and other wanting to change the value of the element identified by key equal to "A2" to "20". Is there any compact way of doing it using MongoDB operations?
It is important to note that:
Clients doesn't know the position of each element in the attr array, only the key of the element which value has to be modified.
Reading the whole attrs array in client space, searching the element to modify at client space, then updating attrs with the new array (in which the element to modify has been changed) would involve race conditions.
Clients also may add and remove elements in the array. Thus, doing a first search at MongoDB to locate the position of the element to modify, then update it using that particular position doesn't work in general, as elements could have been added/removed thus altering of the position previously found.

The process here is really quite simple, it only varies in where you want to "find or create" the elements in the array.
First, assuming the elements for each key are in place already, then the simple case is to query for the element and update with the index returned via the positional $ operator:
db.collection.update(
{
"_id": docId,
"attrs": { "$elemMatch": { "key": "A1", "type": "T1" } }
}
{ "$set": { "attrs.$.value": "20" }
)
That will only modify the element that is matched without affecting others.
In the second case where "find or create" is required and the particular key may not exist, then you use "two" update statements. But the Bulk Operations API allows you to do this in a single request to the server with a single response:
var bulk = db.collection.initializeOrderedBulkOp();
// Try to update where exists
bulk.find({
"_id": docId,
"attrs": { "$elemMatch": { "key": "A1", "type": "T2" } }
}).updateOne({
"$set": { "attrs.$.value": "30" }
});
// Try to add where does noes not exist
bulk.find({
"_id": docId,
"attrs": { "$not": { "$elemMatch": { "key": "A1", "type": "T2" } } }
}).updateOne({
"$push": { "attrs": { "key": "A1", "type": "T2", "value": "30" } }
});
bulk.execute();
The basic logic being that first the update attempt is made to match an element with the required values just as done before. The other condition tests for where the element is not found at all by reversing the match logic with $not.
In the case where the array element was not found then a new one is valid for addition via $push.
I should really add that since we are specifically looking for negative matches here it is always a good idea to match the "document" that you intend to update by some unique identifier such as the _id key. While possible with "multi" updates, you need to be careful about what you are doing.
So in the case of running the "find or create" process then element that was not matched is added to the array correctly, without interferring with other elements, also the previous update for an expected match is applied in the same way:
{
"_id" : ObjectId("55b570f339db998cde23369d"),
"attrs" : [
{
"key" : "A1",
"type" : "T1",
"value" : "20"
},
{
"key" : "A2",
"type" : "T2",
"value" : "14"
},
{
"key" : "A1",
"type" : "T2",
"value" : "30"
}
]
}
This is a simple pattern to follow, and of course the Bulk Operations here remove any overhead involved by sending and receiving multiple requests to and from the server. All of this hapily works without interferring with other elements that may or may not exist.
Aside from that, there are the extra benefits of keeping the data in an array for easy query and analysis as supported by the standard operators without the need to revert to JavaScript server processing in order to traverse the elements.

Related

Update an array item of Mongodb with $and query

Hi I am trying to increment the count of the matching requirement in an array. My sample collection looks like the following:
{
"_id": ObjectId("60760ba2e870fa518f2ae48b"),
"userId": "6075f7289822d94dca8066b4",
"requirements": [
{
"searchText": "zee5",
"planType": "basic",
"mode": "PRIVATE",
"count": 32.0
},
{
"searchText": "sony",
"planType": "standard",
"mode": "PUBLIC",
"count": 12.0
},
{
"searchText": "prime",
"planType": "premium",
"mode": "PRIVATE",
"count": 2
}
]
}
If a user searches for prime, with filter premium and PRIVATE, then the count of the last requirement should be updated. If he searches for prime, with filter standard and PRIVATE, then the new requirement will be inserted with count 1.
I am doing in two steps. First I fire an update with the following query and then if no update, I fire a push query with count 1:
db.getCollection('userProfile').update({ "$and" : [{ "requirements.searchText" : {$eq:"prime"}}, {"requirements.mode" : {$eq: "PUBLIC"}}, {"requirements.planType": {$eq: "standard"}}, { "userId" : "6075f7289822d94dca8066b4"}]}, {$inc: {"requirements.$.count" : 1}})
I was expecting that the above query will not update any requirement, since there is no exact match. Interestingly, it increments the count of the second requirement with (sony, standard, public). What is wrong with the query? How can I get it right?
Demo - with Update - https://mongoplayground.net/p/-ISXaAayxxv
Demo No update - https://mongoplayground.net/p/88bTj3lz7U_
Use $elemMatch to make sure all properties are present in the same object inside the array
The $elemMatch operator matches documents that contain an array field with at least one element that matches all the specified query criteria.
db.collection.update(
{
"requirements": {
$elemMatch: { "searchText": "prime","mode": "PUBLIC", "planType": "standard" }
},
"userId": "6075f7289822d94dca8066b4"
},
{ $inc: { "requirements.$.count": 1 } }
)
Problem -
Your current query will match any document with all these fields in
requirements array in any object, if they match 1 property in 1 index of the array and another match in the next index query will find the document valid.
"searchText": "prime",
"mode": "PUBLIC",
"planType": "standard"

pymongo db query with multiple conditions- $and $exists

An example document looks like this
{
"_id":ObjectId("562e7c594c12942f08fe4192"),
"Type": "f",
"runTime": ISODate("2016-12-21T13:34:00.000+0000"),
"data" : {
"PRICES SPOT" : [
{
"value" : 29.64,
"timeStamp" : ISODate("2016-12-21T23:00:00.000+0000")
},
{
"value" : 29.24,
"timeStamp" : ISODate("2016-12-22T00:00:00.000+0000")
},
{
"value" : 29.81,
"timeStamp" : ISODate("2016-12-22T01:00:00.000+0000")
},
{
"value" : 30.2,
"timeStamp" : ISODate("2016-12-22T02:00:00.000+0000")
},
{
"value" : 29.55,
"timeStamp" : ISODate("2016-12-22T03:00:00.000+0000")
}
]
}
}
My MongoDb has different Type of documents, I'd like to get a cursor for all of the documents that are from a time range that are of type: "f" but that actually exist. There are some documents in the database that broke the code I had previously(which did not check if PRICES SPOT existed).
I saw that I can use $and and $exists from the documentation. However, I am having trouble setting it up because of the range, and the nesting. I am using pyMongo as my python driver and also noticed here that I have to wrap the $and and $exists in quotes.
My code
def grab_forecast_cursor(self, model_dt_from, model_dt_till):
# create cursor with all items that actually exist
cursor = self._collection.find(
{
"$and":[
{'Type': 'f', 'runTime': {"$gte": model_dt_from, "$lte": model_dt_till}
['data']['PRICES SPOT': "$exists": true]}
]})
return cursor
This results in a Key Error it cannot find data. A sample document that has no PRICE SPOT looks exactly like the one I posted in the beginning, just without that respectively.
In short.. Can someone help me set up a query in which I can grab a cursor with all the documents of a certain type but that actually have respected contents nested in.
Update
I added a comma after the model_dt_till and have now a syntax error.
def grab_forecast_cursor(self, model_dt_from, model_dt_till):
# create cursor with all items that actually exist
cursor = self._collection.find(
{
"$and":[
{'Type': 'f', 'runTime': {"$gte": model_dt_from, "$lte": model_dt_till},
['data']['PRICES SPOT': "$exists": true]}
]})
return cursor
You're trying to use Python syntax to denote the path to a data structure, but the "database" want's it's syntax for the "key" using "dot notation":
cursor = self._collection.find({
"Type": "f",
"runTime": { "$gte": model_dt_from, "$lte": model_dt_till },
"data.PRICES SPOT.0": { "$exists": True }
})
You also don't need to write $and like that as ALL MongoDB query conditions are already AND expressions, and part of your statement was actually doing that anyway, so make it consistent.
Also the check for a "non-empty" array is 'data.PRICES SPOT.0' with the added bonus that not only do you know it "exists", but also that it has at least one item to process within it
Python and JavaScript are almost identical in terms of object/dict construction, so you really should be able to just follow the general documentation and the many samples here that are predominantly JavaScript.
I personally even try to notate answers here with valid JSON, so it could be picked up and "parsed" by users of any language. But here, python is just identical to what you could enter into the mongo shell. Except for True of course.
See "Dot Notation" for an overview of the syntax with more information at Query on Embedded / Nested Documents

Find result within Array of Objects and match email address field

I'm trying to match the emailAddress field and the page_slug. Currently I'm using the following which matches just the about page in the modularSequence:
db.getCollection('users').find({"modularSequence.page_slug": "about"}, {"modularSequence.$": 1 })
This works and returns:
{
"_id" : ObjectId("5740c631742da6e83389abb4"),
"modularSequence" : [
{
"page_id" : "1",
"sequence" : "m_1",
"category" : "headers",
"page_slug" : "about"
}
]
}
Which it half what I want. I'm looking to return the emailAddress field as well. I've tried using this but it returns everything and multiple modular elements:
db.getCollection('users').find({$and:[{"emailAddress": 'paul#example.com'}, {"modularSequence.page_slug": "about"}, {"modularSequence": {$elemMatch: {page_slug:'about'}}}]})
[
{
"emailAddress": "paul#example.com",
"modularSequence": [
{
"page_slug": "about",
"category": "headers",
"sequence": "m_1",
"page_id": "1"
},
{
"page_slug": "contact",
"category": "content",
"sequence": "m_4",
"page_id": "2"
}
]
}
]
How do I match both the emailAddress field and the modularSequence.page_slug - only return a result if both the email address matches and the page_slug?
Your $and array is including your field selection parameter as well. But you don't need to use $and here anyway as multiple query terms are implicitly ANDed by default, so you can simplify your query to:
db.users.find({"emailAddress": 'paul#example.com', "modularSequence.page_slug": "about"},
{"emailAddress": 1, "modularSequence.$": 1})
Which is your first query, but with an emailAddress field added to both the query and field selection parameters.
The first parameter of find is the query (which docs), and the second is the projection (which fields within those docs), so that's why those fields are there twice. The $ in the projection represents the modularSequence array element matched in the query.

How to access embedded documents in MongoTemplate when the key is an empty string?

{
"_id" : ObjectId("550add7ee0b4b54a3e7ad53c"),
"day" : "14-03-2015",
"node" : "2G",
"nodeName" : "BLR_SGSN",
"" : {
"A" : 905.84,
"B" : 261.34,
"C" : 2103.94,
"D" : 39.67
}
}
I have this as my data in mongo.
How do I get values of A,B,C,D. ??
You cannot query on this as the sub-document fields cannot be selected.
This can only be a result of a programming error doing something like this ( and probably trying to compute a key name in the process ):
db.collection.insert({
"": {
"A": 1,
"B": 2,
"C": 3
}
})
So you cannot get to sub-elements by standard query ways like:
db.collection.find({ ".A": 905.84 })
You can fix this by updating the documents in the collection affected in this way by giving them a proper key name. But it is of course this is an iterative process. Not sure how to fix this other than with JavaScript from the shell due to the naming problem but:
db.collection.find({ "": { "$exists": true } }).forEach(function(doc) {
if ( doc.hasOwnProperty("") ) {
doc.newprop = doc[""];
delete doc[""];
db.collection.update({ "_id": doc._id }, doc );
}
})
Then at least you can access things by the new "newprop" key ( or whatever you call it ):
db.collection.find({ "newprop.A": 905.84 })
And the same sort of thing will work in other drivers.
My advice here is "go and fix this" and find out the code that caused this key name to be blank in the first place.
There should be a bug report submitted to the MongoDB core project as none of the dirvers handle this well. I thought I could even use $rename here, but you can't.
So blank "" keys are a problem that needs to be fixed.

Updating Value of Array Element in MongoDB

I'd like to know how to update the "value" field of one of the elements identified by the "name" field in the array "array_of_stuff". For example, I want to update the value for "name_of_thing_1" to "new_value_of_thing_1". How can I do this ONLY using the second parameter (i.e. the update parameter) to the update command. I am re-using a class library written in-house I don't have control over the first argument to the update command (i.e. the query parameter). Is this possible?
{
"array_of_stuff": [
{
"name": "name_of_thing_1",
"value": "value_of_thing_1",
},
{
"name": "name_of_thing_2",
"value": "value_of_thing_2",
}
]
}
Thanks for your help!
You can update the value of a single item in an array (if you know its index) like this:
db.stuff.update(/* query ... */, {$set:{"arrayname.<index>":new_value}})
If your array contains documents, you can update a particular field of a document at that index like this:
db.stuff.update(/* query ... */, {$set:{"array_of_stuff.0.value":"new_value_of_thing_1"}})
// If you could use the query parameter and knew something
// about the value in the array you wanted to change:
db.stuff.update({"array_of_stuff.value":"value_of_thing_1"}, {$set:{"array_of_stuff.$.value":"new_value_of_thing_1"}})
See if this example help you:
db.bruno.insert({"array": [{"name": "Hello", "value": "World"}, {"name": "Joker", "value": "Batman"}]})
db.bruno.update({"array.name": "Hello"}, {$set: {"array.$.value": "Change"}})
db.bruno.find().pretty()
output:
db.bruno.find().pretty()
{
"_id" : ObjectId("52389faaafd72821e7b25a73"),
"array" : [
{
"name" : "Hello",
"value" : "Change"
},
{
"name" : "Joker",
"value" : "Batman"
}
]
}
I don't think it is possible. In order to update field of one of the elements in array, you should use positional $ operator, e.g.:
update({'array_of_stuff.name':'name_of_thing_1'},
{ $set: {'array_of_stuff.$.value':'new_value_of_thing_1'}})
But according to documentation: positional $ operator acts as a placeholder for the first element that matches query document, and the array field must appear as part of the query document.