I want to update 10,000 documents in a single web request. I intend to update only one field (which is indexed) in all the documents matched with some criteria with same value.
I see morphia 1.3.2 always set multi=true parameter in update call. Is it enough for updating 10,000 document? Or there are any bulk update functionality in morphia.
The following code should work for you.
Query<Entity> query = datastore.createQuery(Entity.class);
query.filter("name = ", "xxx");
UpdateOperations<Entity> updateOperations = datastore.createUpdateOperations(Entity.class).set
("yyy", 200);
UpdateResults updateResults = datastore.update(query, updateOperations, false, null);
All the documents in the collection with name = 'xxx' will now have all their 'yyy' attribute equal to 200.
Related
In my use case I want to update multiple documents at once, documents that match a query, using spring-data-mongo.
Here is what I have been trying,
Criteria filterCriteria = new Criteria().andOperator(Criteria.where("bac").is("def"));
Update update = new Update();
update.set("status", status);
Query query = new Query();
query.addCriteria(filterCriteria);
mongoOperations.findAndModify(query, update, MyClass.class);
But this is not updating any document.
Plus I have looked up in the mongo documentation but have not anything useful
https://docs.mongodb.com/manual/reference/method/db.collection.findAndModify/#comparisons-with-the-update-method
Here is the version that I am using
Mongodb - 3.6
spring-data-mongodb - 1.5.5.RELEASE
findAndModify(...) method can update a document and return either the old or newly updated document in a single operation.
To update all document that matches the given query use updateMulti(...).
https://docs.spring.io/spring-data/mongodb/docs/current/api/org/springframework/data/mongodb/core/MongoOperations.html#updateMulti-org.springframework.data.mongodb.core.query.Query-org.springframework.data.mongodb.core.query.UpdateDefinition-java.lang.Class-
visit the link and there you will find it.
#Autowire
MongoTemplate mongoTemplate;
Query query = new Query();
Criteria filterCriteria = Criteria.where("bac").is("def");
query.addCriteria(filterCriteria);
Update update = new Update();
update.set("status", status);
mongoTemplate.updateMulti(query, update, MyClass.class);
That is meant to be read as a dual upsert operation, upsert the document then the array element.
So MongoDB is a denormalized store for me (we're event sourced) and one of the things I'm trying to deal with is the concurrent nature of that. The problem is this:
Events can come in out of order, so each update to the database need to be an upsert.
I need to be able to not only upsert the parent document but an element in an array property of that document.
For example:
If the document doesn't exist, create it. All events in this stream have the document's ID but only part of the information depending on the event.
If the document does exist, then update it. This is the easy part. The update command is just written as UpdateOneAsync and as an upsert.
If the event is actually to update a list, then that list element needs to be upserted. So if the document doesn't exist, it needs to be created and the list item will be upserted (resulting in an insert); if the document does exist, then we need to find the element and update it as an upsert, so if the element exists then it is updated otherwise it is inserted.
If at all possible, having it execute as a single atomic operation would be ideal, but if it can only be done in multiple steps, then so be it. I'm getting a number of mixed examples on the net due to the large change in the 2.x driver. Not sure what I'm looking for beyond the UpdateOneAsync. Currently using 2.4.x. Explained examples would be appreciated. TIA
Note:
Reiterating that this is a question regarding the MongoDB C# driver 2.4.x
Took some tinkering, but I got it.
var notificationData = new NotificationData
{
ReferenceId = e.ReferenceId,
NotificationId = e.NotificationId,
DeliveredDateUtc = e.SentDate.DateTime
};
var matchDocument = Builders<SurveyData>.Filter.Eq(s => s.SurveyId, e.EntityId);
// first upsert the document to make sure that you have a collection to write to
var surveyUpsert = new UpdateOneModel<SurveyData>(
matchDocument,
Builders<SurveyData>.Update
.SetOnInsert(f => f.SurveyId, e.EntityId)
.SetOnInsert(f => f.Notifications, new List<NotificationData>())){ IsUpsert = true};
// then push a new element if none of the existing elements match
var noMatchReferenceId = Builders<SurveyData>.Filter
.Not(Builders<SurveyData>.Filter.ElemMatch(s => s.Notifications, n => n.ReferenceId.Equals(e.ReferenceId)));
var insertNewNotification = new UpdateOneModel<SurveyData>(
matchDocument & noMatchReferenceId,
Builders<SurveyData>.Update
.Push(s => s.Notifications, notificationData));
// then update the element that does match the reference ID (if any)
var matchReferenceId = Builders<SurveyData>.Filter
.ElemMatch(s => s.Notifications, Builders<NotificationData>.Filter.Eq(n => n.ReferenceId, notificationData.ReferenceId));
var updateExistingNotification = new UpdateOneModel<SurveyData>(
matchDocument & matchReferenceId,
Builders<SurveyData>.Update
// apparently the mongo C# driver will convert any negative index into an index symbol ('$')
.Set(s => s.Notifications[-1].NotificationId, e.NotificationId)
.Set(s => s.Notifications[-1].DeliveredDateUtc, notificationData.DeliveredDateUtc));
// execute these as a batch and in order
var result = await _surveyRepository.DatabaseCollection
.BulkWriteAsync(
new []{ surveyUpsert, insertNewNotification, updateExistingNotification },
new BulkWriteOptions { IsOrdered = true })
.ConfigureAwait(false);
The post linked as being a dupe was absolutely helpful, but it was not the answer. There were a few things that needed to be discovered.
The 'second statement' in the linked example didn't work
correctly, at least when translated literally. To get it to work, I had to match on the
element and then invert the logic by wrapping it in the Not() filter.
In order to use 'this index' on the match, you have to use a
negative index on the array. As it turns out, the C# driver will
convert any negative index to the '$' character when the query is
rendered.
In order to ensure they are run in order, you must include bulk write
options with IsOrdered set to true.
How can I update a field with the value of other field?
For example, I want to update the field updated_at with the value of the field created_at (it's like cloning fields)
$dm->createQueryBuilder('MyBundle:MyService')
->update()
->multiple(true)
->field('updated_at')->set(CREATED_AT_ATRIBUTE)
->getQuery()
->execute();
Thanks
Option #1: Application side update
The easiest approach when using Doctrine ODM is to perform the update on the application side. So you fetch all the objects you want to update, make the necessary adjustments and flush them.
$services = $dm->createQueryBuilder('MyBundle:MyService')
->getQuery()
->execute();
foreach ($services as $service) {
$service->setUpdatedAt($service->getCreatedAt());
}
$db->flush();
$services represents the mongo cursor, which will fetch the documents as you iterate. You can set eagerCursor to true if you want to fetch all the documents from the collection at once.
Option #2: Database side update
You can also perform the update directly on the database itself. To do so however, you'll need to create the query yourself as the query builder doesn't support this functionality.
// Get MongoDB instance from DocumentManager
$mongo = $dm->getDocumentDatabase('Fully/Qualified/Class/Name')
->getMongoDB();
// Iterate over the collection and update each document
$mongo->execute('
db.MyServiceCollection.find().forEach(function (item) {
db.MyServiceCollection.update(
{_id: item._id},
{$set: {updated_at: item.created_at}}
);
});
');
The function used in forEach is just a regular javascript function, so you can insert some more logic in there if you need more control over what you update.
I have a mongodb collection of the form
{
"_id":"id",
"userEmail":"userEmailFromCustomerCollection",
"customerFavs":[
"www.xyz.com",
"www.xyz.com",
"www.xyz.com"
]
}
I need to add an element to the customers favs array using Jongo , I am using the following code snippet to do so .
String query = "{userEmail:'"+emailId+"'}";
customerFavCollection.update(query).with("{$addToSet:{customerFavs:#}}", favUrl);
My problem , is that I need to upsert the document if the document does not
exist already , how can I do so using Jongo, I know an easier option would be to retrieve the document by Id and if it does not exist then insert the document using save() , but I am trying to avoid the extra retrieve.
You can add upsert() on the query.
customerFavCollection.update("userEmail:#", emailId)
.with("{$addToSet:{customerFavs:#}}", favUrl)
.upsert();
How to make sure that specific fields can be inserted upon creation, but can optionally be excluded when updating the object.
I'm essentially looking for something like the following:
mongoOperations.save(theObject, <fields to ignore>)
From what I see, recently introduced #ReadOnlyProperty will ignore the property both for inserts and updates.
I was able to get the desired behavior by implementing my Custom MongoTemplate and overriding its doUpdate method as follows:
#Override
protected WriteResult doUpdate(String collectionName, Query query,
Update originalUpdate, Class<?> entityClass, boolean upsert, boolean multi) {
Update updateViaSet = new Update();
DBObject dbObject = originalUpdate.getUpdateObject();
Update filteredUpdate = Update.fromDBObject(dbObject, "<fields to ignore>");
for(String key : filteredUpdate.getUpdateObject().keySet()){
Object val = filteredUpdate.getUpdateObject().get(key);
System.out.println(key + "::" + val);
updateViaSet.set(key, filteredUpdate.getUpdateObject().get(key));
}
return super
.doUpdate(collectionName, query, updateViaSet, entityClass, upsert, multi);
}
But the issue is that now it will use Mongo $set form of updates for everything, not just for specific cases.
Please advise if there is any simpler (and correct) way to achieve this.
While creating an update object, use $setOnInsert instead of $set. It is available in spring mongo as well.
If an update operation with upsert: true results in an insert of a document, then $setOnInsert assigns the specified values to the fields in the document. If the update operation does not result in an insert, $setOnInsert does nothing.