I use springData and mongo 4.6. I start a transaction, update one document in one collection and then I need to check if another document in another collection exists.
#Transaction
someService() {
Object res1 = someService1DocuemntUpdate();
// It fails here.
boolean exists = anotherObjectRepository.exists(anotherObjectId);
if (exists) {
process(res1);
}
}
And I get
com.mongodb.MongoCommandException: Command failed with error 50851 (Location50851): 'Cannot run 'count' in a multi-document transaction.
Existance operation works over count operation.
Count has restrictions in multi-document transactions
The following doc describes workarounds
https://docs.mongodb.com/manual/core/transactions/#count-operation
The easiest fix is to use findBy.. != null
More smarter fix is to write agregation request.
Related
I often use findOneAndUpdate to get and update documents based on multiple conditions, for example:
const { value } = await collection.findOneAndUpdate(
{ _id: "abc", name: "xyz" },
{ $set: { ... } }
);
If there is no document that matched the query, I would like to know why the match failed so that I can return useful error messages to users. For example, if no document with _id == "abc" exists, I would like to return a message like "Document not found." If it does exist but has a different name, the message should be "Name is incorrect." At the moment, I can only find out if the entire query matched by checking the returned value.
I could, of course, call findOne({ _id: "abc" }), check if name has the correct value, and call updateOne afterwards but I would prefer one atomic operation.
Is there a way to do this or do I have to use a transaction in this case?
As per the implementation notes, findOneAndUpdate() returns Collection~findAndModifyCallback i.e. the collection result callback. This callback internally has two fields
error (An error instance representing the error during the execution.)
result (The result object if the command was executed successfully.)
This result object, internally holds 3 values,
value (Document returned from findAndModify command.)
lastErrorObject (The raw lastErrorObject returned from the command.)
ok (Is 1 if the command executed correctly.)
You can probably use these values to do the stuff you exactly want.
Links
http://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html#findOneAndUpdate
https://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html#~findAndModifyCallback
https://mongodb.github.io/node-mongodb-native/3.1/api/Collection.html#~findAndModifyWriteOpResult
You can try the update operation and if the returned value is null (no match found) - send a message to the application user.
The findOneAndUpdate(filter, update, options, callback) has the findAndModifyCallback(error, result) and the result value will be null if the update query didn't match. See note below on the findOneAndUpdate option returnOriginal:
The option returnOriginal is boolean true by default. If no documents were found, value will be null by default (returnOriginal: true), even if a document was upserted; when false, returns the updated document rather than the original. Set this as false, and in case you are using the upsert option, so that the value will be null in case there is no match.
So, if the result value is null, this means the query didn't match and the update didn't happen and you can return an appropriate message. How? You still have to run a find query, and:
if (id not found)
send message "Document not found."
else // name not found
send message "Name is incorrect."
If you don't want to run a find query again, just send a generic message to the application user.
Transactions?
If your deployment is a replica-set or sharded cluster then transactions is a possibility. Note transactions are available only with the latest MongoDB versions (4.0 for replica-sets and 4.2 for sharded clusters).
I'm running into duplicate key error in Mongo Sharded Cluster(MSC) which does not happen when working with Mongo Replica Set.
I have simple operation in client app:
// pseudocode - email is unique key
subscriber = db.newsletter.find({email: "john#smith.com"})
if (subscriber == null) {
db.newsletter.insert({email: "john#smith.com", name: "John"})
} else {
db.newsletter.update({email: "john#smith.com"}, {name: "John"})
}
I did not include it in above code but I need to retrieve _id of a document so using upsert is not an option.
Error:
If above process is executed twice for the same and new email it would cause duplicate key error. Second iteration will not be able to find subscriber therefore it will attempt to insert despite document already existing in collection.
As far as I know, having single instance of above client running should not cause duplicate key error, but it does.
This would be the job of the findAndModify command, or the findOneAndUpdate method.
There is an upsert flag, which will create the document if it doesn't exist. You would also have to use the new flag, to return the new document if it is created.
I am using the Mongo DB java driver:
collection.bulkWrite(documents);
I have 1 million records to be inserted. If the insertion of one of the records fails then on the first failure the remaining records won't get inserted.
In order to avoid this I found that there are BulkWriteOptions with ordered as false;
collection.bulkWrite(documents, new BulkWriteOptions().ordered(false) )
If any exception occurs during the above operation, can we get the list of the records for which bulkwrite failed and can we try inserting those records again?
I think you're looking for something similar..
BulkWriteOptions bulkWriteOptions = new BulkWriteOptions();
bulkWriteOptions.ordered(true);
BulkWriteResult bulkWriteResult = null;
try {
bulkWriteResult = mongoCollection.bulkWrite(updateDocuments,
bulkWriteOptions);
} catch (BulkWriteException e) {
List<BulkWriteError> bulkWriteErrors = e.getWriteErrors();
for (BulkWriteError bulkWriteError : bulkWriteErrors) {
int failedIndex = bulkWriteError.getIndex();
Long failedEntity = entityList.get(failedIndex);
System.out.println("Failed record: " + failedEntity);
//handle rollback
}
}
All bulkWrite operations that fail are reported in arrays in the return document, along with the full document that failed to insert. For example, this code via the java driver:
MongoCollection<Document> collection = database.getCollection("bulk");
List<WriteModel<Document>> writes = new ArrayList<WriteModel<Document>>();
writes.add(new InsertOneModel<Document>(new Document("_id", 1)));
writes.add(new InsertOneModel<Document>(new Document("_id", 2)));
writes.add(new InsertOneModel<Document>(new Document("_id", 2)));
writes.add(new InsertOneModel<Document>(new Document("_id", 4)));
BulkWriteResult bulkWriteResult = collection.bulkWrite(writes);
System.out.println(bulkWriteResult.toString());
returns this:
Exception in thread "main" com.mongodb.MongoBulkWriteException:
Bulk write operation error on server localhost:27017. Write errors:
[BulkWriteError{index=2, code=11000, message='E11000 duplicate key
error collection: test.bulk index: _id_ dup key: { : 2 }',
details={ }}].
You didn’t mention what release of MongoDB you are using, but here is the version 3.2 manual entry for unordered bulkWrites: https://docs.mongodb.org/manual/core/bulk-write-operations/ if you want to look into that further.
However, if by “exception” you meant in the middle of the load operation the client or mongod processes crashed, or there was a hardware or network incident, then there won’t be a clean exit from bulkWrite and thus no bulkWriteResult output as above. Unordered bulkWrites execute operations in parallel and not necessarily in the order entered, and it is even more complicated with sharded collections and distributed clusters, so it is impossible to know exactly which operations completed before the crash. The only solution is to repeat the whole job, which usually requires you to remove all documents that inserted successfully the first time.
If you were loading to a new or empty collection you can simply drop and recreate it. If there is a unique key with an index you can just repeat the load as the documents that inserted OK the first time will be rejected as duplicates. Otherwise you will have to run a cleanup job to remove all the documents that could/should have been inserted before starting the reload attempt, which if they aren’t easily identified can be problematic.
The best approach for this scenario is to break large load operations into smaller pieces, so you have four load jobs each with one quarter of the data (or twenty each with 5%, and so on). There is more effort to design the load process this way, but it is much faster to repeat one job with 5% of the total data than have to repeat a single load that failed at the 95% mark.
If you add a loadNum field with a different value for each of the jobs, then count(“loadNum”:n) can be used to confirm that all the documents loaded, and remove(“loadNum”:n) will remove all documents from a job that was only partially successful.
I am using Morphia 0.109 with MongoDB. My document collection uses String values for the _id field, e.g.
#Entity
class MyEntity {
#Id
private String entityId;
...
}
When I call the Datastore or BasicDAO save() method to insert one of these items, I fill in the _id field (entityId) with a unique value as is required. The save() method returns a value of type Key.
My question is, how can I determine, from examining the returned Key, whether the save operation succeeded or failed? I cannot rely on the trick of checking to see if Morphia filled in the _id field, since I had to fill it in prior to calling save().
That's because you're using a String instead of an ObjectId. Is there a particular reason for that?
In general, use the right WriteConcern for your situation, which is a trade-off between speed and data safety. You'll run into an exception if your stated requirements cannot be fulfilled — for example for JOURNAL_SAFE
Exceptions are raised for network issues, and server errors; the write operation waits for the server to group commit to the journal file on disk.
I got two class, one reference another by using #Reference
When inserting I will insert the referenced one first and insert the object with the reference field later.
Everything works fine when I fetch them in the most of time.But sometimes, I got exceptions like
SEVERE: java.lang.RuntimeException:
com.google.code.morphia.mapping.MappingException: The reference({
"$ref" : "UserContactLink", "$id" : "50e92481cde5dadc12ff854b" })
could not be fetched for net.shisoft.db.obj.UserContact.ucs
When I checked the id in UserContactLink and there is no such document with this id. I think this is because I terminate the progress of mongod last time and the transaction (in my viewpoint) didn't finished and the data relation has been corrupted.
Seems mongodb don't have transaction feature, what can I do with this issue?
There are no transactions. In many cases you can restructure your documents to avoid problems with that (embedding documents,...)
You will always need to insert the referenced document first. Upon insert, the MongoDB server creates the ObjectId of the entity which is then used in the reference. You might want to check for the ID before you reference (simple check for null).