I need help to make the emulator behave the same as the cosmos DB when using a custom shard key. The emulator expects the shard key to be in the filter argument when calling the following golang mongo driver API. Otherwise, it will returns error code 61 with message "query in command must target a single shard key"
func (c *collection) replaceDocument(ctx context.Context, filter Filter, doc Document, upsert bool) (bool, error) {
The same code behave correctly on Azure cosmos DB. Below is the document schema I used. I filter on the unique _id only when calling the above API.
{
"_id": ==> unique non NULL key,
"shard_key": ==> non NULL string field for sharding,
other fields,
}
The CosmosDB emulator provides an emulation of CosmosDB service that runs on local workstation. This could result into differences between the emulator behavior versus the service account behavior. More information can be read here.
Related
I am running a mongoDB cluster. A nodejs application connects to this db and I have set readPreference= secondaryPreferred. However there is one critical flow which writes the document and then subsequently reads the same document. While we read it, we are getting stale reads,to avoid this, we wanted to set read preference to primary in that specific query which is fetching the same document. Since the global read preference(via connection string) is set to secondary and one of the query is setting read preference via query, does mongoDb prioritise between two? or will global preference override query specific preference?
I couldn't get this in below mongodb docs.
https://www.mongodb.com/docs/manual/core/read-preference/
We were able to provide the read preference as an option to find function of mongoose.
await UserModel.findOne(query, {}, {readPreference : 'primary'});
The global read preference was set to secondary in connection string as below:
mongodb://db0.example.com,db1.example.com,db2.example.com/?replicaSet=myRepl&readPreference=secondary
The query did honor the parameter sent over query options and was reading data from primary
I'm using Azure CosmosDB with the MongoDB API. I'm trying to execute the following:
db.createCollection('test')
db.test.createIndex({key: 1}, {name: 'key_1', unique: true})
However, doing so fails with the following error:
The unique index cannot be modified. To change the unique index, remove the collection and re-create a new one.
When reading about it in the documentation and on Stack Overflow, it's mentioned that you can only create a unique index for empty collections.
However, when I try the following command, it seems my collection is empty, so this apparantly isn't the reason why it isn't working:
db.test.find()
I tried to recreate the collection several times, but to no avail.
As per the format of the query mentioned, it seems to be like wildcard indexing style and unfortunately if it is true, then wildcard indexing is having limitation in creation of unique index. In Azure Cosmos DB API for MongoDB cannot use wildcard index. Because creating a wildcard index is like executing multiple specific fields.
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/mongodb-indexing
Refer the above-mentioned link for reference.
In my case, the problem was caused by a limitation of the continuous backup of CosmosDb.
As stated in the documentation:
Azure Cosmos DB API for SQL or MongoDB accounts that create unique index after the container is created aren't supported for continuous backup. Only containers that create unique index as a part of the initial container creation are supported. For MongoDB accounts, you create unique index using extension commands.
I had to create my unique index using the custom commands.
We have recently moved to mongo java driver core/sync 4.4.0 from 3.12.1 with spring data mongo db from 2.2.5.RELEASE to 3.3.0 also spring boot version 2.6.2 and mongo server version 4.2.5
We are getting exceptions while hitting upsert queries on sharded collection with above mentioned error
There is way to insert shard key in query filter but that is not feasible for us, hence we tried adding #Sharded annotations to our DTO as we have different shard keys for different collections
Still we are getting the above mentioned error also we are unable to get exact meaning of what is full copy of entity meant in below statement
update/upsert operations replacing/upserting a single existing document as long as the given UpdateDefinition holds a full copy of the entity.
Other queries are working fine also upsert queries are working fine on addition of shard key in query filter but that change is not feasible for us we need quick solution
Please help as not able to find any solution on any platform. Thanks in advance!
So here's the deal:
You cannot upsert on Sharded Collection in MongoDB UNLESS you include the shard key in the filter provided for update operation.
To quote MongoDB Docs:
For a db.collection.update() operation that includes upsert: true and is on a sharded collection, you must include the full shard key in the filter:
For an update operation.
For a replace document operation (starting
in MongoDB 4.2).
If you are using MongoDB 4.4+ then you have a workaround as mentioned below:
However, starting in version 4.4, documents in a sharded collection can be missing the shard key fields. To target a document that is missing the shard key, you can use the null equality match in conjunction with another filter condition (such as on the _id field). For example:
{ _id: <value>, <shardkeyfield>: null } // _id of the document missing shard key
Ref: - https://docs.mongodb.com/manual/reference/method/db.collection.update/#behavior
I have a cosmosdb database with Sharedkey. When my service run, it remove all documents by sharedkey field and insert the next. But during tests I had duplicateds inserts without errors notifys.I't have permission to directly delete the collection. This's customer envirronment and has process to change database.
I'd like to remove all collection documents. But has duplicated SharedKey fields and throw this error message:
MongoError: query in command must target a single shard key
When list by shared key
When try to remove by shared key
Hey friends somebody can "give me a hand"?
Obrigado!!!
Just like what some engineer said in the website I mentioned in the comment,
We have got the same feedback from PG team that you got on the support
ticket that you drop the entire collection using drop() instead of
deletemany(). So please follow the same for getting this issue
resolved.
So the solution to your error is use drop() instead.
To remove all collection documents, you can use the 'drop()' command. For more information on this, you can go through the MongoDB Documentation on drop()
It was a bug. To fix i had to remove all data and move to another collection with once sharedkey.
The best solutions was to change to Atlas MongoDb 😋
I need to kill specific mongodb operations.
But mongo commands (like aggregate or mapReduce) whether they're called from the native node client or shell do not return their opids; so I can't match the values in db.currentOp() with the specific query that I want to kill.
I tried matching my queries with the query property of db.currentOp().inprog, but it's far from reliable and in many cases the value of this property will be "$msg" : "query not recording (too large)" hence it cannot be matched.
How can I get/assign an ID to my async mongodb queries in order to use this ID to find and kill the query from a (different) connection?
Are you using mongodb 2.6? With 2.6 you can set the query timeout via $maxTimeMS
http://docs.mongodb.org/manual/reference/operator/meta/maxTimeMS/