I'm trying to export and import data from an old MongoDB database server to Azure CosmosDB with MongoDB API using mongodump and mongorestore. But i'm having issues with the connection to CosmosDB. I'm using a connection string with the URI flag.
My mongorestore command including connection string is the following:
mongorestore --uri="mongodb://$COSMOS_USERNAME:$COSMOS_PASSWORD#$COSMOS_HOST:$COSMOS_PORT/?maxIdleTimeMS=120000&retrywrites=false&appName=#$DB_NAME#&replicaSet=globaldb&ssl=true" --archive="$ARCHIVE_NAME"
The error message from the command is:
error restoring from archive 'testProdExport.archive': (BadValue) Retryable writes are not supported. Please disable retryable writes by specifying "retrywrites=false" in the connection string or an equivalent driver specific config.
As you can see in the connection string i'm including the retrywrites=false URI parameter, but it looks like CosmosDB doesn't recognize the parameter.
Does anyone have experience with something similar?
//Edit: I've tried and verified that the connection string is working in a mongoose connection as well as in MongoDB Compass.
I found that the various mongotools (mongoimport, mongorestore, etc.) seem to ignore the retrywrites=false or retryWrites=false in the URI, but tacking on the option --writeConcern="{w:0}" allows the command to run successfully on instances that don't support retryable writes
You can try a version of mongorestore that shipped with MongoDB 3.4. This may not be able to read recent dumps though.
The ismaster output you provided includes:
logicalSessionTimeoutMinutes: 30
This advertises session support but https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support-36 says cosmosdb does not support sessions.
CosmosDB is advertising support for a feature it does not implement, the tooling then (correctly) attempts to use it. This is a bug in cosmosdb.
On Windows, didn't have luck with appending on the connectionstring (not necessary if copied from Azure portal)
retryableWrite=false
nor --writeConcern w:0 switch
But I ended up trying all previous archives and the magic combination of mongo tools 100.0.0 worked on CosmosDB with MongoServer v3.6.
Attention: Please take care that this can lead to missing data in the collections you restored. Tested it multiple times with different versions of mongorestore. Use the version which is in the mongoserver archive with version 4.0.27. You don't need to provide writeConcern at all in this version. It will also display how many documents are restored instead of 0 in the newer versions.
This affects CosmosDb with API v4.0.
Related
I'm new to mongodb, and learning as i'm going. In my new team there isn't much documentation so i'm filling in gaps myself
There is mongodb set up that i finally managed to resolce access issues to and connect to with mongodb command line via the connection string below. (And based on my understanding, the community version of mongodb does not support this, i needed to use some server enterprise version which thankfully was free)
While i have minimum access to db now, it is quite a hassle to work with just the command line
mongo.exe
"mongodb://serv1.unix.me:10001,serv2.unix.me:10002,serv3.unix.me:10003/abc?replicaSet=example"
--authenticationMechanism=GSSAPI --authenticationDatabase=$external --username "user#testdomain.me.com" --password "pAssw0rd!" -ssl --sslCAFile C:\files\ca.pem
I have Robo3T - 1.3 but no matter how many times i try to apply above connection string into the robo3t connection window, i keep getting various errors.
Part of me is worried that i need to use the enterprise version of robo3t (which isn't free)
Can anyone give some advice?
I have a MongoDB instance with two databases, let's call them "realdb" and "copydb".
All I want to do is to periodically copy realdb to copydb. The copydb database is our "testing" instance of the actual database "realdb", and we want to periodically update it.
Normally the answer to this question would be "copydb" or "export/import". However, there are some challenges:
The target Mongo instance is running version 1.6 and is not likely to be updated anytime soon.
The target Mongo instance is on a remote server to which I don't have direct access.
This seems like basic enough a function that even 1.6 should have the ability to do it. But when I try anything I get "No such cmd" errors as if the newer Mongo can't communicate with the ancient Mongo.
Any thoughts on how this could be done?
The error "no such command" might mean that:
The command was mistyped, or does not exist
There is a mismatch of the client version (e.g. mongo shell, mongodump, mongoexport, or a mongodb driver) and the server version.
In this case it was a mismatch of the mongo shell version (3.4) vs server version (1.6).
I am in the process of migrating my app from Parse to MongoDB and IBM Bluemix, however I can't seem to reach the server when attempting the migration of data from Parse.
I've been following this tutorial tutorial, and I am currently on the step: Migrating data from Parse.com to your MongoDB instance
I am currently getting this error: No reachable servers when clicking 'Begin the migration'.
These are the connection strings in compose:
And I have tried entering various strings. I assumed this would work:
mongodb://username:password#aws-us-east-1-portal.7.dblayer.com:10803/mtcdatabase
But I get the same error. Obviously I'm changing username and password to my own credentials.
Anyone have any suggestions?
The issue ended up being that I was using the latest version of MongoDB, however Parse doesn't seem to support it. Therefore, I had to use MongoDB Classic version 3.0. Everything works great now.
I have a rails-app which uses both mongoid and mongo. I use mongoid for my own models, and I use mongo because I have ruote with a ruote-mon storage.
In production however; I get
Mongo::ConnectionFailure: Failed to connect to a master node at localhost:27017
when I try to connect to the ruote storage. Even when I just do Mongo::MongoClient.new
Steps I have taken so far to try to resolve this:
I have made my mongodb an explicit master by setting master = true in /etc/mongod.conf
There are no $ENV variables set that could intervene with Mongo::MongoClient.new (double checked)
I have tried to connect using Mongo::MongoClient.new(:slave_ok => true) - same error
I have restarted my mongo database several times (w/o success).
I have checked my firewall settings and I can connect to localhost:27017 with telnet (as said, the mongoid documents can be fetched and stored w/o issue)
I am out of my wits... Any suggestions?
The reason this happened is because we were sending queries with meta operators ($query, $orderby, etc...) for the ismaster command during a connect. This command's output is used to determine whether you are connected to a primary or not and would fail because very old versions of mongodb don't support the use of meta operators.
This fix will be in version 1.8.2 of the gem but I strongly encourage anyone who is still running pre-1.8 versions of mongodb to upgrade. 2.0 is the current legacy release as of the time of this post and even 1.8 is no longer widely supported.
As jmettraux mentioned you can find more details about this on the MongoDB project Jira under Ruby-525
please look at: https://jira.mongodb.org/browse/RUBY-525
Should be fixed by the 1.8.2 mongo gem.
I have a production server running Mongo 2.0.7 and a local machine that runs Mongo 2.0.6.
I exported a database as a JS file and then imported it in my local machine. EVerything seems to be working fine however I am unable to server GridFS files. That is when I try accessing that resource, I get corrupt data and thus no image. Any thoughts? I tried looking at the change log for the version change and saw nothing that could effect that?
Thanks
I'm guessing that you used mongoexport to produce the JSON dumps. mongoexport does not provide the same data fidelity as mongodump, so try that instead.