My customer uses mongoDB 2.4 and as there are some limitations with this version, we have give them the option to upgrade to Latest stable mongoDB 3.4.5.
Initial testing of using
mongodump in MongoDB 2.4
and mongorestore in Mongodb 3.4.5 worked fine as I can see all the collections imported.
From the documentation mongorestore it was not mentioned anywhere that it can restore the dumps from older versions of mongoDB.
As we cannot use the mongorestore , Can I use "mongoexport" to export the data in csv/json format of older mongoDB 2.4 , and import into newer version of mongoDB 3.4 ?
What are the possible problems of using "mongoexport/mongoimport" instead of "mongodump" to upgrade to newer version of mongoDB 3.4 ?
NOTE: I will remove the older version of mongoDB completely and will install the newer version of mongoDB
Mongodump and Mongorestore are better because:
They run faster
They preserve some data formats better than mongoexport and mongoimport, because the data is not translated from BSON into JSON and back.
As described in the MongoDB Docs on MongoImport:
WARNING
Avoid using mongoimport and mongoexport for full instance production backups. They do not reliably preserve all rich BSON data types, because JSON can only represent a subset of the types supported by BSON. Use mongodump and mongorestore as described in MongoDB Backup Methods for this kind of functionality.
In addition, be very careful about the upgrade using mongorestore; just because the data is restored as it was previously, this does not mean that the new version of MongoDB can work with it. For example there were a sequence of changes to the authorisation model after v2.4 which means that you must first upgrade to v2.6, and only then to v3.0. There are similar structural changes at each major version, so it is recommended that you upgrade stepwise, one major version at a time i.e.
v2.4 -> v2.6
v2.6 -> v3.0
v3.0 -> v3.2
v3.2 -> v3.4
From http://www.dba86.com/docs/mongo/2.4/core/import-export.html, mongoexport is supported from 2.4 version. Hence it should be the right tool for that. But still the document has warning message as well.
Warning:
Avoid using mongoimport and mongoexport for full instance production backups. They do not reliably preserve all rich BSON data
types, because JSON can only represent a subset of the types supported
by BSON. Use mongodump and mongorestore as described in MongoDB Backup
Methods for this kind of functionality.
Hope that helps!!!!
Both tools (by default) will just walk the _id index to fetch the data and then write it out to disk. So, yes, both tools will similarly impact your working set which is why I would generally recommend running them against a secondary (preferably a hidden secondary if possible).
I assume you are looking for a mongodump equivalent of the --fields option from mongoexport to only dump out specific fields. The query option can be used to filter results, but it cannot be used with a projection (to select the fields returned) - this is a feature request that is being tracked in TOOLS-28 but is not yet scheduled.
Related
I have a mongodb database with version 3.6.3. I have another mongodb database (on another machine) using version 4.4.5 with no documents in it. I want to put the data from the v3.6.3 into the v4.4.5 database. Can I safetly do this using mongoexport and then mongoimport or do I need to perform more steps?
Yes, mongoexport writes the documents out to a JSON file, and mongoimport can read that file and insert the documents to the new database.
These will transfer only the documents, but not index information. You many want to consider mongodump/mongorestore if you also need to move indexes.
I'm trying to export and import data from an old MongoDB database server to Azure CosmosDB with MongoDB API using mongodump and mongorestore. But i'm having issues with the connection to CosmosDB. I'm using a connection string with the URI flag.
My mongorestore command including connection string is the following:
mongorestore --uri="mongodb://$COSMOS_USERNAME:$COSMOS_PASSWORD#$COSMOS_HOST:$COSMOS_PORT/?maxIdleTimeMS=120000&retrywrites=false&appName=#$DB_NAME#&replicaSet=globaldb&ssl=true" --archive="$ARCHIVE_NAME"
The error message from the command is:
error restoring from archive 'testProdExport.archive': (BadValue) Retryable writes are not supported. Please disable retryable writes by specifying "retrywrites=false" in the connection string or an equivalent driver specific config.
As you can see in the connection string i'm including the retrywrites=false URI parameter, but it looks like CosmosDB doesn't recognize the parameter.
Does anyone have experience with something similar?
//Edit: I've tried and verified that the connection string is working in a mongoose connection as well as in MongoDB Compass.
I found that the various mongotools (mongoimport, mongorestore, etc.) seem to ignore the retrywrites=false or retryWrites=false in the URI, but tacking on the option --writeConcern="{w:0}" allows the command to run successfully on instances that don't support retryable writes
You can try a version of mongorestore that shipped with MongoDB 3.4. This may not be able to read recent dumps though.
The ismaster output you provided includes:
logicalSessionTimeoutMinutes: 30
This advertises session support but https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-feature-support-36 says cosmosdb does not support sessions.
CosmosDB is advertising support for a feature it does not implement, the tooling then (correctly) attempts to use it. This is a bug in cosmosdb.
On Windows, didn't have luck with appending on the connectionstring (not necessary if copied from Azure portal)
retryableWrite=false
nor --writeConcern w:0 switch
But I ended up trying all previous archives and the magic combination of mongo tools 100.0.0 worked on CosmosDB with MongoServer v3.6.
Attention: Please take care that this can lead to missing data in the collections you restored. Tested it multiple times with different versions of mongorestore. Use the version which is in the mongoserver archive with version 4.0.27. You don't need to provide writeConcern at all in this version. It will also display how many documents are restored instead of 0 in the newer versions.
This affects CosmosDb with API v4.0.
Hello I have an ubuntu 14.04 server that is running mongodb 2.4.14. I need to move the mongo instance to a new server. I have installed mongo 3.4.2 on the new server and need to move the databases over. I am pretty new with mongo. I have 2 databases that are pretty big but when I do a mongo dump the file is nowhere near the site of the databases that mongo is showing.I cannot figure out how to get mongoexport to work. What would be the best way to move those databases? If possible can we just export the data from mongo and then import it?
You'll need to give more information on your issue with mongodump and what mongodump parameters you were using.
Since you are doing a migration, you'll want to use mongodump and not mongoexport. mongoexport only outputs a JSON/CSV format of a collection. Because of this, mongoexport cannot retain certain datatypes that exist in BSON and thus MongoDB does not suggest that anyone uses mongoexport for full backups; this consideration is listed on mongo's site.
mongodump will be able to accurately create a backup of your database/collection that mongorestore will be able to restore that dump to your new server.
If you haven't already, check out Back Up and Restore with MongoDB Tools
I find that db.collection.copyTo() and eval() have been deprecated since 3.0. But I do not find what can be instead.
What's the alternatives?
For a server-side solution you can use aggregation...
db.getCollection('source')
.aggregate([
{ $out: 'destination' }
]);
Per this discussion on the MongoDB Group.
The alternative is to implement the equivalent queries/operations using the normal MongoDB query language and client driver API
So that would mean writing your queries in a client environment (e.g. Node.js) and execute them that way. If run on the server connecting to localhost, they should pretty quick, although probably not as quick as using eval.
The rationale for the being deprecated is outlined in this ticket.
https://jira.mongodb.org/browse/SERVER-17453
Create a mongodump of that collection, do mongorestore to a separate or new collection.
This wont stop read/write or wont lock the collection.
mongodump --db db-name --collection collection-name --archive=collection-name.archive
or save as json both works
If saved as archive, to restore
mongorestore --db db-name --collection collection-name --archive=collection-name.archive
I have a mongodb production database running on mongo 2.4.8 and I would like to upgrade to 2.6.x.
The way we want to do that is first load the data to another server running 2.6.3 using mongorestore. However when running the mongorestorecommand we get the following error:
Cannot restore users with schema version 1 to a system with server version 2.5.4 or greater
I cannot find anything related to this issue and do't know what to do. In case it matters, the database itself was not created from scratch with mongo 2.4.x but with previous versions.
What ca I do ? Is there another way of doing this other than using mongorestore ?
Thank you in advance for your help ...
There are two approaches you can take to upgrade your user schema with the 2.4 mongodump.
1) Restore into MongoDB 2.4 and then upgrade to 2.6
This follows the normal 2.6 upgrade path. Instead of trying to mongorestore your 2.4 backup directly into 2.6, restore into a 2.4 instance and then upgrade to 2.6.
It is recommended that before upgrading, you run db.upgradeCheckAllDBs() via a 2.6 mongo shell. This checks for any potential compatibility issues due to changes in MongoDB 2.6. For example, 2.6 implements stronger enforcement of index field definitions and key length restrictions.
2) Restore into MongoDB 2.6 using 2.4 mongorestore and then upgrade the user schema
This approach requires the MongoDB 2.4 version of mongorestore
start up your MongoDB 2.6 mongod without auth enabled
mongorestore your backup using a 2.4 version of mongorestore
run the authSchemaUpgrade command in your 2.6 mongo shell:
db.adminCommand({authSchemaUpgrade: 1 });
restart your 2.6 mongod with auth enabled
Three approaches that I can think of:
First, if you are running as a replica set, I'd upgrade the members
one by one. If you're not running as a replica set that's bad,
MongoDB is really not designed for production usage with a single
instance. Details on converting to a replica set and in a rolling
upgrade here:
http://docs.mongodb.org/manual/release-notes/2.6-upgrade/ and here:
http://docs.mongodb.org/v2.6/tutorial/convert-standalone-to-replica-set/#
Second, if you are running on a file system that supports snapshots
(like Amazon EBS or Linux LVM) you can snapshot the database files,
restore to a new file system and start up a new mongod process using
2.6. http://docs.mongodb.org/manual/tutorial/backup-with-filesystem-snapshots/
Third, try exporting the data via mongoexport and loading it via
mongoimport. It's not the same as mongodump/mongorestore so it has
some limitations (it's not a full backup of the database, just a
text dump of the collections) but might help you get past this issue
with mongorestore: http://docs.mongodb.org/v2.6/core/import-export/