Is it possible to duplicate collection from single server to cluster? I have a collection on test server (single machine) and I will copy the collection to another server (cluster) and I'm not sure if I can use duplicate collection.
I want to copy collection once.
First use mongoexport on the source server to export the collection to a file.
mongoexport --db yourDB --collection yourCollection --out yourCollection.json
When the collection doesn't already exist on the destination shard or isn't yet configured to be sharded, you should do so now by connecting to the mongos instance with the mongo shell and using the command:
sh.shardCollection( "yourDatabase.yourCollection", { yourDesiredShardKey: 1 } )
Then use mongoimport on the destination to import the collection.
mongoimport --db yourDB --collection yourCollection --file yourCollection.json
Both mongoimport and mongoexport have optional --host and --port parameters to import from / export to a remote server. But I would recommend you to copy the file to the destination server yourself. First, this should be faster. Second, in a securely configured network, you shouldn't be able to access both test and production database from the same machine anyway, at least not without authentication.
For a collection to use a cluster's power, you need to have shard options. Duplicating the test collection won't do that.
You can create a new collection with the right sharding options and then copy items of the test collection into it
db.test.mycollection.find().forEach( function(x){db.otherdb.othercollection.insert(x)} );
Related
I'm using Mongo 3.2. I have two databases on my localhost named client1 and client2.
Now client1 contains a collection named users.
I want to clone this collection to client2.
I have tried:-
use client2
db.cloneCollection('localhost:27017', 'client1.users',
{ 'active' : true } )
This outputs
{
"ok" : 0.0,
"errmsg" : "can't cloneCollection from self"
}
Is cloning a collection from one db to another on the same server prohibited?
Few things :
In general cloneCollection is used for different mongo instances but not to copy on same instances.
Also if you're using v4.2 you should stop using copyDB & cloneCollection cause they're deprecated compatibility-with-v4.2 & start using mongodump and mongorestore or mongoexport & mongoimport.
I would suggest to use mongodump & mongorestore :
Cause mongodump would preserve MongoDB's data types i.e.; bson types.
mongodump creates a binary where as mongoexport would convert bson to json & again mongoimport will convert json to bson while writing, which is why they're slow. You can use mongoexport & mongoimport when you wanted to analyze your collections data visually or use json data for any other purpose.
You can run below script in shell
declare - a collections = ("collectionName1" "collectionName2")
for i in "${collections[#]}"
do
echo "$i"
mongodump --host "All-shards" --username=uname --password password --ssl --authenticationDatabase admin --db dbname --collection "$i"
mongorestore --host=host-shard-name --port=27017 --username=uname --password=psswrd --ssl --authenticationDatabase=admin --db=dbname --collection= "$i" ./dump/dbName/"$i".bson;
done
To use mongodump, you must run mongodump against a running mongod or mongos instance. So these commands are being run expecting mongo is properly installed & path setup is good, if not you can navigate to mongo folder & run like ./mongodump & ./mongorestore. Above script will be useful if you wanted to backup multiple collections, You need specify few things in script like :
mongodump--host "All-shards" -> Here you need to specify all shards if your MongoDB is a replica set, if not you can specify localhost:27017.
mongorestore --host=host-shard-name -> You've to specify one shard of replica set, else your localhost, Few things here can be optional --ssl, --username, --password.
So mongodump will create a folder named dump for first time which will have the sub-folders with dbNames & each sub-folder will has bson files respective to their collection names dumped, So you need to refer dbName in restore command & collection name will be taken from variable i -> ./dump/dbName/"$i".bson
Note : MongoDB v3.2 is so old & in cloud based MongoDB service Mongo-atlas it has already reached it's end of lifecycle, So please upgrade asap. If you're looking for a free mongo instance or starting with MongoDB - you can try atlas.
db.cloneCollection() copies data directly between MongoDB instances.
https://docs.mongodb.com/v3.2/reference/method/db.cloneCollection/
That means you cannot clone inside the same mongod instance. Use mongoexport and mongoimport to clone your collection.
Since 4.2 MongoDb introduces $merge operator which allows copy from db1.collection to db2.collection.
I use mongochef as a UI client for my mongo database. Now I have collection which consists of 12,000 records. I want to export them using the mongochef.
I have tried with the export option(available) which is working fine up to 3000 documents. But if the number of records gets increasing the system is getting hung up.
Can you please let me know the best way to export all the documents in a nice way using mongochef.
Thanks.
Finally I came to conclusion to use the mongo using terminal which the best way to use(efficient).
read about the primary and secondary databases and executed the following query:
mongoexport --username user --password pass --host host --db database --collection coll --out file_name.json
Friends,
I'm a MongoDB DBA and I'm totally new to Mongo and also to DBA role.
I want to archive data that is one month old in a 3 node replica set. mongodump is one option I can achieve this but my client asks me if there are any options. So please could you suggest the available options for archiving the data in replica set.
Many thanks!!!
Yes, we have multiple options.
We can able to take from tools like,
OPS Manager
Atlas
Scripting via
Other than that, if you are trying in manually use MongoDUMP
mongodump --archive=test.20150715.gz --gzip --db test
or
mongodump --gzip --db test
EXTRA
if you want to restore same archive file,
mongorestore --archive --port 27017 --host abc.mongo.com
Refer:
https://docs.mongodb.com/manual/reference/program/mongodump/
https://docs.mongodb.com/manual/reference/program/mongorestore/
I have a mongo server receiving data from servers behind an amazon LB, 2 days ago there was an error and part of the servers sent their data to an old mongo server that had a db of the same name, we realized that and fixed it back right away.
Now i have part of my data stored on the wrong machine.
What I need now is a way to compare the data between the 2 dbs (which each have 2 relevant collections) and insert only the missing data to the correct collection.
I do not care about the unique id mongo gives but i do need to compare by the field "uuid" that we create.
mongo version: 2.4.4
I am new to Mongo and any help would be greatly appreciated.
Yes, you can. Follow these steps...
1 mongoexport and then mongoimport on the basis of fields on which you want to compare and import.
mongoexport --db db_name --collection collection_name --type csv --fields field1,field2,field3 --out /var/www/export.csv
once you get the exported CSV at the specified location. Open and remove unwanted fields...
mongoimport --db db_name --collection collection_name --type csv --file /var/www/export.csv --fields field1,field2,field3 --upsertFields field1,field2,field3
NOTE:
1. If you working in production environment, dealing with huge database then free up your mongo from load of queries and then export else it might get stucked.
I'm new in MongoDB and my english is too bad - so, i can't understand mongodb documentation.
I have to dump (backup) several tables (collection) from MongoDB base - how I can do it?
Which utility I have to use for recover collections from backup?
Thank you for your attention for my question!
Start your mongod server. Assuming that your mongod server is running on localhost and port 27017. Now open a command prompt and go to bin directory of your mongodb instance and type the command:
mongodump
This command will back all data of the server to directory /bin/dump/
To backup only specific collection, use:
mongodump --collection YOUR_COLLECTION_NAME --db YOUR_DB_NAME
On a good-bye note, to restore/recover data use command:
mongorestore