I have got mongo db called test and in this db two collections collection1 and collection1_backup.
How to replace content of collection1 with data from collection1_backup.
The best way to have done this (considering the name of the collection ends with _backup) is possibly to have used mongorestore: http://docs.mongodb.org/manual/reference/mongorestore/
However in this case it depends. If the collection is unsharded you can use renameCollection ( http://docs.mongodb.org/manual/reference/command/renameCollection/ ) or you can use a more manual method of (in JavaScript code):
db.collection1.drop(); // Drop entire other collection
db.collection1_backup.find().forEach(function(doc){
db.collection1.insert(doc); // start to replace
});
Those are the most common methods of doing this.
This can be done using simple command:
db.collection1_backup.aggregate([ { $match: {} }, { $out: "collection1" } ])
This command will remove all the documents of collection1 and then make a clone of collection1_backup in collection1.
Generic Command would be
db.<SOURCE_COLLECTION>.aggregate([ { $match: {} }, { $out: "<TARGET_COLLECTION>" } ])
If TARGET_COLLECTION does not exist, the above command will create it.
also usefull:
to export collection to json file
mongoexport --collection collection1_backup --out collection1.json
to import collection from json file
mongoimport --db test --collection collection1 --file collection1.json
to import single collection from backup/dump file one need to convert *.bson file to *.json
by using
bsondump collection1_backup.bson > collection1_backup.json
simply just do this.
//drop collection1
db.collection1.drop();
//copy data from collection1_backup to collection1
db.collection1.insert(db.collection1_backup.find({},{_id:0}).toArray());
Using Java Driver
Try below one:
public void copyTo(String db,String sourceCollection,String destinationCollection,int limit) throws
UnknownHostException {
MongoClient mongo = new MongoClient("localhost", 27017);
DB database = mongo.getDB(db);
DBCollection collection = database.getCollection(sourceCollection);
DBCursor dbCursor = collection.find().limit(limit);
List<DBObject> list = dbCursor.toArray();
DBCollection destination = database.getCollection(destinationCollection);
destination.insert(list, WriteConcern.NORMAL); //WRITE CONCERN is based on your requirment.
}
Better way would be to use .toArray()
db.collection1.drop(); // Drop entire other collection
// creates an array which can be accessed from "data"
db.collection1_backup.find().toArray(function(err, data) {
// creates a collection and inserting the array at once
db.collection1.insert(data);
});
You can use a simple command to Backup MongoDB Collection. It will work only on MongoDB 4.0 or earlier versions.
db.sourceCollectionName.copyTo('targetCollectionName')
Your targetCollectionName must be in Single(') or Double(") Quote
Note:
The db.collection.copyTo() method uses the eval command internally. As
a result, the db.collection.copyTo() operation takes a global lock
that blocks all other read and write operations until the
db.collection.copyTo() completes.
Drop collection1
then use this query
var cursor = db.collection1_backup.find();
var data = [];
while(cursor.hasNest()){
data.push(cursor.next());
}
db.collection1.insertMany(data)
Related
I have a Mongo db with a lot of collection with only 1 document each, I want to search in each of this collection a specific key and and print its value, is possible with a query or need a Python script ?
Easy enough to do in python
from pymongo import MongoClient
db = MongoClient()['yourdatabase']
key_to_find = 'yourkey'
for collection in db.list_collection_names():
records = db[collection].find({key_to_find: { "$exists": True}})
for record in records:
print (record)
I'm using the mongodb native driver v3.2.3 and trying to query how many collections are in the current database. I thought it would be
db.listCollections()
But this does not appear to return a count.
You can get the collection count of a database via the stats method:
let stats = await db.stats();
console.log(stats.collections);
That function returns a cursor which you can then use 'forEach()' on. Like this:
db.listCollections({}, {nameOnly: true}).forEach((_col) => {
console.log(_col.name)
})
You will then have to increment a counter for each collection, or push each collection name to an array and then check it's length.
Another way to get the collection count quickly would be to use the 'collections()' function like this:
db.collections({}, (error, collections) => {
console.log(collections.length)
})
listCollections source
collections source
In the mongo shell, it is possible to insert an array of documents with one call. In a Meteor project, I have tried using
MyCollection = new Mongo.Collection("my_collection")
documentArray = [{"one": 1}, {"two": 2}]
MyCollection.insert(documentArray)
However, when I check my_collection from the mongo shell, it shows that only one document has been inserted, and that document contains the entire array as if it had been a map:
db.my_collection.find({})
{ "_id" : "KPsbjZt5ALZam4MTd", "0" : { "one" : 1 }, "1" : { "two" : 2} }
Is there a Meteor call that I can use to add a series of documents all at once, or must use a technique such as the one described here?
I imagine that inserting multiple documents in a single call would optimize performance on the client side, where the new documents would become available all at once.
You could use the bulk API to do the bulk insert on the server side. Manipulate the array using the forEach() method and within the loop insert the document using bulk insert operations which are simply abstractions on top of the server to make it easy to build bulk operations.
Note, for older MongoDB servers than 2.6 the API will downconvert the operations. However it's not possible to downconvert 100% so there might be some edge cases where it cannot correctly report the right numbers.
You can get raw access to the collection and database objects in the npm MongoDB driver through rawCollection and rawDatabase methods on Mongo.Collection
MyCollection = new Mongo.Collection("my_collection");
if (Meteor.isServer) {
Meteor.startup(function () {
Meteor.methods({
insertData: function() {
var bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp(),
counter = 0,
documentArray = [{"one": 1}, {"two": 2}];
documentArray.forEach(function(data) {
bulkOp.insert(data);
counter++;
// Send to server in batch of 1000 insert operations
if (counter % 1000 == 0) {
// Execute per 1000 operations and re-initialize every 1000 update statements
bulkOp.execute(function(e, rresult) {
// do something with result
});
bulkOp = MyCollection.rawCollection().initializeUnorderedBulkOp();
}
});
// Clean up queues
if (counter % 1000 != 0){
bulkOp.execute(function(e, result) {
// do something with result
});
}
}
});
});
}
I'm currently using the mikowals:batch-insert package.
Your code would work then with one small change:
MyCollection = new Mongo.Collection("my_collection");
documentArray = [{"one": 1}, {"two": 2}];
MyCollection.batchInsert(documentArray);
The one drawback of this I've noticed is that it doesn't honor simple-schema.
I want to save the aggregation framework result to a new collection.
I know that's impossible with the framework at the moment with the command itself.
Is there a workaround in the shell?
Starting with Mongo 2.6.0 you can do this natively without any additional manipulation.
db.<collection>.aggregate( [
{ <operation> },
{ <operation> },
...,
{ $out : "<output-collection>" }
] )
Check the new aggregation operator $out for more detailed example.
P.S. using this way you are not limited to 16Mb size.
Update: See Salvador's answer for a more efficient way to do this in MongoDB 2.6+.
Save the aggregation result in a variable and then insert its result property into a new collection:
var result = db.foo.aggregate(...);
db.bar.insert(result.result);
result.result is no longer working, try toArray().
var result = db.coll.aggregate(...);
db.bar.insert(result.toArray());
Using the mongodb shell, I'm trying to add a new property to each document in a large collection. The collection (Listing) has an existing property called Address. I'm simply trying to add a new property called LowerCaseAddress which can be used for searching so that I don't need to use a case-insensitive regex for address matching, which is slow.
Here is the script I tried to use in the shell:
for( var c = db.Listing.find(); c.hasNext(); ) {
var listing = c.next();
db.Listing.update( { LowerCaseAddress: listing.Address.toLowerCase() });
}
It ran for ~6 hours and then my PC crashed. Is there a better way to add a new property to each documentin a large collection (~4 million records)?
you JavaScript didn't work, but the code below works. But don't knew how long it takes for 4 Million records.
db.Listing.find().forEach(function(item){
db.Listing.update({_id: item._id}, {$set: { LowerCaseAddress: item.Address.toLowerCase() }})
})
You can use updateMany for this too:
try {
db.<collection>.updateMany( {}, {$set: { LowerCaseAddress:
item.Address.toLowerCase() } } );
} catch(e) {
print(e);}