In MongoDB you can convert a collection into a capped collection with the command convertToCapped, but is there a way to revert this change so a capped collection goes back to normal?
It's seems there is only one way to convert from capped collection to normal - just simple copy objects to normal collection and remove original capped collection.
db.createCollection("norm_coll");
var cur = db.cap_col.find()
while (cur.hasNext()) {obj = cur.next(); db.norm_coll.insert(obj);}
same as above without using script.
db.collection.copyTo("collection_temp")
db.collection.drop()
db.collection_temp.renameCollection("collection")
I think there is a way! I'm not sure if this is bullet-proof, but I tried:
db.num_coll.convertToCapped(new_size)
and since then it is working.
Related
Is there a way to get an array slice at the mongo db level? I am trying to do something similar to the following: Model.find({filter: option}, startindex, endindex). Currently the only option I found is to do the following:
let result = await Model.find({filter: option});
returh result.slice(startIndex, endIndex)
Unfortunately, this does not work since I have to pull the full record each time. If I can do this at the mongo level that would be great. Thank you for your help!
UPDATE:
After further research I found a possible solution:
Model.find({filter: option}).skip(skip).limit(limit);
it seems with this method I am able to do slice the document array in the mongo db. If you have any other ideas please let me know. Thank you!
from what i know, there isnt a way to get an slice of an array from an document, but there is the select
PS: skip skips the first documents found by the query, and limit limits the amount returned by the query
Is there a way to duplicate the records in a collection into the same collection ? I am trying to generate lots of records and hence this is needed.
If you just want to duplicate easy way is like below
db.col1.find({},{_id:0}).forEach(function(doc){db.col1.save(doc)});
A quick but maybe not the most efficient way to do that could be:
Get all the documents of the collection
For each one re-write the ObjectId with a new value
Insert the modified document inside the collection
With mongo shell you could do that using the forEach as follows:
db.getCollection('YOUR_COLLECTION').find({}).forEach(
function(doc){
doc._id = new ObjectId();
db.getCollection('YOUR_COLLECTION').insert(doc);
}
)
This way, each time you run this query, all the documents in the collection are duplicated.
In my web scraping project i need to move previous day scraped data from mongo_collection to mongo_his_collection
I am using this query to move data
for record in collection.find():
his_collection.insert(record)
collection.remove()
It works fine but sometimes it break when MongoDB collection contain above 10k rows
Suggest me some optimized query which will take less resources and do the same task
You could use a MapReduce job for this.
MapReduce allows you to specify a out-collection to store the results in.
When you hava a map function which emits each document with its own _id as key and a reduce function which returns the first (and in this case only because _id's are unique) entry of the values array, the MapReduce is essentially a copy operation from the source-collection to the out-collection.
Untested code:
db.runCommand(
{
mapReduce: "mongo_collection",
map: function(document) {
emit(document._id, document);
},
reduce: function(key, values) {
return values[0];
},
out: {
merge:"mongo_his_collection"
}
}
)
If both your collections are in the same database, I believe you're looking for renameCollection.
If not, you unfortunately have to do it manually, using a targeted mongodump / mongorestore command:
mongodump -d your_database -c mongo_collection
mongorestore -d your_database -c mongo_his_collection dump/your_database/mongo_collection.bson
Note that I just typed these two commands from the top of my head without actually testing them, so do make sure you check them before running them in production.
[EDIT]: sorry, I just realised that this was something you needed to do on a regular basis. In that case, mongodump / mongorestore probably isn't the best solution.
I don't see anything wrong with your solution - it would help if you edited your question to explain what you mean by "it breaks".
The query breaks because you are not limiting the find(). When you create a cursor on the server mongod will try to load the entire result set in memory. This will cause problems and/or fail if your collection is too large.
To avoid this use a skip/limit loop. Here is an example in Java:
long count = 0
while (true) {
MongoClient client = new MongoClient();
DBCursor = client.getDB("your_DB_name").getCollection("mongo_collection").find().sort(new BasicDBObject("$natural", 1)).skip(count).limit(100);
while (cursor.hasNext()) {
client.getDB("your_DB_name").getCollection("mongo_his_collection").insert(cursor.next());
count++;
}
}
This will work, but you would get better performance by batching the writes as well. To do that build an array of DBObjects from the cursor and write them all at once with one insert.
Also if the Collection is being altered while you are copying there is no guarantee that you will traverse all documents as some may end up getting moved if they increase in size.
You can try mongodump & mongorestore.
You can use renameCollection to do it directly. Or if on different mongods, use cloneCollection.
References:
MongoDB docs renameCollection: http://docs.mongodb.org/manual/reference/command/renameCollection/#dbcmd.renameCollection
MongoDB docs cloneCollection: http://docs.mongodb.org/manual/reference/command/cloneCollection/
Relevant blog post: http://blog.shlomoid.com/2011/08/how-to-move-mongodb-collection-between.html
Changing the name of a collection in mongodb can be achieved by copy the documents in the collection to a new one and delete the original.
But is there a simpler way to change the name of a collection in mongodb?
db.oldname.renameCollection("newname")
Really? No google search?
http://www.mongodb.org/display/DOCS/renameCollection+Command
> db.oldname.renameCollection("newname")
db.oldCollectionName.renameCollection("NewCollectionName")
Check Manual here.
What I want:
I have a master collection of products, I then want to filter them and put them in a separate collection.
db.masterproducts.find({category:"scuba gear"}).copyTo(db.newcollection)
Of course, I realise the 'copyTo' does not exist.
I thought I could do it with MapReduce as results are created in a new collection using the new 'out' parameter in v1.8; however this new collection is not a subset of my original collection. Or can it be if I use MapReduce correctly?
To get around it I am currently doing this:
Step 1:
/usr/local/mongodb/bin/mongodump --db database --collection masterproducts -q '{category:"scuba gear"}'
Step 2:
/usr/local/mongodb/bin/mongorestore -d database -c newcollection --drop packages.bson
My 2 step method just seems rather inefficient!
Any help greatly appreciated.
Thanks
Bob
You can iterate through your query result and save each item like this:
db.oldCollection.find(query).forEach(function(x){db.newCollection.save(x);})
You can create small server side javascript (like this one, just add filtering you want) and execute it using eval
You can use dump/restore in the way you described above
Copy collection command shoud be in mongodb soon (will be done in votes order)! See jira feature.
You should be able to create a subset with mapreduce (using 'out'). The problem is mapreduce has a special output format so your documents are going to be transformed (there is a JIRA ticket to add support for another format, but I can not find it at the moment). It is also going to be very inefficent :/
Copying a cursor to a collection makes a lot of sense, I suggest creating a ticket for this.
there is also toArray() method which can be used:
//create new collection
db.creatCollection("resultCollection")
// now query for type="foo" and insert the results into new collection
db.resultCollection.insert( (db.orginialCollection.find({type:'foo'}).toArray())