Is it possible to create a capped collection in a read only mode at a single replica in MongoDB? There is a main database with replica set and I need to use a capped collection with a AWS queue to listen for new insertions. In order to avoid possible listening overloads in the main database, I was wonder whether is possible to create a capped in one of the replicas.
This is not possible as at now. Also keep in mind that replication's main purpose it to have similar data through out the replica set.
Related
I created MongoDB replica set. It works perfectly fine. But, how do I create the replica set only for specific collections?
Business Case: We have many collections with lots of writes going on
but it can be volatile, i.e. we don't need to replicate all these
large collections. We just need replication/failsafe only for one
collection.
Is this possible?
I have a MongoDB running with 3 member replica set. All the members are up & running.
I have one database with 4-5 collections and I want to delete that database.
What is the best way to do it. Can I just use db.dropDatabase() on primary?
Do I need to stop secondary before I drop the database? After I drop the database will secondary members sync automatically to primary? What about the memory, will it get free after I delete the database?
Yes. Just drop the database using the command db.dropDatabase() in the primary and the changes will be propagated to secondaries as well. You don't need to shutdown the secondaries.
as Anand Jayabalan said, you just need to drop the database on primary node. The replication do the rest reading the oplog for the secondaries.
From Reference links and official documentation.
The primary is the only member in the replica set that receives write operations. MongoDB applies write operations on the primary and then records the operations on the primary’s oplog. Secondary members replicate this log and apply the operations to their data sets.
Reference:
https://docs.mongodb.org/manual/core/replication-introduction/
https://docs.mongodb.org/manual/core/replica-set-primary/
Is there any way to selectively index on secondary replica-set collection fields only as I don't need those index on primary's collection?
Eg. I have Product collection with Product-Category as field. I don't want to index this field on primary, I only want index on replica-set secondary as I will be reading data only from secondary.
I realize this is an old thread, but I was recently digging around on the same topic. I want to run data reporting tools against my live data, and I assume I'll want different indexes for that, and but I don't want to introduce them in a way that impacts the performance of my primary nodes that are serving application requests (since more indexes increase memory consumption and I believe can impact write performance in a way that would be visible to the consuming application depending on the chosen MongoDB Write Concern).
I came across two things:
This feels like a hack to me, but some people recommend using a hidden secondary that is configured to never become a primary, and they temporarily take it out of the replica set, add additional indexes, and then add it back into the replica set - Different indexes on different replica set members
And that stackoverflow posting also referenced the following open MongoDB enhancement request "Allow different indexes on replica set p=0 nodes" https://jira.mongodb.org/browse/SERVER-3664. Please vote it up!
Pete
We are using Mongo to host a multi-tenant application. Each tenant is going to have their own database. To get around resource utilization issues the approach that we are taking is to shard by database (as opposed to by collection - if that is the correct term to use).
This means for every x tenants we will create a new 3-node replica set. So we may have for example 1000 tenants on 1 shard and another 1000 tenants on another shard.
My question is regarding the placement of the databases for new signups. The approach we were going to take was to flag a shard as being the 'active' shard and creating all new tenants on that shard. When it reaches capacity, create a new shard, flag that as the active shard and continue on.
Can you choose which shard you create a new database on in Mongo directly? If left to Mongo, from what I understand, it will do it in round robin fashion when there is more then one shard which may leave our shards imbalanced.
Is this the right approach or is there an alternative better approach?
You can use shard tags to force some collections to reside only on specific shards. So you could, for example, tag each shard with its serial number, and tag the collections/databases you want to have on that shard with that tag, until it runs full at which point you create a new shard, increase the counter and use that for new data.
Another option then is to not enable sharding on the individual databases at all, and use the movePrimary command to force a specific shard to act as the primary shard for a specific database. Since the database won't be sharded, all its data will remain on its designated primary shard, which is exactly what you want.
That being said, it seems to me like this approach conflicts with the very concept of sharding, which is meant to evenly distribute data across multiple machines automatically.
I have capped local MongoDB collection A, and I'd like to replicate to a MongoDB collection B on a cloud server. Now, I want to keep the documents from A that will be deleted due to its capped size.
Collection A replicate Collection B
-------------- ----------> --------------
Capped at 50MB Infinite size!
local cloud server
Is this possible in MongoDB (as it is possible in CouchDB using filters)?
Or should I search for a totally different approach?
Thanks for your advice!
The deletes in a capped collection are not operations, and so they are not replicated via the oplog. Hence, all you need to do is make the collection non-capped on a secondary and it will simply continue to grow as you add data to the capped collection and those ops are replicated. Try something like this:
Add the secondary on the cloud server as normal
Stop that new secondary, restart it outside the set with no replset argument
Drop the capped collection, recreate with same name as a regular non-capped collection
(Optional) Re-import the data manually using mongodump/mongorestore
Restart the secondary with the original replica set parameters
The new normal collection will just keep growing
If you want to delete the collection or make other changes you will need to take the secondary out each time, but otherwise this should behave as you want. I haven't done this myself explicitly, but I have seen this happen accidentally and had to do the reverse :)