I am trying to use the oplog monitoring class in casbah
https://github.com/mongodb/casbah/blob/master/casbah-core/src/main/scala/util/OpLog.scala
What i want to do is monitor the oplog entries at a production mongo db on
production.someserver.com
and get the entries and send them to the storage DB at
test.someotherserver.com
and replicate all the data that is in the production server to the test server. I cannot use replica sets to do this as i cannot redeploy now. I am trying to build a scala app to do this. Casbah the official scala driver for mongo as the above mentioned class which i m trying to instantiate using
val mongoColl = MongoConnection() ("test") ("test_data")
val oLog = new MongoOpLog(mongoColl)
But im not even able to instantiate it, getting an error that mongooplog is not found. Ive imported the necessary package. But even if im able to do this i have no clue on how to do what i want to do. can any one pls point me in a right direction on how to achieve this. I am pretty new to scala so a bit of detailed explanation or a link containing it would be helpful for me.
You need to have replication enabled on the server for the oplog to be created; as either a member of a replica set or in master mode for master/slave.
Otherwise, MongoDB does not waste CPU cycles and disk space maintaining an oplog. Please see the documentation on Replication for more info - http://www.mongodb.org/display/DOCS/Replication
You should really never be running any database with a single server in production, incidentally.
Related
I'm fairly new to MongoDB (Atlas - free tier), where I have created a project using it for storing my data. I had it set up and working fine for a couple of weeks, when suddenly I received an email with: An alert is open for your Atlas project: Replica set has no primary. I have no idea what this means and I don't believe I have done anything in the last couple of days/weeks that could warrant this alert. However, after checking my project, it seems that I can no longer connect to my cluster and access my data.
After checking on MongoDB Cloud, it seems that my cluster has stopped working and only the secondary shard (don't know if this is the right terminology) is running, while the other two seem to be down. Can anyone explain what this means, why it is happening or how to fix it? Thanks.
To troubleshoot issues like this, read the server logs and act based on the information therein.
For free and shared tiers in Atlas the logs are apparently not available. Therefore:
For a free tier cluster (M0), delete this cluster and create a new one. If you don't have a backup you should be able to dump via a direct connection to any of the operational secondary nodes or using the secondary read preference.
For a shared tier cluster (M2/M5), use the official MongoDB support channels for assistance.
I am trying to make a PoC for MongoDB to DocDB migration with DMS.
I've set up a MongoDB instance with some dummy data and an empty DocDB. Source and Target endpoints are also set in DMS and both of them are connecting successfully to my databases.
When I create a migration task in DMS everything seems to be working fine. All existing data is successfully replicated from the MongoDB instance to DocDB and the migration task state is at "Load complete, replication ongoing".
At this point I tried creating new entries in existing collections as well as creating new empty collections in MongoDB but nothing happens in DocDB. If I understand correctly the replication should be real time and anything I create should be replicated instantly?
Also there are no indication of errors or warnings whatsoever... I don't suppose its a connectivity issue to the databases since the initial data is being replicated.
Also the users I am using for the migration have admin privileges in both databases.
Does anyone have any suggestions?
#PPetkov - could you check the following?
1. Check if right privileges were assigned to the user in the MongoDB endpoint according to https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html.
2. Check if replicate set to capture changes was appropriately configured in the MongoDB instance.
3. Once done, try to search for "]E:" or "]W:" in the CloudWatch logs to understand if there are any noticeable failures/warnings.
I have a mongo 2.4.8 database setup and running in a live environment. I am wanting to add a replica however I would like to use the latest version 3.2.9 for the replica.
Is the only way for me to do this to upgrade the current node to version 3.2.9 then add the replica?
My plan would be sync all the data to the new node make it primary then update the old node to the latest version is this possible?
yes, you can create a new node and make a replica, and update the old node.
few things to keep in mind are:-
The default storage engine for 3.2.9 will be wiredtiger and for 2.4.8 it will be mmapv1, so you would have to change the configuration so that you can keep on using mmapv1 as your storage engine.
Do replication very carefully. if not done properly, there are chances that the whole database is blown. i recommend you to take the backup of the database before doing replication
I would definitely go with the first method that you mentioned. Upgrade the current stand alone database and then create a replica set. I tried to find the best practice from Mongodb, but I couldn't find an answer. So, I asked Adam ex employee of MongoDB and creator M202 course to find his opinion.
Source: Adam, ex employee of Mongodb
I have gone with the route of a full mongo backup then restore into the new nodes.
The replication old to new was very fragile plus the backup is very fast to do as long as you allowed to bring the server down.
ShardingTest() object could be used to create a testing sharding, which I would like to use for simple development and testing.
Howevere, I could not restart the testing db I have created before, as using following commands:
mongo --nodb
cluster = new ShardingTest({shards:3, chunksize:1, config:3, rs:{nodes:{r1:{startClean:false},r2:{startClean:false},a:{startClean:false}}}})
I have tried with the following with no luck:
cluster = new ShardingTest({shards:3, chunksize:1, restart:true, config:3, rs:{nodes:{r1:{startClean:false},r2:{startClean:false},a:{startClean:false}}}})
Does there is any way to restart the previous setup shard, please?
I have to insert every records everytimes I restart the shard, which is very time-consuming.
Thank you so much!
ShardingTest doesn't persist the data in shards like a real MongoDB setup so if your testing needs need persistence, another tool is warranted.
A nice tool for quickly setting up development MongoDB clusters is mlaunch.
I have used NORM driver in production. New year holidays - it is pretty cool, so my project get high loading and i want to set up a replication set, but have a problem - Norm does not support replication set :( . as far as i understand sharding too?
Help me :) Who did use mongodb csharp or official 10gen driver with replset? Is there any problem on production? If i choose another driver I'll have to rewrite the repository, but I do not want it to be in vain. Is there some issues?
Sharding should not depend on driver-specific support. When you shard, you connect to a router application mongos and this router behaves exactly like mongod.
So you should be able to shard. But you will probably need to change the "connection string". The suggested setup is to have one mongos per application server (instead of your current single mongod).