server :ubuntu 16.04
database :mongodb 3.2.10
configuration (replica set 3 nodes)
engine WiredTiger
we are having performance related to querying , the query run time degraded over the time and it does not seem to be related to load as no users accessing the node , the current fix is by restarting the mongo instance. i had the same issue on 3.2.9 version , it seems to keep happening on 3.2.10 as well the degradation is happening on all nodes.
Few tips to verify performance issues in MongoDB
In MongoDB Profiler you can check the slow running queries.
You can try indexing the documents (use inputs from above step)
Since the instances are replicated please revisit the write concern part https://docs.mongodb.com/manual/core/replica-set-write-concern/
can you check whether mongodb in-memory implementation can help https://docs.mongodb.com/manual/core/inmemory/
You can see few important tips here - https://docs.mongodb.com/manual/administration/analyzing-mongodb-performance/
Related
I am running tests on the server, as a result of which mongodb starts using a lot of RAM. Mongo version 3.6.8, arm architecture on the server, there is no swap file. I've tried all the solutions from here Is there any option to limit mongodb memory usage? . In particular, when using cgroups, mongo exceeds the limit and dies.
I will also say that the memory that mongodb uses for the cache is not returned if necessary.
I will be glad of any help.
In the server structure, primary, secondary, and arbiter are each physically operated.
mongo db version is 4.2.3.
Some of the documents were deleted in the oldest order because too many documents were accumulated in a specific collection.
However, even deleting documents did not release the storage area.
Upon checking, I found that mongodb's mechanism retains reusable bytes even if the document is deleted.
Also, I found out that unnecessary disk space can be freed with the compact command in the WiredTiger engine.
Currently, all clients connected to the db are querying using the arbiter ip and port.
Since the DB is composed only of replication, not sharding, if each individual executes the compact command independently, Even if each instance is locked, it is expected that the arbiter will distribute the query to the currently available instances.
Is this possible?
Or, Should I shutdown each instance, run it standalone, run the compact command, and then reconfigure psa?
You may upgrade your MonogDB to latest version 4.4. Documentation of compact:
Blocking
Changed in version 4.4.
Starting in v4.4, on WiredTiger, compact only blocks the following
metadata operations:
db.collection.drop
db.collection.createIndex and db.collection.createIndexes
db.collection.dropIndex and db.collection.dropIndexes
compact does not block MongoDB CRUD Operations for the database it is
currently operating on.
Before v4.4, compact blocked all operations for the database it was
compacting, including MongoDB CRUD Operations, and was therefore
recommended for use only during scheduled maintenance periods.
Starting in v4.4, the compact command is appropriate for use at any
time.
To anyone looking for the answer with 4.4 please see this bug and the documentation entry as the compact routine still forces the node to recovery state if you are running in replica set (and I assume this is the default use case for most projects)
I'm using Mongodb on my Windows server 2012 for more than two years. Since the last update some weird issues started to happen which in the end lead to usage of the entire RAM memory.
The service Iv'e configured for Mongodb is as follows:
logpath=d:\data\log\mongod.log
dbpath=d:\data\db
storageEngine=wiredTiger
rest=true
#override port
port=27017
#configsvr = true
shardsvr = true
And in order to limit the Cache memory usage Iv'e added the following line:
wiredTigerCacheSizeGB=10
And this is where the weird stuff started happening. When I check the task manager it says that now Mongodb is really limited to 10GB as I defined in the service but it is actually using a lot more than 10GB.
In the first image you can see the memory consumption sorted by RAM consumption
While in fact the machine I'm using has 28GB in total
This crazy consumption leads to failure in the scripts I'm running, even the most basic ones, even when I only run simple queries like 'count' or 'distinct', I believe that this is a direct results of the memory consumption.
When I checked the log files I saw that there are many open connections that even when the session ends it indicates that still the same amount of connections is opened:
So in the end I have two major questions:
1. Is there a way of solving this issue without downgrading the Mongodb version?
2. The config file looks right? is everything there is necessary?
Memory usage in WiredTiger is a two-level cache:
First is the WiredTiger cache as controlled by --wiredTigerCacheSizeGB
Second is the Operating System filesystem cache. MongoDB automatically uses all free memory that is not used by the WiredTiger cache or by other processes
See also WiredTiger memory usage
For OS filesystem cache, MongoDB doesn't manage the memory it uses directly - it lets the OS manage it. Windows will try to use every last scrap of physical memory if it can - but lots of it should and will be thrown out if other processes request memory.
An alternative is to run mongod in a container (e.g. lxc, cgroups, Docker, etc.) that does not have access to all of the RAM available in a system.
Having said the above:
You are also running another database in the server i.e. mysqld. MongoDB, like some databases will perform better on a dedicated server to reduce memory contention.
Task Manager shows mongod is using 10GB, although the machine is using up to ~28GB. This may or may not be mongod as you have other processes as well.
Useful resources:
FAQ: Memory diagnostics for WiredTiger
FAQ: MongoDB Cache Handling
MongoDB Production Notes
I'm running MongoDB 2.4.5 and recently I've started digging into Replica Set to get some kind of redundancy.
I started same mongo instance with --replSet parameter and also added an Arbiter to running Replica Set. What happened was writing to mongo slowed down significantly (from 15ms to 30-60ms, sometimes even around 300ms). As soon as I restarted it in not-replicaset-mode performance went back to normal.
I also set up the newest 3.0 version of MongoDB with no data and run same tester as before and the result was quite similar - writes were at least 50% slower while running the ReplicaSet mode.
I could not find many examples of such behaviour online so I guess something is wrong with my mongo configuration or OS configuration.
Any ideas? Thanks for help.
It sounds like you're using "replica acknowledged" write concern, which means that the operation will not return until the data has been written to both the primary and replica. The write concern can be set when doing any write operation (from 2.6 onwards - it looks from the 2.4 documentation that calling getLastError causes a write concern of replica acknowledged in 2.4, are you doing that in your test code?).
Read this section (v3)) or this section (v2.4) of the MongoDB documentation to understand the implications of different write concerns and try explicitly setting it to Acknowledged.
Okay so the problem was C# library. I used a native C# driver (works fine even with 2.4.5 MongoDB) and there seems to be no difference in performance. Thanks for help.
I am trying to perform performance testing for one of my application using MongoDB. I am using 2.5.5 development version. Sharding works fine when I try to read and write data using mongos.
To perform performance testing I need to start 600-700 mongoconnection threads to the mongos. Each thread queries around 2000 documents which is distributed on two shards. This test runs fine for few minutes but after sometime it stops working with the error "Connection refused by one of the shard". Looking closely at it I found that server runs out of ports when these many threads request data.
Could anyone please have a look and let me know if it is a MongoDB bug in the dev version or is it something which I am doing wrong while connecting to database.
your help will be much appreciated.
Thanks,
Vibhu