I've implemented a replica set that I'm using globally. I have my master in Oregon, US and 4 secondaries. California and Virginia, Frankfurt and Sydney. I also have web servers in those same regions as well. Those web servers connect to mongo using mongoose:
var mongoose = require("mongoose");
var dbUrl = "mongodb://***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017/exampleDb";
var dbOptions : {
"replSet": {
"rs_name": "exampleRepSet",
"readPreference": "ReadPreference.SECONDARY_PREFERRED",
"read_preference": "ReadPreference.SECONDARY_PREFERRED",
"w":0,
"slaveOk": true
}
}
mongoose.connect(dbUrl, dbOptions);
My problem is that my client's have higher latency to the database depending on how far away they are from the master. California get 40ms while Sydney gets 400ms. I don't understand why this is happening since they should be reading off of the secondary database in their region.
I understand that writes must be done to the primary but even if I perform a find then shouldn't it be done on the regional secondary and return pretty quick?
I realize there are some redundant options in that config but I'm getting desperate. I've also tried the option "ReadPreference.NEAREST" to no avail.
Try using the following options:
var mongoose = require("mongoose");
var dbUrl = "mongodb://***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017,***.***.***.***:27017/exampleDb";
mongoose.connect(dbUrl, {
server: {
readPreference: "nearest",
strategy: "ping"
},
replset: {
rs_name: "exampleRepSet",
readPreference: "nearest",
strategy: "ping"
}
});
Whilst the documentation specifies ping as the default strategy, it seems Mongoose mandates that you specify one when you use readPreference.
Also note that secondaryPreferred is not the same thing as nearest. secondaryPreferred prefers reads off of secondary members (as the name suggests) regardless of network latency, where nearest prioritizes reads to the member with the lowest amount of network latency.
Short of a misconfiguration in your replica set, make sure your primary is online and reachable - by default Mongoose will refuse to use a secondary if the primary is offline.
Edit
Try setting the read preference on the connection string itself with mongodb://connection/db/?readPreference=secondary and not in the dbOptions. I can't find anything in the node-mongodb-native that says a read preference can be added to the replset config. http://mongodb.github.io/node-mongodb-native/2.0/api/ReplSet.html
Old Answer
You may need to set the setting to nearest and not secondary preferred. http://docs.mongodb.org/manual/reference/read-preference/#nearest
dbOptions.db = {
readPreference: "secondaryPreferred",
};
This worked for me
Related
Is there such possibility from database backend to force user to read only from SECONDARY members ?
I would like to restrict some users to not be able to impact performance in PRIMARY replicaset members in my on-premise deployment ( not atlas )
Issue is easy to solve if customer agree adding to the URI
readPreference=secondary
But I am checking if there is option to force from the database side without asking the customer ...
the only option I have found is to restrict by server IP address:
use admin
db.createUser(
{
user: "dbuser",
pwd: "password"
roles: [ { role: "readWrite", db: "reporting" } ],
authenticationRestrictions: [ {
clientSource: ["192.0.2.0"],
serverAddress: ["198.51.100.1","192.51.100.2"]
} ]
}
)
There are currently no supported ways to enforce this from within MongoDB itself apart from the authenticationRestrictions configurations for defining users which is noted in the question itself.
Regarding the comments - ANALYTICS tag in Atlas are a (automatic) Replica Set Tag. Replica set tags themselves can be used in on-premise deployments. But tags are used in conjunction with read preference which is set by the client application (at least in the connection string). So that approach/solution really doesn't provide any additional enforcement from read preference alone for the purposes of this question. Additional information about tags can be found here and here.
In an 'unsupported'/hacky fashion, you could create the user(s) directly and only on the SECONDARY members that you want the client to read from. This would be accomplished by taking the member out of the replica set, starting it up as a standalone, creating the user, and then joining it back to the replica set. While it would probably work, there are a number of implications that don't make this a particularly good approach. For example, elections (for high availability purposes) would change the PRIMARY (therefore where the client can read from) among other things.
Other approaches to this would be in redirecting/restricting traffic at the network layer. Again not a great approach.
Use case
I'd like to be able to read from the 'nearest' mongodb node when it is just a read operation, but to be able to make transactions (which require a 'primary' readPreference).
Currently I have to set readPreference="primary" on the connection, to be able to make transactions.
I did try to set readPreferene to "nearest" on the connection, and when making the transaction calls involving the session to set it to primary, but didn't work (not that I ever found any documentation saying it's supposed to work).
mongoose.model("cars").create([{name: "subaruo"], {
session: session,
readPreference: "primary",
});
Is there a way to do this?
The solution is to specify the readPreference="primary" in the call to session.startTransaction()
session.startTransaction({
readPreference: "primary",
});
I'm just getting started with mongoDB. I am trying to understand how to set up my secondary database servers so that when there is no primary, the secondaries can be used to read data. I believe the read preference I'm going for is preferredPrimary.
Now that I kinda understand which of the read preferences I want to test out, I'm trying to understand how to set up my replica set for preferredPrimary.
I've been reading through the following documentation:
http://docs.mongodb.org/manual/tutorial/configure-replica-set-tag-sets/
Questions:
Is this the right doc to follow to set up read preferences?
Assuming that it is, I want to verify that the tags names / values are anything that I come up with? So specifically, the key used in the example "dc" is NOT a keyword in mongo. Is that correct?
once I set up these tags, in my client, when I'm connecting to the mongo database, do i have to specify any settings? I'm using a php front end, and I found this:
http://php.net/manual/en/mongodb.setreadpreference.php
can you confirm that these tags replace the rs.slaveOK() method?
Environment:
mongoDB version 2.6.5
replica set with 3 members - one primary and 2 secondary servers
Yes
Yes
Yes, but the link that you provided is only for readPreference
You also need to supply custom writeConcern (extract from link in a question):
db.users.insert( { id: "xyz", status: "A" }, { writeConcern: { w: "MultipleDC" } } )
Look into php driver documentation how to do that.
Yes, you may skip call to slaveOK in this case (especially, that in 95% cases you will be reading from primary)
According to the mongodb website, I should be able to connect to a replica set if I just give it one member from the replica set:
"The C# Driver is able to connect to a replica set even if the seed list is incomplete. It will find the primary server even if it is not in the seed list as long as at least one of the servers in the seed list responds (the response will contain the full replica set and the name of the current primary)." http://www.mongodb.org/display/DOCS/CSharp+Driver+Tutorial#CSharpDriverTutorial-Connectionstrings
However, I cannot get my driver to connect if I just give it a secondary member.
This is my current connection statement:
m_server = MongoServer.Create(new MongoServerSettings { ConnectionMode = ConnectionMode.ReplicaSet, Server = new MongoServerAddress(connection) });
The 'connection' variable is: mongodb://servername/?safe=true
I saw this: https://jira.mongodb.org/browse/CSHARP-500, and I did run rs.status(), and did use the correct server name. Any help is appreciated!
At this moment I’m learning MongoDB and I’ve being playing around replica set connections. I like to contribute with 2 ways that I have used to connect to the database that I found useful, if doesn’t help anyone, at least I will have a place to refer to in the future (I’m sure I’m going to need it at some point)
first:
var connString = "mongodb://localhost:27029,localhost:27027,localhost:27028?connect=replicaSet";
var client = new MongoClient(connString);
var db = client.GetDatabase("test");
second:
var settings = new MongoClientSettings
{
Servers = new[]
{
new MongoServerAddress("localhost", 27027),
new MongoServerAddress("localhost", 27028),
new MongoServerAddress("localhost", 27029)
},
ConnectionMode = ConnectionMode.Automatic,
ReplicaSetName = "m101",
WriteConcern = new WriteConcern(WriteConcern.WValue.Parse("3"),wTimeout:TimeSpan.Parse("10"))
};
var client = new MongoClient(settings);
The first, allows me to connect to the database through the servers specified in the list of server. This allows the driver to connect automatically to the new principal node in the replica set in the case of failure with the principal.
With the second, I send the list of servers in the replica set, the connection type. The name of the replica set, and the write concern configuration. With this settings, I’m forcing the driver to wait for an acknowledge of writing from the 3 servers in the replica set (WValue:3) and to wait at the most 10 seconds for the confirmation of writing.
So, the connection variable is a full connection string, not something to pass to MongoServerAddress. Also, you can specify the connection mode on the connection string as well. Try this:
connection = "mongodb://servername/?safe=true&connect=replicaset";
m_server = MongoServer.Create(connectionString);
I have MongoDB replica Set as:
Host1: 10.10.10.1:27017
Host2: 10.10.10.1:27018
Host3: 10.10.10.2:27017
Host4: 10.10.10.2:27018
When I set the connection (e.g. IN Mongoose), I have to do this:
mongoose.connectSet('mongodb://10.10.10.1:271017/mydb,
mongodb://10.10.10.1:27018/mydb,
mongodb://10.10.10.2:27107/mydb, mognodb://10.10.10.27108/mydb);
can I just do this?
mongoose.connectSet('mongodb://10.10.10.1/mydb, mongodb://10.10.10.2/mydb');
How about express-session-mongo and mongolian? how to set up the connection to replica set?
Why I have to define all the host names? shouldn't the driver know all the primary and secondary and keep the lookup table, instead of manually define it during setting up the connection?
so the mongodb driver that is used by mongoose will find all members of the replicaset if you have at least one member in the original set. This is due to each member in a replicaset knowing about all the other members. But the more servers you know about the better as it makes it more likely to find a server in case your only passed in server is down.