Created an AWS DMS pipeline:
Source endpoint - MongoDB
Target endpoint - RDS Postgres SQL
Successfully did all the security configuration, and both endpoints returned successful while testing it.
For the MongoDB source, I am using one of the three replicas sets with a username and a password that is not the admin username.
I also added the privilege "changeStream" in the replica set user.
But when starting the DMS migration task getting this error in cloud watch.
Encountered an error while initializing change stream: 'not authorized on admin to execute command
{ aggregate: 1, pipeline: [ { $changeStream: { fullDocument: "updateLookup", startAtOperationTime: Timestamp(1656005815, 0),
allChangesForCluster: true } }, "ok" : { "$numberDouble" : "0.0" },
"errmsg" : "not authorized on admin to execute command { aggregate: 1, pipeline: [ { $changeStream: { fullDocument:
\"updateLookup\", startAtOperationTime: Timestamp(1656005815, 0), allChangesForCluster: true } },
74f1-4aab-9ca1-f964ab655777\ (change_streams_capture.c:356)
Assuming this is due to some missing privileges in mongo replica sets USER.
Related
For known all databases (even with just users declared) I use with pymongo (pymongo==3.10.1 with mongodb 4.2)
db.command({'usersInfo': { 'forAllDBs': True },
'showCredentials': True})
this command need userAdminAnyDatabase builtin-role on client user.
But this role provide to much privileges. If someone crack the user password he can upgrade role to dbAdminAnyDatabases.
So I failed to create a any database role with authorization (role with only viewUser action on any databases) for userinfos previous command.
any can help me for this role definition?
You could try anyResource to get all the databases.
Thank's i create role :
with pymongo.MongoClient('mongodb://localhost:27017/',
username='SuperAdmin',
password='XXXXXXXXXX',
authSource='admin',
authMechanism='SCRAM-SHA-256') as client:
extendRole="showUsersAnyBase"
crole=client['admin'].command('rolesInfo',extendRole)
if reinit and extendRole in [x['role'] for x in crole['roles']]:
client['admin'].command('dropRole',extendRole)
if reinit or extendRole not in [x['role'] for x in crole['roles']]:
client['admin'].command({'createRole': extendRole,
'privileges': [{'resource': { 'anyResource': True },
'actions': [ "viewUser" ]}],
'roles':[ ]
})
create connector user:
usersInfos=client['admin'].command({'usersInfo': [{'user' : 'connector','db': 'admin'}]})
if reinitConnector and 'connector' in [x['user'] for x in usersInfos['users']]:
client['admin'].command('dropUser',"connector")
if reinitConnector or 'connector' not in [x['user'] for x in usersInfos['users']]:
client['admin'].command("createUser", "connector",
pwd="XXXXXXXXXX",
roles=[extendRole])
use the connector user with for userInfos command:
with pymongo.MongoClient('mongodb://localhost:27017/',
username='connector',
password='XXXXXXXXXX',
authSource='admin',
authMechanism='SCRAM-SHA-256') as client2:
usersInfos=client2['admin'].command({
'usersInfo': { 'forAllDBs': True },
'showCredentials': True
})
for user in usersInfos['users']:
print("user:",user['user'],
"db:",user['db'],
"roles:",[x['db']+'->'+x['role'] for x in user['roles']])
Works fine !!
and connector user can't grant role:
client2['admin'].command('grantRolesToUser','connector',
roles=['dbAdminAnyDatabase'])
throw exception :
.....
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: not authorized on admin to execute command { grantRolesToUser: "connector", .....
I'm trying to enable PLAIN authentication security over a mongodb replica shard managed with OpsManager following their documentation https://docs.opsmanager.mongodb.com/v4.0/tutorial/enable-ldap-authentication-for-group/ .
The issue I'm facing is at the automation-agent trying to get mongoS status while restarting after enabling security. Please see the error output below:
<mongos_5> [09:18:19.711] Failed to compute states :
<mongos_5> [09:18:19.711] Error calling ComputeState : <mongos_5> [09:18:19.632] Error getting current config from running mongo using conn params = mongos01:27017 (local=false) :
<mongos_5> [09:18:19.632] Error getting pid for mongos01:27017 (local=false) :
<mongos_5> [09:18:19.632] Error running command for runCommandWithTimeout(dbName=admin, cmd=[{serverStatus 1} {locks false} {recordStats false}]) :
result={"$clusterTime":{"clusterTime":6808443558471663617,"signature": {"hash":"e44BxV30B7dTpampo4VZsVuio7E=","keyId":6808441655801151517}},"code":13,"codeName":"Unauthorized",
"errmsg":"command serverStatus requires authentication","ok":0,"operationTime":6808443558471663617} connection=&{mongos01:27017 (local=false) 2 true 0xc4207b21a0 2020-03-26 09:18:19.627337419 +0000 UTC 0xc4207bdef0 <nil> }
identityUsed= : command serverStatus requires authentication
I noticed that even if opsmanager is not able to get the status the security was enabled successfully and PLAIN authentication mechanism works but the status hangs at
Start the process ... Start MongoDB process
I tried this over the API following mongodb-labs repo https://github.com/mongodb-labs/mms-api-examples/blob/master/automation/api_usage_example/configs/security_ldap_cluster.json but also manually following mongodb docs but everytime I'm facing the same error.
After all I enabled LDAP(PLAIN) only for mongo in mongoconfig file (see below the ops manager API snippet call example), and avoid enable in opsmanager for the agents also.
{
"args2_6": {
"net": {
"port": 28001
},
"replication": {
"replSetName": "rs0"
},
"storage": {
"dbPath": "/data/mongo"
},
"systemLog": {
"destination": "file",
"path": "/data/mongo/mongodb.log"
},
"security": {
"authorization": "enabled"
},
"setParameter": {
"saslauthdPath": "",
"authenticationMechanisms": "PLAIN,MONGO-CR,SCRAM-SHA-256",
}
}, ...
I have a trouble.
I'm trying to apply a role to user.
I want the user only find, insert, remove and update specific collection
I'm doing this.
Create role:
db.createRole({role:'user_role_col',privileges:[{resource:{db:'something_else', collection: 'col_something_else'}, actions: ['find','remove','insert','update']}], roles: []});
.
Create user with that role:
db.createUser({user: 'user_col',pwd: '1234',roles: [{role: "user_role_col", db: "something_else"}]})
.
When I created role and user, I stay on something_else database (use something_else)
0 errors got, but I can only read col_something_else, I cannot remove, update, or insert :(
What am I doing wrong?
I believe the following commands replicates what you did. Here I'm creating the role and user in the test database:
> use test
> db.createRole({role: 'testRole', privileges: [{resource: {db:'test', collection:'test'}, actions:['find','update','insert','remove']}], roles: []})
> db.createUser({user:'testUser',pwd:'password',roles:[{role:'testRole', db:'test'}]})
I would then have to quit the mongo shell and re-authenticate using the new credentials. However, the authenticationDatabase parameter must point to the test database, since that's where I created the role and the user:
$ mongo -u testUser -p password --authenticationDatabase test
> db.test2.insert({a:1}) //try inserting to a non-"test" collection
unauthorized
> db.test2.find() //try find on a non-"test" collection
unauthorized
> db.test.insert({a:1}) //insert into "test" collection
Inserted 1 record(s)
> db.test.updateMany({},{$set:{a:2}}) //update "test" collection
{
"acknowledged": true,
"matchedCount": 1,
"modifiedCount": 1
}
> db.test.find() //find on "test" collection
{
"_id": ObjectId("5a0bcfa4322032cfcc3a69c6"),
"a": 2
}
> db.test.deleteOne({a:2}) //delete from "test" collection
{
"acknowledged": true,
"deletedCount": 1
}
> db.test.drop() //try to drop "test" collection
2017-11-15T16:32:07.715+1100 E QUERY [thread1] Error: drop failed: {
"ok": 0,
"errmsg": "not authorized on test to execute command { drop: \"test\" }",
"code": 13,
"codeName": "Unauthorized"
}
I found that the new custom role (testRole) is authorized correctly. This is using MongoDB 3.4.10.
I have an existing mongodump of a single collection that I am trying to restore. After running mongo restore, no errors show up and the data is not in the collection. Are there any known reasons how this could happen? I would expect if the data weren't inserted for some reason, an error would be provided in the log.
To create and attempt to restore the dump, I followed the answer provided for this question:
How to use mongodump for 1 collection
I've created a new database on a different server and it has an empty collection. I've checked the mongo log file and there are no errors, it shows the connection open and authenticate, then disconnect on the next line.
mongorestore -vvvvv -u user -p 'password' --db=MyDatabase --collection=MyCollection dump1/MyCollection.bson
2015-03-04T18:20:31.331+0000 creating new connection to:127.0.0.1:27017
2015-03-04T18:20:31.332+0000 [ConnectBG] BackgroundJob starting: ConnectBG
2015-03-04T18:20:31.332+0000 connected to server 127.0.0.1:27017 (127.0.0.1)
2015-03-04T18:20:31.332+0000 connected connection!
connected to: 127.0.0.1
2015-03-04T18:20:31.333+0000 drillDown: dump1/MyCollection.bson
2015-03-04T18:20:31.333+0000 dump1/MyCollection.bson
2015-03-04T18:20:31.333+0000 going into namespace [MyDatabase.MyCollection]
Restoring to MyDatabase.MyCollection without dropping. Restored data will be inserted without raising errors; check your server log
file size: 94876
130 objects found
2015-03-04T18:20:31.336+0000 Creating index: { key: { _id: 1 }, name: "_id_", ns: "MyDatabase.MyCollection" }
2015-03-04T18:20:31.340+0000 Creating index: { key: { geometry: "2dsphere" }, name: "geometry_2dsphere", ns: "MyDatabase.MyCollection", 2dsphereIndexVersion: 2 }
Log file:
2015-03-04T18:20:31.333+0000 [conn874] authenticate db: MyDatabase { authenticate: 1, nonce: "xxx", user: "user", key: "xxx" }
2015-03-04T18:20:31.342+0000 [conn874] end connection 127.0.0.1:59420 (25 connections now open)
The query I am using on the origin and destination is:
db.MyCollection.find()
On the origin server, the collection has 130 elements, which is what is also shown in the mongorestore output "130 objects found".
Edit:
I added the --drop option to the mongorestore command. The log file output clearly shows that it is creating the index on an empty collection.
2015-03-20T15:03:57.565+0000 [conn61965] authenticate db: MyDatabase { authenticate: 1, nonce: "xxx", user: "user", key: "xxx" }
2015-03-20T15:03:57.566+0000 [conn61965] CMD: drop MyDatabase.MyCollection
2015-03-20T15:03:57.631+0000 [conn61965] build index on: MyDatabase.MyCollection properties: { v: 1, key: { _id: 1 }, name: "_id_", ns: "MyDatabase.MyCollection" }
2015-03-20T15:03:57.631+0000 [conn61965] added index to empty collection
2015-03-20T15:03:57.652+0000 [conn61965] build index on: MyDatabase.MyCollection properties: { v: 1, key: { geometry: "2dsphere" }, name: "geometry_2dsphere", ns: "MyDatabase.MyCollection", 2dsphereIndexVersion: 2 }
2015-03-20T15:03:57.652+0000 [conn61965] added index to empty collection
2015-03-20T15:03:57.654+0000 [conn61965] end connection 127.0.0.1:59456 (21 connections now open)
So the issue ended up being that the user I was trying to do the restore with only had the read and dbAdmin roles. I had made a separate user so that the regular user used by the application did not have administrative rights. After changing my user's role from read to readWrite, it worked as expected.
To be honest, if the user didn't have the correct permissions, I really would have expected the log to show an error of some sort when it tries to run the restore without the correct permission.
I have created a user "mongo01testro" in the mongo01test database.
use mongo01test
db.addUser( "mongo01testro", "pwd01", true );
db.system.users.find();
{ "_id" : ObjectId("53xyz"), "user" : "mongo01testro", "readOnly" : true, "pwd" : "b9eel61" }
When I logged in from another session as this newly created user,
I am able to insert documents into the collection which is strange.
I am looking to do the following:
Create 2 separate users one for read only and one for read write for
each database.
Create an admin user which have sysadmin/dba access to all the
databases in MongoDB instance used for Backup/Recovery or admin
purpose.
Please kindly help.
Regards,
Parag
You forgot --auth to enable
Create Users
// ensure that we have new db, no roles, no users
use products
db.dropDatabase()
// create admin user
use products
db.createUser({
"user": "prod-admin",
"pwd": "prod-admin",
"roles": [
{"role": "clusterAdmin", "db": "admin" },
{"role": "readAnyDatabase", "db": "admin" },
"readWrite"
]},
{ "w": "majority" , "wtimeout": 5000 }
)
// login via admin acont in order to create readonly user
// mongo --username=prod-admin --password=prod-admin products
db.createUser({
"user": "prod-r",
"pwd": "prod-r",
"roles": ["read"]
})
Enable auth:
sudo vim /etc/mongod.conf # edit file
# Turn on/off security. Off is currently the default
#noauth = true
auth = true
sudo service mongod restart # reload configuiration
Check write restriction:
# check if write operation for readonly user
$ mongo --username=prod-r --password=prod-r products
MongoDB shell version: 2.6.4
connecting to: products
> db.laptop.insert({"name": "HP"})
WriteResult({
"writeError" : {
"code" : 13,
"errmsg" : "not authorized on products to execute command { insert: \"laptop\", documents: [ { _id: ObjectId('53ecb7115f0bfc61d8b1113e'), name: \"HP\" } ], ordered: true }"
}
})
# check write operation for admin user
$ mongo --username=prod-admin --password=prod-admin products
MongoDB shell version: 2.6.4
connecting to: products
> db.laptop.insert({"name": "HP"})
WriteResult({ "nInserted" : 1 })
# check read operation for readonly user
$ mongo --username=prod-r --password=prod-r products
MongoDB shell version: 2.6.4
connecting to: products
> db.laptop.findOne()
{ "_id" : ObjectId("53ecb54798da304f99625d05"), "name" : "HP" }
MongoDB changed the way it handles users in versions >= 2.2 and 2.6 and if you are updating mongodb you will have to Upgrade User Authorization Data to 2.6 Format.
In versions < 2.2 (legacy) you use the db.addUser(username, password, readOnly) function as #Hüseyin BABAL suggested. If you are using version > 2.2 or 2.6 you have a lot more control over what you can do with roles and privileges.
So assuming you use mongo v > 2.6 you can create something like:
use admin
db.createUser({
user: "myUsername",
pwd: "mypwd",
roles: [{
role: "userAdminAnyDatabase",
db: "admin"
}]
})
db.addUser({
user: "username",
pwd: "password",
roles: [{
role: "readWrite",
db: "myDBName"
}]
})
You can use the Build in Roles or you can even create custom roles.
So as you can see you clearly have a lot more options when using v > 2.6 when it comes to authentication and role management.
if the Access control is enabled on the MongoDB deployment, Then you should login by authenticating to the relevant database with admin user (or user with userAdmin role) which you want to control the access. to do that
mongo <db_name> -u <user_name> -p <user_password>
ex: mongo mongo01test -u admin -p admin_paaswrod
then execute the following query to create a read only user for current connected database,
db.createUser({user:"user_name",pwd:"password",roles:[{role:"read", db:"db_name"}]});
ex:db.createUser({user:"user_rd",pwd:"password",roles:[{role:"read", db:"mongo01test"}]});
create a user with both readWrite access,
db.createUser({user:"user_name",pwd:"password",roles:[{role:"readWrite", db:"db_name"}]});
ex:db.createUser({user:"user_rw",pwd:"pass_rw",roles:[{role:"readWrite", db:"mongo01test"}]});
to create a user with admin privileges,
use admin;
db.createUser({user:"admin_user_name",pwd:"password", roles:[{role:"dbAdmin", db:"admin"},{role:"readWriteAnyDatabase", db:"admin"},{role:"backup", db:"admin"},{role:"restore", db:"admin"}]});
Here is my scneario;
// Create admin user
use admin
db.addUser('root', 'strong_password');
// Read only user
use dbforuser1
db.addUser('user1', 'user1_pass', true);
// Read / Write access
use dbforuser2
db.addUser('user2', 'user2_pass');
If you want to login to dbforuser1
use dbforuser1
db.auth('user1', 'user1_pass')