wolkenkit: DATABASE access & backup - mongodb

I just cloned wolkenkit-todomvc from repo. I tried to execute it with "wolkenkit start" then
docker exec -it todomvc-mongodb mongo admin
show dbs
and got
{
"operationTime" : Timestamp(1514838628, 1),
"ok" : 0,
"errmsg" : "not authorized on admin to execute command { listDatabases: 1.0 }",
"code" : 13,
"codeName" : "Unauthorized"
}
My questions are
What is the user and password for access these container's database?
how do I access Mongodb and Postgres data from docker container?. such as
db.todos.find()
SELECT * FROM todos

To execute wolkenkit-todomvc you first need to install wolkenkit. The concrete steps depend on your platform. Refer to installing wolkenkit on macOS, installing wolkenkit on Linux, or installing wolkenkit on Windows.
Then run the application by running the following command, as described in the quick start:
$ wolkenkit start
If you want to access the database explicitly, as I assume from reading your question, e.g. to create a backup, you need to use the wolkenkit user. The password is the so-called "shared key", which is shown when running wolkenkit start. If you want to, you can also set the shared key manually, otherwise it is created randomly.
Please note that there are two databases, as you can see from the architecture of wolkenkit, one using PostgreSQL to store the events, one using MongoDB to store the denormalized lists. In the events database, there is only one large table for all the events. This is the raw data. If you are interested in the interpreted views, take a look at the MongoDB.
If you want to create a backup, backup the event store, as this is the single source of truth. The read database can be rebuilt from the event store, but not vice-versa.
Hope this helps.
PS: Please note that I am one of the authors of wolkenkit.

Related

I followed the mongoDB docs and now cannot access my DB I get this error: connect ECONNREFUSED 127.0.0.1:27017

Below I list the directions I followed from the docs
Problem
After following the docs listed below I am unable to access my database. I most definitely did not forget my password as I actually saved the query that I ran to create the user I will list the query I ran below.
I get the following error when I try to connect my local host instance in MongoDB compass
connect ECONNREFUSED 127.0.0.1:27017
I think the issue might lie in the fact that I was running these command in the mongodb compass mongosh terminal and not the mongod
use admin
db.createUser(
{
user: "max",
pwd: "max",
roles: [
{ role: "userAdminAnyDatabase", db: "admin" },
{ role: "readWriteAnyDatabase", db: "admin" }
]
}
)
What I need Help with:
I need help accessing my local database and if possible setting up authentication on the schema
Below this line is the docs I followed
Start MongoDB without access control
Start a standalone
mongod
instance without access control.
Open a terminal and run the following command as the mongod user:
mongod --port 27017 --dbpath /var/lib/mongodb
The
mongod instance in this tutorial uses
port 27017
and the /var/lib/mongodb data directory.
The tutorial assumes that the /var/lib/mongodb directory exists and is the default
dbPath
. You may specify a different data directory or port as needed.
TIP
When
mongod
starts, it creates some system files in the /var/lib/mongodb directory. To ensure the system files have the correct ownership, follow this tutorial as the mongod user. If you start
mongod
as the root user you will have to update file ownership later.
Connect to the instance
Open a new terminal and connect to the database deployment with
mongosh
mongosh --port 27017
If you are connecting to a different deployment, specify additional command line options, such as
--host
, as needed to connect.
Create the user administrator
IMPORTANT
Localhost Exception
You can create the user administrator either before or after enabling access control. If you enable access control before creating any user, MongoDB provides a localhost exception which allows you to create a user administrator in the admin database. Once created, you must authenticate as the user administrator to create additional users.
I made sure that I made a user
Using
mongosh switch to the admin database add the myUserAdmin user with the userAdminAnyDatabase
and
readWriteAnyDatabase
roles":
use admin
db.createUser(
{
user: "myUserAdmin",
pwd: passwordPrompt(), // or cleartext password
roles: [
{ role: "userAdminAnyDatabase", db: "admin" },
{ role: "readWriteAnyDatabase", db: "admin" }
]
}
)
With respect to the technical issue, the error message that was provided (connect ECONNREFUSED 127.0.0.1:27017) suggests that the server is actively refusing the connection. Searching with this specific text surfaces lots of results that can probably help you solve the problem. This one, for example, suggests that one common reason for this error message is that the mongod process (the database itself) is not running. This is easy to test and reproduce, such as by trying to connect to a host and port where there is no mongod process running and listening. So I'd recommend taking a look at that question, or similar ones, to help troubleshoot and resolve your issue.
Also from the technical perspective, it is really important that we understand the different between the different processes associated with using MongoDB. As was noted in the comments:
The mongod process itself is the actual executable that needs to be up and running for the database to be accessible. Apart from starting this process with the appropriate parameters you don't do anything else with this. Typically this process needs to be --forked so that it does not stop when the command line is closed (which is what #Wernfried Domscheit was getting at in the comments). More information about this process is here in the documentation.
The shell(s) that are used to connect to and interact with the running database. The legacy one was mongo and the newer one (which is in Compass) is mongosh. This is an interface for connecting to and performing various tasks against the running database. This can include things like creating users or querying data.
This then bridges nicely toward the general advice part of this answer. It is important to keep in mind that, as of the time of writing, this question has been viewed more than 100 times. That represents a significant number of people who are volunteering their time in an attempt to help you without expecting anything in return. Said another way, we are trying to help you.
But in order to do so, we need to have an appropriate level of detail and context. It is great that you attempted to provide that information in the question itself, but ultimately it is not really enough for us to assist you effectively. Moreover, it is important to keep in mind that it is not a goal of this site (as far as I understand) to provide full tutorials or completely walkthrough configuring a new environment. Such guides can be found elsewhere, including in the documentation from the vendor that you linked. Having basic knowledge of the technology that you are working with is the foundation that allows for a "good" question. Such questions are those that allow us to help you in a focused and otherwise reasonable manner.
For MongoDB specifically, if you wanted to try to get additional support then you may try posting to the developer forums here.
The final point to make here is that it is worth considering what your ultimate goal here is. Operationally managing a database is often a task (or full job) that requires a lot of specific knowledge to perform correctly. This includes things like configuring the system properly to provide the high availability, data security, and other requirements needed by your application. Taking the time to learn these things yourself is certainly something that can be done, but the actual place that you should learn about them should probably be done primarily in a different place than this site. If you don't want to spend your time doing this, which is perfectly fine, then I would suggest looking into a hosted solution. Places like DigitalOcean and MongoDB itself offer such 'fully managed' solutions. Usually such solutions help extract away most of these underlying configuration/management details to help get you up and running faster with little upfront knowledge about operating databases being required. It may be something that is worth looking into, especially if you are just starting out on your MongoDB journey and have other tasks that you want to be focusing your time on.
I just uninstalled and reinstalled both MongoDB and MongoDB compass

Error 13 when trying to authenticate to MongoDB

I'm trying to connect to MongoDB through JDBC. The connection string is like below,
mongodb://localhost:27017/games?authSource=admin
However I'm getting the following trace:
{ "ok" : 0.0, "errmsg" : "not authorized on admin to execute command { listDatabases: 1 }", "code" : 13 }
My intention is not listing all the databases, but the user has to authenticate against admin database and can read/write on games database. What mistake I'm making here?
I want user X to authenticate against admin DB but read just games DB so not sure why it asks for listDatabase privilege.
First, I assume you are using the MongoDB Java Driver, which is actually not JDBC.
It would be helpful for you to share:
How you created your user
The Java code that you are executing
The version of mongo-java-driver and MongoDB that you are using
But based on the error, it appears that you are successfully authenticating. I strongly suspect that you are either directly calling listDatabases() or listDatabaseNames().
The other thing that does not look quite right is the fact that you are specifying authSource=admin in your MongoClientURI. But that issue should have given you an Autentication Failed error. You should be either leaving the authSource off of the connection string or specify authSource=games.
Based on what you described, when you created your user, you should have created the user in the games database (users will actually be stored in the admin database, but you would be authenticating against the games database).

MongoDB: how do I authenticate when executing the `copydb` command

I have a replica set on a remote host which requires authentication in order to connect. The original (root) user was created in the admin database which I have used in order to remotely connect. I building some sort of a "backup" script which copies a db into the replica set and in a later time I should be able to copy a db from the remote location into other MongoDB instances.
So I wrote the script to copy a database by connecting TO the remote location, authenticating and then running the db.runCommand using copydb: 1. It works great, no problems here.
When I try to copy a db back into my local machine that's when things go wrong, mainly because I have to authenticate as part of the copydb command. I originally tried to use the same technique (db.runCommand) but since the nonce and key authentication are messy by themselves I tried to solve the problem first by writing the commands manually into mongo's shell using db.copyDatabase, according to the documentation it should do this process for me.
This is the command:
db.copyDatabase('from_db', 'to_db', 'remote.host.example.com', 'my_user', 'my_password')
Which responds with:
{ "ok" : 0, "errmsg" : "Authentication failed.", "code" : 18 }
I tried switching roles (root, userAdmin, readWrite, ...) but nothing works. I tried creating another user inside the db I am trying to copy, but that didn't seems to do much other than change the response a little into:
{
"ok" : 0,
"errmsg" : "unable to login { ok: 0.0, code: 18, errmsg: \"Authentication failed.\" }"
}
I searched everywhere, went over anything in the manual which seemed remotely relevant and I still can't figure it out.
How am I suppose to copy a db from a remote location which requires an authentication??

MongoDB-CR Authentication failed

I am getting following error while authenticating user : purchase_user#purchase failed. MongoDB-CR Authentication failed. Missing credentials in user document when I access webservice through browser.
But I am able to authenticate purchase_user from mongo it returns 1 .
go to mongoDB console and delete your current user & set authSchema version to 3 instead of 5 , follow these commands in mongo console -
mongo
use admin
db.system.users.remove({}) <== removing all users
db.system.version.remove({}) <== removing current version
db.system.version.insert({ "_id" : "authSchema", "currentVersion" : 3 })
Now restart the mongod and create new user then it should work fine.
Note: use remove commands in test db only, if in production use update.
Authentication information for Kubernetes Helm Chart
If you delete the all users and authentication is enabled in the configuration (or --auth param which is set per default on the Kubernetes helm chart), it's not possible to access MongoDB any more. Its required to disable authentication, create a new user and then re-enable it.
On Kubernetes you need to edit the parameters and add --noauth as argument, since it's not the default there as on a classic installed MongoDB. Please see the CLI documentation for more information about --noauth and the corresponding --auth.
Had the same issue. What was happening to me was that when I use MongoDB 3 to create my user, it was using SCRAM-SHA-1 as it's authentication mechanism instead of MongoDB-CR. What I had to do was:
List item
Delete the created user.
Modify the collection admin.system.version such that the authSchema's currentVersion is 3 instead of 5 (3 is using MongoDB-CR).
Recreate your user.
Should work without problems now.
The step number 2. above is not detailed explicitly, I found this solution and worked for me.
var schema = db.system.version.findOne({"_id" : "authSchema"})
schema.currentVersion = 3
db.system.version.save(schema)
I think this is the answer you need:
1) Start 3.0 without auth enabled. (Auth needs to be disabled otherwise you'll get the not authorized error).
2) Run (after selecting "admin"use db):
var schema = db.system.version.findOne({"_id" : "authSchema"})
schema.currentVersion = 3
db.system.version.save(schema)
3) restart mongodb with auth enabled.
4) Create a new admin user (the old one, the one you created before this workaround won't work).
Things should work now. This issue was driving me crazy as well.
Answer came from here: https://jira.mongodb.org/browse/SERVER-17459
Adding to above solution by Vivek & explanation taken from here
use admin
db.system.users.remove({}) <== removing all users
db.system.version.remove({}) <== removing current version
db.system.version.insert({ "_id" : "authSchema", "currentVersion" : 3 })
you only need to downgrade the schema to create MONGODB-CR users.
Once they are there the old drivers will work regardless of the value
of authSchemaVersion. However if you run authSchemaUpgrade to change
from "3" to "5" the users will obviously be upgraded.
My comment regarding new users was that if you have existing SCRAM
users and change the schema manually to "3" the user documents won't
be consistent with the new schema. This is not enforced however but
the SCRAM users will still work for any driver supporting SCRAM.
Upgrade mongo-java-driver to 3.0.3 and use :-
MongoCredential.createScramSha1Credential instead of MongoCredential.createMongoCRCredential
MongoCredential createMongoCRCredential = MongoCredential.createScramSha1Credential(mongoConfiguration.getDatabaseUserName(), mongoConfiguration.getAuthenticationDatabase(),mongoConfiguration.getDatabasePassword().toCharArray());
http://docs.mongodb.org/master/release-notes/3.0-scram/
For me I was using a mongo 2 client trying to connect to a mongo 3 server. Upgrading the client fixed the issue.
I was getting this error as well.
Check your Spring Config file.. I had a constructor arg named "MONGODB-CR" which I swapped to "SCRAM-SHA-1" and it fixed my issue.
tailing the mongodb log file helped me diagnose this.
uninstall mongodb-clients packages provided by Ubuntu
install mongodb-org-shell provided by official MongoDB
This solved the problem, because The unofficial mongodb package provided by Ubuntu is not maintained by MongoDB. You should always use the official MongoDB mongodb-org packages, which are kept up-to-date with the most recent major and minor MongoDB releases.
Probably old news, and problem solved, but adding my experience with the same error:
I had the exact same problem (using MongoDB 3.0), and a C# driver that was setup to use a pre 3.0 db.
In C# I used "MongoDB.Driver.CreateMongoCRCredentials()", which caused the error the OP was getting.
The fix (for me), was to switch the command above to "MongoDB.Driver.CreateCredential()".
I guess this could be caused by using "old" users (from pre 3.0) on an upgraded system. Which either forces you to upgrade your users to the new authentication mechanism, or downgrade the authentication mechanism on your server.
June 2018 I got this error after trying to connect to my Mongodb version 3.6 from an ancient client installed in /usr/bin. I installed the mongo DB in a separate folder outside of the OS standard directory, and so my installation was conflicting with the ancient version installed by the package manager.
For those who is struggling to update auth schema (see the accepted answer) in MongoDB 3.6 due to the not authorized on admin to execute command and removing FeatureCompatibilityVersion document is not allowed errors, this is what's worked for me.
To resolve the first error:
> db.system.version.remove({})
WriteResult({
"writeError" : {
"code" : 13,
"errmsg" : "not authorized on admin to execute command { update: \"system.version\", ordered: true, lsid: { id: UUID(\"58e86006-d889-440a-bd83-ad09fcd81747\") }, $db: \"admin\" }"
}
})
I had to create a custom role that permits any action on any resource and a user with this role, then login to the admin database with that new user:
mongo admin
db.createUser({user: 'admin', pwd: 'mypwd', roles: ['root']})
exit
mongo admin -u admin -p
db.createRole({role: 'fullaccess', privileges: [{resource: {anyResource: true}, actions: ["anyAction"]}], roles: []})
db.createUser({user: 'superadmin', pwd: 'mypwd', roles: ['fullaccess']})
exit
mongo admin -u superadmin -p
(Just using the admin user with root role or disabling security.authorization in config didn't work for me and still had the same error when trying to update the system.version table.)
After that I had another error:
> db.system.version.remove({})
WriteResult({
"nRemoved" : 0,
"writeError" : {
"code" : 40670,
"errmsg" : "removing FeatureCompatibilityVersion document is not allowed"
}
})
To resolve it, we should only update the authSchema document instead of removing the whole collection.
(Generally speaking, you shouldn't blindly remove everything from system tables in production and always check what would be the implications of updating them, so that's another reason to update the needed record only.)
db.system.version.update({"_id": "authSchema"}, {currentVersion: 3})
Now you should be able to create a user with the old authentication mechanism. You also might need to switch to your database first, so that the user is created in that database rather than in admin one. Otherwise you'd have to use the authSource=admin parameter in your connection string.
(I'm actually lying here - it still will be created in admin database, just with mydb.myuser id instead of admin.myuser. But I use the same way of describing these things that's being used in MongoDB documentation. I suppose this is how it actually used to work in previous versions and in general we shouldn't care about the internal implementation details.)
use mydb
db.createUser({user: 'myuser', pwd: 'mypwd', roles: [{role: 'dbOwner', db: 'mydb'}]})
And don't forget to cleanup:
use admin
db.system.version.update({"_id": "authSchema"}, {currentVersion: 5})
exit
mongo admin -u admin -p
db.dropUser('superadmin')
db.dropRole('fullaccess')
You may want to keep the admin user - I was not able to create it again even with security.authorization setting disabled. It looks like if there are any records in admin.system.users table, the setting does not work anymore and mongo requires authentication to do something.
I had the same error with a Spring Boot app using a new MongoDB 3.2.8 database. By upgrading to the latest version of the Java Mongo driver (3.2.2) and then adding the authentication mechanism param to the URI in my application.properties, I was able to get it working:
spring.data.mongodb.uri=mongodb://myusername:mypassword#localhost/?authSource=admin&authMechanism=SCRAM-SHA-1
spring.data.mongodb.database=test

Migrate mongodb to Google Cloud Compute Cluster

I have a mongodb in a replica set running with a cloud provider called compose.io.
I just created a new Google cloud compute mongodb cluster using these instructions
I want to copy all the data in my compose database to the compute instance.
One path I have been following has led me to get a file system backup of the running database and store it locally. I have opened that database locally and executed mongodump (I didn't seem to have permission to do that against the remote database) so I have the output of mongodump and a file system copy of the database stored on my machine.
I have no idea how to get any of this in to the compute cluster I created. I don't seem to be able to run mongorestore although figuring that out is still my main path at the moment. I am getting authentication errors which may be my not getting the command right or a database configuration issue. I am not sure yet.
I tried mongorestore from my local machine to the machine holding the primary database in the replica set.
Edit:
The last thing I tried was copy scp the mongodump output on to that machine and run mongorestore there.
I got this error:
2015-01-28T23:35:40.303+0000 Creating index: { key: { _id: 1 }, ns: "admin.system.users", name: "_id_" }
Error creating index admin.system.users: 13 err: "not authorized to create index on admin.system.users"
Aborted
Now I don't seem to be able to run any commands in mongo that require any kind of privileges, such as list database. Tried passing credentials for users that existed in the original database but that is not working so far.
Here is one possible fix.
Turn off auth in mongod.conf:
# mongod.conf
#auth=false
noauth=true
Run the mongorestore, then restart mongod with auth enabled.