Mongodb authentication using elixir-mongo - mongodb

I have just started using Elixir, so I figure I have some basic misunderstanding going on here. Here is the code...
defmodule Mdb do
def connect(collection, this_db \\ "db-test") do
{:ok, mongo} = Mongo.connect("db-test.some-mongo-server.com", 12345)
db = mongo |> Mongo.db(this_db)
db |> Mongo.auth("user", "secretpassword")
db
end
end
I start with iex -S mix
and when I try db = Mdb.connect("users") I get
** (UndefinedFunctionError) undefined function: Mongo.auth/3
Mongo.auth(%Mongo.Db{auth: nil, mongo: %Mongo.Server{host: 'db-test.some-mongo-server.com', id_prefix: 12641, mode: :passive, opts: %{}, port: 12345, socket: #Port<0.5732>, timeout: 6000}, name: "db-stage", opts: %{mode: :passive, timeout: 6000}}, "user", "secretpassword")
(mdb_play) lib/mdb.ex:7: Mdb.connect/2
I looks like Mongo.auth/3 is undefined, but that makes no sense to me. Can any one point me towards my error?
thanks for the help

I just played around it, and faced the same error. As in the error message, Mongo.auth seems not defined, and it might be Mongo.Db.auth instead. However, I faced another error (ArgumentError) on Mongo.Db.auth too. It may be certain issue in the library.
** (ArgumentError) argument error
:erlang.byte_size
...
(mongo) lib/mongo_request.ex:43: Mongo.Request.cmd/3
(mongo) lib/mongo_db.ex:44: Mongo.Db.auth/1
I'm not familiar with the library, but after small change in Mongo.Db.auth, normal call seems started working.
I tried with the following sequence.
mongo = Mongo.connect!(server, port)
db = mongo |> Mongo.db(db_name)
db |> Mongo.Db.auth(user_name, password)
collection = db |> Mongo.Db.collection(collection_name)
collection |> Mongo.Collection.count()
The change I tried is in the following fork-repo.
https://github.com/parroty/elixir-mongo

Related

How to connect to Mongo DB in Docker from Scala Play Framework?

I have a Mongo DB Docker container running on 192.168.0.229. From another computer, I can access it via:
> mongo "mongodb://192.168.0.229:27017/test"
But when I add that configuration string (host="192.168.0.229") to my Play Framework app, I get a timeout error:
[debug] application - Login Form Success: UserData(larry#gmail.com,testPW)
[error] p.a.h.DefaultHttpErrorHandler -
! #7m7kggikl - Internal server error, for (POST) [/] ->
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[TimeoutException: Future timed out after [30 seconds]]]
In the past, the connection was successful with host="localhost" and even an Atlas cluster (host="mycluster.gepld.mongodb.net") for the hostname, so there were no problems connecting previously with the same code. For some reason, Play Framework does not want to connect to this endpoint!
Could it be because the hostname is an IP address? Or, maybe Play/ Akka is doing something under the covers to stop the connection (or something to make Mongo/Docker refuse to accept it?)?
I'm using this driver:
"org.mongodb.scala" %% "mongo-scala-driver" % "4.4.0"
Perhaps I should switch to the Reactive Scala Driver? Any help would be appreciated.
Clarifications:
The Mongo DB Docker container is running on a linux machine on my local network. This container is reachable from within my local network at 192.168.0.229. The goal is to set my Play Framework app configuration to point to the DB at this address, so that as long as the Docker container is running, I can develop from any computer on my local network. Currently, I am able to access the container through the mongo shell on any computer:
> mongo "mongodb://192.168.0.229:27017/test"
I have a Play Framework app with the following in the Application.conf:
datastore {
# Dev
host: "192.168.0.229"
port: 27017
dbname: "test"
user: ""
password: ""
}
This data is used in a connection helper file called DataStore.scala:
package model.db.mongo
import org.mongodb.scala._
import utils.config.AppConfiguration
trait DataStore extends AppConfiguration {
lazy val dbHost = config.getString("datastore.host")
lazy val dbPort = config.getInt("datastore.port")
lazy val dbUser = getConfigString("datastore.user", "")
lazy val dbName = getConfigString("datastore.dbname", "")
lazy val dbPasswd = getConfigString("datastore.password", "")
//MongoDB Atlas Method (Localhost if DB User is empty)
val uri: String = s"mongodb+srv://$dbUser:$dbPasswd#$dbHost/$dbName?retryWrites=true&w=majority"
//val uri: String = "mongodb+svr://192.168.0.229:27017/?compressors=disabled&gssapiServiceName=mongodb"
System.setProperty("org.mongodb.async.type", "netty")
val mongoClient: MongoClient = if (getConfigString("datastore.user", "").isEmpty()) MongoClient() else MongoClient(uri)
print(mongoClient.toString)
print(mongoClient.listDatabaseNames())
val database: MongoDatabase = mongoClient.getDatabase(dbName)
def close = mongoClient.close() //Do this when logging out
}
When you start the app, you open localhost:9000 which is simply a login form. When you fill out the data that corresponds with the data in the users collection, the Play app times out:
[error] p.a.h.DefaultHttpErrorHandler -
! #7m884abc4 - Internal server error, for (POST) [/] ->
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[TimeoutException: Future timed out after [30 seconds]]]
at play.api.http.HttpErrorHandlerExceptions$.$anonfun$convertToPlayException$2(HttpErrorHandler.scala:381)
at scala.Option.map(Option.scala:242)
at play.api.http.HttpErrorHandlerExceptions$.convertToPlayException(HttpErrorHandler.scala:380)
at play.api.http.HttpErrorHandlerExceptions$.throwableToUsefulException(HttpErrorHandler.scala:373)
at play.api.http.DefaultHttpErrorHandler.onServerError(HttpErrorHandler.scala:264)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:430)
at play.core.server.AkkaHttpServer$$anonfun$2.applyOrElse(AkkaHttpServer.scala:422)
at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:454)
at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
Caused by: java.util.concurrent.TimeoutException: Future timed out after [30 seconds]
at scala.concurrent.impl.Promise$DefaultPromise.tryAwait0(Promise.scala:212)
at scala.concurrent.impl.Promise$DefaultPromise.result(Promise.scala:225)
at scala.concurrent.Await$.$anonfun$result$1(package.scala:201)
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread$$anon$3.block(ThreadPoolBuilder.scala:174)
at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118)
at akka.dispatch.MonitorableThreadFactory$AkkaForkJoinWorkerThread.blockOn(ThreadPoolBuilder.scala:172)
at akka.dispatch.BatchingExecutor$BlockableBatch.blockOn(BatchingExecutor.scala:116)
at scala.concurrent.Await$.result(package.scala:124)
at model.db.mongo.DataHelpers$ImplicitObservable.headResult(DataHelpers.scala:27)
at model.db.mongo.DataHelpers$ImplicitObservable.headResult$(DataHelpers.scala:27)
The call to the Users collection is defined in UserAccounts.scala:
case class UserAccount(_id: String, fullname: String, username: String, password: String)
object UserAccount extends DataStore {
val logger: Logger = Logger("database")
//Required for using Case Classes
val codecRegistry = fromRegistries(fromProviders(classOf[UserAccount]), DEFAULT_CODEC_REGISTRY)
//Using Case Class to get a collection
val coll: MongoCollection[UserAccount] = database.withCodecRegistry(codecRegistry).getCollection("users")
//Using Document to get a collection
val listings: MongoCollection[Document] = database.getCollection("users")
def isValidLogin(username: String, password: String): Boolean = {
findUser(username) match {
case Some(u: UserAccount) => if(password.equals(u.password)) { true } else {false }
case None => false
}
}
Just an FYI if anyone runs into this problem. I had a bad line in my DataStore.scala file:
val mongoClient: MongoClient = if (getConfigString("datastore.user", "").isEmpty()) MongoClient() else MongoClient(uri)
Since I was trying to connect without a username (there's no auth on my test db), the above line was saying, "If there's no username, you must be trying to connect to the default MongoClient() location, which is localhost". My mistake.
I simply changed the above line to this:
val mongoClient: MongoClient = MongoClient(uri)

"Could not lookup Ecto repo" error during tests

I've been getting the following exception when trying to setup a new ecto repo. Currently only using it for testing (since i am just setting this up).
** (RuntimeError) could not lookup Ecto repo Lipwig.AnnotatedUnit.Repo because it was not started or it does not exist
This happens when my tests use Lipwig.AnnotatedUnit.DataCase. However, when i use Lipwig.DataCase (which references the Lipwig.Repo`), the tests run fine.
I have the following setup in the project.
defmodule Lipwig.Repo do
use Ecto.Repo,
otp_app: :lipwig,
adapter: Ecto.Adapters.Postgres
end
defmodule Lipwig.AnnotatedUnit.Repo do
use Ecto.Repo,
otp_app: :lipwig,
adapter: Ecto.Adapters.Postgres
end
This here is my test setup
defmodule Lipwig.DataCase do
use ExUnit.CaseTemplate
using do
quote do
alias Lipwig.Repo
import Ecto
import Ecto.Changeset
import Ecto.Query
import Lipwig.DataCase
end
end
setup tags do
:ok = Ecto.Adapters.SQL.Sandbox.checkout(Lipwig.Repo)
unless tags[:async] do
Ecto.Adapters.SQL.Sandbox.mode(Lipwig.Repo, {:shared, self()})
end
:ok
end
def errors_on(changeset) do
Ecto.Changeset.traverse_errors(changeset, fn {message, opts} ->
Regex.replace(~r"%{(\w+)}", message, fn _, key ->
opts |> Keyword.get(String.to_existing_atom(key), key) |> to_string()
end)
end)
end
end
defmodule Lipwig.AnnotatedUnit.DataCase do
use ExUnit.CaseTemplate
using do
quote do
alias Lipwig.AnnotatedUnit.Repo
import Ecto
import Ecto.Changeset
import Ecto.Query
import Lipwig.AnnotatedUnit.DataCase
end
end
setup tags do
:ok = Ecto.Adapters.SQL.Sandbox.checkout(Lipwig.AnnotatedUnit.Repo)
unless tags[:async] do
Ecto.Adapters.SQL.Sandbox.mode(Lipwig.AnnotatedUnit.Repo, {:shared, self()})
end
:ok
end
def errors_on(changeset) do
Ecto.Changeset.traverse_errors(changeset, fn {message, opts} ->
Regex.replace(~r"%{(\w+)}", message, fn _, key ->
opts |> Keyword.get(String.to_existing_atom(key), key) |> to_string()
end)
end)
end
end
My other datacase for Lipwig.Repo is almost identical, just referencing the other repo.
My config.exs config defines
config :lipwig,
ecto_repos: [Lipwig.Repo, Lipwig.AnnotatedUnit.Repo]
My test.exs config is
config :lipwig, Lipwig.Repo,
username: "postgres",
password: "postgres",
database: "lipwig_test",
hostname: "localhost",
pool: Ecto.Adapters.SQL.Sandbox
config :lipwig, Lipwig.AnnotatedUnit.Repo,
username: "postgres",
password: "postgres",
database: "lipwig_test",
hostname: "localhost",
pool: Ecto.Adapters.SQL.Sandbox
As per my knowledge, the configurations seem fine. As the error suggests, I think the issue might be because you are not starting the Lipwig.AnnotatedUnit.Repo in with you application.
Look for the application.ex file in the /lib/lipwig_web folder and make sure that Lipwig.AnnotatedUnit.Repo is added in the children array.
Hope this resolves 🥂

Phoenix Repo.insert out of controller

I'm making a task that get an json in an api and insert in MongoDb. I'm using phoenix and Mongodb_Ecto.
I have a model Group and this code in a controller works like a charm:
HTTPoison.start
group = %Group{ param1: "value", param2: "value" } |> Repo.insert!
But,in a task I don't have the Repo defmodule. I tried to make this:
HTTPoison.start
group = %Group{ param1: "value", param2: "value"} |> MyApp.Repo.insert!
Using MyApp.Repo instead of only Repo.
I'm receiving this error:
** (exit) exited in: :gen_server.call(MyApp.Repo.Pool, {:checkout, #Reference<0.0.1.11>, true}, 5000)
** (EXIT) no process
:erlang.send(MyApp.Repo.Pool, {:"$gen_cast", {:cancel_waiting, #Reference<0.0.1.11>}}, [:noconnect])
(stdlib) gen_server.erl:416: :gen_server.do_send/2
(stdlib) gen_server.erl:232: :gen_server.do_cast/2
src/poolboy.erl:58: :poolboy.checkout/3
(stdlib) timer.erl:197: :timer.tc/3
lib/mongo/pool/poolboy.ex:33: Mongo.Pool.Poolboy.run/2
lib/mongo/pool.ex:142: Mongo.Pool.run_with_log/5
lib/mongo.ex:220: Mongo.insert_one/4
lib/mongo_ecto/connection.ex:124: Mongo.Ecto.Connection.catch_constraint_errors/1
lib/mongo_ecto.ex:522: Mongo.Ecto.insert/6
lib/ecto/repo/model.ex:253: Ecto.Repo.Model.apply/4
lib/ecto/repo/model.ex:83: anonymous fn/10 in Ecto.Repo.Model.do_insert/4
lib/ecto/repo/model.ex:14: Ecto.Repo.Model.insert!/4
How can I access Repo.insert in the correct way to save my data in mongoDb?
Thanks for this.
Your application (and therefore your database pool) is not started automatically in a Mix task. You can start it manually by adding the following:
:application.ensure_all_started(:my_app)
If :httpoison is listed in your Mixfile's :applications, you don't need to do HTTPoison.start anymore as the above line will ensure :httpoison is started before :my_app.

PyMongo sending "authenticate" for every query

I am getting an authentication to MongoDb for every query I run using PyMongo MongoClient. This seems expensive / unnecessary:
2015-02-13T09:38:08.091-0800 [conn375243] authenticate db: { authenticate: 1, user: "", nonce: "xxx", key: "xxx" }
2015-02-13T09:38:08.876-0800 [conn375243] end connection xxx (15 connections now open)
2015-02-13T09:38:08.962-0800 [initandlisten] connection accepted from xxx:42554 #375244 (16 connections now open)
2015-02-13T09:38:08.966-0800 [conn375244] authenticate db: { authenticate: 1, user: "", nonce: "xxx", key: "xxx" }
As far as I can tell, I'm using the same MongoClient (although it's hidden behind MongoEngine) and not intentionally disconnecting it at any point:
19:20:45 {'default': MongoClient('xxx-a0.mongolab.com', 39931)}
19:20:45 [139726027002480]
19:28:35 {'default': MongoClient('xxx-a0.mongolab.com', 39931)} # print mongo_client_instance
19:28:35 [139726027002480] # print id(mongo_Client_instance)
When I set a pdb breakpoint in the authenticate function, this is the stacktrace. I cannot figure out why asking the cursor to refresh requires a fresh authentication. Am I misunderstanding, and that is part of the MongoDb protocol? My goal is to have as few "authenticate" commands sent as possible, since right now they're 50% of my logged commands on the server.
/home/ubuntu/workspace//metadata/jobs.py(24)get()
-> b = Item.objects.get_or_create(id=i['id'])[0]
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/mongoengine/queryset/base.py(241)get_or_create()
-> doc = self.get(*q_objs, **query)
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/mongoengine/queryset/base.py(182)get()
-> result = queryset.next()
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/mongoengine/queryset/base.py(1137)next()
-> raw_doc = self._cursor.next()
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/cursor.py(1058)next()
-> if len(self.__data) or self._refresh():
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/cursor.py(1002)_refresh()
-> self.__uuid_subtype))
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/cursor.py(915)__send_message()
-> res = client._send_message_with_response(message, **kwargs)
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/mongo_client.py(1194)_send_message_with_response()
-> sock_info = self.__socket(member)
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/mongo_client.py(922)__socket()
-> self.__check_auth(sock_info)
/home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/mongo_client.py(503)__check_auth()
-> sock_info, self.__simple_command)
> /home/ubuntu/workspace//venv/local/lib/python2.7/site-packages/pymongo/auth.py(239)authenticate()
-> mechanism = credentials[0]
Additional information that might be useful is that these calls are from a Python RQ worker. I am trying to set up the connection before the fork step, but it's possible something is happening there to cause this.
(Pdb) os.getpid()
10507
... next query...
(Pdb) os.getpid()
10510
Got it!
The default Python-RQ worker uses the fork model, and the forking blocked PyMongo from sharing connection sockets.
I switched to the GeventWorker and now the sockets are shared by default.

How do I resolve a MongoDB timeout error when connecting via the Scala Play! framework?

I am connecting to MongoDB while using the Scala Play! framework. I end up getting this timeout error:
! #6j672dke5 - Internal server error, for (GET) [/accounts] ->
play.api.Application$$anon$1: Execution exception[[MongoTimeoutException: Timed out while waiting to connect after 10000 ms]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10-2.2.1.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10-2.2.1.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:165) [play_2.10-2.2.1.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:162) [play_2.10-2.2.1.jar:2.2.1]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.4.jar:na]
at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.4.jar:na]
Caused by: com.mongodb.MongoTimeoutException: Timed out while waiting to connect after 10000 ms
at com.mongodb.BaseCluster.getDescription(BaseCluster.java:131) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.getClusterDescription(DBTCPConnector.java:396) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.getType(DBTCPConnector.java:569) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBTCPConnector.isMongosConnection(DBTCPConnector.java:370) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.Mongo.isMongosConnection(Mongo.java:645) ~[mongo-java-driver-2.12.3.jar:na]
at com.mongodb.DBCursor._check(DBCursor.java:454) ~[mongo-java-driver-2.12.3.jar:na]
Here is my Scala code for connecting to the database:
//models.scala
package models.mongodb
//imports
package object mongoContext {
//context stuff
val client = MongoClient(current.configuration.getString("mongo.host").toString())
val database = client(current.configuration.getString("mongo.database").toString())
}
Here is the actual model that is making the connection:
//google.scala
package models.mongodb
//imports
case class Account(
id: ObjectId = new ObjectId,
name: String
)
object AccountDAO extends SalatDAO[Account, ObjectId](
collection = mongoContext.database("accounts")
)
object Account {
def all(): List[Account] = AccountDAO.find(MongoDBObject.empty).toList
}
Here's the Play! framework MongoDB conf information:
# application.conf
# mongodb connection details
mongo.host="localhost"
mongo.port=27017
mongo.database="advanced"
Mongodb is running on my local machine. I can connect to it by typing mongo at the terminal window. Here's the relevant part of the conf file:
# mongod.conf
# Where to store the data.
# Note: if you run mongodb as a non-root user (recommended) you may
# need to create and set permissions for this directory manually,
# e.g., if the parent directory isn't mutable by the mongodb user.
dbpath=/var/lib/mongodb
#where to log
logpath=/var/log/mongodb/mongod.log
logappend=true
#port = 27017
# Listen to local interface only. Comment out to listen on all interfaces.
#bind_ip = 127.0.0.1
So what's causing this timeout error and how do I fix it? Thanks!
I figured out that I needed to change:
val client = MongoClient(current.configuration.getString("mongo.host").toString())
val database = client(current.configuration.getString("mongo.database").toString())
to:
val client = MongoClient(conf.getString("mongo.host"))
val database = client(conf.getString("mongo.database"))