trying to upsert with the new Scala Async Driver using this code, but the DB never gets created even though this is called many times:
override def insertOrUpdateOne(user: UserNew): Future[Unit] = {
users.replaceOne(equal("_id", user._id), user)
.toFuture
.map(_ => ())
}
mongo client created fine, collection created fine, but the above causes no write to be performed in fact the db is never created (which should happen on the first write). I have other code that uses casbah and that is working synchronously.
users is the Mongo collection with no syntactic sugar from me. (edited)
toFuture seems to do the subscribe (which should trigger the write?) but it still doesn't work. I'm mixing some collections with casbah and one with the async client, the casbah ones work but not the one above using async.
Any ideas?
it should be
override def insertOrUpdateOne(user: UserNew): Future[Unit] = {
users.replaceOne(equal("_id", user._id), user, UpdateOptions().upsert(true))
.toFuture
.map(_ => ())
}
Nothing in the docs but when I read the code I see it defaults upsert to false.
BTW this allows us to ignore all the Observable and subscribe stuff and setup of callbacks. Just process the results in a Scala idiomatic someFuture.map() This makes the UsersDaonon-Mongo specific code as an interface. This in turn allows all code in the UsersDaoImplto handle Mongo specific stuff, which enables the Impl to be injected so a runtime choice can be made about which Impl to use. Mongo can be replaced without dependent code caring or knowing about the Impl.
Related
I successfully inserted data into a mongodb database, but I don't know how to extract data out of a query. I use the default scala mongodb drive :
"org.mongodb.scala" %% "mongo-scala-driver" % "1.1.1"
The documentation seems to contains errors, by the way. This line rises a compilation error while this is copy pasted from the doc :
collection.find().first().printHeadResult()
This is how I query a collection:
collection.find()
How to convert it to a scala collection of object on which I can iterate and process ? Thanks
Yes, I agree with the compilation error. I think "collection.find().first().printHeadResult()" is not part of scala driver 1.1.1 release. The current scala driver github which uses this code is "1.2.0-SNAPSHOT" version.
You can get the results using the below code. However, you may experience some async behavior using the below code. Please refer the driver documentation.
val observable: FindObservable[Document] = collection.find();
observable.subscribe ( new Observer[Document] {
override def onNext(result: Document): Unit = println(result.toJson())
override def onError(e: Throwable): Unit = println("Failed" + e.getMessage)
override def onComplete(): Unit = println("Completed")
})
Mongo driver Observables link
This is answered from the best of my current knowledge. I spent a lot of time using casbah and I've recently switched to using the new async scala driver, so there may be more ergonomic ways to do some of this stuff that I don't know yet.
Basically you need to transform the result of the observable and then eventually, you'll probably want to turn it into something that isn't an observable so you can have synchronous code interact with it (maybe, depending on what you're doing).
In the current Mongo Scala API (2.7.0 as of writing this), you might process a list of documents like this:
coll.find(Document("head" -> 1)).map(dbo => dbo.getInteger("head"))
That takes the list of documents where head is equal to one and then applies the map function to convert it from the Document (dbo) into an Int by extracting the "head" element (note this will fall in ugly fashion if there isn't a head field or the field is not an int. There are more robust ways to get values out using get[T]).
You can find a full list of the operations that an Observable supports here:
https://mongodb.github.io/mongo-scala-driver/2.7/reference/observables/
under the list of Monadic operators.
The other part is how do you get the good stuff out the Observable because you want to do something synchronous with them. The best answer I have found so far is to dump the Observable into a Future and then calling Await.result on that.
val e = coll.find(Document("head" -> 1)).map(dbo => dbo.getInteger("head"))
val r = Await.result(e.toFuture(), Duration.Inf)
println(r)
That will print out the List[Int] that was created by evaluating the map function for each Document in the Observable.
Getting started learning scala and designing/implementing for asynchronous execution. My questions is around how to design APIs (and then call) them for operations that return Unit but may not right away. For example in the snippet below, the function (using Slick 3.0) inserts a user into the DB. Is Unit the correct return type for this function, and if so, how do callers know if/when the newly inserted user is successful?
override def insertOrUpdate(entity: User): Unit = {
database.run(users.insertOrUpdate(entity))
}
For example, if the above executes asynchronously and a caller looks something like
//create and insert new user with id = 7
val newUser = User(7, "someName")
userRepo.insertOrUpdate(newUser)
How does a caller know whether or not it is safe to do
userRepo.findById(7)
In unit testing, I know that if I follow up the insert call immediately by a findById call, the findById will return nothing, but if I introduce some latency between the insert and find call, it finds the new user. To summarize, what is the proper way to design an API for a function that executes asynchronously but has no natural return value to wrap in a Future?
Generally when working with Futures, you'll want to do any further processing via methods called on the returned Future. Eg.:
val newUser = User(7, "someName")
val future = userRepo.insertOrUpdate(newUser)
future.onSuccess { outcome => // Here, 'outcome' will hold whatever was contained in the Future - Unit in your description above.
val storedUser = userRepo.findById(7) // This will only execute once the future completes (successfully).
...
}
There are plenty of other useful methods for manipulating a Future (or a collection of them), such as "onFailure", "recover", "map" and "flatMap".
Try not to wait on the Future until as late as possible - preferably let Play or Spray or whatever other framework you might happen to be using take care of it for you (see here for Play documentation on doing this, for example).
Finally, in terms of your DB call to insert, I'd look into having the call return at least a boolean, or better still the primary key the new entry was inserted with, rather than Unit.
I have a service that makes use of the Scala Async library. I'm using this library primarily to time my database calls. The method that I want to test contains multiple calls to the database using the async await mechanism. A pseudo code of what I have is as below:
def myDbMethod() = async {
val firstCall = await(call the db and get the result)
val secondCall = await(all the db and get the result)
val thirdCall = await(all the db and get the result)
...
}
In my Scala test unit test, I have
Await.result(myDbMethod(), 10.seconds)
I was just trying to debug myMethod by running my unit test which would return with a test success even before getting into the secondCall. I mean I had breakpoints in all the 3 calls to the database, but the IntelliJ debugger would just exit out as soon as it finishes the first call to the database. Why is this? How can I test this behaviour using IntelliJ debugger?
I'm not sure that my answer would suffice your expectations, but it's a known issue. The problem is that async/await is quite a complicated macro, which does heavy transformations on the trees (you can check the output by enabling -Xprint:<phase_name_after_typer> flag). Unfortunately neither of existing IDEs (I'm working with Intellij and Ensime) can debug it, but I'm not familiar with their internals to explain why they can't in details.
From my experience I couldn't find any neat pros over the native for-comprehension, so you can stick with th native syntax or explicit flatmap calls, which is nicely debuggable.
This construct could be used for depdendant asynchronous calls.
async / await adds some sugar to make that easier, but to formalate that by hand you can do it like this:
def dbMethodDependant : Future[T] = for {
res1 <- firstCall
res2 <- secondCall(res2)
res3 <- thirdCall(res3)
} yield res3
Await.result(dbMethodDependant, forever)
I'm using Slick 3.0 and (of course) almost all the examples out there cover Slick 2.x. Things have changed and frankly seem more complicated, not less.
Here's an example: I want to get an object (a GPPerson) by id. This is what I have right now, and it seems very verbose... more so than Slick 2.x:
def get(id: GPID): Option[GPPerson] = Await.result(
db.run(
people.filter(_.id === id).result), Duration.Inf
).headOption
In Slick 2.x things were easier because of the implicits, among other things. But the above seems to be the most concise expression I've come up with.
It also doesn't really address exception handling, which I would need to add.
I started to use Slick 3.0 in a new project a few months ago and I had the same questions. This is what I understood:
Slick 3.0 was designed for non-blocking asynchronous (reactive) applications. Obviously it means Akka + Play / Spray nowadays. In this world you mostly interact with Futures, that's why Slick's db.run returns Future. There is no point in using Await.result - if you need blocking calls it's better to return to 2.x.
But if you use reactive stack you'll get benefits immediately. For example, Spray is completely non-blocking library that works with Futures nicely using onComplete directive. You can call a method that returns Future with a result from Slick in a Spray route and then use that result together with onComplete. In this case the whole response-reply pipeline is non-blocking.
You also mentioned exception handling, so this is exactly how you do it - using Futures.
So based on my experience I would write your method in a following way:
def get(id: GPID): Future[Option[GPPerson]] = db.run(
people.filter(_.id === id).result.map(_.headOption)
)
and then work with a Future.
you can do this.
def syncResult[R](action:slick.dbio.DBIOAction[R, slick.dbio.NoStream, scala.Nothing]):R = {
import scala.concurrent.duration.Duration
val db = Database.forConfig("db")
try {
Await.result(db.run(action), Duration.Inf)
} finally db.close
}
def get(id: GPID): Option[GPPerson] = syncResult { people.filter(_.id === id).result.headOption }
Came across a problem I did not find an answer yet.
Running on playframework 2 with Scala.
Was required to write an Action method that performs multiple Future calls.
My question:
1) Is the attached code non-blocking and hence looking the way it should be ?
2) Is there a guarantee that both DAO results are caught at any given time ?
def index = Action.async {
val t2:Future[Tuple2[List[PlayerCol],List[CreatureCol]]] = for {
p <- PlayerDAO.findAll()
c <- CreatureDAO.findAlive()
}yield(p,c)
t2.map(t => Ok(views.html.index(t._1, t._2)))
}
Thanks for your feedback.
Is the attached code non-blocking and hence looking the way it should be ?
That depends on a few things. First, I'm going to assume that PlayerDAO.findAll() and CreatureDAO.findAlive() return Future[List[PlayerCol]] and Future[List[CreatureCol]] respectively. What matters most is what these functions are actually calling themselves. Are they making JDBC calls, or using an asynchronous DB driver?
If the answer is JDBC (or some other synchronous db driver), then you're still blocking, and there's no way to make it fully "non-blocking". Why? Because JDBC calls block their current thread, and wrapping them in a Future won't fix that. In this situation, the most you can do is have them block a different ExecutionContext than the one Play is using to handle requests. This is generally a good idea, because if you have several db requests running concurrently, they can block Play's internal thread pool used for handling HTTP requests, and suddenly your server will have to wait to handle other requests (even if they don't require database calls).
For more on different ExecutionContexts see the thread pools documentation and this answer.
If you're answer is an asynchronous database driver like reactive mongo (there's also scalike-jdbc, and maybe some others), then you're in good shape, and I probably made you read a little more than you had to. In that scenario your index controller function would be fully non-blocking.
Is there a guarantee that both DAO results are caught at any given time ?
I'm not quite sure what you mean by this. In your current code, you're actually making these calls in sequence. CreatureDAO.findAlive() isn't executed until PlayerDAO.findAll() has returned. Since they are not dependent on each other, it seems like this isn't intentional. To make them run in parallel, you should instantiate the Futures before mapping them in a for-comprehension:
def index = Action.async {
val players: Future[List[PlayerCol]] = PlayerDAO.findAll()
val creatures: Future[List[CreatureCol]] = CreatureDAO.findAlive()
val t2: Future[(List[PlayerCol], List[CreatureCol])] = for {
p <- players
c <- creatures
} yield (p, c)
t2.map(t => Ok(views.html.index(t._1, t._2)))
}
The only thing you can guarantee about having both results being completed is that yield isn't executed until the Futures have completed (or never, if they failed), and likewise the body of t2.map(...) isn't executed until t2 has been completed.
Further reading:
Are there any benefits in using non-async actions in Play Framework 2.2?
Understanding the Difference Between Non-Blocking Web Service Calls vs Non-Blocking JDBC