I'm inserting a record in MongoDB :
val observable: Observable[Completed] = collection.insertOne(doc)
observable.subscribe(new Observer[Completed] {
override def onNext(result: Completed): Unit = { println("Inserted"); }
override def onError(e: Throwable): Unit = { println(" \n\nFailed " + e + "\n\n"); fail() }
override def onComplete(): Unit = { println("Completed"); }
})
The test passes even though the onError callback is invoked. This is because insertOne is an asynchronous method and the test completes before the onError is invoked. I would like to wrap the insertOne method into a blocking call, so subscribe is not invoked until after observable value is set.
Is there an idiomatic method to achieve this is in Scala ?
The simplest approach to synchronously block the async operation is using Await.result on a Future. Since MongoCollection.insertOne returns an Observable[Complete], you can use the implicit ScalaObservable.toFuture on it:
val observable = collection.insertOne(doc)
Await.result(observable.toFuture, Duration.Inf)
observable.subscribe(new Observer[Completed] {
override def onNext(result: Completed): Unit = { println("Inserted"); }
override def onError(e: Throwable): Unit = { println(" \n\nFailed " + e + "\n\n"); fail() }
override def onComplete(): Unit = { println("Completed"); }
})
Related
Trying to wrap my head around how async tasks are chained together, for ex futures and flatMap
val a: Future[Int] = Future { 123 }
val b: Future[Int] = a.flatMap(a => Future { a + 321 })
How would i implement something similar where i build a new Future by waiting for a first and applying a f on the result. I tried looking a scala's source code but got stuck here: https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/Future.scala#L216
i would imagine some code like:
def myFlatMap(a: Future[Int])(f: a => Future[Int]): Future[Int] = {
Future {
// somehow wait for a to complete and the apply 'f'
a.onComplete {
case Success(x) => f(a)
}
}
}
but i guess above will return a Future[Unit] instead. I would just like to understand the "pattern" of how async tasks are chained together.
(Disclaimer: I'm the main maintainer of Scala Futures)
You wouldn't want to implement flatMap/recoverWith/transformWith yourself—it is a very complex feature to implement safely (stack-safe, memory-safe, and concurrency-safe).
You can see what I am talking about here:
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L442
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L307
https://github.com/scala/scala/blob/2.13.x/src/library/scala/concurrent/impl/Promise.scala#L274
There is more reading on the topic here: https://viktorklang.com/blog/Futures-in-Scala-2.12-part-9.html
I found that looking at the source for scala 2.12 was easier for me to understand how Future / Promise was implemented, i learned that Future is in fact a Promise and by looking at the implementation i now understand how "chaining" async tasks can be implemented.
Im posting the code that i wrote to help me better understand what was going on under the hood.
import java.util.concurrent.{ScheduledThreadPoolExecutor, TimeUnit}
import scala.collection.mutable.ListBuffer
trait MyAsyncTask[T] {
type Callback = T => Unit
var result: Option[T]
// register a new callback to be called when task is completed.
def onComplete(callback: Callback): Unit
// complete the given task, and invoke all registered callbacks.
def complete(value: T): Unit
// create a new task, that will wait for the current task to complete and apply
// the provided map function, and complete the new task when completed.
def flatMap[B](f: T => MyAsyncTask[B]): MyAsyncTask[B] = {
val wrapper = new DefaultAsyncTask[B]()
onComplete { x =>
f(x).onComplete { y =>
wrapper.complete(y)
}
}
wrapper
}
def map[B](f: T => B): MyAsyncTask[B] = flatMap(x => CompletedAsyncTask(f(x)))
}
/**
* task with fixed pre-calculated result.
*/
case class CompletedAsyncTask[T](value: T) extends MyAsyncTask[T] {
override var result: Option[T] = Some(value)
override def onComplete(callback: Callback): Unit = {
// we already have the result just call the callback.
callback(value)
}
override def complete(value: T): Unit = () // noop nothing to complete.
}
class DefaultAsyncTask[T] extends MyAsyncTask[T] {
override var result: Option[T] = None
var isCompleted = false
var listeners = new ListBuffer[Callback]()
/**
* register callback, to be called when task is completed.
*/
override def onComplete(callback: Callback): Unit = {
if (isCompleted) {
// already completed just invoke callback
callback(result.get)
} else {
// add the listener
listeners.addOne(callback)
}
}
/**
* trigger all registered callbacks to `onComplete`
*/
override def complete(value: T): Unit = {
result = Some(value)
listeners.foreach { listener =>
listener(value)
}
}
}
object MyAsyncTask {
def apply[T](body: (T => Unit) => Unit): MyAsyncTask[T] = {
val task = new DefaultAsyncTask[T]()
// pass in `complete` as callback.
body(task.complete)
task
}
}
object MyAsyncFlatMap {
/**
* helper to simulate async task.
*/
def delayedCall(body: => Unit, delay: Int): Unit = {
val scheduler = new ScheduledThreadPoolExecutor(1)
val run = new Runnable {
override def run(): Unit = {
body
}
}
scheduler.schedule(run, delay, TimeUnit.SECONDS)
}
def main(args: Array[String]): Unit = {
val getAge = MyAsyncTask[Int] { cb =>
delayedCall({ cb(66) }, 1)
}
val getName = MyAsyncTask[String] { cb =>
delayedCall({ cb("John") }, 2)
}
// same as: getAge.flatMap(age => getName.map(name => (age, name)))
val result: MyAsyncTask[(Int, String)] = for {
age <- getAge
name <- getName
} yield (age, name)
result.onComplete {
case (age, name) => println(s"hello $name $age")
}
}
}
I´m learning the monad IO of scalaZ and I cannot understand how catchAll and catchSome operators works.
I was expecting so see a behave like the onError or onErrorrResumeNext of RxJava, but instead is not catching the throwable, and it´s just breaking the test and throwing the NullPointerException..
Here my two examples
#Test
def catchAllOperator(): Unit = {
val errorSentence: IO[Throwable, String] =
IO.point[Throwable, String](null)
.map(value => value.toUpperCase())
.catchAll(error => IO.fail(error))
println(unsafePerformIO(errorSentence))
}
And catchSome example
#Test
def catchSomeOperator(): Unit = {
val errorFunction = new PartialFunction[Throwable /*Entry type*/ , IO[Throwable, String] /*Output type*/ ] {
override def isDefinedAt(x: Throwable): Boolean = x.isInstanceOf[NullPointerException]
override def apply(v1: Throwable): IO[Throwable, String] = IO.point("Default value")
}
val errorSentence = IO.point[Throwable, String](null)
.map(value => value.toUpperCase())
.catchSome(errorFunction)
println(unsafePerformIO(errorSentence))
}
Any idea what I´m doing wrong?.
Regards
I wrote this code
val myEc = new ExecutionContext {
val tp = Executors.newFixedThreadPool(2)
def execute(r: Runnable) { tp.submit(r) }
def reportFailure(t: Throwable) { println("ho ha ha ... something broke!") }
}
val f : Future[Int] = Future(throw new Exception("123"))(myEc)
Await.result(f, Duration.Inf)
The program doesn't print "ho ha ha"... so what is the purpose of reportFailure then? shouldn't it get called when the future fails?
I am intending to fetch all records that match a criteria from Mongo using Scala Mongo Driver.
Using Observables, you can access the stream by creating a subscription:
val MaxBuffer: Long = 100
var docs: Queue[Document] = Queue.empty
var sub: Option[Subscription] = None
val q: Observable[Document]
def fetchMoreRecords: Unit = sub.get.request(MaxBuffer)
q.subscribe(new Observer[Document] {
override def onSubscribe(subscription: Subscription): Unit = {
sub = Some(subscription)
fetchMoreRecords
}
override def onError(e: Throwable): Unit = fail(out, e)
override def onComplete(): Unit = {
println("Stream is complete")
complete(out)
}
override def onNext(result: Document): Unit = {
if (doc.size == maxBuffer) {
fail(out, new RuntimeException("Buffer overflow"))
} else {
docs = docs :+ result
}
}
})
(this code is incomplete)
I would need a function like:
def isReady: Future[Boolean] = {}
Which completes whenever onNext was called at least once.
The bad way to do this would be:
def isReady: Future[Boolean] = {
Future {
def wait: Unit = {
if (docs.nonEmpty) {
true
} else { wait }
}
wait
}
}
What would be the best way to achieve this?
You want to use Promise:
val promise = Promise[Boolean]()
...
override def onNext() = {
...
promise.tryComplete(Success(true))
}
override def onError(e: Throwable) =
promise.tryComplete(Failure(e))
val future = promise.future
You should do something to handle the case when there are no result (as it is now, the future will never be satisfied ...
Let's say I have a function
def request(url: String): Future[String]
and need to write a function
def requestFirst(urls: List[String]): Future[String]
that calls request(url) in sequence untill it completes successfully (in this case the successful value is returned) or requests for all urls fail (in this case all failures are returned).
How can this be done in Scala?
def requestFirst(urls: List[String]): Future[String] = {
val default: Future[String] = Future.failed(new scala.Exception("all failed"))
urls.foldLeft(default)((prevFuture, currentUrl) => {
prevFuture fallbackTo (request(currentUrl))
})
}
OR
def requestFirst(urls: List[String]): Future[String] = {
def requestFirstInternal(urlSubset: List[String]): Future[String] = {
if(urlSubset.isEmpty) {
Future.failed(new Exception("Exhausted all urlSubset"))
} else {
request(urlSubset.head) fallbackTo {
requestFirstInternal(urlSubset.tail)
}
}
}
requestFirstInternal(urls)
}
This is what I came up with based on #parnav-shukla answer:
def requestFirst(urls: List[String]): Future[String] = {
def requestFirstInternal(urlSubset: List[String], errors: Seq[String]): Future[String] = {
if(urlSubset.isEmpty) {
Future.failed(new Exception(errors.mkString("\n")))
} else {
request(urlSubset.head) recoverWith { t =>
requestFirstInternal(urlSubset.tail, errors ++ Seq(t.getMessage))
}
}
}
requestFirstInternal(urls, Seq.empty)
}