This method throws exception
#throws(classOf[QuestionNotFoundException])
def getQuestionFromQuestionID(questionKey: PracticeQuestionKeys) = {
logger.trace(s"getting question with keys ${questionKey}")
val practiceQuestionFuture: Future[Option[PracticeQuestion]] = findOne(questionKey)
for (questionOption <- practiceQuestionFuture) yield {
questionOption.fold(throw QuestionNotFoundException())(question => {
logger.trace("got question " + question)
question
})
}
}
I use it like follows
def function1 = {
...
val res = for{existingQuestion <- questionsRepository.getQuestionFromQuestionID(questionKey){..}
res.recover {
case exception => {
logger.trace(s"exception ${exception}")
Ok(Json.toJson(JsonResultError(messagesApi("error.answerAdditionFail")(langs.availables(0))+". "+exception.getMessage())))
}
...
}
I want to test that function1 handles the thrown exception. I have written the following test case
"function" should {
"should return error if the question id is not correct" in {
...
when(answerTestEnv.mockPracticeQuestionsRepository.getQuestionFromQuestionID(ArgumentMatchers.any[PracticeQuestionKeys])).thenThrow(
QuestionNotFoundException()
)
val response:Accumulator[ByteString,Result] = controller.function1(request)
//note that run is how accumulator gets the data and starts processing it. no parameter means no data
val resultOfAccumulatorRun:Future[Result] = response.run(ByteString(body))
val responseBody = contentAsJson(resultOfAccumulatorRun)(Timeout(Duration(5000,"millis")),TestEnvImplicits.mat)
println(s"received response ${responseBody}")
val result = (responseBody \ "result").get.as[String]
val additionalInfo = (responseBody \ "additional-info").get.as[String]
result mustBe "error"
additionalInfo mustBe components.messagesApi("error.answerAdditionFail")(components.langs.availables(0))+". "+QuestionNotFoundException().msg
}
}
But when I run my test case, the Exception gets thrown but it seems that res.recover doesn't handle it. I see this in the console.
Question not found
utilities.QuestionNotFoundException: Question not found
at controllers.AnswerController.$anonfun$newAnswer$1(AnswerController.scala:428)
Why doestres.recover` doesn't handle it.
the partial function passed to recover() gets invoked on a failed Future, but your Future isn't a failed Future, it is a Future that contains an exception as its 'successful' result type (actually it will be forced to be something more like Future[AnyRef], as it could be either an Exception or a PracticeQuestion).
You probably don't want to be using Option.fold() like that. Instead try:
practiceQuestionFuture.flatMap { qOpt =>
qOpt.map(q => Future(q))
.getOrElse(Future.failed(QuestionNotFoundException()))
}
(NB: I'm not near a compiler to try this out right now, so I may have made a syntactical mistake or two, but there should be enough there to get you going).
Related
The Scala Try construct together with its flatMap does not work as I would expect or want it to. The TL;DR is that I want to do a series of operations that can fail in two ways: either by raising an exception, which should be promoted and caught higher up in the call stack, or by returning Failure, as the failure must logically be handled in different parts of the program.
I would expect something like this to do the trick:
def firstStepSucceeds(): Try[Int] = Try {
1
}
def secondStepThrows(input: Int) = {
throw new Exception("Exception thrown in second step")
}
// I expect this to propagate the exception thrown in secondStepThrows
firstStepSucceeds() flatMap (secondStepThrows _)
(Full Scastie with example)
However, in this case, the flatMap() call actually implicitly catches the uncaught exception thrown by secondStepThrows, which is not what I want (which is why I left out the Try block). Is there a way to get the same behaviour without the implicit exception-catching?
What happens in a Try should stay in a Try. Most Scala programmers would be very surprised if a function returning a Try also sometimes threw an exception.
The typical pattern if you want to handle exceptions in different places is to differentiate by the type of the exception. So
val partiallyRecoveredTry = originalTry.recover{
case _: SecondStepException => "second step had an exception"
}
// Further up the call stack
partiallyRecoveredTry.getOrElse("first step had an exception")
Try.flatMap() did not caught exceptions implicitely, it is the essence of Try.
When you use it, it is very explicit, and that's the goal.
I don't really understand what you want, but is something like that is possible for you ?
try {
val first = firstStepSucceeds()
val second = first.map(secondStepThrows).get
val third = secondStepFails(second)
// ...
}
catch {
case e: Exception => ???
}
I did some further experimentation, and what I ended up with was this reimplementation of Try as (the now right-biased and hence monadic) Either:
object CatchAll {
def apply[SomeType](block: => SomeType) = try { Right(block) }
catch { case e: Throwable => Left(e) }
}
def firstStepSucceeds() = CatchAll {
1
}
def firstStepFails() = CatchAll {
throw new Exception("First step failed")
}
def secondStepSucceeds(input: Int) = CatchAll {
input + 1
}
def secondStepFails(input: Int) = CatchAll {
throw new Exception("Second step failed in try block!")
}
def secondStepThrows(input: Int) = {
throw new Exception("Second step failed unexpectedly!")
}
firstStepSucceeds() flatMap (secondStepSucceeds _)
firstStepFails() flatMap (secondStepSucceeds _)
firstStepSucceeds() flatMap (secondStepFails _)
// This now throws an exception as expected
//firstStepSucceeds() flatMap (secondStepThrows _)
I am trying to read incremental data from my data source using Scala-Spark. Before hitting the source tables, I am trying to calculate the min & max of partition column that I use in my code in a Future which is present in a class: GetSourceMeta as given below.
def getBounds(keyIdMap:scala.collection.mutable.Map[String, String]): Future[scala.collection.mutable.Map[String, String]] = Future {
var boundsMap = scala.collection.mutable.Map[String, String]()
keyIdMap.keys.foreach(table => if(!keyIdMap(table).contains("Invalid")) {
val minMax = s"select max(insert_tms) maxTms, min(insert_tms) minTms from schema.${table} where source='DB2' and key_id in (${keyIdMap(table)})"
println("MinMax: " + minMax)
val boundsDF = spark.read.format("jdbc").option("url", con.getConUrl()).option("dbtable", s"(${minMax}) as ctids").option("user", con.getUserName()).option("password", con.getPwd()).load()
try {
val maxTms = boundsDF.select("minTms").head.getTimestamp(0).toString + "," + boundsDF.select("maxTms").head.getTimestamp(0).toString
println("Bounds: " + maxTms)
boundsMap += (table -> maxTms)
} catch {
case np: java.lang.NullPointerException => { println("No data found") }
case e: Exception => { println(s"Unknown exception: $e") }
}
}
)
boundsMap.foreach(println)
boundsMap
}
I am calling the above method in my main method as:
object LoadToCopyDB {
val conf = new SparkConf().setAppName("TEST_YEAR").set("some parameters")
def main(args: Array[String]): Unit = {
val spark = SparkSession.builder().config(conf).master("yarn").enableHiveSupport().config("hive.exec.dynamic.partition", "true").config("hive.exec.dynamic.partition.mode", "nonstrict").getOrCreate()
val gsm = new GetSourceMeta()
val minMaxKeyMap = gsm.getBounds(keyIdMap).onComplete {
case Success(values) => values.foreach(println)
case Failure(f) => f.printStackTrace
}
.
.
.
}
Well, the onComplete didn't print any values so I used andThen as below and that didn't help as well.
val bounds: Future[scala.collection.mutable.Map[String, String]] = gpMetaData.getBounds(incrementalIds) andThen {
case Success(outval) => outval.foreach(println)
case Failure(e) => println(e)
}
Earlier the main thread exits without letting the Future: getBounds execute. Hence I couldn't find any println statements from the Future displayed on the terminal. I found out that I need to keep the main thread Await inorder to complete the Future. But when I use Await in main along with onComplete:
Await.result(bounds, Duration.Inf)
The compiler gives an error:
Type mismatch, expected: Awaitable[NotInferedT], actual:Unit
If I declare the val minMaxKeyMap as Future[scala.collection.mutable.Map[String, String] the compiler says: Expression of type Unit doesn't conform to expected type Future[mutable.map[String,String]]
I tried to print the values of bounds after the Await statement but that just prints an empty Map.
I couldn't understand how can to fix this. Could anyone let me know what do I do to make the Future run properly ?
In this kind of cases, is always better to follow the types. The method onComplete only returns Unit, it won´t return a future hence it can´t be passed using Await.
In case you want to return a Future of any type you will have to map or flatmap the value and return an option, for example. In this case, does not matter what you return, you only want Await method to wait for this result and print a trace. You can treat the possible exception in the recover. It would be like that in your code:
val minMaxKeyMap:Future[Option[Any] = gsm.getBounds(keyIdMap).map { values =>
values.foreach(println)
None
}.recover{
case e: Throwable =>
e. printStackTrace
None
}
Note that the recover part has to return an instance of the type.
After that, you can apply the Await to the Future, and you will get the results printed. Is not the prettiest solution but it will work in your case.
I am having problem with my play-framework API server. I need to have some processing running in the background that is returning a Future with the result and then write the result as response. However, the request thread goes all out and return before my Future completes. Here is the code...
def requestAction(): Action[AnyContent] = Action.async { implicit request =>
var fResult: Future[String] = Future { "initial value" }
try {
fResult = doSomethingAsyncAndGetResponseString(); // return "great, everything is done"
}
catch {
case t: Throwable {
fResult = Future { "failed" }
}
}
// done, return response, but the problem is, this is executed first because doSomethingAsyncAndGetResponseString() is still executing and returns later
fResult.map( res => {
// problem here, because I get "initial value" which is not what I want
Ok(res)
}
}
Is there a way to get "great, everything is done" or "failed" without Async.await ? I have been using this format all over in my API server, but today it broke because in a new API that I write, the doSomethingAsyncAndGetResponseString is a bit longer. I didn't expect that, so something must be wrong with how I understand the structure.
Thanks in advance!
You are trying to write Java like code using Scala.
You are doing it wrong. Read about Futures and How to use them.
Here is the tutorial
Futures can be composed using map and flatMap constructs. recover and recoverWith will give the user access to exception happened in the computation pipeline
You have to do something like this, given below
def requestAction(): Action[AnyContent] = Action.async { implicit request =>
Future { "initial value" }.flatMap { _ =>
doSomethingAsyncAndGetResponseString() // returns Future
}.map { res =>
Ok(res)
}.recover { case th =>
Ok(th.getMessage)
}
}
Handling exceptions and recovering from exceptions
Exception handling is inbuilt into Future.
recover gives access to exception and also helps the user to provide the alternative success value in case of exception.
recoverWith gives access to an exception and also helps the user to provide/chain alternative Future computation which can succeed or fail.
Future {
throw new Exception("foo exception")
}.recover {
case th => println(s"msg: ${th.getMessage}")
}
I want the following code to return a custom message when one of the method, callfuture1() or callfuture2(), throws an exception. My understanding was if either of the future fails, f would be a failed future.
However, when callfuture1 throws an exception. f.onFailure is not executed. Instead I see the call stack stopped at line of code in callFuture1() where exception occurred and a standard internalError is returned. Why does that happen?
val f = for {
x <- callfuture1()
y <- callfuture2()
} yield y
f.onFailure {
//send an internalserver error with some custom message
}
f.map {
//send data back
}
====update====
i see from the responses, that potential issue is that Exception is being thrown outside the Future and hence my code fails to catch that failed future.
So i changed the code such that Exception only occurs inside the future. I still am not able to explain the behavior i am seeing. (I wonder if it has to anything to do with Play framework.)
def controllerfunction(id: String) = Action.async{
val f = for{
x <- callfuture1(id)
y <- callfuture2(x)
} yield y
y.onFailure{case t =>
println("This gets printed");
Ok("shit happened, but i am still ok")}
y.map{resp:String => Ok(resp)}
}
def callfuture1(id: String):Future[Obj1] = {
for {
val1 <- callfuture1.1(id)
val2 <- callfuture1.2(val1)
} yield val2
}
def callfuture1.2:Future[Obj3] = Future{
thrown new Exception("TEST ME");
}
def callfuture 1.1:Future[Obj4] = {...}
def callfuture2: Future[String] = {....}
Expectation.
The method callfuture1.2 throws an exception inside the future, so my expectation is onFailure should be executed, (which does get executed), and the response returned should "Shit happened, but i am still ok"
Actuality
The play framework returns InternalServerError and i see error stack on my console. I see the printlin("This gets printed") is getting executed.
Cant understand what is happening. Any insights?
==== update 2 =====
I verified that the issue only happens when called inside controller of play framework ( i am using play 2.5). As a standalone scala program everthing works as expected. I believe play error handling catches the unhandaled exception and prints the stack trace. I think this should only be happening in development environment.
This can happen if callfuture1 throws "outside of a future".
Your for comprehension is desugared into this:
val f = callfuture1.flatMap{ x =>
callfuture2.map{ y =>
y
}
}
If callfuture2 throws right away (as opposed to returning a failed future), you will still end up with a failed future because callfuture2 is called inside Future.flatMap, which catches exceptions and turns them into failed futures (same for Future.map).
The situation is different for callfuture1: if it throws right away, there is no enclosing Future.map or Future.flatMap to turn it into a failed future.
In general you should try to avoid having a method that returns a Future and can also throw an error.
This means that if callfuture1 does anything that can throw, it should catch that and turn the exception in a failed future that you then return.
UPDATE: Concerning your update about how you expected "Shit happened, but i am still ok" to be returned:
As already hinted by Dima in a comment, Future.onFailure can only be used for side effects. Futures are immutable. If you want to recover from a failed exception, there is no way to modify the original (failed) future and all you can actually do is transform it into a new future.
Have a look at Future.recover. It does exactly what you need, namely it allows to transform an input future by matching the failed result (if any) and transforming it into a successful future. It is the equivalent of a catch clause, but for futures. Concretely what you really meant to do is something like this:
def controllerfunction(id: String) = Action.async{
val f = for{
x <- callfuture1(id)
y <- callfuture2(x)
} yield y
f.map{ resp: String =>
Ok(resp)
}.recover{
case t: Throwable =>
println("This gets printed");
Ok("shit happened, but i am still ok")
}
}
It seems that inside callfuture1() you're not wrapping all your process inside the Future constructor like
def callfuture1(): Future[?] = Future {
val x = ...
x
}
but your code seems to be
def callfuture1(): Future[?] = {
val x = ... // some error happen here
Future(x)
}
so because it's outside the future, your error is throwing directly into your program code
Appears I am missing something but here is what I have got (posting only relevant piece). where MyService.save returns Future[Option[MyCompany] ].
def myPost = Action.async(parse.json) { request =>
val mn = Json.fromJson[MyEntity](request.body)
mn.map{
case m : MyEntity => MyService.save(m).map{f=>
f.map(mm=> Ok(mm ) )
}
}.recoverTotal {
e => Future { BadRequest("Detected error:" + JsError.toFlatJson(e)) }
}
}
Although I have defined
implicit val companyWriter: Writes[MyCompany] = (...)
And this implicit is in the scope, it shows compile error
Cannot write an instance of MyCompany to HTTP response. Try to define
a Writeable[MyCompany]
FYI: This writer is used elsewhere where I do Json.toJson(myCompany) and over there it finds and works fine.
Anything in particular to async Ok that it's missing?
EDIT
It appears that Ok() method cannot figure out the MyCompany needs to be transformed to json. following seems to have worked.
Ok(Json.toJson(mm) )
Is this because arguments to Ok can vary? Separately there are too many "map" in the above code. Any recommendation for improvement and making it more concise ?
Your compiler error is about a Writeable, not a Writes. Writeables are used to convert whatever you have to something that can be written to an HTTP response, Writes are used to marshall objects to JSON. The names can be a little confusing.
As for style...
def myPost = Action.async(parse.json) { request =>
request.body.validate[MyEntity] map { myEntity =>
MyService.save(myEntity).map { maybeCompany =>
maybeCompany match {
case Some(company) => Ok(Json.toJson(company))
case None => NoContent // or whatever's appropriate
}
}
} recoverTotal { t =>
Future { BadRequest("Detected error: " + JsError.toFlatJson(e)) }
}
}