In scala.util.control.Exception, there are many functions to create catchers and the likes. However, in some cases, I find that I would like to translate/rename/wrap an exception like this:
class MyBaseException...
class MyDerivedException extends MyBaseException ...
class MyOtherDerivedException extends MyBaseException ...
try {
// some code throw new MyDerivedException
// some other code throw new java.lang.IllegalArgumentException
} catch {
case e: MyBaseException => throw e // Let every exceptions derived from MyBaseException go through unchanged
case NonFatal(e) => new MyOtherDerivedException(e)
}
This can also be done with a catch like that:
try{ ... } catch {
case NonFatal(e) if (!classOf[MyBaseException].isAssignableFrom(e.getClass)) => new MyOtherDerivedException(e)
}
So now, fitting that into scala.util.control.Exception catching syntax, I didn't find any way of doing that. In practice, I want something like that:
def wouldMatch(x: Throwable, classes: scala.collection.Seq[Class[_]]): Boolean =
classes exists (_ isAssignableFrom x.getClass)
def shouldRethrowIfMatchOrElse(x: Throwable, classes: scala.collection.Seq[Class[_]]): Boolean = {
if (wouldMatch(x, classes)) true
else shouldRethrow(x)
}
def catchingWithRethrow[T](c: Catcher[T])(exceptions: Class[_]*): Catch[T] = new Catch(c, None, { shouldRethrowIfMatchOrElse(_, exceptions) })
And be used like that:
val myCatching = catchingWithRethrow(nonFatalCatcher)(classOf[MyBaseException]).withApply(e => new MyOtherDerivedException(e))
myCatching {
// some code throw new MyDerivedException
// some other code throw new java.lang.IllegalArgumentException
}
I find that Catch[T] should have a withRethrow(...) function to override the third parameters. This probably would be more elegant like that:
val myCatching = catching(nonFatalCatcher).withRethrow(...).withApply(e => new MyOtherDerivedException(e))
Did I miss something in scala.util.control.Exception that could accomplish that without custom code?
What are your thoughts on this?
I find that Catch[T] should have a withRethrow(...) function to override the third parameters. [...] Did I miss something in scala.util.control.Exception that could accomplish that without custom code?
Maybe there is something like a withRethrow in the standard library, but I haven't seen it either. Luckily, Scala allows us to enrich existing interfaces using implicit classes. Assuming you do not mind adding three lines of custom code, you could achieve the desired syntax:
implicit class WithRethrow[T](theCatch: Catch[T]) {
def withRethrow(exceptions: Class[_]*): Catch[T] = new Catch[T](theCatch.pf, theCatch.fin, t => exceptions exists (_.isAssignableFrom(t.getClass)))
}
With your use-case
// may need to import the implicit class if it is not already in scope
val myCatching = catching(nonFatalCatcher).withRethrow(classOf[MyBaseException]).withApply(e => throw new MyOtherDerivedException(e))
myCatching {
throw new OutOfMemoryError() // OutOfMemoryError is thrown because it is not non-fatal
throw new IllegalArgumentException // MyOtherDerivedException is thrown because IllegalArgumentException is not rethrown
throw new MyDerivedException // MyDerivedException is thrown because MyDerivedException is rethrown
}
It seems like a very specialized/unusual use case. It violates LSP. And using exceptions at all is unidiomatic in scala - scala.util.control.Exception is mainly about catching exceptions that library functions might throw and translating them into more idiomatic expressions of potential failures.
If you want to do this then write your own code for it - it should be pretty straightforward. I really don't think this is a common enough use case for there to be a standard library function for it.
Related
The following code returns a Future.
val findUserFuture: Future[Option[User]] = userRepo.findOne(userKeys)
Then I process the Future
findUserFuture.flatMap {....}
.recover{...}
fineOne returns the Future and the Future wraps call to getOneById
def findOne(userKeys:UserKeys):Future[Option[User]] = {
Future{
//val loginInfo:LoginInfo = LoginInfo(userKeys.providerID,userKeys.authProvider)
val userOption:Option[User] = getOneById(userKeys)
userOption
}
}
I suppose that recover will be called if Future returned by findOne fails i.e. throws an Exception. So I am simulating that by making getOneById throw an exception.
when(mockUserRepository.findOne(userKeys)).thenReturn(Future(Some(user)))
when(mockUserRepository.getOneById(userKeys)).thenThrow(classOf[RuntimeException]) //simulating database error
But the unit test doesn't throw an exception and the test proceeds using value Future(Some(User)).
I also tried throwing the exception from findOne - when(mockUserRepository.findOne(userKeys)).thenThrow(classOf[RuntimeException]) but the test case stops
with the following two prints and the .recover of the Future is not called
java.lang.RuntimeException was thrown.
java.lang.RuntimeException
This findUserFuture: Future[Option[User]] or userRepo.findOne returns future,
hence you need to return Future.failed in your mock.
For ex.
when(mockUserRepository.findOne(userKeys)).thenReturn(Future(Some(user)))
when(mockUserRepository.getOneById(userKeys)).thenReturn(Future.failed(new RuntimeException("network failure"))
Find below complete working test to simulate your use case :
test("mock future test") {
case class User(name: String)
case class UserNotFoundException(name: String) extends Exception
trait UserRepo {
def findOne(name: String): Future[Option[User]]
}
val name = "bob"
val dummyUser = User("dummy")
val userRepo = mock[UserRepo]
when(userRepo.findOne(name)).thenReturn(Future.failed(new RuntimeException()))
val userF = userRepo
.findOne(name)
.flatMap {
case Some(user) ⇒ Future.successful(user)
case None ⇒ Future.failed(UserNotFoundException(name))
}
.recover {
case NonFatal(_) ⇒ dummyUser
}
userF.futureValue shouldBe dummyUser
}
Update *
After looking at the original post closely, I found small mistake in the way you are mocking.
try below:
when(mockUserRepository.findOne(userKeys)).thenCallRealMethod()
when(mockUserRepository.getOneById(userKeys)).thenThrow(classOf[RuntimeException])
Notice thenCallRealMethod() here, earlier you were mocking findOne to return future with successful value which means original method was not getting called which in turn was not calling getOneById
You can't mock the type you want to test, a mock has nothing of the original behaviour of that type.
If you wanna stub just some behaviour of the class under test (or any other type), you should use a spy, then you would only do
when(spyUserRepository.getOneById(userKeys)).thenThrow(classOf[RuntimeException])
then call spyUserRepository.findOne(userKeys) and assert that it returns a Failed future
That said, it seems you have the responsabilities a bit mixed up here, I'd suggest having a second look to your design as having to resort to use spys for this looks like a big code smell for me.
I'm trying to construct class which receives a parser as an argument and uses this parser on each line. Below is a minimal example that you can paste into spark-shell.
import scala.util.{Success,Failure,Try}
import scala.reflect.ClassTag
class Reader[T : ClassTag](makeParser: () => (String => Try[T])) {
def read(): Seq[T] = {
val rdd = sc.parallelize(Seq("1","2","oops","4")) mapPartitions { lines =>
// Since making a parser can be expensive, we want to make only one per partition.
val parser: String => Try[T] = makeParser()
lines flatMap { line =>
parser(line) match {
case Success(record) => Some(record)
case Failure(_) => None
}
}
}
rdd.collect()
}
}
class IntParser extends (String => Try[Int]) with Serializable {
// There could be an expensive setup operation here...
def apply(s: String): Try[Int] = Try { s.toInt }
}
However, when I try to run something like new Reader(() => new IntParser).read() (which type-checks just fine) I get the dreaded org.apache.spark.SparkException: Task not serializable error relating to closures.
Why is there an error and is there a way to re-engineer the above to avoid this (while keeping Reader generic)?
The problem is that makeParser is variable to class Reader and since you are using it inside rdd transformations spark will try to serialize the entire class Reader which is not serializable. So you will get task not serializable exception.
Adding Serializable to the class Reader will work with your code. But that is not a good practice since it will serialize entire class variables which might not be needed.
In general you could use the functions instead of method to avoid serialization issues. Because in scala functions are actually objects and it will be serialized.
Refer to this answer :
Task not serializable: java.io.NotSerializableException when calling function outside closure only on classes not objects
Is there any way I can convert a variable of type com.ning.http.client.ListenableFuture[A] into a type scala.concurrent.Future[A]
in other words what would be the content of the function
def toFuture[A](a: com.ning.http.client.ListenableFuture[A]):scala.concurrent.Future[A] = ???
I am specifically in the case where A = com.ning.http.client.Response
Note that com.ning.http.client.ListenableFuture[A] is not the same as com.google.common.util.concurrent.ListenableFuture (and hence this proposed duplicate does not solve the issue)
The idea is the same as with guava's ListenableFuture, although a little more constrained due to more constrained signature.
First, you need to get a java.util.concurrent.Executor to add a callback. Since your Scala code interacts with a Java library, I'd suggest to define your pool of scala.concurrent.ExecutorServices based on Java Executors - that way you can retain both the instance of an Executor and an ExecutorService, something like the following:
import java.util.concurrent.Executors
import scala.concurrent.ExecutionContext
val executor = Executors.newFixedThreadPool(5) // use it for Java futures
implicit val executionContext = ExecutionContext.fromExecutor(executor) // use it for Scala futures
The above steps are not needed if you want to process everything in different pools. In case you want to use an existing ExecutionContext, here's a snippet I googled.
Then, to convert the ListenableFuture into a Future, I'd do something like this (considering some exception semantics of java.util.concurrent.Future):
def toFuture[A](a: ListenableFuture[A]): Future[A] = {
val promise = Promise[A]()
a.addListener(new Runnable {
def run() = {
try {
promise.success(a.get)
} catch {
case ex: ExecutionException => promise.failure(ex.getCause)
case ex => promise.failure(ex)
}
}
}, executor)
promise.future
}
I'm experiencing strange behavior and I'm wondering if this is a bug or if I'm missing something.
The following code:
class Foo extends SpecificationWithJUnit {
"This test should pass" in new ctx {
Bar(Zoo("id")) must haveInstanceZoo
}
trait ctx extends Scope {
def haveInstanceZoo : Matcher[Bar] =
beAnInstanceOf[Zoo] ^^ { (_: Bar).z aka "instanceOfZoo" }
}
}
case class Bar(z: Zoo)
case class Zoo(id: String)
fails with the following Exception:
'org.specs2.matcher.ThrownExpectationsCreation$$anon$1#48072f8c:
org.specs2.matcher.ThrownExpectationsCreation$$anon$1'
is not an instance of 'com.test.Zoo'
If I remove the "aka" from the custom matcher everything works.
Thoughts?
Thanks
Netta
You cannot use aka like this because you are effectively trying to assert that an Expectation, the object you create with aka is an instance of Zoo.
If you want to specify a different failure message on a matcher you can write this:
def haveInstanceZoo: Matcher[Bar] = (bar: Bar) =>
(bar.z.isInstanceOf[Zoo], "bar.z is not an instance of Zoo")
I recently started developing an Application in Play Scala. Although I have used Play Java for several applications already, I am also new to Scala and Play Scala.
I use DAO pattern to abstract the database interaction. The DAO contains methods for insert, update delete. After reading async and thread-pool related documentation, I figured that making database interaction async was highly important, unless you tweak the Play default thread pool to have many threads.
To ensure that all database calls are handled asynchronously, I made all the call to return a Future instead of a value directly. I have created a separate execution context for the database interactions.
trait Dao[K, V] {
def findById(id: K): Future[Option[V]]
def update(v: V): Future[Boolean]
[...]
}
This has lead to very complex and deeply nested code in actions.
trait UserDao extends Dao[Long, User] {
def existsWithEmail(email: String): Future[Boolean]
def insert(u: User) Future[Boolean]
}
object UserController extends Controller {
def register = Action {
[...]
userDao.existsWithEmail(email).flatMap { exists =>
exits match {
case true =>
userDao.insert(new User("foo", "bar")).map { created =>
created match {
case true => Ok("Created!")
case false => BadRequest("Failed creation")
}
}
case false =>
Future(BadRequest("User exists with same email"))
}
}
}
}
Above is a sample of simplest of actions. Level of nesting gets deeper as I have more database calls involved. Although I figured that some of the nesting can be reduced with the use of for comprehension, I am doubting if my approach itself is fundamentally wrong?
Consider a case where I need to create a user,
a. If none exists already with same email address.
b. If none exists already with same mobile number.
I can create two futures,
f(a) checking if user exists with email.
f(b) checking if user exists with mobile.
I cannot go and insert a new user unless I verify that both conditions evaluate false. I can actually have f(a) and f(b) running in parallel. The parallel execution maybe undesirable in case f(a) evaluates to true, and may work in favor otherwise. Step 3 of creating user depends on both these futures, so I wonder if following is equally good?
trait UserDao extends Dao[Long, User] {
def existsWithEmail(email: String): Boolean
def existsWithMobile(mobile: String): Boolean
def insert(u: User): Unit
}
def register = Action {
implicit val dbExecutionContext = myconcurrent.Context.dbExceutionContext
Future {
if (!userDao.existsWithEmail(email) && !userDao.existsWithMobile(mobile) {
userDao.insert(new User("foo", "bar")
Ok("Created!")
} else {
BadRequest("Already exists!")
}
}
}
Which one is a better approach? Does the approach of using a single Future with multiple calls to database have any downside?
You are correct when you say that a for comprehension can make for less nesting.
To solve the dual-future problem, consider:
existsWithEmail(email).zip(existsWithMobile(mobile)) map {
case (false, false) => // create user
case _ => // already exists
}
If you have a lot of these, you can use Future.sequence( Seq(future1, future2, ...) ) to turn a sequence of futures into a future sequence.
You may want to take a look at more functional idioms for DB access than DAO, e.g., Slick or Anorm. Usually those will compose better and end up being more flexible than DAO.
A side note: it is more efficient to use if/else for a simple true/false test than it is to use match/case, and is the preferred style.
I solved this problem using for comprehension in scala. I added a few implicit type converters to help with error handling.
Initially I did something like,
def someAction = Action.async {
val result =
for {
student <- studentDao.findById(studentId)
if (student.isDefined)
parent <- parentDao.findById(student.get.parentId)
if (parent.isDefined)
address <- addressDao.findById(parent.get.addressId)
if (address.isDefined)
} yield {
// business logic
}
result fallbackTo Future.successful(BadRequest("Something went wrong"))
}
This is how the code was initially structured to counter the dependency between futures. Note that each subsequent future depends on the previous future. Also, each findById is returning a Future[Option[T]] so if within for comprehension is required to handle cases where the methods return None. I used fallbackTo method on the Future to fallback to a BadRequest result if any of the futures evaluated to None (In event of any if condition failing within for comprehension, it returns a failed future) Another issue with above approach was that it would suppress any kind of exception encountered (even exceptions as trivial as NPE) and simply fallback to BadRequest instead, which was very bad.
Above method was able to counter future of options and handle the failed cases, although it was not helpful to figure out exactly which of the futures in the for comprehension had failed. To overcome this limitation, I used implicit type converters.
object FutureUtils {
class FutureProcessingException(msg: String) extends Exception(msg)
class MissingOptionValueException(msg: String) extends FutureProcessingException(msg)
protected final class OptionFutureToOptionValueFuture[T](f: Future[Option[T]]) {
def whenUndefined(error: String)(implicit context: ExecutionContext): Future[T] = {
f.map { value =>
if (value.isDefined) value.get else throw new MissingOptionValueException(error)
}
}
}
import scala.language.implicitConversions
implicit def optionFutureToValueFutureConverter[T](f: Future[Option[T]]) = new OptionFutureToOptionValueFuture(f)
}
The above implicit conversions allowed me to write readable for comprehensions chaining multiple futures.
import FutureUtils._
def someAction = Action.async {
val result =
for {
student <- studentDao.findById(studentId) whenUndefined "Invalid student id"
parent <- parentDao.findById(student.get.parentId) whenUndefined "Invalid parent id"
address <- addressDao.findById(parent.get.addressId) whenUndefined "Invalid address id"
} yield {
// business logic
}
result.recover {
case fpe: FutureProcessingException => BadRequest(fpe.getMessage)
case t: Throwable => InternalServerError
}
}
The above approach ensured that all exceptions caused by missing Option value are handled as a BadRequest with specific message about what exactly failed. All other failures are treated as InternalServerError. You can log the exact exception with stack trace in order to help debug.