I am trying to find the best way to handle jedis commands from scala. I am trying to implement a finally block, and prevent the java exceptions from bubbling up to my caller.
Does the following code make sense, and is it the best I can do performance wise, if I want to ensure that I handle exceptions when redis may be down temporarily? This trait would be extended by an object, and I'd call objectname.del(key). I feel like I'm combining too many concepts (Either, Option, Try, feels like there should be a cleaner way)
trait MyExperiment {
implicit class TryOps[T](val t: Try[T]) {
def eventually[Ignore](effect: => Ignore): Try[T] = {
val ignoring = (_: Any) => { effect; t }
t transform (ignoring, ignoring)
}
}
val jpool:JedisPool = initialize()
// init the pool at object creation
private def initialize(): JedisPool =
{
val poolConfig = new JedisPoolConfig()
poolConfig.setMaxIdle(10)
poolConfig.setMinIdle(2)
poolConfig.setTestWhileIdle(true)
poolConfig.setTestOnBorrow(true)
poolConfig.setTestOnReturn(true)
poolConfig.setNumTestsPerEvictionRun(10)
new JedisPool( poolConfig , "localhost" )
}
// get a resource from pool. This can throw an error if redis is
// down
def getFromPool: Either[Throwable,Jedis] =
Try(jpool.getResource) match {
case Failure(m) => Left(m)
case Success(m) => Right(m)
}
// return an object to pool
// i believe this may also throw an error if redis is down?
def returnToPool(cache:Jedis): Unit =
Try(jpool.returnResource(cache))
// execute a command -- "del" in this case, (wrapped by
// the two methods above)
def del(key: String) : Option[Long] = {
getFromPool match {
case Left(m) => None
case Right(m) => Try(m.del(key)) eventually returnToPool(m) match {
case Success(r) => Option(r)
case Failure(r) => None
}
}
}
}
Not an exact answer, but I moved on after doing some performance testing. Using the standard java-ish exception blocks ended up being much faster at high iterations (at 10,000 iterations, it was about 2.5x faster than the (bad) code above). That also cleaned up my code, although it's more verbose.
So the answer I arrived at is to use the Java-style exception blocks which provide for the finally construct. I believe it should be significantly faster, as long as exceptions are a very rare occurance.
Related
I am trying to write a custom Akka SnapshotStore plugin.
I am at the point where I want to implement this method:
def loadAsync(persistenceId: String, criteria: SnapshotSelectionCriteria): Future[Option[SelectedSnapshot]]
This is what I have so far:
import cats.data.OptionT
import cats.implicits._
...
override def loadAsync(
persistenceId: String,
criteria: SnapshotSelectionCriteria
): Future[Option[SelectedSnapshot]] = {
// same as in the original plugin
val metadata = snapshotMetadatas(persistenceId, criteria).sorted.takeRight(maxLoadAttempts)
// need to get rid of this one!
import scala.concurrent.ExecutionContext.Implicits.global
val getSnapshotAndReportMetric = for {
snapshot <- OptionT.fromOption[Future](getMaybeSnapshotFromMetadata(metadata))
_ <- OptionT.liftF(Future {
observabilityService
.recordMetric(
LongCounterMetric(readVehicleSnapshotFromDiskCounter, SnapShotDirectoryScannerCommandOptions)
)
}(directorySnapshotScanningDispatcher))
} yield snapshot
getSnapshotAndReportMetric.value.recoverWith {
// retry if we listed an older snapshot that was deleted before loading
case _: NoSuchFileException => loadAsync(persistenceId, criteria)
}(streamDispatcher)
}
For reference here is the signature of getMaybeSnapshotFromMetadata. It's signature can be modified to add a Future, but ultimately the wrapped response must be of type Option[SelectedSnapshot].
private def getMaybeSnapshotFromMetadata(metadata: Seq[SnapshotMetadata]): Option[SelectedSnapshot]
The code as is compiles, but only because I have imported the implicit global ExecutionContext. My goal is to use different explicit execution contexts (different configurable dispatchers), but I don't see how to do it for the first line (the one that calls getMaybeSnapshotFromMetadata) within for-comprehensions. If I used OptionT.liftF then I could do it, e.g.
...
snapshot <- OptionT.liftF( Future { getMaybeSnapshotFromMetadata(metadata)}(streamDispatcher))
...but then I get a OptionT[Future, Option[SelectedSnapshot] as a result.
Is there a solution to what I want to achieve? If not I can work around with mere Futures and it's andThen chain method:
Future {
getMaybeSnapshotFromMetadata(metadata)
}(streamDispatcher)
.andThen(selectedSnapshot => {
observabilityService
.recordMetric(
LongCounterMetric(readVehicleSnapshotFromDiskCounter, SnapShotDirectoryScannerCommandOptions)
)
selectedSnapshot
})(opentelemetryDispatcher)
.recoverWith {
// retry if we listed an older snapshot that was deleted before loading
case _: NoSuchFileException => loadAsync(persistenceId, criteria)
}(streamDispatcher)
Update For the second line within for-comprehension liftF is actually a viable solution - I updated the code block.
Do not overuse cats. It's a nice tool for some things, but when used improperly, it just adds complexity and hurts readability without any benefit.
Future(getMaybeSnapshotFromMetadata(metadata))(streamDispatcher)
.andThen { case _ =>
observabilityService.recordWiseMetric(...)
}(directorySnapshotScanningDispatcher)
.recoverWith(...)(streamDispatcher)
This is equivalent to your code without all the complication ...
It is not a "workaround", but an actual proper way to write this.
Consider a Model for Master/Slave election for a cluster.
Member{ id: Long, isMaster: Boolean }
I have a Dao/Repo with following methods:
MemberDao.findById(id:Long):Future[Option[Member]]
MemberDao.update(id:Long, member: Member):Future[Unit]
MemberDao.all() : Future[List[Member]]
Within the MemberService, I'm trying to write a function to set isMaster to false for all existing members, and I'm ending up with this crazily bloated code:
class MemberService ... {
def demoteAllMembers() : Future[Boolean] = {
val futures = memberDao.all.map{ memberFuture =>
memberFuture.map{ member =>
memberDao.findById(member.id).map { existingMemberFuture =>
existingMemberFuture.map { existingMember =>
memberDao.update(existingMember.id, existingMember.copy(isMaster = false)
}
}
}
val results = Await.result(futures, 10 seconds)
// return something here
}
}
}
My Questions are:
1. How should the return statement be written to handle success / errors? e.g. On success, return Future(true) and on failure, return Future(false)
2. Is this way of repetitively mapping future the correct way of doing async programming in scala? I understand this could be written differently in Actor paradigm and probably much better, but in case of OOP, is this the best Scala can do?
Thanks.
Why are you doing MemberDao.findById when you are already holding a member in hand??? (You are also treating the return as a Member, while it should really be an Option[Member]).
Also, update does not need to take an id as a separate parameter (there is one available inside member).
You don't need to Await your result, because your function is returning a Future, and you don't need to return a Boolean: just throw an exception to signal failure.
Consider something like this:
def demoteAllMembers: Future[Unit] = memberDao.all.flatMap {
Future.sequence(_.foreach {
memberDao.update(_.copy(isMaster = false))
})
}.map ( _ => () )
Not all that bloated, is it? :)
Scala's Try is very useful.
I'd like to use that pattern, but log all exceptions.
How can I do this?
Define the following helper:
import scala.util.{Try, Failure}
def LogTry[A](computation: => A): Try[A] = {
Try(computation) recoverWith {
case e: Throwable =>
log(e)
Failure(e)
}
}
Then you can use it as you would use Try, but any exception will be logged through log(e).
Starting Scala 2.13, the chaining operation tap can be used to apply a side effect (in this case some logging) on any value while returning the original value:
import util.chaining._
val x = Try("aa".toInt).tap(_.failed.foreach(println))
// java.lang.NumberFormatException: For input string: "aa"
// x: Try[Int] = Failure(java.lang.NumberFormatException: For input string: "aa")
Or an equivalent pattern matching version:
val x = Try("aa".toInt).tap { case Failure(e) => println(e) case _ => }
// java.lang.NumberFormatException: For input string: "aa"
// x: Try[Int] = Failure(java.lang.NumberFormatException: For input string: "aa")
The tap chaining operation applies a side effect (in this case println or some logging) on a value (in this case a Try) while returning the original unmodified value on which tap is applied (the Try):
def tap[U](f: (A) => U): A
You can tweak it even further using implicit class
def someMethod[A](f: => A): Try[A] = Try(f)
implicit class LogTry[A](res: Try[A]) {
def log() = res match {
case Success(s) => println("Success :) " + s); res
case Failure(f) => println("Failure :( " + f); res
}
}
Now you can call someMethod and on its result call log like this:
scala> someMethod(1/0).log
Failure :( java.lang.ArithmeticException: / by zero
and
scala> someMethod(1).log
Success :) 1
Of course println method inside implicit class can be substituted with any logging you want.
You used the term "exceptions" which is ambiguous. (java.lang.)Throwable is the root of anything that can be placed behind the throw term. java.lang.Exception is one of the two descendants of Throwable (the other being java.lang.Error). Further making this ambiguous is java.lang.RuntimeException, a descendant of Exception, which is probably where you mostly want to spend your logging time (unless you are doing lower level application framework or hardware driver implementations).
Assuming you are wanting to log literally ALL instances of Throwable, then you would need something like this (NOT RECOMMENDED):
def logAtThrowable(f: => A): Try[A] =
try
Try(f) match {
case failure # Failure(throwable) =>
log(s"Failure: {throwable.getMessage}")
failure
case success # _ =>
//uncomment out the next line if you want to also log Success-es
//log(s"Success: {throwable.getMessage}")
success
}
catch throwable: Throwable => {
//!NonFatal pathway
log(s"Failure: {throwable.getMessage}")
throw throwable
}
The external try/catch is required to capture all the Throwable instances which are filtered away by scala.util.control.NonFatal within the Try's try/catch block.
That said...there is a Java/JVM rule: you should never define a catch clause at the resolution of Throwable (again, unless you are doing lower level application framework or hardware driver implementations).
Following the intention of this rule, you would need to narrow the Throwable to you only emitted logging at the finer grained level, say something more refined, like java.lang.RuntimeException. If so, the code would look like this (recommended):
def logAtRuntimeException(f: => A): Try[A] =
Try(f) match {
case failure # Failure(throwable) =>
throwable match {
case runtimeException: RuntimeException =>
log(s"Failure: {runtimeException.getMessage}")
}
failure
case success # _ =>
success
}
In both code snippets above, you will notice that I used match as opposed to .recoverWith. This is to facilitate easily adding a rethrow that works. It turns out that all the methods on Try are themselves also wrapped with try/catch blocks. This means that if you want to log the Throwable and then rethrow it, if you are using one of the Try methods like recoverWith, the rethrow is immediately recaught and placed into a Failure thereby completely undermining the value of the intentional rethrow. By using match, the rethrow is guaranteed to succeed as it remains outside any of the Try methods.
If you would like to see more of the rabbit holes around this particular area, I created a blog post of my own exploration.
I am developing an application using Play framework and scala. I am using anorm for data-access layer. And I've got a problem I could not solve.
Brief: I want to be able to have methods in data-access objects (dao) to work inside transactions as well as being called alone.
Details:
I have data-access layer consist of class with methods that only executes particular SQL over database. Traditionally they looks like:
def list() = DB.withConnection { implicit cn =>
...
}
Now I want to have some methods to be executed in a transaction scope. Like traditional select-update service methods but still be able to run them alone. So, what I have in my mind is like this:
class Service {
def fooTransacted() = {
inTransaction {
val old = dao.select(id = 2)
val newObj = old.copy(isActive = true)
dao.update(newObj)
}
}
def fooSinle() = {
dao.select(id = 2)
}
}
I tried around several ways, but could not come up with any solution.
What about
class Dao {
def foo(id: Long)(implicit connection: Connection) = {
SQL("select * from foo where id={id}").on('id->id).as(...)
}
}
class Service{
def withConnection = {
DB.withConnection {implicit connection =>
Dao.foo(1)
Dao.foo(2)
}
}
def withTransaction = {
DB.withTransaction {implicit connection =>
Dao.foo(1)
Dao.foo(2)
}
}
The solution I've seen used elsewhere (principally in Squeryl), is roughly the following:
import java.sql.Connection
object Helper {
private val conn: ThreadLocal[Connection] = new ThreadLocal
def inTransaction[X](f: Connection => X) = {
conn.get() match {
case null =>
DB.withConnection { newConn =>
conn.set(newConn)
try f(newConn)
finally conn.set(null)
}
case c => f(c)
}
}
}
This way, the inTransaction method is re-entrant, so there's no harm in calling it redundantly inside dao.select.
If you prefer, you can expose conn via a public method, and change the signature of f to => X - you lose some compile-time safety, but the API is a little cleaner.
One pitfall with this approach is that connections are tied to threads, which may cause problems if you're using futures or actors, and a process can resume on a different thread (this is a tricky area anyway, but one you should be aware of).
You might want to look into Squeryl too - it may already do what you need.
I need to use some Java library, which might throw some exceptions in one method and return error codes in another set of methods. So far it leads to the ugly code like
val txn = mgr.prepareTransaction()
val accessRecord = txn.readByQuery(...)
var state : Either[MyError, Result] = null //
try {
// do something here
val result = txn.runCodeWithin(new Callable[Result]() {...})
if (result == -1) {
state = Left(CanNotReadRecord)
} else {
state = Right(txn.getCachedRecord())
}
} catch {
case e: Exception => state = Left(GeneralError(e))
} finally {
state match {
case Right(_) => txn.commit();
case _ => txn.rollback();
}
}
I mostly interested in getting rid of state as var and ability to check the state in finally block. Please advice.
Scala 2.10 introduced the Try class, which is a more functional replacement to the use case of Either[Throwable, Result]. It's got all of the usual monad ops (the things that make for-comprehensions work), and some other helpful methods. (check out the docs for Try here)
Here's a possible re-implementation of your code, using Try, and replacing CanNotReadRecord with a CanNotReadRecordException. It should be functionally equivalent to your example, with the exception of that replacement.
def txResults(txn: Transaction): Try[Record] = for {
result <- Try{ txn.runCodeWithin(...) }
checked <- result match {
case -1 => Failure( new CanNotReadRecordException )
case _ => Success( txn.getCachedRecord )
}
} yield checked
txResults(txn) match {
case Success(record) => txn.commit()
case Failure(e) => txn.rollback() //and maybe handle `e`
}
The Scala ARM (Automatic Resource Management) library handles all this sort of thing elegantly and in a completely air-tight manner.
Check it out.