Consider a Model for Master/Slave election for a cluster.
Member{ id: Long, isMaster: Boolean }
I have a Dao/Repo with following methods:
MemberDao.findById(id:Long):Future[Option[Member]]
MemberDao.update(id:Long, member: Member):Future[Unit]
MemberDao.all() : Future[List[Member]]
Within the MemberService, I'm trying to write a function to set isMaster to false for all existing members, and I'm ending up with this crazily bloated code:
class MemberService ... {
def demoteAllMembers() : Future[Boolean] = {
val futures = memberDao.all.map{ memberFuture =>
memberFuture.map{ member =>
memberDao.findById(member.id).map { existingMemberFuture =>
existingMemberFuture.map { existingMember =>
memberDao.update(existingMember.id, existingMember.copy(isMaster = false)
}
}
}
val results = Await.result(futures, 10 seconds)
// return something here
}
}
}
My Questions are:
1. How should the return statement be written to handle success / errors? e.g. On success, return Future(true) and on failure, return Future(false)
2. Is this way of repetitively mapping future the correct way of doing async programming in scala? I understand this could be written differently in Actor paradigm and probably much better, but in case of OOP, is this the best Scala can do?
Thanks.
Why are you doing MemberDao.findById when you are already holding a member in hand??? (You are also treating the return as a Member, while it should really be an Option[Member]).
Also, update does not need to take an id as a separate parameter (there is one available inside member).
You don't need to Await your result, because your function is returning a Future, and you don't need to return a Boolean: just throw an exception to signal failure.
Consider something like this:
def demoteAllMembers: Future[Unit] = memberDao.all.flatMap {
Future.sequence(_.foreach {
memberDao.update(_.copy(isMaster = false))
})
}.map ( _ => () )
Not all that bloated, is it? :)
Related
I have an existing code which returns an instance of certain type.
def myMethod(inputs) = {
.....some calculations
MyInstance(....)
}
I have to make a change in it now. Change calls some service which returns a Future of some value, which I need to use to update MyInstance.
def myMethod(inputs) = {
.....some calculations
val futureWithSomeValue = someexternalservice.getData(....)
futureWithSomeValue.onComplete {
case Success(value) => ....create MyInstance
case Failure => ....throw error
}
}
But onComplete returns Unit, and hence it breaks the code.
What is best way to do it without changing method signature?
If myMethod call a Future then myMethod must return a Future
Thus, as was said in the comments, you may rather use map instead of onComplete to produce a new Future with your instance and return that.
def myMethod(inputs): Future[MyInstance] = {
// some calculations.
val futureWithSomeValue = someexternalservice.getData(....)
futureWithSomeValue.map { value =>
MyInstance(...)
}
No need to pay attention to the purpose of the function here, it's only for demonstration:
def readAllByPersonOrFail(person: Person, otherPersonId: Long): Future[List[Person]] = {
val personSiblingsFuture: Future[List[Person]] = personSiblingsDomain.readAllByPersonId(person.id)
personSiblingsFuture.map { persons =>
persons.find(_.id == otherPersonId) match {
case Some(person) =>
person.isActive match {
case true => person
case false => throw new IllegalArgumentException("something inactive")
}
case None => throw new IllegalArgumentException("something wrong ehre")
}
}
personSiblingsFuture
}
I would like to return personSiblingsFuture above iff it validates (makes sure correct person is in the list and is active), otherwise throw the exception. I don't think the above code is doing the right thing as it is not existing upon failure.
Take a look at scala.concurrent.Future.map. This creates a new future, whose value is resolved by applying a function to the successful result of this future.
Note that here you're throwing away the resulting future you just created with .map() too.
There are a few areas to solve your problem, though you should question more deeply the use of exceptions with Futures. Scala provides concepts like Future, Option, and Try specifically to avoid throwing exceptions and have a clearer control flow.
Option 1, return the mapped future
In your funciton,
def func(...): Future[List[Person]] {
val personSiblingsFuture = ...;
personSiblingsFuture.map { persons =>
...
}
}
// note we're not returning personSiblingsFuture,
// but the mapped result
When someone actually tries to get the value of the future, e.g. by using .value, they might see an exception intead:
def main() {
val future = func(...); // this is fine
val my_list = future.value; // awaits Future, might throw here
}
Option 2, actually await the list and throw in the function
Returning a future that might throw is strange, it might be a bit easier if the you actually explicitly a had a function that might throw, e.g.
/** jsdoc describing function **/
def funcMightThrow(...): List[Person] {
val personSiblingsFuture = ...;
val personSiblings = personSiblingsFuture.value;
personSiblings.find(_.id == otherPersonId) match {
case Some(person) =>
person.isActive match {
case true => personSiblings
case false => throw new IllegalArgumentException("something inactive")
}
case None => throw new IllegalArgumentException("something wrong ehre")
}
}
Option 3, consider making return types more explicit
def func(...): Future[Try[List[Person]]] {
val personSiblingsFuture = ...;
personSiblingsFuture.map { persons =>
...
// successful case returns 'persons' (List[Person])
// fail cases return Failure(...) instead
}
} // return the mapped future
You can also return Try[List[Person]] rather than a Future[] of that, by using .value, which makes func a blocking function.
I am new to Scala and Play!, but have a reasonable amount of experience of building webapps with Django and Python and of programming in general.
I've been doing an exercise of my own to try to improve my understanding - simply pull some records from a database and output them as a JSON array. I'm trying to use the Enumarator/Iteratee functionality to do this.
My code follows:
TestObjectController.scala:
def index = Action {
db.withConnection { conn=>
val stmt = conn.createStatement()
val result = stmt.executeQuery("select * from datatable")
logger.debug(result.toString)
val resultEnum:Enumerator[TestDataObject] = Enumerator.generateM {
logger.debug("called enumerator")
result.next() match {
case true =>
val obj = TestDataObject(result.getString("name"), result.getString("object_type"),
result.getString("quantity").toInt, result.getString("cost").toFloat)
logger.info(obj.toJsonString)
Future(Some(obj))
case false =>
logger.warn("reached end of iteration")
stmt.close()
null
}
}
val consume:Iteratee[TestDataObject,Seq[TestDataObject]] = {
Iteratee.fold[TestDataObject,Seq[TestDataObject]](Seq.empty[TestDataObject]) { (result,chunk) => result :+ chunk }
}
val newIteree = Iteratee.flatten(resultEnum(consume))
val eventuallyResult:Future[Seq[TestDataObject]] = newIteree.run
eventuallyResult.onSuccess { case x=> println(x)}
Ok("")
}
}
TestDataObject.scala:
package models
case class TestDataObject (name: String, objtype: String, quantity: Int, cost: Float){
def toJsonString: String = {
val mapper = new ObjectMapper()
mapper.registerModule(DefaultScalaModule)
mapper.writeValueAsString(this)
}
}
I have two main questions:
How do i signal that the input is complete from the Enumerator callback? The documentation says "this method takes a callback function e: => Future[Option[E]] that will be called each time the iteratee this Enumerator is applied to is ready to take some input." but I am unable to pass any kind of EOF that I've found because it;s the wrong type. Wrapping it in a Future does not help, but instinctively I am not sure that's the right approach.
How do I get the final result out of the Future to return from the controller view? My understanding is that I would effectively need to pause the main thread to wait for the subthreads to complete, but the only examples I've seen and only things i've found in the future class is the onSuccess callback - but how can I then return that from the view? Does Iteratee.run block until all input has been consumed?
A couple of sub-questions as well, to help my understanding:
Why do I need to wrap my object in Some() when it's already in a Future? What exactly does Some() represent?
When I run the code for the first time, I get a single record logged from logger.info and then it reports "reached end of iteration". Subsequent runs in the same session call nothing. I am closing the statement though, so why do I get no results the second time around? I was expecting it to loop indefinitely as I don't know how to signal the correct termination for the loop.
Thanks a lot in advance for any answers, I thought I was getting the hang of this but obviously not yet!
How do i signal that the input is complete from the Enumerator callback?
You return a Future(None).
How do I get the final result out of the Future to return from the controller view?
You can use Action.async (doc):
def index = Action.async {
db.withConnection { conn=>
...
val eventuallyResult:Future[Seq[TestDataObject]] = newIteree.run
eventuallyResult map { data =>
OK(...)
}
}
}
Why do I need to wrap my object in Some() when it's already in a Future? What exactly does Some() represent?
The Future represents the (potentially asynchronous) processing to obtain the next element. The Option represents the availability of the next element: Some(x) if another element is available, None if the enumeration is completed.
I am trying to find the best way to handle jedis commands from scala. I am trying to implement a finally block, and prevent the java exceptions from bubbling up to my caller.
Does the following code make sense, and is it the best I can do performance wise, if I want to ensure that I handle exceptions when redis may be down temporarily? This trait would be extended by an object, and I'd call objectname.del(key). I feel like I'm combining too many concepts (Either, Option, Try, feels like there should be a cleaner way)
trait MyExperiment {
implicit class TryOps[T](val t: Try[T]) {
def eventually[Ignore](effect: => Ignore): Try[T] = {
val ignoring = (_: Any) => { effect; t }
t transform (ignoring, ignoring)
}
}
val jpool:JedisPool = initialize()
// init the pool at object creation
private def initialize(): JedisPool =
{
val poolConfig = new JedisPoolConfig()
poolConfig.setMaxIdle(10)
poolConfig.setMinIdle(2)
poolConfig.setTestWhileIdle(true)
poolConfig.setTestOnBorrow(true)
poolConfig.setTestOnReturn(true)
poolConfig.setNumTestsPerEvictionRun(10)
new JedisPool( poolConfig , "localhost" )
}
// get a resource from pool. This can throw an error if redis is
// down
def getFromPool: Either[Throwable,Jedis] =
Try(jpool.getResource) match {
case Failure(m) => Left(m)
case Success(m) => Right(m)
}
// return an object to pool
// i believe this may also throw an error if redis is down?
def returnToPool(cache:Jedis): Unit =
Try(jpool.returnResource(cache))
// execute a command -- "del" in this case, (wrapped by
// the two methods above)
def del(key: String) : Option[Long] = {
getFromPool match {
case Left(m) => None
case Right(m) => Try(m.del(key)) eventually returnToPool(m) match {
case Success(r) => Option(r)
case Failure(r) => None
}
}
}
}
Not an exact answer, but I moved on after doing some performance testing. Using the standard java-ish exception blocks ended up being much faster at high iterations (at 10,000 iterations, it was about 2.5x faster than the (bad) code above). That also cleaned up my code, although it's more verbose.
So the answer I arrived at is to use the Java-style exception blocks which provide for the finally construct. I believe it should be significantly faster, as long as exceptions are a very rare occurance.
I have the following snippet of code:
val oldStep = step
try {
someDangerousActionPotentiallyModifyingStep()
} finally {
step = oldStep
}
Is it possible to write a generic helper method to abstract away the pattern? Then I could use the helper method like this:
remember(step) {
someDangerousActionPotentiallyModifyingStep()
}
Of course the following approach does not work, because Scala does not support pass by reference:
def remember[T](x: T)(action: => Unit) {
val previousValue = x
try {
action
} finally {
x = previousValue
}
}
I solved the problem by changing the step data member into an implicit parameter. That way, I never have to restore the old value of step, because its value never changes inside the same scope :)
The problem is that a var does not give you enough control to do what you want to do.
As a direct answer to your question: you need an object with an update method instead of a field. Something like this:
class Box[T](var value:T) {
def apply = value
def update(newValue:T) { value = newValue }
override def toString = value.toString
}
Then you can solve your problem like this:
def remember[T](box:Box[T])(action: =>Unit) {
val prev = box.apply
try {
action
} catch {
case _ => box() = prev
}
}
val step = new Box(0)
// working update
remember(step) { step() = 4 }
// step is now 4
// aborted update
remember(step) { step() = 5; throw new Exception }
// step is still 4
But note that this is really not very idiomatic scala. You should try to make your someDangerousActionPotentiallyModifyingStep a side-effect-free function that returns the new step.
Basically like this:
try {
step = someDangerousFunctionOfStep(step)
} catch {
case _ => // we don't have to do anything because step is still the same
}
If that is not possible for some reason, you might want to investigate Akka agents, which are conceptually similar to the Box above, except with the difference that they are thread-safe and can be used in a transactional way.
Here is how you would use akka agents to solve the problem:
First you need an actor system:
implicit val actorSystem = akka.actor.ActorSystem("test")
Then you can define an agent containing the step value
val step = akka.agent.Agent(0)
Now you can update it in a transaction:
import scala.concurrent.stm._
atomic { txn => step() = 4 }
// step.get will now return 4
atomic { txn => step() = 5; throw new Exception }
// step.get will still return 5. You will have to catch the exception if you don't want
// it to propagate outward
The real power of akka agents comes when you have multiple agents and update them atomically. See the akka agents docs for the canonical "account transfer" transaction example.
It's pretty ugly (and an affront to immutability), but you could try something like this:
def remember[T](newT:T, get: => T, set:(T) => Unit)(action: => Unit) = {
var old = get
set(newT)
try{
action
}
finally{
set(old)
}
}
And then use it like this:
var x = "hello"
remember[String]("world", x, x = _){
//do something dangerous here
}
I think you're looking for Loan pattern.
http://blog.knoldus.com/2012/11/16/scalaknol-understanding-loan-pattern/