Inspection error in scala method / play framework / rest - scala

I'm still learning scala so this might be a question with an easy answer, but I've been stuck on writing a single method over and over for almost a day, unable to get this code to compile.
I'm playing with the Play Framework and a reactive mongo template to learn how Scala and Play work.
I have a controller with a few methods, endpoints for a REST service.
The issue is about the following method, which accepts a list of json objects and updates those objects using the mongo reactive driver. The class has one member, citiesFuture which is of type Future[JSONCollection].
The original class code which I'm adding this method to can be found here for context: CityController on github
def updateAll() = Action.async(parse.json) { request =>
Json.fromJson[List[City]](request.body) match {
case JsSuccess(givenCities, _) =>
citiesFuture onComplete[Future[Result]] { cities =>
val updateFutures: List[Future[UpdateWriteResult]] = for {
city <- givenCities
} yield cities.get.update(City.getUniqueQuery(city), Json.obj("$set" -> city))
val promise: Promise[Result] = Promise[Result] {
Future.sequence(updateFutures) onComplete[Result] {
case s#Success(_) =>
var count = 0
for {
updateWriteResult <- s.value
} yield count += updateWriteResult.n
promise success Ok(s"Updated $count cities")
case Failure(_) =>
promise success InternalServerError("Error updating cities")
}
}
promise.future
}
case JsError(errors) =>
Future.successful(BadRequest("Could not build a city from the json provided. " + Errors.show(errors)))
}
}
I've managed to get this far with alot of trial and error, but I'm starting to understand how some of the mechanics of scala and Futures work, I think :) I think I'm close, but my IDE still gives me a single Inspection error just at the single closing curly brace above the line promise.future.
The error reads: Expression of type Unit doesn't conform to expected type Nothing.
I've checked the expected return values for the Promise and onComplete code blocks, but I don't believe they expect Nothing as a return type.
Could somebody please explain to me what I'm missing, and also, I'm sure this can be done better, so let me know if you have any tips I can learn from!

You're kinda on the right track but as #cchantep said, once you're operating in Future-land, it would be very unusual to need to create your own with Promise.future.
In addition, it's actually quite unusual to see onComplete being used - idiomatic Scala generally favors the "higher-level" abstraction of mapping over Futures. I'll attempt to demonstrate how I'd write your function in a Play controller:
Firstly, the "endpoint" just takes care of one thing - interfacing with the outside world - i.e. the JSON-parsing part. If everything converts OK, it calls a private method (performUpdateAll) that actually does the work:
def updateAll() = Action.async(parse.json) { request =>
Json.fromJson[List[City]](request.body) match {
case JsSuccess(givenCities, _) =>
performUpdateAll(givenCities)
case JsError(errors) =>
Future.successful(BadRequest("Could not build a city from the json provided. "))
}
}
Next, we have the private function that performs the update of multiple cities. Again, trying to abide by the Single Responsibility Principle (in a functional sense - one function should do one thing), I've extracted out updateCity which knows how to update exactly one city and returns a Future[UpdateWriteResult]. A nice side-effect of this is code-reuse; you may find you'll be able to use such a function elsewhere.
private def performUpdateAll(givenCities:List[City]):Future[Result] = {
val updateFutures = givenCities.map { city =>
updateCity(city)
}
Future.sequence(updateFutures).map { listOfResults =>
if (listOfResults.forall(_.ok)) {
val count = listOfResults.map(_.n).sum
Ok(s"Updated $count cities")
} else {
InternalServerError("Error updating cities")
}
}
}
As far as I can tell, this will work in exactly the same way as you intended yours to work. But by using Future.map instead of its lower-level counterpart Future.onComplete and matching on Success and Failure you get much more succinct code where (in my opinion) it's much easier to see the intent because there's less boilerplate around it.
We still check that every update worked, with this:
if (listOfResults.forall(_.ok))
which I would argue reads pretty well - all the results have to be OK!
The other little trick I did to tidy up was replace your "counting" logic which used a mutable variable, with a one-liner:
var count = 0
for {
updateWriteResult <- s.value
} yield count += updateWriteResult.n
Becomes:
val count = listOfResults.map(_.n).sum
i.e. convert the list of results to a list of integers (the n in the UpdateWriteResult) and then use the built-in sum function available on lists to do the rest.

Related

How to handle a failed future in a for-comprehension

i have the following for comprehension. It is supposed to delete a row in my database but only if the row exists (So if there is a news for the given id):
override def deleteNews(newsId: Long): Int = {
val getAndDelete = for {
Some(news) <- newsDao.get(newsId)// returns Future[Option[News]]
delete <- newsDao.remove(news) // returns Future[Int]
} yield delete
Await.result(getAndDelete, responseTimeout)
}
But i don't know how to handle the case when there is no element for a given id. Currently this exception is thrown:
Unexpected exception[NoSuchElementException: Future.filter predicate is not satisfied]
I hope my approach is not to awful :D
I'm relatively new to scala.
Using Await is not that great of an idea: it's best to delay the blocking as long as you possibly can.
IMO, no element for a given ID shouldn't be a failure. newsDao.get should return a successful future of None if there's nothing with that ID, you shouldn't call newsDao.remove on an ID which doesn't exist if you can help it, and the overall result should just be successfully deleted zero rows (as I'd look at the contract of deleteNews as ensuring that at some point between the call and the return there were no rows associated with newsId (a little bit of handwaving here around data races, of course...)).
So with that, assuming you can't change newsDao's implementation:
val getFut: Future[Option[News]] =
newsDao.get(newsId).recover {
// can still fail for other reasons
case _: NoSuchElementException => None
}
// I really prefer map/flatMap directly vs. for-comprehension sugar, especially when dealing with multiple monadicish things
// Not the most succinct, but leaving meaningful names in for documentation
val getAndRemove =
getFut.flatMap { newsOpt =>
newsOpt.map { news =>
newsDao.remove(news)
}.getOrElse(Future.successful(0))
}
If you still need deleteNews to return a bare Int, you can Await.result and accept that you'll sometimes get exceptions thrown and that this is probably suboptimal.
As Levi mentioned, always try to avoid blocking and when you pattern match, make sure you handle all the cases.
You can do this using for-comprehension like below:
def deleteNews(newsId: Long): Future[Option[Int]] =
for {
news <- newsDao.get(newsId)
delete <- Future.sequence(news.map(id => newsDao.remove(id)).toList)
} yield delete.headOption
Honestly I have not used this trick to go from Option[Future] to Future[Option]. I would be interested to see what others says!

Functional scala log accumulator

I'm working on a Scala project using cats library, mainly. In there, we have calls like
for {
_ <- initSomeServiceAndLog("something from a far away service")
_ <- initSomeOtherServiceAndLog("something from another far away service")
a <- b()
c <- d(a)
} yield c
Imagine that b also logs something or might throw a business error (I know, we avoid to throw in Scala, but it's not the case right now). I'm looking for a solution to accumulate logs and print them all in the end, in a single message.
For a happy path, I saw that Writer Monad from Cats might be an acceptable solution.
But what if b method throws? The requirements are to logs everything - all previous logs and the error message, in a single message, with some kind of unique trace ID.
Any thoughts? Thanks in advance
Implementing functional logging (in a way that preserves logs even if error happened) using monad transformers like Writer (WriterT) or State (StateT) is hard. However, if we don't be anal about FP approach we could do the following:
use some IO monad
with it create something like in-memory storage for logs
however implement in in a functional way
Personally I would pick either cats.effect.concurrent.Ref or monix.eval.TaskLocal.
Example using Ref (and Task):
type Log = Ref[Task, Chain[String]]
type FunctionalLogger = String => Task[Unit]
val createLog: Task[Log] = Ref.of[Task, Chain[String]](Chain.empty)
def createAppender(log: Log): FunctionalLogger =
entry => log.update(chain => chain.append(entry))
def outputLog(log: Log): Task[Chain[String]] = log.get
with helpers like that I could:
def doOperations(logger: FunctionalLogger) = for {
_ <- operation1(logger) // logging is a side effect managed by IO monad
_ <- operation2(logger) // so it is referentially transparent
} yield result
createLog.flatMap { log =>
doOperations(createAppender(log))
.recoverWith(...)
.flatMap { result =>
outputLog(log)
...
}
}
However, making sure that output is called is a bit of a pain so we could use some form of Bracket or Resource to handle it:
val loggerResource: Resource[Task, FunctionalLogger] = Resource.make {
createLog // acquiring resource - IO operation that accesses something
} { log =>
outputLog(log) // releasing resource - works like finally in try-catchso it should
.flatMap(... /* log entries or sth */) // be called no matter if error occured
}.map(createAppender)
loggerResource.use { logger =>
doSomething(logger)
}
If you don't like passing this appender around explicitly you could use Kleisli to inject it:
type WithLogger[A] = Kleisli[Task, FunctionalLogger, A]
// def operation1: WithLogger[A]
// def operation2: WithLogger[B]
def doSomething: WithLogger[C] = for {
a <- operation1
b <- operation2
} yield c
loggerResource.use { logger =>
doSomething(logger)
}
TaskLocal would be used in a very similar way.
At the end of the day you would end up with:
type that says that it is logging
mutability managed through IO, so referential transparency would not be lost
certainty that even if IO fails, log will be preserved and the results sent
I believe some purist would not like this solution, but it has all the benefits of FP, so I would personally use it.

Scala adding elements to seq and handling futures, maps, and async behavior

I'm still a newbie in scala and don't quite yet understand the concept of Futures/Maps/Flatmaps/Seq and how to use them properly.
This is what I want to do (pseudo code):
def getContentComponents: Action[AnyContent] = Action.async {
contentComponentDTO.list().map( //Future[Seq[ContentComponentModel]] Get all contentComponents
contentComponents => contentComponents.map( //Iterate over [Seq[ContentComponentModel]
contentComponent => contentComponent.typeOf match { //Match the type of the contentComponent
case 1 => contentComponent.pictures :+ contentComponentDTO.getContentComponentPicture(contentComponent.id.get) //Future[Option[ContentComponentPictureModel]] add to _.pictures seq
case 2 => contentComponent.videos :+ contentComponentDTO.getContentComponentVideo(contentComponent.id.get) //Future[Option[ContentComponentVideoModel]] add to _.videos seq
}
)
Ok(Json.toJson(contentComponents)) //Return all the contentComponents in the end
)
}
I want to add a Future[Option[Foo]] to contentComponent.pictures: Option[Seq[Foo]] like so:
case 2 => contentComponent.pictures :+ contentComponentDTO.getContentComponentPicture(contentComponent.id.get) //contentComponent.pictures is Option[Seq[Foo]]
and return the whole contentComponent back to the front-end via json in the end.
I know this might be far away from the actual code in the end, but I hope you got the idea. Thanks!
I'll ignore your code and focus on what is short and makes sense:
I want to add a Future[Option[Foo]] to contentComponent.pictures: Option[Seq[Foo]] like so:
Let's do this, focusing on code readability:
// what you already have
val someFuture: Future[Option[Foo]] = ???
val pics: Option[Seq[Foo]] = contentComponent.pictures
// what I'm adding
val result: Future[Option[Seq[Foo]]] = someFuture.map {
case None => pics
case Some(newElement) =>
pics match {
case None => Some(Seq(newElement)) // not sure what you want here if pics is empty...
case Some(picsSequence) => Some(picsSequence :+ newElement)
}
}
And to show an example of flatMap let's say you need the result of result future in another future, just do:
val otherFuture: Future[Any] = ???
val everything: Future[Option[Seq[Foo]]] = otherFuture.flatmap { otherResult =>
// do something with otherResult i.e., the code above could be pasted in here...
result
}
My answer will attempt to help with some of the conceptual sub-questions which form parts of your overall larger question.
flatMap and for-yield
One of the points of flatMap is to help with the problem of the Pyramid of Doom. This happens when you have
structures nested within structures nested within structures ...
doA().map { resultOfA =>
doB(resultOfA).map { resolutOfB =>
doC(resultOfB).map { resultOfC =>
...
}
}
}
If you use for-yield you get flatMap out of the box and it allows you to
flatten the pyramid
so that your code looks more like a linear structure
for {
resultOfA <- doA
resultOfB <- doB(resultOfA)
resultOfC <- doC(resultOfB)
...
} yield {...}
There is a rule of thumb in software engineering that deeply nested structures are harder to debug and reason about, so
we strive to minimise the nesting. You will hit this issue especially when dealing with Futures.
Mapping over Future vs. mapping over sequence
Mapping is usually first thought in terms of iteration over a sequence, which might lead to understanding of
mapping over a Future in terms of iterating over a sequence of one. My advice would be not to use the iteration concept when
trying to understand mapping over Futures, Options etc. In these cases it might be better to think of mapping as a process of destructing the structure
so that you get at the element inside the structure. One could visualise mapping as
breaking the shell of a walnut so you get at the delicious kernel inside and then rebuilding the shell.
Futures and monads
As you try to learn more about Futures and when you begin to deal with types like Future[Option[SomeType]] you will inevitably
stumble upon documentation about monads and its cryptic terminology might scare you away. If this happens, it might help to think of monads (of which Future is a particular instance) as simply
something you can stick into a for-yield so that you can get at the
delicious walnut kernels whilst avoiding the pyramid of doom.

What is the advantage of using Option.map over Option.isEmpty and Option.get?

I am a new to Scala coming from Java background, currently confused about the best practice considering Option[T].
I feel like using Option.map is just more functional and beautiful, but this is not a good argument to convince other people. Sometimes, isEmpty check feels more straight forward thus more readable. Is there any objective advantages, or is it just personal preference?
Example:
Variation 1:
someOption.map{ value =>
{
//some lines of code
}
} orElse(foo)
Variation 2:
if(someOption.isEmpty){
foo
} else{
val value = someOption.get
//some lines of code
}
I intentionally excluded the options to use fold or pattern matching. I am simply not pleased by the idea of treating Option as a collection right now, and using pattern matching for a simple isEmpty check is an abuse of pattern matching IMHO. But no matter why I dislike these options, I want to keep the scope of this question to be the above two variations as named in the title.
Is there any objective advantages, or is it just personal preference?
I think there's a thin line between objective advantages and personal preference. You cannot make one believe there is an absolute truth to either one.
The biggest advantage one gains from using the monadic nature of Scala constructs is composition. The ability to chain operations together without having to "worry" about the internal value is powerful, not only with Option[T], but also working with Future[T], Try[T], Either[A, B] and going back and forth between them (also see Monad Transformers).
Let's try and see how using predefined methods on Option[T] can help with control flow. For example, consider a case where you have an Option[Int] which you want to multiply only if it's greater than a value, otherwise return -1. In the imperative approach, we get:
val option: Option[Int] = generateOptionValue
var res: Int = if (option.isDefined) {
val value = option.get
if (value > 40) value * 2 else -1
} else -1
Using collections style method on Option, an equivalent would look like:
val result: Int = option
.filter(_ > 40)
.map(_ * 2)
.getOrElse(-1)
Let's now consider a case for composition. Let's say we have an operation which might throw an exception. Additionaly, this operation may or may not yield a value. If it returns a value, we want to query a database with that value, otherwise, return an empty string.
A look at the imperative approach with a try-catch block:
var result: String = _
try {
val maybeResult = dangerousMethod()
if (maybeResult.isDefined) {
result = queryDatabase(maybeResult.get)
} else result = ""
}
catch {
case NonFatal(e) => result = ""
}
Now let's consider using scala.util.Try along with an Option[String] and composing both together:
val result: String = Try(dangerousMethod())
.toOption
.flatten
.map(queryDatabase)
.getOrElse("")
I think this eventually boils down to which one can help you create clear control flow of your operations. Getting used to working with Option[T].map rather than Option[T].get will make your code safer.
To wrap up, I don't believe there's a single truth. I do believe that composition can lead to beautiful, readable, side effect deferring safe code and I'm all for it. I think the best way to show other people what you feel is by giving them examples as we just saw, and letting them feel for themselves the power they can leverage with these sets of tools.
using pattern matching for a simple isEmpty check is an abuse of pattern matching IMHO
If you do just want an isEmpty check, isEmpty/isDefined is perfectly fine. But in your case you also want to get the value. And using pattern matching for this is not abuse; it's precisely the basic use-case. Using get allows to very easily make errors like forgetting to check isDefined or making the wrong check:
if(someOption.isEmpty){
val value = someOption.get
//some lines of code
} else{
//some other lines
}
Hopefully testing would catch it, but there's no reason to settle for "hopefully".
Combinators (map and friends) are better than get for the same reason pattern matching is: they don't allow you to make this kind of mistake. Choosing between pattern matching and combinators is a different question. Generally combinators are preferred because they are more composable (as Yuval's answer explains). If you want to do something covered by a single combinator, I'd generally choose them; if you need a combination like map ... getOrElse, or a fold with multi-line branches, it depends on the specific case.
It seems similar to you in case of Option but just consider the case of Future. You will not be able to interact with the future's value after going out of Future monad.
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Promise
import scala.util.{Success, Try}
// create a promise which we will complete after sometime
val promise = Promise[String]();
// Now lets consider the future contained in this promise
val future = promise.future;
val byGet = if (!future.value.isEmpty) {
val valTry = future.value.get
valTry match {
case Success(v) => v + " :: Added"
case _ => "DEFAULT :: Added"
}
} else "DEFAULT :: Added"
val byMap = future.map(s => s + " :: Added")
// promise was completed now
promise.complete(Try("PROMISE"))
//Now lets print both values
println(byGet)
// DEFAULT :: Added
println(byMap)
// Success(PROMISE :: Added)

Is there an easy way to get a Stream as output of a RowParser?

Given rowParser of type RowParser[Photo], this is how you would parse a list of rows coming from a table photo, according to the code samples I have seen so far:
def getPhotos(album: Album): List[Photo] = DB.withConnection { implicit c =>
SQL("select * from photo where album = {album}").on(
'album -> album.id
).as(rowParser *)
}
Where the * operator creates a parser of type ResultSetParser[List[Photo]]. Now, I was wondering if it was equally possible to get a parser that yields a Stream (thinking that being more lazy is always better), but I only came up with this:
def getPhotos(album: Album): Stream[Photo] = DB.withConnection { implicit c =>
SQL("select * from photo where album = {album}").on(
'album -> album.id
)() collect (rowParser(_) match { case Success(photo) => photo })
}
It works, but it seems overly complicated. I could of course just call toStream on the List I get from the first function, but my goal was to only apply rowParser on rows that are actually read. Is there an easier way to achieve this?
EDIT: I know that limit should be used in the query, if the number of rows of interest is known beforehand. I am also aware that, in many cases, you are going to use the whole result anyway, so being lazy will not improve performance. But there might be a case where you save a few cycles, e.g. if for some reason, you have search criteria that you cannot or do not want to express in SQL. So I thought it was odd that, given the fact that anorm provides a way to obtain a Stream of SqlRow, I didn't find a straightforward way to apply a RowParser on that.
I ended up creating my own stream method which corresponds to the list method:
def stream[A](p: RowParser[A]) = new ResultSetParser[Stream[A]] {
def apply(rows: SqlParser.ResultSet): SqlResult[Stream[A]] = rows.headOption.map(p(_)) match {
case None => Success(Stream.empty[A])
case Some(Success(a)) => {
val s: Stream[A] = a #:: rows.tail.flatMap(r => p(r) match {
case Success(r) => Some(r)
case _ => None
})
Success(s)
}
case Some(Error(msg)) => Error(msg)
}
}
Note that the Play SqlResult can only be either Success/Error while each row can also be Success/Error. I handle this for the first row only, assuming the rest will be the same. This may or may not work for you.
You're better off making smaller (paged) queries using limit and offset.
Anorm would need some modification if you're going to keep your (large) result around in memory and stream it from there. Then the other concern would be the new memory requirements for your JVM. And how would you deal with caching on the service level? See, previously you could easily cache something like photos?page=1&size=10, but now you just have photos, and the caching technology would have no idea what to do with the stream.
Even worse, and possibly on a JDBC-level, wrapping Stream around limited and offset-ed execute statements and just making multiple calls to the database behind the scenes, but this sounds like it would need a fair bit of work to port the Stream code that Scala generates to Java land (to work with Groovy, jRuby, etc), then get it on the approved for the JDBC 5 or 6 roadmap. This idea will probably be shunned as being too complicated, which it is.
You could wrap Stream around your entire DAO (where the limit and offset trickery would happen), but this almost sounds like more trouble than it's worth :-)
I ran into a similar situation but ran into a Call Stack Overflow exception when the built-in anorm function to convert to Streams attempted to parse the result set.
In order to get around this I elected to abandon the anorm ResultSetParser paradigm, and fall back to the java.sql.ResultSet object.
I wanted to use anorm's internal classes for the parsing result set rows, but, ever since version 2.4, they have made all of the pertinent classes and methods private to their package, and have deprecated several other methods that would have been more straight-forward to use.
I used a combination of Promises and Futures to work around the ManagedResource that anorm now returns. I avoided all deprecated functions.
import anorm._
import java.sql.ResultSet
import scala.concurrent._
def SqlStream[T](sql:SqlQuery)(parse:ResultSet => T)(implicit ec:ExecutionContext):Future[Stream[T]] = {
val conn = db.getConnection()
val mr = sql.preparedStatement(conn, false)
val p = Promise[Unit]()
val p2 = Promise[ResultSet]()
Future {
mr.map({ stmt =>
p2.success(stmt.executeQuery)
Await.ready(p.future, duration.Duration.Inf)
}).acquireAndGet(identity).andThen { case _ => conn.close() }
}
def _stream(rs:ResultSet):Stream[T] = {
if (rs.next()) parse(rs) #:: _stream(rs)
else {
p.success(())
Stream.empty
}
}
p2.future.map { rs =>
rs.beforeFirst()
_stream(rs)
}
}
A rather trivial usage of this function would be something like this:
def getText(implicit ec:ExecutionContext):Future[Stream[String]] = {
SqlStream(SQL("select FIELD from TABLE")) { rs => rs.getString("FIELD") }
}
There are, of course, drawbacks to this approach, however, this got around my problem and did not require inclusion of any other libraries.