Compile error on a Future[Option[BasicProfile]] method in play - scala

i'm writing a play 2.3 application using secure social and reactivemongo library, with scala.
Now i'm trying to implement the UserService[T] trait but i'm getting compile errors on the updatePasswordInfo method.
This is the method:
def updatePasswordInfo(user: LoginUser,info: PasswordInfo): scala.concurrent.Future[Option[BasicProfile]] = {
implicit val passwordInfoFormat = Json.format[PasswordInfo]
//the document query
val query = Json.obj("providerId" -> user.providerId,
"userId" -> user.userId
)
//search if the user exists
val futureUser: Future[Option[LoginUser]] = UserServiceLogin.find(query).one
futureUser map {
case Some(x) => val newPassword = Json.obj("passswordInfo" -> info)// the new password
UserServiceLogin.update(query, newPassword) //update the document
val newDocument: Future[Option[LoginUser]] = UserServiceLogin.find(query).one
newDocument map {
case Some(x) => x
case None => None
} //return the new LoginUser
case None => None
}
}
And this is the compiler error:
/Users/alberto/git/recommendation-system/app/security/UserService.scala:203: type mismatch;
[error] found : scala.concurrent.Future[Product with Serializable]
[error] required: Option[securesocial.core.BasicProfile]
[error] newDocument map {
What's wrong?

If you map over a Future[A] you'll end up with a Future[B], where T is the type returned from the lambda you pass to map.
Since that lambda is returning a Future[B] in this case you end up with a Future[Future[B]], which doesn't match the expected type.
The easy fix is to use a flatMap, which takes a lambda going from A to Future[B].
Also, you're returning an Option[LoginUser] but the method is expected to return an Option[BasicProfile]. The compiler inferred a common supertype, which in this case is Product with Serializable, since they're both case classes.
To sum it up
scala.concurrent.Future[Product with Serializable]
^_____________________^^_________________________^
1 2
use flatMap instead of map
return a BasicProfile instead of LoginUser, or change the method return type to Future[Option[LoginUser]]
By the way, there's a lot of room for improvement, as you could use a for-comprehension and the OptionT monad transformer from scalaz to make the whole thing prettier.
Here's an example
import scalaz._; import Scalaz._
val newPassword = Json.obj("passswordInfo" -> info)
(for {
// this is done only for failing fast in case the user doesn't exist
_ <- optionT(UserServiceLogin.find(query).one)
_ <- optionT(Future.successful(Some(UserServiceLogin.update(query, newPassword))))
updatedUser <- optionT(UserServiceLogin.find(query).one)
} yield updatedUser).run
By the way, this works under the assumption that update is a sync call, which might (and I hope) not be the case. If it returns a Future[T] just change the code to
_ <- optionT(UserServiceLogin.update(query, newPassword).map(Some(_)))
or if it already returns a Future[Option[T]]
_ <- optionT(UserServiceLogin.update(query, newPassword))

If you really want to do the find to fail fast (though it is not so useful) and then reload the updated user from the database, something like this should do without the need for using scalaz :
def updatePasswordInfo(user: LoginUser,info: PasswordInfo): scala.concurrent.Future[Option[BasicProfile]] = {
implicit val passwordInfoFormat = Json.format[PasswordInfo]
//the document query
val query = Json.obj("providerId" -> user.providerId,
"userId" -> user.userId)
val newPassword = Json.obj("passswordInfo" -> info)
//update the document
for {
userO <- UserServiceLogin.find(query).one[BasicProfile] //fail fast (not sure this is really useful)
updatedUser<-UserServiceLogin.update(query, newPassword).map(_=>userO).recover{case _ =>None}
actualUser <- UserServiceLogin.find(query).one[BasicProfile]
} yield actualUser
}

There are several ways in which your code can be improved.
For example, you don't need to find the user before firing the query.
Also, it would be good to check if your query actually succeeded (if the API allows it).
Third, I am not sure in which way LoginUser corresponds to the BasicProfile. Your code doesn't seem to do any kind of conversion. If LoginUser is a subclass of BasicProfile, or can be cast to BasicProfile somehow, you can try something like this:
def updatePasswordInfo(user: LoginUser,info: PasswordInfo): scala.concurrent.Future[Option[BasicProfile]] = {
implicit val passwordInfoFormat = Json.format[PasswordInfo]
//the document query
val query = Json.obj("providerId" -> user.providerId,
"userId" -> user.userId
)
UserServiceLogin.update(query, newPassword) //update the document
for {
user <- UserServiceLogin.find(query).one
} yield user.map(_.asInstanceOf[BasicProfile]) //return the new LoginUser
}

Related

How to functionally handle a logging side effect

I want to log in the event that a record doesn't have an adjoining record. Is there a purely functional way to do this? One that separates the side effect from the data transformation?
Here's an example of what I need to do:
val records: Seq[Record] = Seq(record1, record2, ...)
val accountsMap: Map[Long, Account] = Map(record1.id -> account1, ...)
def withAccount(accountsMap: Map[Long, Account])(r: Record): (Record, Option[Account]) = {
(r, accountsMap.get(r.id))
}
def handleNoAccounts(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
tuple
}
def toRichAccount(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
a.map(acct => RichAccount(r, acct))
}
records
.map(withAccount(accountsMap))
.map(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
So there are multiple issues with this approach that I think make it less than optimal.
The tuple return type is clumsy. I have to destructure the tuple in both of the latter two functions.
The logging function has to handle the logging and then return the tuple with no changes. It feels weird that this is passed to .map even though no transformation is taking place -- maybe there is a better way to get this side effect.
Is there a functional way to clean this up?
I could be wrong (I often am) but I think this does everything that's required.
records
.flatMap(r =>
accountsMap.get(r.id).fold{
logger.error(s"no account for ${r.id}")
Option.empty[RichAccount]
}{a => Some(RichAccount(r,a))})
If you're using scala 2.13 or newer you could use tapEach, which takes function A => Unit to apply side effect on every element of function and then passes collection unchanged:
//you no longer need to return tuple in side-effecting function
def handleNoAccounts(tuple: (Record, Option[Account]): Unit = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
}
records
.map(withAccount(accountsMap))
.tapEach(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
In case you're using older Scala, you could provide extension method (updated according to Levi's Ramsey suggestion):
implicit class SeqOps[A](s: Seq[A]) {
def tapEach(f: A => Unit): Seq[A] = {
s.foreach(f)
s
}
}

“required: scala.collection.GenTraversableOnce[?]” error from this for-comprehension

I’m a new scala developer kind of bogged down with a type problem. Sometimes I’m still tripped up by handling futures, and I think this is one of those times. This section of code…
// do some stuff with a collection of List[Future[ClientArticle]]
Future.sequence(listFutureClonedArticles).map( clonedArticles =>
for {
// create persistent records of the cloned client articles, and discard the response
_ <- clonedArticles.map(clonedArticle => clientArticleDAO.create(clonedArticle))
// add cloned articles to a batch of articles, and discard the response
_ <- batchDAO.addArticlesToExistingBatch(destinationBatch._id, clonedArticles.map(_._id))
} yield {
// ultimately just return the cloned articles
clonedArticles
}
)
… is producing this compiler error:
[error] /.../app/services/BatchServiceAPI.scala:442: type mismatch;
[error] found : scala.concurrent.Future[List[model.ClientArticle]]
[error] required: scala.collection.GenTraversableOnce[?]
[error] _ <- batchDAO.addArticlesToExistingBatch(destinationBatch._id, clonedArticles.map(_._id))
[error] ^
The arguments to addArticlesToExistingBatch() appear to be the correct type for the method signature:
/** Adds a list of id's to a batch by it's database ID. */
def addArticlesToExistingBatch(batchId: ID, articleIds: List[ID])(implicit ec: ExecutionContext): Future[Return]
Of course, I might be misunderstanding how a for comprehension works too. I don’t understand how an error can occur at the <- operator, nor how/why there would be type expectations at that point.
Can anyone help me understand what needs to be done here?
=== 21 minutes later... ===
Interesting. When I stop using the for comprehension and break these out into two separate maps, it compiles.
// create cloned client articles
Future.sequence(listFutureClonedArticles).map(clonedArticles =>
clonedArticles.map(clonedArticle => clientArticleDAO.create(clonedArticle)))
// add cloned articles to destination batch
Future.sequence(listFutureClonedArticles).map(clonedArticles =>
batchDAO.addArticlesToExistingBatch(destinationBatch._id, clonedArticles.map(_._id)))
So yeah, I guess I still don't quite understand for-comprehensions. I thought they could be used to roll up several Future operations. Why doesn't that work in this scenario
for comprehension is a combination of flatMap and map. Every line with <- is converted into a flatMap but the last line which is converted to a map.
So, you code
for {
// create persistent records of the cloned client articles, and discard the response
_ <- clonedArticles.map(clonedArticle => clientArticleDAO.create(clonedArticle))
// add cloned articles to a batch of articles, and discard the response
_ <- batchDAO.addArticlesToExistingBatch(destinationBatch._id, clonedArticles.map(_._id))
} yield {
// ultimately just return the cloned articles
clonedArticles
}
is converted to
clonedArticles.map(clonedArticle => clientArticleDAO.create(clonedArticle)).flatMap { _ =>
batchDAO.addArticlesToExistingBatch(destinationBatch._id, clonedArticles.map(_._id)).map { _ =>
clonedArticles
}
}
as clonedArticles is a List and the signature for flatMap for the list is
final override def flatMap[B, That](f: A => GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That = ???
If you look at the parameter required by flatMap it needs a function A => GenTraversableOnce but in your function you are passing a function A => Future that is the problem.
I have tried to imitate your problem with simple functions you can try that:
import scala.concurrent._
import scala.concurrent.ExecutionContext.Implicits.global
val listOfFuture: List[Future[Int]] = (1 to 10).map(Future(_)).toList
def f(i: List[Int]): Future[String] = Future(s"very complex logic: ${i.sum}")
def create(i: Int): Future[Unit] = Future(println(s"creating something complex: $i"))
Future.traverse(listOfFuture){ futureX =>
for {
x <- futureX
_ <- create(x)
} yield x
}.flatMap(f)

Scala's for comprehension for Futures and Options

I have recently read Manuel Bernhardt's new book Reactive Web Applications. In his book, he states that Scala developers should never use .get to retrieve an optional value.
I want to pick up his suggestions but I am struggling to avoid .get when using for comprehensions for Futures.
Let's say I have the following code:
for {
avatarUrl <- avatarService.retrieve(email)
user <- accountService.save(Account(profiles = List(profile.copy(avatarUrl = avatarUrl)))
userId <- user.id
_ <- accountTokenService.save(AccountToken.create(userId, email))
} yield {
Logger.info("Foo bar")
}
Normally, I would have used AccountToken.create(user.id.get, email) instead of AccountToken.create(userId, email). However, when trying to avoid this bad practice, I get the following exception:
[error] found : Option[Nothing]
[error] required: scala.concurrent.Future[?]
[error] userId <- user.id
[error] ^
How can I solve this?
First option
If you really want to use for comprehension you'll have to separate it to several fors, where each works with the same monad type:
for {
avatarUrl <- avatarService.retrieve(email)
user <- accountService.save(Account(profiles = List(profile.copy(avatarUrl = avatarUrl)))
} yield for {
userId <- user.id
} yield for {
_ <- accountTokenService.save(AccountToken.create(userId, email))
}
Second option
Another option is to avoid Future[Option[T]] altogether and use Future[T] which can materialize into Failure(e) where e is a NoSuchElementException whenever you expect a None (in your case, the accountService.save() method):
def saveWithoutOption(account: Account): Future[User] = {
this.save(account) map { userOpt =>
userOpt.getOrElse(throw new NoSuchElementException)
}
}
Then you'll have:
(for {
avatarUrl <- avatarService.retrieve(email)
user <- accountService.saveWithoutOption(Account(profiles = List(profile.copy(avatarUrl = avatarUrl)))
_ <- accountTokenService.save(AccountToken.create(user.id, email))
} yield {
Logger.info("Foo bar")
}) recover {
case t: NoSuchElementException => Logger.error("boo")
}
Third option
Fall back to map/flatMap and introduce intermediate results.
Let's take a step back and explore the meaning of our expression:
A Future is "eventually a value (but might fail)"
An Option is "maybe a value"
What are the semantics of Future[Option]? Let's explore the values to gain some intuition:
Future[Option]
Success(Some(x)) => Good. Let's do stuff with x
Success(None) => Finished but got nothing => This is probably an application-level error
Failure(_) => Something went wrong, so we don't have a value
We can flatten Success(None) into a Failure(SomeApplicationException) and eliminate the need of handling the Option separately.
For that, we can define a generic function to turn an Option into a Future and use the for-comprehension to apply the flattening.
def optionToFuture[T](opt:Option[T], ex: ()=>Exception):Future[T] = opt match {
case Some(v) => Future.successful(v)
case None => Future.failed(ex())
}
We can now express our computation fluently with a for-comprehension:
for {
avatarUrl <- avatarService.retrieve(email)
user <- accountService.save(Account(profiles = List(profile.copy(avatarUrl = avatarUrl)))
userId <- optionToFuture(user.id, () => new UserNotFoundException(email))
_ <- accountTokenService.save(AccountToken.create(userId, email))
} yield {
Logger.info("Foo bar")
}
Stop Option propogation by failing the Future when option is None
Fail the future when id is none and abort
for {
....
accountOpt <-
user.id.map { id =>
Account.create(id, ...)
}.getOrElse {
Future.failed(new Exception("could not create account."))
}
...
} yield result
Better to have a custom exception like
case class NoIdException(msg: String) extends Exception(msg)
invoking .get on Option should be done only if you are sure that option is Some(x) otherwise .get will throw an exception.
Thats by using .get is not good practise because it may cause an exception in the code.
Instead of .get its good practice to use getOrElse.
You can map or flatMap the option to get access to the inner value.
Good practice
val x: Option[Int] = giveMeOption()
x.getOrElse(defaultValue)
Get can be used here
val x: Option[Int] = giveMeOption()
x.OrElse(Some(1)).get

Scala/Play/Squeryl Retrieve multiple params

I have the following url : http://localhost/api/books/?bookId=21&bookId=62?authorId=2
I want to retrieve all the bookId values with Scala and then use Squeryl to do a fetch in a the database.
I'm using the PlayFrameWork as the WebServer, so here's my code :
val params = request.queryString.map { case (k, v) => k -> v(0) } // Retrieve only one the first occurence of a param
So params.get("bookId") will only get the last value in the bookId params. e-g : 62.
To retrieve all my bookId params i tried this :
val params = request.queryString.map { case (k, v) => k -> v } so i can get a Seq[String], but what about the authorId which is not a Seq[String]? .
At the end i want to fetch the bookIds and authorId in my DB using Squeryl :
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map(bookIds: Seq[String] => b.bookId in bookIds))
In my controller i get the params and open the DB connection :
val params = request.queryString.map { case (k, v) => k -> v(0) }
DB.withTransaction() { where(Library.whereHelper(params)}
In my model i use the queries :
def whereHelper(params : Map[String,String]) = {
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map{bookIds: Seq[String] => b.bookId in bookIds})
}
Since bookIds is a list, i need to use the Seq[String]. There's a way to use request.queryString.map { case (k, v) => k -> v } for both a string (authorId) and a list of strings (bookIds) ?
Thanks,
If I really understand what you are trying to do, you want to know how to get the parameters from queryString. This is pretty simple and you can do the following at your controller:
def myAction = Action { request =>
// get all the values from parameter named bookId and
// transforming it to Long. Maybe you don't want the map
// and then you can just remove it.
val bookIds: Seq[Long] = request.queryString("bookId").map(_.toLong)
// Notice that now I'm using getQueryString which is a helper
// method to access a queryString parameter. It returns an
// Option[String] which we are mapping to a Option[Long].
// Again, if you don't need the mapping, just remove it.
val authorId: Option[Long] = request.getQueryString("authorId").map(_.toLong)
DB.withTransaction() { where(Library.whereHelper(authorId, bookIds) }
// Do something with the result
}
At your model you will have:
def whereHelper(authorId: Option[Long], booksId: List[Long]) = authorId match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in bookIds)
case None =>
(b.bookId in bookIds)
}
I've left explicit types to help you understand what is happen. Now, since you have both values, you can just use the values at your query.
Edit after chat:
But, since you want to receive a params: Map[String, Seq[String]] at your models and is just having problems about how to get the authorId, here is what you can do:
def whereHelper(params: Map[String, Seq[String]]) = {
// Here I'm being defensive to the fact that maybe there is no
// "booksIds" key at the map. So, if there is not, an Seq.empty
// will be returned. map method will run only if there is something
// at the Seq.
val booksIds = params.getOrElse("booksIds", Seq.empty).map(_.toLong)
// The same defensive approach is being used here, and also getting
// the head as an Option, so if the Seq is empty, a None will be
// returned. Again, the map will be executed only if the Option
// is a Some, returning another Some with the value as a Long.
val authorId = params.getOrElse("authorId", Seq.empty).headOption
authorId.map(_.toLong) match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in booksIds)
case None =>
(b.bookId in booksIds)
}
}
Of course, more parameters you have, more complicated this method will be.

Wait for a list of futures with composing Option in Scala

I have to get a list of issues for each file of a given list from a REST API with Scala. I want to do the requests in parallel, and use the Dispatch library for this. My method is called from a Java framework and I have to wait at the end of this method for the result of all the futures to yield the overall result back to the framework. Here's my code:
def fetchResourceAsJson(filePath: String): dispatch.Future[json4s.JValue]
def extractLookupId(json: org.json4s.JValue): Option[String]
def findLookupId(filePath: String): Future[Option[String]] =
for (json <- fetchResourceAsJson(filePath))
yield extractLookupId(json)
def searchIssuesJson(lookupId: String): Future[json4s.JValue]
def extractIssues(json: org.json4s.JValue): Seq[Issue]
def findIssues(lookupId: String): Future[Seq[Issue]] =
for (json <- searchIssuesJson(componentId))
yield extractIssues(json)
def getFilePathsToProcess: List[String]
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val finalResultPromise = Promise[Map[String, Seq[Issue]]]()
// (1) inferred type of issuesByFile not as expected, cannot get
// the type system happy, would like to have Seq[Future[(String, Seq[Issue])]]
val issuesByFile = getFilePathsToProcess map { f =>
findLookupId(f).flatMap { lookupId =>
(f, findIssues(lookupId)) // I want to yield a tuple (String, Seq[Issue]) here
}
}
Future.sequence(issuesByFile) onComplete {
case Success(x) => finalResultPromise.success(x) // (2) how to return x here?
case Failure(x) => // (3) how to return null from here?
}
//TODO transform finalResultPromise to Java Map
}
This code snippet has several issues. First, I'm not getting the type I would expect for issuesByFile (1). I would like to just ignore the result of findLookUpId if it is not able to find the lookUp ID (i.e., None). I've read in various tutorials that Future[Option[X]] is not easy to handle in function compositions and for expressions in Scala. So I'm also curious what the best practices are to handle these properly.
Second, I somehow have to wait for all futures to finish, but don't know how to return the result to the calling Java framework (2). Can I use a promise here to achieve this? If yes, how can I do it?
And last but not least, in case of any errors, I would just like to return null from thisIsCalledByJavaFramework but don't know how (3).
Any help is much appreciated.
Thanks,
Michael
Several points:
The first problem at (1) is that you don't handle the case where findLookupId returns None. You need to decide what to do in this case. Fail the whole process? Exclude that file from the list?
The second problem at (1) is that findIssues will itself return a Future, which you need to map before you can build the result tuple
There's a shortcut for map and then Future.sequence: Future.traverse
If you cannot change the result type of the method because the Java interface is fixed and cannot be changed to support Futures itself you must wait for the Future to be completed. Use Await.ready or Await.result to do that.
Taking all that into account and choosing to ignore files for which no id could be found results in this code:
// `None` in an entry for a file means that no id could be found
def entryForFile(file: String): Future[(String, Option[Seq[Issue]])] =
findLookupId(file).flatMap {
// the need for this kind of pattern match shows
// the difficulty of working with `Future[Option[T]]`
case Some(id) ⇒ findIssues(id).map(issues ⇒ file -> Some(issues))
case None ⇒ Future.successful(file -> None)
}
def thisIsCalledByJavaFramework(): java.util.Map[String, java.util.List[Issue]] = {
val issuesByFile: Future[Seq[(String, Option[Seq[Issue]])]] =
Future.traverse(getFilePathsToProcess)(entryForFile)
import scala.collection.JavaConverters._
try
Await.result(issuesByFile, 10.seconds)
.collect {
// here we choose to ignore entries where no id could be found
case (f, Some(issues)) ⇒ f -> issues
}
.toMap.mapValues(_.asJava).asJava
catch {
case NonFatal(_) ⇒ null
}
}