I'm trying to learn Futures and ReactiveMongo.
In my case I have a couple of invite objects and want to filter out the ones that already exist in the db. I do not want to update or upsert the ones already in the db. Therefore I have created a filter method:
filter method:
def isAllowedToReview(invite: Invite): Future[Boolean] = {
ReviewDAO.findById(invite.recoId, invite.invitedUserId).map {
maybeReview => {
maybeReview match {
case Some(review) => false
case None => true
}
}
}
}
DAO:
def findById(rId: Long, userId: Long): Future[Option[Review]] = findOne(Json.obj("rId" -> recoId, "userId" -> userId))
def findOne(query: JsObject)(implicit reader: Reads[T]): Future[Option[T]] = {
collection.find(query).one[T]
}
and then call:
val futureOptionSet: Set[Future[Option[Invite]]] = smsSet.filter(isAllowedToReview)
save the filtered set somehow...
this doesn't work since filter expects in this case Invite => Boolean but I'm sending Invite => Future(Boolean). How would you filter and save this?
smsSet.map(sms => isAllowedToReview(sms).map(b => sms -> b)) will have type Set[Future[(Invite, Boolean)]]. You should be able to call Future.sequence to turn it into a Future[Set[(Invite, Boolean)]]. Then you can collect the results .map(_.collect{ case (sms, true) => sms}).
So putting everything together a solution may look like this:
val futures = smsSet.map(sms => isAllowedToReview(sms).map(b => sms -> b))
val future = Future.sequence(futures)
val result = future.map(_.collect{ case (sms, true) => sms})
When you see map and sequence you may be able to refactor to:
val filteredSet = Future.traverse(smsSet){ sms =>
isAllowedToReview(sms).map(b => sms -> b)
}.map(_.collect{ case (sms, true) => sms})
Note that instead of returning the set, you may just want to save your sms there. But the way I wrote this, all will be wrapped in a Future and you can still compose with other operations.
You could try something like this:
val revsFut = Future.sequence(smsSet.map(invite => ReviewDAO.findById(invite.recoId, invite.invitedUserId)))
val toSave = for(revs <- revsFut) yield {
val flatRevs = revs.flatten
smsSet.filter{ invite =>
flatRevs.find(review => /*Add filter code here */).isDefined
}
}
What I'm doing here is first fetching the Set of reviews matching the the invites by mapping over the smsSet, fetching each individually and then sequencing that into one singe Future. Then, in the for-comprehension I flatten the Set of Option[Review] and then filter down the smsSet based on what's in that flatRevs Set. Since I don't know your object model, I had to leave the impl of the flatRevs.find up to you, but it should be pretty easy as that point.
Related
I want to log in the event that a record doesn't have an adjoining record. Is there a purely functional way to do this? One that separates the side effect from the data transformation?
Here's an example of what I need to do:
val records: Seq[Record] = Seq(record1, record2, ...)
val accountsMap: Map[Long, Account] = Map(record1.id -> account1, ...)
def withAccount(accountsMap: Map[Long, Account])(r: Record): (Record, Option[Account]) = {
(r, accountsMap.get(r.id))
}
def handleNoAccounts(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
tuple
}
def toRichAccount(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
a.map(acct => RichAccount(r, acct))
}
records
.map(withAccount(accountsMap))
.map(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
So there are multiple issues with this approach that I think make it less than optimal.
The tuple return type is clumsy. I have to destructure the tuple in both of the latter two functions.
The logging function has to handle the logging and then return the tuple with no changes. It feels weird that this is passed to .map even though no transformation is taking place -- maybe there is a better way to get this side effect.
Is there a functional way to clean this up?
I could be wrong (I often am) but I think this does everything that's required.
records
.flatMap(r =>
accountsMap.get(r.id).fold{
logger.error(s"no account for ${r.id}")
Option.empty[RichAccount]
}{a => Some(RichAccount(r,a))})
If you're using scala 2.13 or newer you could use tapEach, which takes function A => Unit to apply side effect on every element of function and then passes collection unchanged:
//you no longer need to return tuple in side-effecting function
def handleNoAccounts(tuple: (Record, Option[Account]): Unit = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
}
records
.map(withAccount(accountsMap))
.tapEach(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
In case you're using older Scala, you could provide extension method (updated according to Levi's Ramsey suggestion):
implicit class SeqOps[A](s: Seq[A]) {
def tapEach(f: A => Unit): Seq[A] = {
s.foreach(f)
s
}
}
I have the following url : http://localhost/api/books/?bookId=21&bookId=62?authorId=2
I want to retrieve all the bookId values with Scala and then use Squeryl to do a fetch in a the database.
I'm using the PlayFrameWork as the WebServer, so here's my code :
val params = request.queryString.map { case (k, v) => k -> v(0) } // Retrieve only one the first occurence of a param
So params.get("bookId") will only get the last value in the bookId params. e-g : 62.
To retrieve all my bookId params i tried this :
val params = request.queryString.map { case (k, v) => k -> v } so i can get a Seq[String], but what about the authorId which is not a Seq[String]? .
At the end i want to fetch the bookIds and authorId in my DB using Squeryl :
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map(bookIds: Seq[String] => b.bookId in bookIds))
In my controller i get the params and open the DB connection :
val params = request.queryString.map { case (k, v) => k -> v(0) }
DB.withTransaction() { where(Library.whereHelper(params)}
In my model i use the queries :
def whereHelper(params : Map[String,String]) = {
(a.author_id === params.get("authorId").?) and
(params.get("bookId").map{bookIds: Seq[String] => b.bookId in bookIds})
}
Since bookIds is a list, i need to use the Seq[String]. There's a way to use request.queryString.map { case (k, v) => k -> v } for both a string (authorId) and a list of strings (bookIds) ?
Thanks,
If I really understand what you are trying to do, you want to know how to get the parameters from queryString. This is pretty simple and you can do the following at your controller:
def myAction = Action { request =>
// get all the values from parameter named bookId and
// transforming it to Long. Maybe you don't want the map
// and then you can just remove it.
val bookIds: Seq[Long] = request.queryString("bookId").map(_.toLong)
// Notice that now I'm using getQueryString which is a helper
// method to access a queryString parameter. It returns an
// Option[String] which we are mapping to a Option[Long].
// Again, if you don't need the mapping, just remove it.
val authorId: Option[Long] = request.getQueryString("authorId").map(_.toLong)
DB.withTransaction() { where(Library.whereHelper(authorId, bookIds) }
// Do something with the result
}
At your model you will have:
def whereHelper(authorId: Option[Long], booksId: List[Long]) = authorId match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in bookIds)
case None =>
(b.bookId in bookIds)
}
I've left explicit types to help you understand what is happen. Now, since you have both values, you can just use the values at your query.
Edit after chat:
But, since you want to receive a params: Map[String, Seq[String]] at your models and is just having problems about how to get the authorId, here is what you can do:
def whereHelper(params: Map[String, Seq[String]]) = {
// Here I'm being defensive to the fact that maybe there is no
// "booksIds" key at the map. So, if there is not, an Seq.empty
// will be returned. map method will run only if there is something
// at the Seq.
val booksIds = params.getOrElse("booksIds", Seq.empty).map(_.toLong)
// The same defensive approach is being used here, and also getting
// the head as an Option, so if the Seq is empty, a None will be
// returned. Again, the map will be executed only if the Option
// is a Some, returning another Some with the value as a Long.
val authorId = params.getOrElse("authorId", Seq.empty).headOption
authorId.map(_.toLong) match {
case Some(author_id) =>
(a.author_id === author_id) and
(b.bookId in booksIds)
case None =>
(b.bookId in booksIds)
}
}
Of course, more parameters you have, more complicated this method will be.
I have a collection of type Future[Option[String]] and I map it to a function that returns Future[Option[Profile]], but this create a return type of Future[Option[Future[Option[Profile]]]] because queryProfile return type is `Future[Option[Profile]]'
val users: Future[Option[User]] = someQuery
val email: Future[Option[String]] = users map(opSL => opSL map(_.email) )
val userProfile = email map {opE => opE map {E => queryProfile(E)}}
I need to use the Profile object contained deep inside val userProfile without unpacking all these levels, what would be the right way to use flatMap or `flatten', or is there a better approach all together ?
You can get a "partial Future" with something like this:
val maybeProfile: Future[Profile] = users
.collect { case Some(u) => u.email }
.flatMap { email => queryProfile(email) }
.collect { case Some(p) => p }
Now maybeProfile contains the (completely "naked"/unwrapped) Profile instance, but only if it was able to find it. You can .map it as usual to do something else with it, that'll work in the usual ways.
If you want to ever block and wait for completion, you will have to handle the missing case at some point. For example:
val optionalProfile: Option[Profile] = Await.result(
maybeProfile
.map { p => Some(p) } // Or just skip the last `collect` above
.recover { case _:NoSuchElementException => None },
1 seconds
)
If you are happy with just having Future[Option[Profile]], and would prefer to have the "unwrapping" magic, and handling the missing case localized in one place, you can put the two fragments from above together like this:
val maybeProfile: Future[Option[Profile]] = users
.collect { case Some(u) => u.email }
.flatMap { email => queryProfile(email) }
.recover { case _:NoSuchElementException => None }
Or use Option.fold like the other answer suggested:
val maybeProfile: Future[Option[Profile]] = users
.map { _.map(_.email) }
.flatMap { _.fold[Future[Option[Profile]]](Future.successful(None))(queryProfile) }
Personally, I find the last option less readable though.
Personally I think a monad transformer such as OptionT provided by scalaz/cats would be the cleanest approach:
val users = OptionT[Future,User](someQuery)
def queryProfile(email:String) : OptionT[Future,Profile] = ...
for {
u <- users
p <- queryProfile(u.email)
} yield p
I'd just create a helper method like this:
private def resolveProfile(optEmail: Option[String]): Future[Option[Profile] =
optEmail.fold(Future.successful(None)) { email =>
queryProfile(email).map(Some(_))
}
which then allows you to just flatMap your original email future like so:
val userProfile = email.flatMap(resolveProfile)
Hoping someone can offer an opinion on a solution for this issue I'm having.
I'll try to simplify the issue so save bringing in domain issues, etc.
I have a list of Optional strings. I'm using the collect method to basically filter out strings that don't exist.
names collect {
case Some(value) => value
}
Simple enough. I'm homing to actually go one further. If a value is a None I'd like to call a function and use its response in place of the None. For example
names collect {
case Some(value) => value
case _ => getData(_)
}
The catch is the getData method returns a future. I understand that conventions for futures advise accessing the value within a callback, so something like the map method or on complete, but the issue is that I don't know if I need to call the getData method until I'm in the collect and have the value, so I can't simply wrap all my logic in a map method on getData. It doesn't feel like using Await and blocking is a good idea.
Any idea how I could reasonably handle this would be greatly appreciated. Very new to Scala, so I'd love to hear opinions and options.
EDIT:
I was trying to simplify the problem but I think I've instead missed out on key information.
Below is the actual implementation of my method:
def calculateTracksToExport()(
implicit exportRequest: ExportRequest,
lastExportOption: Option[String]
): Future[List[String]] = {
val vendorIds = getAllFavouritedTracks().flatMap { favTracks =>
Future.sequence {
favTracks.map { track =>
musicClient.getMusicTrackDetailsExternalLinks(
track,
exportRequest.vendor.toString.toLowerCase
).map { details =>
details.data.flatMap { data =>
data.`external-links`.map { link =>
link.map(_.value).collect {
case Some(value) => value
case None => getData(track)
}
}
}.getOrElse(List())
}
}
}.map(_.flatten)
}
vendorIds
}
You can use Future.sequence for collecting values:
def collect(list:List[Option[String]]):Future[List[String]] = Future.sequence(
list.map {
case Some(item) => Future.successful(item)
case _ => getData()
}
)
If something can be in future, you will have to always treat it like future. So have sequence of Futures as return value:
def resolve[T](input: Seq[Option[T]], supplier: => Future[T]): Seq[Future[T]] = {
input.map(option => option.map(Future.successful).getOrElse(supplier))
}
Usage example:
// Input to process
val data = Seq(Some(1), None, Some(2), None, Some(5))
//Imitates long-running background process producing data
var count = 6
def getData: Future[Int] = Future( {
Thread sleep (1000)
count += 1
count
})
resolve(data, getData) // Resolve Nones
.map(Await.result(_, 10.second)).foreach( println ) // Use result
Outputs:
1
8
2
7
5
http://ideone.com/aa8nJ9
I am using play2 and reactivemongo to fetch a result from mongodb. Each item of the result needs to be transformed to add some metadata. Afterwards I need to apply some sorting to it.
To deal with the transformation step I use enumerate():
def ideasEnumerator = collection.find(query)
.options(QueryOpts(skipN = page))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate()
Then I create an Iteratee as follows:
val processIdeas: Iteratee[Idea, Unit] =
Iteratee.foreach[Idea] { idea =>
resolveCrossLinks(idea) flatMap { idea =>
addMetaInfo(idea.copy(history = None))
}
}
Finally I feed the Iteratee:
ideasEnumerator(processIdeas)
And now I'm stuck. Every example I saw does some println inside foreach, but seems not to care about a final result.
So when all documents are returned and transformed how do I get a Sequence, a List or some other datatype I can further deal with?
Change the signature of your Iteratee from Iteratee[Idea, Unit] to Iteratee[Idea, Seq[A]] where A is the type. Basically the first param of Iteratee is Input type and second param is Output type. In your case you gave the Output type as Unit.
Take a look at the below code. It may not compile but it gives you the basic usage.
ideasEnumerator.run(
Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
) // returns Future[List[MyObject]]
As you can see, Iteratee is a simply a state machine. Just extract that Iteratee part and assign it to a val:
val iteratee = Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
and feel free to use it where ever you need to convert from your Idea to List[MyObject]
With the help of your answers I ended up with
val processIdeas: Iteratee[Idea, Future[Vector[Idea]]] =
Iteratee.fold(Future(Vector.empty[Idea])) { (accumulator: Future[Vector[Idea]], next:Idea) =>
resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
} flatMap (ideaWithMeta => accumulator map (acc => acc :+ ideaWithMeta))
}
val ideas = collection.find(query)
.options(QueryOpts(page, perPage))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate(perPage).run(processIdeas)
This later needs a ideas.flatMap(identity) to remove the returning Future of Futures but I'm fine with it and everything looks idiomatic and elegant I think.
The performance gained compared to creating a list and iterate over it afterwards is negligible though.