Using a Future's response - scala

Hoping someone can offer an opinion on a solution for this issue I'm having.
I'll try to simplify the issue so save bringing in domain issues, etc.
I have a list of Optional strings. I'm using the collect method to basically filter out strings that don't exist.
names collect {
case Some(value) => value
}
Simple enough. I'm homing to actually go one further. If a value is a None I'd like to call a function and use its response in place of the None. For example
names collect {
case Some(value) => value
case _ => getData(_)
}
The catch is the getData method returns a future. I understand that conventions for futures advise accessing the value within a callback, so something like the map method or on complete, but the issue is that I don't know if I need to call the getData method until I'm in the collect and have the value, so I can't simply wrap all my logic in a map method on getData. It doesn't feel like using Await and blocking is a good idea.
Any idea how I could reasonably handle this would be greatly appreciated. Very new to Scala, so I'd love to hear opinions and options.
EDIT:
I was trying to simplify the problem but I think I've instead missed out on key information.
Below is the actual implementation of my method:
def calculateTracksToExport()(
implicit exportRequest: ExportRequest,
lastExportOption: Option[String]
): Future[List[String]] = {
val vendorIds = getAllFavouritedTracks().flatMap { favTracks =>
Future.sequence {
favTracks.map { track =>
musicClient.getMusicTrackDetailsExternalLinks(
track,
exportRequest.vendor.toString.toLowerCase
).map { details =>
details.data.flatMap { data =>
data.`external-links`.map { link =>
link.map(_.value).collect {
case Some(value) => value
case None => getData(track)
}
}
}.getOrElse(List())
}
}
}.map(_.flatten)
}
vendorIds
}

You can use Future.sequence for collecting values:
def collect(list:List[Option[String]]):Future[List[String]] = Future.sequence(
list.map {
case Some(item) => Future.successful(item)
case _ => getData()
}
)

If something can be in future, you will have to always treat it like future. So have sequence of Futures as return value:
def resolve[T](input: Seq[Option[T]], supplier: => Future[T]): Seq[Future[T]] = {
input.map(option => option.map(Future.successful).getOrElse(supplier))
}
Usage example:
// Input to process
val data = Seq(Some(1), None, Some(2), None, Some(5))
//Imitates long-running background process producing data
var count = 6
def getData: Future[Int] = Future( {
Thread sleep (1000)
count += 1
count
})
resolve(data, getData) // Resolve Nones
.map(Await.result(_, 10.second)).foreach( println ) // Use result
Outputs:
1
8
2
7
5
http://ideone.com/aa8nJ9

Related

How to Check If Some Futures in a Collection Have Failed

I used traverse to execute a collection of futures like this:
val result: Future[List[Either[Error, Int]]] = Future.traverse(urls)(foo(_))
I end up with a Future[List[Either[Error, Int]]]. How can I check that one of these futures resulted in an Error?
I tried to do this but I think it is wrong because I am reading that you cannot substitute variables for futures?
val check: Future[Boolean] = result.map{
fut => fut.exists(c => c.isLeft)
}
check.map{
b => b match {
case true => // do something
case false => // do something
}
}
You can convert the result to a list of errors like this:
val errors: Future[List[Error]] = result.map(_.collect{ case Left(err) => err })
It is then possible to use Await.result to extract these error values, but that is nearly always a bad idea because it blocks the current thread.
It is better to ask "What do I want to do once the Future is complete but returns errors?". Then implement that behaviour in a map or foreach on the errors Future.

Is it possible to call a generically-typed function by passing in a class specified in a map?

Asking here because I'm pulling out my hair trying to figure out what exactly it is I need to do here.
I'm writing a batch-processing endpoint that attempts to convert the body of a request to a specific Scala case class before executing the logic within the endpoint itself.
This is as far as I currently got. First, I have a function executeWithType that takes a generic type, a controller method, and the request body and executes the controller method after converting the request body to the provided type. The request param is available in a scope outside this function.
def executeWithType[A](action: () => Action[A], batchRequest: BatchRequest): Future[Result] = {
action()(request.map(_ => batchRequest.body.map(_.as[A]).get))
}
Then, I have some code that checks what endpoint to call and what type to cast to depending on what's in the BatchRequest itself.
val res: Future[Result] = (batchRequest.method, batchRequest.endpoint) match {
case ("POST", "/endpoint1") => {
executeWithType[EndpointOneType](controller.endpointOne _, batchRequest)
}
case ("POST", "/endpoint2") => {
executeWithType[EndpointTwoType](controller.endpointTwo _, batchRequest)
}
case _ => Future.successful(NotFound)
}
The above works perfectly fine - however, I want to avoid this sort of tuple-matching with individual cases if possible, and specify a Map that does this instead. In my ideal world, the end result of the code block immediately above would look like this:
val actions = Map(
Seq("POST", "/endpoint1") -> (controller.endpointOne _, EndpointOneType),
Seq("POST", "/endpoint2") -> (controller.endpointTwo _, EndpointTwoType)
)
val res = actions.get(Seq(batchRequest.method, batchRequest.endpoint)) match {
case Some(action) => {
executeWithType[action._2](action._1, batchRequest)
}
case _ => Future.successful(NotFound)
}
Is this possible at all? I've been trying to fight with it but my understanding of reflection in Scala is really weak, so I'm not sure exactly how I'd go about doing this. I've tried a bunch of classOf and typeTag and Class[_] stuff but I'm basically swinging in the dark. Hoping someone more knowledgeable than I am could help me out.
The big things are:
What needs to go in the second space of the tuple in the value of the Map? How do you pass a Class variable?
How do we use that class-as-a-variable to call a generically typed method?
How do we use that class-as-a-variable to call a generically typed method?
You can't. But I'd like to suggest an alternate solution.
Just define a local class instead of tuples:
class RequestAction[A](action: () => Action[A]) {
def apply(request: BatchRequest) = executeWithType(action, request)
}
val actions = Map(
Seq("POST", "/endpoint1") -> new RequestAction(controller.endpointOne _), // type parameter is inferred
Seq("POST", "/endpoint2") -> new RequestAction(controller.endpointTwo _)
)
val res = actions.get(Seq(batchRequest.method, batchRequest.endpoint)) match {
case Some(action) => action(batchRequest)
case _ => Future.successful(NotFound)
}
(while this depends on code not shown in the question, it looks likely that you can simplify by passing Action[A] instead of () => Action[A]).

Using Futures in Scala?

I am trying to incorporate a database into my http-microservice.
The microservice has a function getValueFromInternet(val: Foo): Future[Value] which was being called by my microservice on a GET request. Now, I want it to happen such that, a function getValue(val: Foo): Future[Value] would first query a db and if the database returns no results, call getValueFromInternet. The database query returns a Future[Seq[Value2]] where I can convert Value2 to Value using a function. And if no entry is found corresponding to that value, an empty Vector is returned.
This is what I have tried so far:
def getValue(val: Foo): Future[Value] = {
val resultFuture = db.getValue(val)
// 1st attempt. Clearly wrong
resultFuture onComplete {
case Success(Vector()) => getValueFromInternet(val)
case Success(vec) => convertValue2to1(vec.head)
}
// 2nd attempt. This is also wrong
resultFuture match {
case Future(Success(Vector())) => getValueFromInternet(val)
case Future(Success(vec)) => convertValue2to1(vec.head)
}
}
I would be grateful for any help suggesting how I can do this.
I have implemented the database and microservice independently and you can find them here and here
You have to use flatMap, since the thing you want to do if the first operation does not return a result also returns a future.
This is as close to your code as possible while still compiling. Note that you can't have identifiers called val in scala, since that is a keyword.
def getValue(v: Foo)(implicit ec: ExecutionContext): Future[Value] = {
val resultFuture: Future[Seq[Value2]] = db.getValue(v)
resultFuture.flatMap { vec =>
if(vec.isEmpty)
getValueFromInternet(v)
else
Future.successful(convertValue2to1(vec.head))
}
}

How to get a result from Enumerator/Iteratee?

I am using play2 and reactivemongo to fetch a result from mongodb. Each item of the result needs to be transformed to add some metadata. Afterwards I need to apply some sorting to it.
To deal with the transformation step I use enumerate():
def ideasEnumerator = collection.find(query)
.options(QueryOpts(skipN = page))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate()
Then I create an Iteratee as follows:
val processIdeas: Iteratee[Idea, Unit] =
Iteratee.foreach[Idea] { idea =>
resolveCrossLinks(idea) flatMap { idea =>
addMetaInfo(idea.copy(history = None))
}
}
Finally I feed the Iteratee:
ideasEnumerator(processIdeas)
And now I'm stuck. Every example I saw does some println inside foreach, but seems not to care about a final result.
So when all documents are returned and transformed how do I get a Sequence, a List or some other datatype I can further deal with?
Change the signature of your Iteratee from Iteratee[Idea, Unit] to Iteratee[Idea, Seq[A]] where A is the type. Basically the first param of Iteratee is Input type and second param is Output type. In your case you gave the Output type as Unit.
Take a look at the below code. It may not compile but it gives you the basic usage.
ideasEnumerator.run(
Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
) // returns Future[List[MyObject]]
As you can see, Iteratee is a simply a state machine. Just extract that Iteratee part and assign it to a val:
val iteratee = Iteratee.fold(List.empty[MyObject]) { (accumulator, next) =>
accumulator + resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
}
}
and feel free to use it where ever you need to convert from your Idea to List[MyObject]
With the help of your answers I ended up with
val processIdeas: Iteratee[Idea, Future[Vector[Idea]]] =
Iteratee.fold(Future(Vector.empty[Idea])) { (accumulator: Future[Vector[Idea]], next:Idea) =>
resolveCrossLinks(next) flatMap { next =>
addMetaInfo(next.copy(history = None))
} flatMap (ideaWithMeta => accumulator map (acc => acc :+ ideaWithMeta))
}
val ideas = collection.find(query)
.options(QueryOpts(page, perPage))
.sort(Json.obj(sortField -> -1))
.cursor[Idea]
.enumerate(perPage).run(processIdeas)
This later needs a ideas.flatMap(identity) to remove the returning Future of Futures but I'm fine with it and everything looks idiomatic and elegant I think.
The performance gained compared to creating a list and iterate over it afterwards is negligible though.

How should I handle Filter and Futures in play2 and Scala

I'm trying to learn Futures and ReactiveMongo.
In my case I have a couple of invite objects and want to filter out the ones that already exist in the db. I do not want to update or upsert the ones already in the db. Therefore I have created a filter method:
filter method:
def isAllowedToReview(invite: Invite): Future[Boolean] = {
ReviewDAO.findById(invite.recoId, invite.invitedUserId).map {
maybeReview => {
maybeReview match {
case Some(review) => false
case None => true
}
}
}
}
DAO:
def findById(rId: Long, userId: Long): Future[Option[Review]] = findOne(Json.obj("rId" -> recoId, "userId" -> userId))
def findOne(query: JsObject)(implicit reader: Reads[T]): Future[Option[T]] = {
collection.find(query).one[T]
}
and then call:
val futureOptionSet: Set[Future[Option[Invite]]] = smsSet.filter(isAllowedToReview)
save the filtered set somehow...
this doesn't work since filter expects in this case Invite => Boolean but I'm sending Invite => Future(Boolean). How would you filter and save this?
smsSet.map(sms => isAllowedToReview(sms).map(b => sms -> b)) will have type Set[Future[(Invite, Boolean)]]. You should be able to call Future.sequence to turn it into a Future[Set[(Invite, Boolean)]]. Then you can collect the results .map(_.collect{ case (sms, true) => sms}).
So putting everything together a solution may look like this:
val futures = smsSet.map(sms => isAllowedToReview(sms).map(b => sms -> b))
val future = Future.sequence(futures)
val result = future.map(_.collect{ case (sms, true) => sms})
When you see map and sequence you may be able to refactor to:
val filteredSet = Future.traverse(smsSet){ sms =>
isAllowedToReview(sms).map(b => sms -> b)
}.map(_.collect{ case (sms, true) => sms})
Note that instead of returning the set, you may just want to save your sms there. But the way I wrote this, all will be wrapped in a Future and you can still compose with other operations.
You could try something like this:
val revsFut = Future.sequence(smsSet.map(invite => ReviewDAO.findById(invite.recoId, invite.invitedUserId)))
val toSave = for(revs <- revsFut) yield {
val flatRevs = revs.flatten
smsSet.filter{ invite =>
flatRevs.find(review => /*Add filter code here */).isDefined
}
}
What I'm doing here is first fetching the Set of reviews matching the the invites by mapping over the smsSet, fetching each individually and then sequencing that into one singe Future. Then, in the for-comprehension I flatten the Set of Option[Review] and then filter down the smsSet based on what's in that flatRevs Set. Since I don't know your object model, I had to leave the impl of the flatRevs.find up to you, but it should be pretty easy as that point.