I am refactoring a scala http4s application to remove some pesky side effects causing my app to block. I'm replacing .unsafeRunSync with cats.effect.IO. The problem is as follows:
I have 2 lists: alreadyAccessible: IO[List[Page]] and pages: List[Page]
I need to filter out the pages that are not contained in alreadyAccessible.
Then map over the resulting list to "grant Access" in the database to these pages. (e.g. call another method that hits the database and returns an IO[Page].
val addable: List[Page] = pages.filter(p => !alreadyAccessible.contains(p))
val added: List[Page] = addable.map((p: Page) => {
pageModel.grantAccess(roleLst.head.id, p.id) match {
case Right(p) => p
}
})
This is close to what I want; However, it does not work because filter requires a function that returns a Boolean but alreadyAccessible is of type IO[List[Page]] which precludes you from removing anything from the IO monad. I understand you can't remove data from the IO so maybe transform it:
val added: List[IO[Page]] = for(page <- pages) {
val granted = alreadyAccessible.flatMap((aa: List[Page]) => {
if (!aa.contains(page))
pageModel.grantAccess(roleLst.head.id, page.id) match { case Right(p) => p }
else null
})
} yield granted
this unfortunately does not work with the following error:
Error:(62, 7) ';' expected but 'yield' found.
} yield granted
I think because I am somehow mistreating the for comprehension syntax, I just don't understand why I cannot do what I'm doing.
I know there must be a straight forward solution to such a problem, so any input or advice is greatly appreciates. Thank you for your time in reading this!
granted is going to be an IO[List[Page]]. There's no particular point in having IO inside anything else unless you truly are going to treat the actions like values and reorder them/filter them etc.
val granted: IO[List[Page]] = for {
How do you compute it? Well, the first step is to execute alreadyAccessible to get the actual list. In fact, alreadyAccessible is misnamed. It is not the list of accessible pages; it is an action that gets the list of accessible pages. I would recommend you rename it getAlreadyAccessible.
alreadyAccessible <- getAlreadyAccessible
Then you filter pages with it
val required = pages.filterNot(alreadyAccessible.contains)
Now, I cannot decipher what you're doing to these pages. I'm just going to assume you have some kind of function grantAccess: Page => IO[Page]. If you map this function over required, you will get a List[IO[Page]], which is not desirable. Instead, we should traverse with grantAccess, which will produce a IO[List[Page]] that executes each IO[Page] and then assembles all the results into a List[Page].
granted <- required.traverse(grantAccess)
And we're done
} yield granted
Related
I am trying to loop over inputs and process them to produce scores.
Just for the first input, I want to do some processing that takes a while.
The function ends up returning just the values from the 'else' part. The 'if' part is done executing after the function returns the value.
I am new to Scala and understand the behavior but not sure how to fix it.
I've tried inputs.zipWithIndex.map instead of foreach but the result is the same.
def getscores(
inputs: inputs
): Future[Seq[scoreInfo]] = {
var scores: Seq[scoreInfo] = Seq()
inputs.zipWithIndex.foreach {
case (f, i) => {
if (i == 0) {
// long operation that returns Future[Option[scoreInfo]]
getgeoscore(f).foreach(gso => {
gso.foreach(score => {
scores = scores.:+(score)
})
})
} else {
scores = scores.:+(
scoreInfo(
id = "",
score = 5
)
)
}
}
}
Future {
scores
}
}
For what you need, I would drop the mutable variable and replace foreach with map to obtain an immutable list of Futures and recover to handle exceptions, followed by a sequence like below:
def getScores(inputs: Inputs): Future[List[ScoreInfo]] = Future.sequence(
inputs.zipWithIndex.map{ case (input, idx) =>
if (idx == 0)
getGeoScore(input).map(_.getOrElse(defaultScore)).recover{ case e => errorHandling(e) }
else
Future.successful(ScoreInfo("", 5))
})
To capture/print the result, one way is to use onComplete:
getScores(inputs).onComplete(println)
The part your missing is understanding a tricky element of concurrency, and that is that the order of execution when using multiple futures is not guaranteed.
If your block here is long running, it will take a while before appending the score to scores
// long operation that returns Future[Option[scoreInfo]]
getgeoscore(f).foreach(gso => {
gso.foreach(score => {
// stick a println("here") in here to see what happens, for demonstration purposes only
scores = scores.:+(score)
})
})
Since that executes concurrently, your getscores function will also simultaneously continue its work iterating over the rest of inputs in your zipWithindex. This iteration, especially since it's trivial work, likely finishes well before the long-running getgeoscore(f) completes the execution of the Future it scheduled, and the code will exit the function, moving on to whatever code is next after you called getscores
val futureScores: Future[Seq[scoreInfo]] = getScores(inputs)
futureScores.onComplete{
case Success(scoreInfoSeq) => println(s"Here's the scores: ${scoreInfoSeq.mkString(",")}"
}
//a this point the call to getgeoscore(f) could still be running and finish later, but you will never know
doSomeOtherWork()
Now to clean this up, since you can run a zipWithIndex on your inputs parameter, I assume you mean it's something like a inputs:Seq[Input]. If all you want to do is operate on the first input, then use the head function to only retrieve the first option, so getgeoscores(inputs.head) , you don't need the rest of the code you have there.
Also, as a note, if using Scala, get out of the habit of using mutable vars, especially if you're working with concurrency. Scala is built around supporting immutability, so if you find yourself wanting to use a var , try using a val and look up how to work with the Scala's collection library to make it work.
In general, that is when you have several concurrent futures, I would say Leo's answer describes the right way to do it. However, you want only the first element transformed by a long running operation. So you can use the future return by the respective function and append the other elements when the long running call returns by mapping the future result:
def getscores(inputs: Inputs): Future[Seq[ScoreInfo]] =
getgeoscore(inputs.head)
.map { optInfo =>
optInfo ++ inputs.tail.map(_ => scoreInfo(id = "", score = 5))
}
So you neither need zipWithIndex nor do you need an additional future or join the results of several futures with sequence. Mapping the future just gives you a new future with the result transformed by the function passed to .map().
How do I replace my first conditional with the require function in the context of a Future? Should I wrap the entire inRange method in a Future, and if I do that, how do I handle the last Future so that it doesn't return a Future[Future[List[UserId]], or is there a better way?
I have a block of code that looks something like this:
class RetrieveHomeownersDefault(depA: DependencyA, depB: DependencyB) extends RetrieveHomeowners {
def inRange(range: GpsRange): Future[List[UserId]] = {
// I would like to replace this conditional with `require(count >= 0, "The offset…`
if (count < 0) {
Future.failed(new IllegalArgumentException("The offset must be a positive integer.")
} else {
val retrieveUsers: Future[List[UserId]] = depA.inRange(range)
for (
userIds <- retrieveUsers
homes <- depB.homesForUsers(userIds)
) yield FilterUsers.withoutHomes(userIds, homes)
}
}
}
I started using the require function in other areas of my code, but when I tried to use it in the context of Futures I ran into some hiccups.
class RetrieveHomeownersDefault(depA: DependencyA, depB: DependencyB) extends RetrieveHomeowners {
// Wrapped the entire method with Future, but is that the correct approach?
def inRange(range: GpsRange): Future[List[UserId]] = Future {
require(count >= 0, "The offset must be a positive integer.")
val retrieveUsers: Future[List[UserId]] = depA.inRange(range)
// Now I get Future[Future[List[UserId]]] error in the compiler.
for (
userIds <- retrieveUsers
homes <- depB.homesForUsers(userIds)
) yield FilterUsers.withoutHomes(userIds, homes)
}
}
Any tips, feedback, or suggestions would be greatly appreciated. I'm just getting started with Futures and still having a tough time wrapping my head around many concepts.
Thanks a bunch!
Just remove the outer Future {...} wrapper. It's not necessary. There's no good reason for the require call to go inside the Future. It's actually better outside since then it will report immediately (in the same thread) to the caller that the argument is invalid.
By the way, the original code is wrong too. The Future.failed(...) is created but not returned. So essentially it didn't do anything.
Scala is pretty new for me and I have problems as soon as a leave the gatling dsl.
In my case I call an API (Mailhog) which responds with a lot of mails in json-format. I can’t grab all the values.
I need it with “jsonPath” and I need to “regex” as well.
That leads into a map and a list which I need to iterate through and save each value.
.check(jsonPath("$[*]").ofType[Map[String,Any]].findAll.saveAs("id_map"))
.check(regex("href=3D\\\\\"(.*?)\\\\\"").findAll.saveAs("url_list"))
At first I wanted to loop the “checks” but I did’nt find any to repeat them without repeating the “get”-request too. So it’s a map and a list.
1) I need every value of the map and was able to solve the problem with the following foreach loop.
.foreach("${id_map}", "idx") {
exec(session => {
val idMap = session("idx").as[Map[String,Any]]
val ID = idMap("ID")
session.set("ID", ID)
})
.exec(http("Test")
.get("/{ID}"))
})}
2) I need every 3rd value of the list and make a get-request on them. Before I can do this, I need to replace a part of the string. I tried to replace parts of the string while checking for them. But it won’t work with findAll.
.check(regex("href=3D\\\\\"(.*?)\\\\\"").findAll.transform(raw => raw.replace("""=\r\n""","")).saveAs("url"))
How can I replace a part of every string in my list?
also how can I make a get-request on every 3rd element in the list.
I can't get it to work with the same foreach structure above.
I was abole to solve the problem by myself. At first I made a little change to my check(regex ...) part.
.check(regex("href=3D\\\\\"(.*?)\\\\\"").findAll.transform(_.map(raw => raw.replace("""=\r\n""",""))).saveAs("url_list"))
Then I wanted to make a Get-Request only on every third element of my list (because the URLs I extracted appeared three times per Mail).
.exec(session => {
val url_list =
session("url_list").as[List[Any]].grouped(3).map(_.head).toList
session.set("url_list", url_list)
})
At the end I iterate through my final list with a foreach-loop.
foreach("${url_list}", "urls") {
exec(http("Activate User")
.get("${urls}")
)
}
I'm refactoring some scala code to teach my coworkers about for-comprehensions, and I've got a line like:
for {
// ...
result <- components.collectFirst({ case section if section.startsWith(DESIRED_SUBSTRING) => section.substring(section.indexOf(DELIM) + 1).trim() == "true" })
} yield result
That's a bit long.
At first, I wished I could just skip the result <- ... followed by the immediate yield, as I can in Haskell, but then I noticed the processing going on inside collectFirst.
So I thought it'd be much easier to read as I should better do this as
for {
// ...
section <- components.filter(_.startsWith(DESIRED_SUBSTRING)).headOption
} yield section.substring(section.indexOf(DELIM) + 1).trim() == "true"
Which works, but it is less efficient, since filter has to process all the elements. I'd like to be able to use a lazy filter:
components.withFilter(_.startsWith(DESIRED_SUBSTRING)).headOption
But FilterMonadic doesn't seem to support headOption, and I can't figure out a way to derive it from the operations it does support. I'm sure there's a way with flatMap and some bf, but I'm too unfamiliar with the scala ecosystem at the moment.
If I want to stick with standard library tricks, am I stuck with
for {
// ...
section <- components.collectFirst({ case section if section.startsWith(DESIRED_SUBSTRING) => section })
} yield section.substring(section.indexOf(DELIM) + 1).trim() == "true"
Or is there something better I can use?
If you use components.find(_.startsWith(DESIRED_SUBSTRING)) that will give you an Option with the first element that meets the condition. Then, you can just map over it with any subsequent processing you need.
I have some validation on input data which I really would prefer to handle in the controller code, because:
It only applies in very specific circumstances, so cluttering the verifying function in the form definition would lower code cohesion.
It produces collateral results which I need to use elsewhere in the controller.
What's a clean way form producing a new Form like the one just bound with an additional (field or general) error message in the success branch of Form.fold?
To illustrate, I would like something like the (non-existent) Form.withError method I'm calling here:
val form= myForm.bindFromRequest
form.fold(
errors => BadRequest(view(errors))
{
case(data, button) =>
button match {
case Some("save") =>
val r= costlyFunction(data)
if (r.isOk) {
doSomethingWith(r)
Ok(...)
}
else {
val f= form.withError("my custom error")
BadRequest(view(f))
}
case ...
}
}
Found it myself:
val f= Form(form.mapping, form.data,
Seq(new play.api.data.FormError("error.key", "my error")), form.value)
Apologies for the noise -- leaving it here in case someone else gets stuck as I did.
Shorter alternative:
val f= form.withError("error.key", "my error")), form.value)