Given Futures fa, fb, fc, I can use f: Function1[(A,B,C), Future[D]], to return a Future[D] either by:
(for {
a <- fa
b <- fb
c <- fc
} yield (a,b,c)).flatMap(f)
which has the unenviable property of declaring the variables a,b,c twice.
or
a.zip(b).zip(c).flatMap{ case (a, (b, c)) => f(a, b, c) }
which is terser, but the nesting of the futures into pairs of pairs is weird.
It would be great to have a form of the for-expression where the yield returns a flattened result. Is there such a thing?
There's no reason to flatMap in the yield. It should be another line in the for-comprehension.
for {
a <- fa
b <- fb
c <- fc
d <- f(a, b, c)
} yield d
I don't think it can get more concise than that.
Related
I have a List "a, b, c,d", and this is the expected result
a
ab
abc
abcd
b
bc
bcd
c
cd
d
I tried bruteforce, but I think, there might be any other efficient solution, provided I am having very long list.
Here's a one-liner.
"abcd".tails.flatMap(_.inits.toSeq.init.reverse).mkString(",")
//res0: String = a,ab,abc,abcd,b,bc,bcd,c,cd,d
The mkString() is added just so we can see the result. Otherwise the result is an Iterator[String], which is a pretty memory efficient collection type.
The reverse is only there so that it comes out in the order you specified. If the order of the results is unimportant then that can be removed.
The toSeq.init is there to remove empty elements left behind by the inits call. If those can be dealt with elsewhere then this can also be removed.
This may not be the best solution but one way of doing this is by using sliding function as follow,
val lst = List('a', 'b', 'c', 'd')
val groupedElements = (1 to lst.size).flatMap(x =>
lst.sliding(x, 1))
groupedElements.foreach(x => println(x.mkString("")))
//output
/* a
b
c
d
ab
bc
cd
abc
bcd
abcd
*/
It may not be the best solution, but I think is a good one, and it's tailrec
First this function to get the possible sublists of a List
def getSubList[A](lista: Seq[A]): Seq[Seq[A]] = {
for {
i <- 1 to lista.length
} yield lista.take(i)
}
And then this one to perform the recursion calling the first function and obtain all the sublists possible:
def getSubListRecursive[A](lista: Seq[A]): Seq[Seq[A]] = {
#tailrec
def go(acc: Seq[Seq[A]], rest: Seq[A]): Seq[Seq[A]] = {
rest match {
case Nil => acc
case l => go(acc = acc ++ getSubList(l), rest= l.tail)
}
}
go(Nil, lista)
}
The ouput:
getSubListRecursive(l)
res4: Seq[Seq[String]] = List(List(a), List(a, b), List(a, b, c), List(a, b, c, d), List(b), List(b, c), List(b, c, d), List(c), List(c, d), List(d))
def map2[A,B,C] (a: Par[A], b: Par[B]) (f: (A,B) => C) : Par[C] =
(es: ExecutorService) => {
val af = a (es)
val bf = b (es)
UnitFuture (f(af.get, bf.get))
}
def map3[A,B,C,D] (pa :Par[A], pb: Par[B], pc: Par[C]) (f: (A,B,C) => D) :Par[D] =
map2(map2(pa,pb)((a,b)=>(c:C)=>f(a,b,c)),pc)(_(_))
I have map2 and need to produce map3 in terms of map2. I found the solution in GitHub but it is hard to understand. Can anyone put a sight on it and explain map3 and also what this does (())?
On a purely abstract level, map2 means you can run two tasks in parallel, and that is a new task in itself. The implementation provided for map3 is: run in parallel (the task that consist in running in parallel the two first ones) and (the third task).
Now down to the code: first, let's give name to all the objects created (I also extended _ notations for clarity):
def map3[A,B,C,D] (pa :Par[A], pb: Par[B], pc: Par[C]) (f: (A,B,C) => D) :Par[D] = {
def partialCurry(a: A, b: B)(c: C): D = f(a, b, c)
val pc2d: Par[C => D] = map2(pa, pb)((a, b) => partialCurry(a, b))
def applyFunc(func: C => D, c: C): D = func(c)
map2(pc2d, pc)((c2d, c) => applyFunc(c2d, c)
}
Now remember that map2 takes two Par[_], and a function to combine the eventual values, to get a Par[_] of the result.
The first time you use map2 (the inside one), you parallelize the first two tasks, and combine them into a function. Indeed, using f, if you have a value of type A and a value of type B, you just need a value of type C to build one of type D, so this exactly means that partialCurry(a, b) is a function of type C => D (partialCurry itself is of type (A, B) => C => D).
Now you have again two values of type Par[_], so you can again map2 on them, and there is only one natural way to combine them to get the final value.
The previous answer is correct but I found it easier to think about like this:
def map3[A, B, C, D](a: Par[A], b: Par[B], c: Par[C])(f: (A, B, C) => D): Par[D] = {
val f1 = (a: A, b: B) => (c: C) => f(a, b, c)
val f2: Par[C => D] = map2(a, b)(f1)
map2(f2, c)((f3: C => D, c: C) => f3(c))
}
Create a function f1 that is a version of f with the first 2 arguments partially applied, then we can map2 that with a and b to give us a function of type C => D in the Par context (f1).
Finally we can use f2 and c as arguments to map2 then apply f3(C => D) to c to give us a D in the Par context.
Hope this helps someone!
So I'm running into a speed issue where I have a dataset that needs to be aggregated multiple times.
Initially my team had set up three accumulators and were running a single foreach loop over the data. Something along the lines of
val accum1:Accumulable[a]
val accum2: Accumulable[b]
val accum3: Accumulable[c]
data.foreach{
u =>
accum1+=u
accum2 += u
accum3 += u
}
I am trying to switch these accumulations into an aggregation so that I can get a speed boost and have access to accumulators for debugging. I am currently trying to figure out a way to aggregate these three types at once, since running 3 separate aggregations is significantly slower. Does anyone have any thoughts as to how I can do this? Perhaps aggregating agnostically then pattern matching to split into two RDDs?
Thank you
As far as I can tell all you need here is aggregate with zeroValue, seqOp and combOp corresponding to the operations which are performed by your accumulators.
val zeroValue: (A, B, C) = ??? // (accum1.zero, accum2.zero, accum3.zero)
def seqOp(r: (A, B, C), t: T): (A, B, C) = r match {
case (a, b, c) => {
// Apply operations equivalent to
// accum1.addAccumulator(a, t)
// accum2.addAccumulator(c, t))
// accum3.addAccumulator(c, t)
// and return the first argument
// r
}
}
def combOp(r1: (A, B, C), r2: (A, B, C)): (A, B, C) = (r1, r2) match {
case ((a1, b1, c1), (a2, b2, c2)) => {
// Apply operations equivalent to
// acc1.addInPlace(a1, a2)
// acc2.addInPlace(b1, b2)
// acc3.addInPlace(c1, c2)
// and return the first argument
// r1
}
}
val rdd: RDD[T] = ???
val accums: (A, B, C) = rdd.aggregate(zeroValue)(seqOp, combOp)
This is the code from scalaz document, so basically, if the value is left value, it will take this left value as the final result and stop evaluating the rest, which is very useful.
for {
e1 <- "event 1 ok".right
e2 <- "event 2 failed!".left[String]
e3 <- "event 3 failed!".left[String]
} yield (e1 |+| e2 |+| e3) // event 2 failed
However, I have this code and I am using reduceLeft to append the value
object processor {
def evaluate(x: Int): \/[String, Int] = {
if (x <= 3) x.right[String] else ("some exception about " + x).left[Int]
}
}
val result = (1 to 6).map(processor.evaluate).reduceLeft(_ |+| _)
//\/-(-\/(some exception about 4some exception about 5some exception about 6))
The left value is accumulated, which is not what I want. I guess the different behaviour caused by reduceLeft is "left" value is already evaluated while for operation (flatMap and map) would not.
How should I change this code to take the left result as the final result
You can use traverse syntax
val result = (1 to 6).toList.traverseU(processor.evaluate)
I convert range to List to get List type class in scope
I think there may be some confusion about what is exactly going on in the following code
for {
e1 <- "event 1 ok".right
e2 <- "event 2 failed!".left[String]
e3 <- "event 3 failed!".left[String]
} yield (e1 |+| e2 |+| e3) // event 2 failed
In the code above, the for comprehension is using map and flatMap for \/[+A, +B]. The function defined by the yield statement will never be evaluated because of the implementation of map/flatMap for \/[+A, +B]. In this case it is map/flatMap which is doing the merging of the different \/[String, _]. The |+| operator used by the function in yield, is defined in SemiGroup syntax is simply using Semigroup[String] to combine the strings on the right side into one string. In the case above, it may as well be using String.append. I get why one might want to use Semigroup[String] to combine these opposed to simply String.append here, but the important part is that the function defined by yield is using Semigroup[String] and not something like Semigroup[A \/ B].
In the case below, you are using Semigroup[A \/ B] to combine the String \/ Int instances into a single String \/ Int. reduceLeft (or foldLeft if you chose that route instead) is simply doing the wiring of each element in the list to the accumulation function it is passed.
object processor {
def evaluate(x: Int): \/[String, Int] = {
if (x <= 3) x.right[String] else ("some exception about " + x).left[Int]
}
}
val result: String \/ Int = (1 to 6).map(processor.evaluate).reduceLeft(_ |+| _)
From the definition of Semigroup[A \/ B], we can see that it requires Semigroup[A] and Semigroup[B].
If you instead used Monad[A \/ _] or Applicative[A \/ _] to combine the A \/ B in the function passed to reduceLeft, the As would not be combined.
The following uses Applicative[A \/ _]
val result: String \/ Int = (1 to 6).map(processor.evaluate).reduceLeft {
(xs, x) => (xs |#| x)(_ |+| _)
}
The following uses map/flatMap defined for A \/ B and is most similar to the code at the top.
val result: String \/ Int = (1 to 6).map(processor.evaluate).reduceLeft {
(xs, x) => for {
xsA <- xs
xA <- x
} yield xsA |+| xA
}
foldMapM might do what you want, but uses Foldable[List].foldsRight so your error will be different than if you used foldLeft. Unfortunately, an ugly type lambda, or the type alias below is needed.
type StringEither[B]=String \/ B
val result: String \/ Int = (1 to 6).toList.foldMapM[StringEither, Int](processor.evaluate)
I have what I hope is a simple question about Slick. Apologies if this is well documented - I may have overlooked something in my searching.
I have an aggregate query built as follows:
def doQuery(/* ... */) = for {
a <- Query(TableA)
b <- a.relationship.where // ...
c <- b.relationship.where // ...
} yield (a, b, c)
This returns me a Query[(A, B, C)].
I also have a case class:
case class Aggregate(a: A, b: B, c: C)
I'd like to transform my query to a Query[Aggregate] so my fellow developers can call .list() or .firstOption() and get a List or Option as appropriate.
I naturally went for the .map() method on Query, but it has an implicit Shape argument that I'm not sure how to handle.
Is this straightforward in Slick? We're using v1.0.1 at the moment but upgrading to 2.0 is also a possibility.
Best regards,
Dave
After a lot of playing around, I have concluded that this is not possible in Slick 1.
In Slick 2 you can use the <> operator to transform a projection assembled in the yield portion of the for comprehension:
def doQuery(/* ... */) = for {
a <- Query(TableA)
b <- a.relationship.where // ...
c <- b.relationship.where // ...
} yield (a, b, c) <> (Aggregate.tupled, Aggregate.unapply)
This works as expected in conjunction with .list and .firstOption. I'm unsure what the consequences are of trying to use .insert, .update and .delete.
If you can modify doQuery, then you just want to do yield Aggregate(a, b, c) instead of yield (a, b, c).
Or, if you want to transform the result without modifying doQuery, then you can call .map { case (a, b, c) => Aggregate(a, b, c) } on the result of doQuery.