The goal is to lazily evaluate a collection of functions stopping on and returning the first happy result. Computation should be sequential. Here is my attempt
def f1(i: Int): Either[String, Int] = {println(s"f1($i)"); Left("boom-f1") }
def f2(i: Int): Either[String, Int] = {println(s"f2($i)"); Left("boom-f2") }
def f3(i: Int): Either[String, Int] = {println(s"f3($i)"); Right(i) }
val in = 42
(f1(in) #:: f2(in) #:: f3(in) #:: Stream.empty) collectFirst { case Right(x) => x } toRight("boom")
which outputs
f1(42)
f2(42)
f3(42)
res0: Either[String,Int] = Right(42)
where we see all three executed, whilst
def f1(i: Int): Either[String, Int] = {println(s"f1($i)"); Right(i) }
def f2(i: Int): Either[String, Int] = {println(s"f2($i)"); Right(i) }
def f3(i: Int): Either[String, Int] = {println(s"f3($i)"); Right(i) }
would output
f1(42)
res0: Either[String,Int] = Right(42)
where we see only one executed.
Does cats provide abstraction for such lazy failure-biased traversal?
You can use the Ior data type which is an inclusive-or relationship between two data types.
import cats.data.Ior
import cats.implicits._
import scala.annotation.tailrec
object Main2 {
def main(args: Array[String]) : Unit = {
def f1(i: Int): Either[String, Int] = {println(s"f1($i)"); Left("boom-f1") }
def f2(i: Int): Either[String, Int] = {println(s"f2($i)"); Left("boom-f2") }
def f3(i: Int): Either[String, Int] = {println(s"f3($i)"); Right(i) }
def traverseLazy(input: Int, list: List[Int => Either[String, Int]]): Ior[List[String], Int] = {
#tailrec
def go(ls: List[Int => Either[String, Int]], acc: List[String]): Ior[List[String], Int] = ls match {
case x :: xs => x(input) match {
case Left(error) => go(xs, error :: acc)
case Right(value) => if (ls.isEmpty) value.rightIor else Ior.both(acc, value)
}
case Nil => acc.leftIor
}
go(list, List.empty)
}
val res = traverseLazy(42, List(f1, f2, f3)).fold(
_.intercalate("\n"),
res => s"succeeded with $res",
(errors, res) => s"completed successfully with res $res but some errors were also found: ${errors.intercalate(", ")}")
println(res)
}
}
Ah I was overcomplicating, orElse already implements such semantics and gives simply
f1(in) orElse f2(in) orElse f3(in) orElse Left("boom")
as it takes by-name parameter.
Related
def merge(bigrams1: Map[String, mutable.SortedMap[String, Int]],
bigrams2: Map[String, mutable.SortedMap[String, Int]]): Map[String, mutable.SortedMap[String, Int]] = {
bigrams2 ++ bigrams1
.map(entry1 => entry1._1 -> (entry1._2 ++ bigrams2.getOrElse(entry1._1, mutable.SortedMap())
.map(entry2 => entry2._1 -> (entry2._2 + entry1._2.getOrElse(entry2._1, 0)))))
}
At compile time, I get these errors:
Error:(64, 114) diverging implicit expansion for type scala.math.Ordering[T1]
starting with method Tuple9 in object Ordering
bigrams2 ++ bigrams1.map(entry1 => entry1._1 -> (entry1._2 ++ bigrams2.getOrElse(entry1._1, mutable.SortedMap()).map(entry2 => entry2._1 -> (entry2._2 + entry1._2.getOrElse(entry2._1, 0)))))
Error:(64, 114) not enough arguments for method apply: (implicit ord: scala.math.Ordering[A])scala.collection.mutable.SortedMap[A,B] in class SortedMapFactory.
Unspecified value parameter ord.
bigrams2 ++ bigrams1.map(entry1 => entry1._1 -> (entry1._2 ++ bigrams2.getOrElse(entry1._1, mutable.SortedMap()).map(entry2 => entry2._1 -> (entry2._2 + entry1._2.getOrElse(entry2._1, 0)))))
Specifying the types of the sorted map solves the problem:
def merge(bigrams1: Map[String, mutable.SortedMap[String, Int]],
bigrams2: Map[String, mutable.SortedMap[String, Int]]): Map[String, mutable.SortedMap[String, Int]] = {
bigrams2 ++ bigrams1
.map(entry1 => entry1._1 -> (entry1._2 ++ bigrams2.getOrElse(entry1._1, mutable.SortedMap[String, Int]())
.map(entry2 => entry2._1 -> (entry2._2 + entry1._2.getOrElse(entry2._1, 0)))))
}
Why do these type parameters need to be specified? Why can they not be inferred without problems with the implicit ordering?
Full code:
import java.io.File
import scala.annotation.tailrec
import scala.collection.mutable
import scala.io.Source
import scala.util.matching.Regex
case class Bigrams(bigrams: Map[String, mutable.SortedMap[String, Int]]) {
def mergeIn(bigramsIn: Map[String, mutable.SortedMap[String, Int]]): Bigrams = {
Bigrams(Bigrams.merge(bigrams, bigramsIn))
}
def extractStatistics(path: String): Bigrams = {
val entry: File = new File(path)
if (entry.exists && entry.isDirectory) {
val bigramsFromDir: Map[String, mutable.SortedMap[String, Int]] = entry
.listFiles
.filter(file => file.isFile && file.getName.endsWith(".sgm"))
.map(Bigrams.getBigramsFrom)
.foldLeft(Map[String, mutable.SortedMap[String, Int]]())(Bigrams.merge)
val bigramsFromSubDirs: Bigrams = entry
.listFiles
.filter(entry => entry.isDirectory)
.map(entry => extractStatistics(entry.getAbsolutePath))
.foldLeft(Bigrams())(Bigrams.merge)
bigramsFromSubDirs.mergeIn(bigramsFromDir)
} else if (entry.exists && entry.isFile) {
Bigrams(Bigrams.getBigramsFrom(entry))
} else
throw new RuntimeException("Incorrect path")
}
def getFreqs(word: String): Option[mutable.SortedMap[String, Int]] = {
bigrams.get(word)
}
}
object Bigrams {
def fromPath(path: String): Bigrams = {
new Bigrams(Map[String, mutable.SortedMap[String, Int]]()).extractStatistics(path)
}
def apply(): Bigrams = {
new Bigrams(Map())
}
val BODY: Regex = "(?s).*<BODY>(.*)</BODY>(?s).*".r
// Return a list with the markup for each article
#tailrec
def readArticles(remainingLines: List[String], acc: List[String]): List[String] = {
if (remainingLines.size == 1) acc
else {
val nextLine = remainingLines.head
if (nextLine.startsWith("<REUTERS ")) readArticles(remainingLines.tail, nextLine +: acc)
else readArticles(remainingLines.tail, (acc.head + "\n" + nextLine) +: acc.tail)
}
}
def merge(bigrams1: Map[String, mutable.SortedMap[String, Int]],
bigrams2: Map[String, mutable.SortedMap[String, Int]]): Map[String, mutable.SortedMap[String, Int]] = {
bigrams2 ++ bigrams1
.map(entry1 => entry1._1 -> (entry1._2 ++ bigrams2.getOrElse(entry1._1, mutable.SortedMap[String, Int]())
.map(entry2 => entry2._1 -> (entry2._2 + entry1._2.getOrElse(entry2._1, 0)))))
}
def merge(bigrams1: Bigrams, bigrams2: Bigrams): Bigrams = {
new Bigrams(merge(bigrams1.bigrams, bigrams2.bigrams))
}
def getBigramsFrom(path: File): Map[String, mutable.SortedMap[String, Int]] = {
val file = Source.fromFile(path)
val fileLines: List[String] = file.getLines().toList
val articles: List[String] = Bigrams.readArticles(fileLines.tail, List())
val bodies: List[String] = articles.map(extractBody).filter(body => !body.isEmpty)
val sentenceTokens: List[List[String]] = bodies.flatMap(getSentenceTokens)
sentenceTokens.foldLeft(Map[String, mutable.SortedMap[String, Int]]())((acc, tokens) => addBigramsFrom(tokens, acc))
}
def getBigrams(tokens: List[String]): List[(String, String)] = {
tokens.indices.
map(i => {
if (i < tokens.size - 1) (tokens(i), tokens(i + 1))
else null
})
.filter(_ != null).toList
}
// Return the body of the markup of one article
def extractBody(article: String): String = {
try {
val body: String = article match {
case Bigrams.BODY(bodyGroup) => bodyGroup
}
body
}
catch {
case _: MatchError => ""
}
}
def getSentenceTokens(text: String): List[List[String]] = {
val separatedBySpace: List[String] = text
.replace('\n', ' ')
.replaceAll(" +", " ") // regex
.split(" ")
.map(token => if (token.endsWith(",")) token.init.toString else token)
.toList
val splitAt: List[Int] = separatedBySpace.indices
.filter(i => i > 0 && separatedBySpace(i - 1).endsWith(".") || i == 0)
.toList
groupBySentenceTokens(separatedBySpace, splitAt, List()).map(sentenceTokens => sentenceTokens.init :+ sentenceTokens.last.substring(0, sentenceTokens.last.length - 1))
}
#tailrec
def groupBySentenceTokens(tokens: List[String], splitAt: List[Int], sentences: List[List[String]]): List[List[String]] = {
if (splitAt.size <= 1) {
if (splitAt.size == 1) {
sentences :+ tokens.slice(splitAt.head, tokens.size)
} else {
sentences
}
}
else groupBySentenceTokens(tokens, splitAt.tail, sentences :+ tokens.slice(splitAt.head, splitAt.tail.head))
}
def addBigramsFrom(tokens: List[String], bigrams: Map[String, mutable.SortedMap[String, Int]]): Map[String, mutable.SortedMap[String, Int]] = {
var newBigrams = bigrams
val bigramsFromTokens: List[(String, String)] = Bigrams.getBigrams(tokens)
bigramsFromTokens.foreach(bigram => { // TODO: This code uses side effects to get the job done. Try to remove them.
val currentFreqs: mutable.SortedMap[String, Int] = newBigrams.get(bigram._1)
.map((map: mutable.SortedMap[String, Int]) => map)
.getOrElse(mutable.SortedMap())
val incrementedWordFreq = currentFreqs.get(bigram._2)
.map(freq => freq + 1)
.getOrElse(1)
val newFreqs = currentFreqs + (bigram._2 -> incrementedWordFreq)
newBigrams = newBigrams - bigram._1 + (bigram._1 -> newFreqs)
})
newBigrams
}
}
The thing is that method Map#getOrElse in 2.13 (or MapLike#getOrElse in 2.12) has signature
def getOrElse[V1 >: V](key: K, default: => V1): V1
https://github.com/scala/scala/blob/2.13.x/src/library/scala/collection/Map.scala#L132-L135
https://github.com/scala/scala/blob/2.12.x/src/library/scala/collection/MapLike.scala#L129-L132
i.e. it expects not necessarily default of the same type as V in Map[K, +V] (or MapLike[K, +V, +This <: MapLike[K, V, This] with Map[K, V]]) but possibly of a supertype of V. In your case V is mutable.SortedMap[String, Int] and there are so many its supertypes (you can look at inheritance hierarchy yourself). mutable.SortedMap() can be of any type mutable.SortedMap[A, B] or their super types.
If you replace method getOrElse with having fixed V
implicit class MapOps[K, V](m: Map[K, V]) {
def getOrElse1(key: K, default: => V): V = m.get(key) match {
case Some(v) => v
case None => default
}
}
or in 2.12
implicit class MapOps[K, V, +This <: MapLike[K, V, This] with Map[K, V]](m: MapLike[K, V, This]) {
def getOrElse1(key: K, default: => V): V = m.get(key) match {
case Some(v) => v
case None => default
}
}
then in your code
... bigrams2.getOrElse1(entry1._1, mutable.SortedMap())
all types will be inferred and it will compile.
So sometimes types should be just specified explicitly when compiler asks.
Suppose I've got a function op: (Int, Int) => Future[Int] and need to write a new function foo :
def foo(xs: Seq[Int],
zero: Int,
op: (Int, Int) => Future[Int]): Future[Int] = ???
foo should work as foldLeft and apply op sequentially to all elements in xs, e.g.:
val op: (Int, Int) => Future[Int] = (x, y) => Future(x + y)
val xs = (1 to 10)
val fut = foo(xs, 0, op) // should return Future of 55
fut.value // Some(Success(55))
How would you implement foo ?
I'm not sure why the other answer was deleted - but with plain Scala this works for me:
def foo(xs: Seq[Int], zero: Int, op: (Int, Int) => Future[Int]): Future[Int] =
xs.foldLeft(Future.successful(zero))((a, b) => a.flatMap(op(_, b)))
Do I miss something?
Try foldM from cats:
import cats._
import cats.implicits._
def foo(xs: Seq[Int], zero: Int, op: (Int, Int) => Future[Int]): Future[Int] =
Foldable[List].foldM(xs.toList, zero)(op)
Without using an external library:
Implement a "special" foldLeft:
def foldLeft[Int](xs: Seq[Int], z: Int)(op: (Int, Int) => Future[Int]): Future[Int] = {
def f(xs: Seq[Int], accF: Future[Int]): Future[Int] = xs match {
case Seq() => accF
case x +: xs => accF.flatMap(acc => f(xs, op(acc, x)))
}
f(xs, Future.successful(z))
}
And using it:
def foo(xs: Seq[Int],
zero: Int,
op: (Int, Int) => Future[Int]): Future[Int] = foldLeft(xs, zero)(op)
Suppose I've got partial function parf
val parf: PartialFunction[Int, String] = { case 0 => "!!!" }
Now I've got also case class A(x: Int) and I need a function to transform PartialFunction[Int, String] to PartialFunction[A, String]:
def foo(pf: PartialFunction[Int, String]): PartialFunction[A, String] = ???
For example, foo(parf) should return {case A(0) => "!!!" }. How would you write function foo ?
To maintain the correct functionality, you need to check if the inner partial function is defined on a parameter you're going to pass:
val parf: PartialFunction[Int, String] = { case 0 => "!!!" }
case class A(x: Int)
def foo(pf: PartialFunction[Int, String]): PartialFunction[A, String] = {
case A(i) if pf.isDefinedAt(i) => pf(i)
}
If you plan to do it on a larger scale, you might want to convert a partial function to an extractor object, so it can be used in pattern matches directly with a better syntax:
trait Extractor[A, B] {
def unapply(a: A): Option[B]
}
object Extractor {
implicit def partialFunctionAsExtractor[A, B](pf: PartialFunction[A, B]): Extractor[A, B] =
new Extractor[A, B] {
def unapply(a: A) = if (pf.isDefinedAt(a)) Some(pf(a)) else None
}
}
def foo2(pf: Extractor[Int, String]): PartialFunction[A, String] = {
case A(pf(str)) => str
}
foo2(parf) // implicit conversion magic
I don't see what got you confused about it? You just need to match-extract the Int out of A and then let the PF behave as it wants to behave.
scala> case class A(x: Int)
// defined class A
scala> val parf: PartialFunction[Int, String] = { case 0 => "!!!" }
// parf: PartialFunction[Int,String] = <function1>
scala> def foo(pf: PartialFunction[Int, String]): PartialFunction[A, String] = {
| case A(x) if pf.isDefinedAt(x) => pf(x)
| }
// foo: (pf: PartialFunction[Int,String])PartialFunction[A,String]
scala> val parfA = foo(parf)
// parfA: PartialFunction[A,String] = <function1>
scala> parfA(A(0))
//res0: String = !!!
scala> parfA(A(1))
// scala.MatchError: A(1) (of class A)
// at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:254)
// at scala.PartialFunction$$anon$1.apply(PartialFunction.scala:252)
// at $anonfun$1.applyOrElse(<console>:11)
// at $anonfun$1.applyOrElse(<console>:11)
// at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
// at $anonfun$foo$1.applyOrElse(<console>:13)
// at $anonfun$foo$1.applyOrElse(<console>:13)
// at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
// ... 28 elided
#Oleg Pyzhcov already provided a great solution. Another approach would be to create a PartialFunction[A, Int] that is defined at A(0), and use andThen to chain it with parf:
val parf: PartialFunction[Int, String] = { case 0 => "!!!" }
case class A(n: Int)
val bar: PartialFunction[A, Int] = { case a: A if a.n == 0 => a.n }
def foo(pf: PartialFunction[Int, String]): PartialFunction[A, String] =
bar andThen pf
// foo: (pf: PartialFunction[Int,String])PartialFunction[A,String]
foo(parf)
// res1: PartialFunction[A,String] = <function1>
Suppose I am refactoring a function like this:
def check(ox: Option[Int]): Unit = ox match {
case None => throw new Exception("X is missing")
case Some(x) if x < 0 => throw new Exception("X is negative")
case _ => ()
}
I'd like to get rid of the exceptions but I need to keep the check signature and behavior as is, so I'm factoring out a pure implementation -- doCheck:
import scala.util.{Try, Success, Failure}
def doCheck(ox: Option[Int]): Try[Unit] = ???
def check(ox: Option[Int]): Unit = doCheck(ox).get
Now I am implementing doCheck as follows:
def doCheck(ox: Option[Int]): Try[Unit] = for {
x <- ox toTry MissingX()
_ <- (x > 0) toTry NegativeX(x)
} yield ()
Using the following implicits:
implicit class OptionTry[T](o: Option[T]) {
def toTry(e: Exception): Try[T] = o match {
case Some(t) => Success(t)
case None => Failure(e)
}
}
implicit class BoolTry(bool: Boolean) {
def toTry(e: Exception): Try[Unit] = if (bool) Success(Unit) else Failure(e)
}
The implicits certainly add more code. I hope to replace OptionTry with an implicit from scalaz/cats someday and maybe find an analog for BoolTry.
You could refactor with a loan pattern and Try.
def withChecked[T](i: Option[Int])(f: Int => T): Try[T] = i match {
case None => Failure(new java.util.NoSuchElementException())
case Some(p) if p >= 0 => Success(p).map(f)
case _ => Failure(
new IllegalArgumentException(s"negative integer: $i"))
}
Then it can be used as following.
val res1: Try[String] = withChecked(None)(_.toString)
// res1 == Failure(NoSuchElement)
val res2: Try[Int] = withChecked(Some(-1))(identity)
// res2 == Failure(IllegalArgumentException)
def foo(id: Int): MyType = ???
val res3: Try[MyType] = withChecked(Some(2)) { id => foo(id) }
// res3 == Success(MyType)
How can I apply implement the following function?
def wrapIntoOption(state: State[S, A]): State[Option[S], Option[A]]
The bigger picture is this:
case class Engine(cylinders: Int)
case class Car(engineOpt: Option[Engine])
val engineOptLens = Lens.lensu((c, e) => c.copy(engineOpt = e), _.engineOpt)
def setEngineCylinders(count: Int): State[Engine, Int] = State { engine =>
(engine.copy(cylinders = count), engine.cylinders)
}
def setEngineOptCylinders(count: Int): State[Option[Engine], Option[Int]] = {
??? // how to reuse setEngineCylinders?
}
def setCarCylinders(count: Int): State[Car, Option[Int]] = {
engineOptLens.lifts(setEngineOptCylinders)
}
Is there nicer way to deal with Option properties?
One way to create the wrapIntoOption function would be to call run on the passed State[S, A] and convert the resulting (S, A) tuple into (Option[S], Option[A]).
import scalaz.State
import scalaz.std.option._
import scalaz.syntax.std.option._
def wrapIntoOption[S, A](state: State[S, A]): State[Option[S], Option[A]] =
State(_.fold((none[S], none[A])){ oldState =>
val (newState, result) = state.run(oldState)
(newState.some, result.some)
})
You could then define setEngineOptCylinders as :
def setEngineOptCylinders(count: Int): State[Option[Engine], Option[Int]] =
wrapIntoOption(setEngineCylinders(count))
Which you can use as :
scala> val (newEngine, oldCylinders) = setEngineOptCylinders(8).run(Engine(6).some)
newEngine: Option[Engine] = Some(Engine(8))
oldCylinders: Option[Int] = Some(6)