I am working in a project that use the following code:
case class R(f: Vector[String], s: Vector[String]) {
def apply(name: String): String = f(schema indexOf name)
def apply(names: S): Vector[String] = names map (this apply _)
}
def processCSV(file: String)(yld: R => Unit): Unit = {
val in = new Scanner(file)
val s= in.next('\n').split(",").toVector
while (in.hasNext) {
val f = schema map (n => in.next(if (n == s.last) '\n' else ','))
yld(R(f, s))
}
}
def execOp(op: Operator)(yld: R => Unit): Unit = op match {
case Scan(file, _, _, _) => processCSV(file)(yld)}
Then My question is what is the meaning of yld? That is the same of yield?
Exactly how works, someone can help me to understand how works this yld?
yield is a scala keyword used with for-comprehensions.
yld in that code is VERY different: it is just the name that the author of the code gave one of the parameters to the functions processCSV and execOp. Any other name could have been given to those parameters: fn, callback, cb, etc. Nothing special there. Given the type R => Unit, it is just a function that takes R as input and return Unit (equivalent to void in java). Essentially, a callback where the work happens as side-effects.
Related
Background
I have been reading the book Functional Programming in Scala, and have some questions regarding the content in Chapter 7: Purely functional parallelism.
Here is the code for the answers in the book: Par.scala, but I am confused about certain part of it.
Here is the first part of the code of Par.scala, which stands for Parallelism:
import java.util.concurrent._
object Par {
type Par[A] = ExecutorService => Future[A]
def unit[A](a: A): Par[A] = (es: ExecutorService) => UnitFuture(a)
private case class UnitFuture[A](get: A) extends Future[A] {
def isDone = true
def get(timeout: Long, units: TimeUnit): A = get
def isCancelled = false
def cancel(evenIfRunning: Boolean): Boolean = false
}
def map2[A, B, C](a: Par[A], b: Par[B])(f: (A, B) => C): Par[C] =
(es: ExecutorService) => {
val af = a(es)
val bf = b(es)
UnitFuture(f(af.get, bf.get))
}
def fork[A](a: => Par[A]): Par[A] =
(es: ExecutorService) => es.submit(new Callable[A] {
def call: A = a(es).get
})
def lazyUnit[A](a: => A): Par[A] =
fork(unit(a))
def run[A](es: ExecutorService)(a: Par[A]): Future[A] = a(es)
def asyncF[A, B](f: A => B): A => Par[B] =
a => lazyUnit(f(a))
def map[A, B](pa: Par[A])(f: A => B): Par[B] =
map2(pa, unit(()))((a, _) => f(a))
}
The simplest possible model for Par[A] might be ExecutorService => Future[A], and run simply returns the Future.
unit promotes a constant value to a parallel computation by returning a UnitFuture, which is a simple implementation of Future that just wraps a constant value.
map2 combines the results of two parallel computations with a binary function.
fork marks a computation for concurrent evaluation. The evaluation won’t actually occur until forced by run. Here is with its simplest and most natural implementation of it. Even though it has its problems, let's first put them aside.
lazyUnit wraps its unevaluated argument in a Par and marks it for concurrent evaluation.
run extracts a value from a Par by actually performing the computation.
asyncF converts any function A => B to one that evaluates its result asynchronously.
Questions
The fork is the function confuses me a lot here, because it takes a lazy argument, which will be evaluated later when it is called. Then my questions are more about when we should use this fork, i.e., when we need lazy-evaluation and when we need to have the value directly.
Here is an exercise from the book:
EXERCISE 7.5
Hard: Write this function, called sequence. No additional primitives are required. Do not call run.
def sequence[A](ps: List[Par[A]]): Par[List[A]]
And here is the answers (offered here).
First
def sequence_simple[A](l: List[Par[A]]): Par[List[A]] =
l.foldRight[Par[List[A]]](unit(List()))((h, t) => map2(h, t)(_ :: _))
What is the different between above code and the following:
def sequence_simple[A](l: List[Par[A]]): Par[List[A]] =
l.foldLeft[Par[List[A]]](unit(List()))((t, h) => map2(h, t)(_ :: _))
Additionally
def sequenceRight[A](as: List[Par[A]]): Par[List[A]] =
as match {
case Nil => unit(Nil)
case h :: t => map2(h, fork(sequenceRight(t)))(_ :: _)
}
def sequenceBalanced[A](as: IndexedSeq[Par[A]]): Par[IndexedSeq[A]] = fork {
if (as.isEmpty) unit(Vector())
else if (as.length == 1) map(as.head)(a => Vector(a))
else {
val (l,r) = as.splitAt(as.length/2)
map2(sequenceBalanced(l), sequenceBalanced(r))(_ ++ _)
}
}
In sequenceRight, fork is used when recursive function is directly called. However, in sequenceBalanced, fork is used outside of the whole function body.
Then, what is the differences or above code and the following (where we switched the places of fork):
def sequenceRight[A](as: List[Par[A]]): Par[List[A]] = fork {
as match {
case Nil => unit(Nil)
case h :: t => map2(h, sequenceRight(t))(_ :: _)
}
}
def sequenceBalanced[A](as: IndexedSeq[Par[A]]): Par[IndexedSeq[A]] =
if (as.isEmpty) unit(Vector())
else if (as.length == 1) map(as.head)(a => Vector(a))
else {
val (l,r) = as.splitAt(as.length/2)
map2(fork(sequenceBalanced(l)), fork(sequenceBalanced(r)))(_ ++ _)
}
Finally, given the sequence defined above, we have the following function:
def parMap[A,B](ps: List[A])(f: A => B): Par[List[B]] = fork {
val fbs: List[Par[B]] = ps.map(asyncF(f))
sequence(fbs)
}
I would like to know, can I also implement the function in the following way, which is by applying the lazyUnit defined in the beginning? Is this implementation lazyUnit(ps.map(f)) lazy?
def parMapByLazyUnit[A, B](ps: List[A])(f: A => B): Par[List[B]] =
lazyUnit(ps.map(f))
I did not completely understand your doubt. But I see a major problem with the following solution,
def parMapByLazyUnit[A, B](ps: List[A])(f: A => B): Par[List[B]] =
lazyUnit(ps.map(f))
To understand the problem lets look at def lazyUnit,
def fork[A](a: => Par[A]): Par[A] =
(es: ExecutorService) => es.submit(new Callable[A] {
def call: A = a(es).get
})
def lazyUnit[A](a: => A): Par[A] =
fork(unit(a))
So... lazyUnit takes an expression of type => A and submits it to ExecutorService to get evaluated. And returns the wrapped result of this parallel computation as Par[A].
In parMap for every element of ps: List[A], we not only have to evaluate the corresponding mapping using the function f: A => B but we have to do these evaluations in parallel.
But our solution lazyUnit(ps.map(f)) will submit the whole { ps.map(f) } evaluation as a single task to our ExecutionService. Which means we are not doing it in parallel.
What we need to do is make sure that for each element a in ps: [A], the function f: A => B is executed as a separate task for our ExecutorService.
Now, as we learned from our implementation is that we can run an expression of type exp: => A by using lazyUnit(exp) to get a result: Par[A].
So, we will do exactly that for every a: A in ps: List[A],
val parMappedTmp = ps.map( a => lazyUnit(f(a) ) )
// or
val parMappedTmp = ps.map( a => asyncF(f)(a) )
// or
val parMappedTmp = ps.map(asyncF(f))
But, Now our parMappedTmp is a List[Par[B]] and whereas we needed a Par[List[B]]
So, you will need a function with the following signature to get what you wanted,
def sequence[A](ps: List[Par[A]]): Par[List[A]]
Once you have it,
val parMapped = sequence(parMappedTmp)
Is there any way to write a function(lst: List(T), fun: ) that iterates through lst and partially applies each element to fun and returning a new function each time and recursively doing this until the result of the function application is :Future[T] as desired and not a function type?
fun is a curried function
Something like this.
def partialAppRec(lst : List[T], fun: ?) =
//pardon the non-exhaustive pattern match
lst match {
case x::xs =>
val test = fun(x)
if (test: Future[T]) return test
partialAppRec(xs, (fun(x) _) )
}
But what type would fun be? Is there anyway to say that fun: , disregarding parameters that it could take. I want to be able to take in a fun of variable parameters but that returns Future[T]. f : ..=>Future[T] but I'm not sure something like this exists.
Any tips/suggestions? Thanks.
How about Either?
trait Fun[T] extends Function[T, Either[Fun[T], Future[T]]]
object Fun {
def apply[T](f: T => Either[Fun[T], Future[T]]) =
new Fun[T] { def apply(t: T) = f(t) }
}
def partialAppRec[T](lst: List[T], fun: Fun[T]): Future[T] = lst match {
case Nil => ???
case head :: tail => fun(head) match {
case Right(fu) => fu
case Left(f) => partialAppRec(tail, f)
}
}
I am currently reading Hutton's and Meijer's paper on parsing combinators in Haskell http://www.cs.nott.ac.uk/~pszgmh/monparsing.pdf. For the sake of it I am trying to implement them in scala. I would like to construct something easy to code, extend and also simple and elegant. I have come up with two solutions for the following haskell code
/* Haskell Code */
type Parser a = String -> [(a,String)]
result :: a -> Parser a
result v = \inp -> [(v,inp)]
zero :: Parser a
zero = \inp -> []
item :: Parser Char
item = \inp -> case inp of
[] -> []
(x:xs) -> [(x,xs)]
/* Scala Code */
object Hutton1 {
type Parser[A] = String => List[(A, String)]
def Result[A](v: A): Parser[A] = str => List((v, str))
def Zero[A]: Parser[A] = str => List()
def Character: Parser[Char] = str => if (str.isEmpty) List() else List((str.head, str.tail))
}
object Hutton2 {
trait Parser[A] extends (String => List[(A, String)])
case class Result[A](v: A) extends Parser[A] {
def apply(str: String) = List((v, str))
}
case object Zero extends Parser[T forSome {type T}] {
def apply(str: String) = List()
}
case object Character extends Parser[Char] {
def apply(str: String) = if (str.isEmpty) List() else List((str.head, str.tail))
}
}
object Hutton extends App {
object T1 {
import Hutton1._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
object T2 {
import Hutton2._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
T1.run
T2.run
}
Question 1
In Haskell, zero is a function value that can be used as it is, expessing all failed parsers whether they are of type Parser[Int] or Parser[String]. In scala we achieve the same by calling the function Zero (1st approach) but in this way I believe that I just generate a different function everytime Zero is called. Is this statement true? Is there a way to mitigate this?
Question 2
In the second approach, the Zero case object is extending Parser with the usage of existential types Parser[T forSome {type T}] . If I replace the type with Parser[_] I get the compile error
Error:(19, 28) class type required but Hutton2.Parser[_] found
case object Zero extends Parser[_] {
^
I thought these two expressions where equivalent. Is this the case?
Question 3
Which approach out of the two do you think that will yield better results in expressing the combinators in terms of elegance and simplicity?
I use scala 2.11.8
Note: I didn't compile it, but I know the problem and can propose two solutions.
The more Haskellish way would be to not use subtyping, but to define zero as a polymorphic value. In that style, I would propose to define parsers not as objects deriving from a function type, but as values of one case class:
final case class Parser[T](run: String => List[(T, String)])
def zero[T]: Parser[T] = Parser(...)
As shown by #Alec, yes, this will produce a new value every time, since a def is compiled to a method.
If you want to use subtyping, you need to make Parser covariant. Then you can give zero a bottom result type:
trait Parser[+A] extends (String => List[(A, String)])
case object Zero extends Parser[Nothing] {...}
These are in some way quite related; in system F_<:, which is the base of what Scala uses, the types _|_ (aka Nothing) and \/T <: Any. T behave the same (this hinted at in Types and Programming Languages, chapter 28). The two possibilities given here are a consequence of this fact.
With existentials I'm not so familiar with, but I think that while unbounded T forSome {type T} will behave like Nothing, Scala does not allow inhertance from an existential type.
Question 1
I think that you are right, and here is why: Zero1 below prints hello every time you use it. The solution, Zero2, involves using a val instead.
def Zero1[A]: Parser[A] = { println("hi"); str => List() }
val Zero2: Parser[Nothing] = str => List()
Question 2
No idea. I'm still just starting out with Scala. Hope someone answers this.
Question 3
The trait one will play better with Scala's for (since you can define custom flatMap and map), which turns out to be (somewhat) like Haskell's do. The following is all you need.
trait Parser[A] extends (String => List[(A, String)]) {
def flatMap[B](f: A => Parser[B]): Parser[B] = {
val p1 = this
new Parser[B] {
def apply(s1: String) = for {
(a,s2) <- p1(s1)
p2 = f(a)
(b,s3) <- p2(s2)
} yield (b,s3)
}
}
def map[B](f: A => B): Parser[B] = {
val p = this
new Parser[B] {
def apply(s1: String) = for ((a,s2) <- p(s1)) yield (f(a),s2)
}
}
}
Of course, to do anything interesting you need more parsers. I'll propose to you one simple parser combinator: Choice(p1: Parser[A], p2: Parser[A]): Parser[A] which tries both parsers. (And rewrite your existing parsers more to my style).
def choice[A](p1: Parser[A], p2: Parser[A]): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = { p1(s) ++ p2(s) }
}
def unit[A](x: A): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = List((x,s))
}
val character: Parser[Char] = new Parser[Char] {
def apply(s: String): List[(Char,String)] = List((s.head,s.tail))
}
Then, you can write something like the following:
val parser: Parser[(Char,Char)] = for {
x <- choice(unit('x'),char)
y <- char
} yield (x,y)
And calling parser("xyz") gives you List((('x','x'),"yz"), (('x','y'),"z")).
In Scala we have a by-name-parameters where we can write
def foo[T](f: => T):T = {
f // invokes f
}
// use as:
foo(println("hello"))
I now want to do the same with an array of methods, that is I want to use them as:
def foo[T](f:Array[ => T]):T = { // does not work
f(0) // invokes f(0) // does not work
}
foo(println("hi"), println("hello")) // does not work
Is there any way to do what I want? The best I have come up with is:
def foo[T](f:() => T *):T = {
f(0)() // invokes f(0)
}
// use as:
foo(() => println("hi"), () => println("hello"))
or
def foo[T](f:Array[() => T]):T = {
f(0)() // invokes f(0)
}
// use as:
foo(Array(() => println("hi"), () => println("hello")))
EDIT: The proposed SIP-24 is not very useful as pointed out by Seth Tisue in a comment to this answer.
An example where this will be problematic is the following code of a utility function trycatch:
type unitToT[T] = ()=>T
def trycatch[T](list:unitToT[T] *):T = list.size match {
case i if i > 1 =>
try list.head()
catch { case t:Any => trycatch(list.tail: _*) }
case 1 => list(0)()
case _ => throw new Exception("call list must be non-empty")
}
Here trycatch takes a list of methods of type ()=>T and applies each element successively until it succeeds or the end is reached.
Now suppose I have two methods:
def getYahooRate(currencyA:String, currencyB:String):Double = ???
and
def getGoogleRate(currencyA:String, currencyB:String):Double = ???
that convert one unit of currencyA to currencyB and output Double.
I use trycatch as:
val usdEuroRate = trycatch(() => getYahooRate("USD", "EUR"),
() => getGoogleRate("USD", "EUR"))
I would have preferred:
val usdEuroRate = trycatch(getYahooRate("USD", "EUR"),
getGoogleRate("USD", "EUR")) // does not work
In the example above, I would like getGoogleRate("USD", "EUR") to be invoked only if getYahooRate("USD", "EUR") throws an exception. This is not the intended behavior of SIP-24.
Here is a solution, although with a few restrictions compared to direct call-by-name:
import scala.util.control.NonFatal
object Main extends App {
implicit class Attempt[+A](f: => A) {
def apply(): A = f
}
def tryCatch[T](attempts: Attempt[T]*): T = attempts.toList match {
case a :: b :: rest =>
try a()
catch {
case NonFatal(e) =>
tryCatch(b :: rest: _*)
}
case a :: Nil =>
a()
case Nil => throw new Exception("call list must be non-empty")
}
def a = println("Hi")
def b: Unit = sys.error("one")
def c = println("bye")
tryCatch(a, b, c)
def d: Int = sys.error("two")
def e = { println("here"); 45 }
def f = println("not here")
val result = tryCatch(d, e, f)
println("Result is " + result)
}
The restrictions are:
Using a block as an argument won't work; only the last expression of the block will be wrapped in an Attempt.
If the expression is of type Nothing (e.g., if b and d weren't annotated), the conversion to Attempt is not inserted since Nothing is a subtype of every type, including Attempt. Presumably the same would apply for an expression of type Null.
As of Scala 2.11.7, the answer is no. However, there is SIP-24, so in some future version your f: => T* version may be possible.
I am trying to translate examples from this article to Scala.
So I defined a monadic class Parser with success as return function.
class Parser[A](val run: String => List[(A, String)]) {
def apply(s: String) = run(s)
def flatMap[B](f: A => Parser[B]): Parser[B] = {
val runB = {s: String => for((r, rest) <- run(s); rb <- f(r)(rest)) yield rb}
new Parser(runB)
}
}
def success[A](a:A):Parser[A] = {
val run = {s:String => List((a, s))}
new Parser(run)
}
I defined also a new parser item to return the 1st character of the input.
def item(): Parser[Char] = {
val run = {s: String => if (s.isEmpty) Nil else List((s.head, s.tail))}
new Parser(run)
}
Now I am defining a new parser item12: Parser[(Char, Char)] to return a pair of 1st and 2nd characters
def item12():Parser[(Char, Char)] =
item().flatMap(a => (item().flatMap(b => success(a, b))))
I would like to write it with for-comprehension instead of nested flatMap calls. I understand that I need to define a map method for the Parser. How would you do that ?
I understand that I need to define a map method for the Parser. How would you do that ?
def map[B](f: A => B): Parser[B] = {
val runB = {s: String => ???} // you need to call run and f here
new Parser(runB)
}
I am leaving the body of runB in case you want to do it yourself.