I'm experimenting with Free monad in Scalaz and trying to build simple interpreter to parse and evaluate expressions like:
dec(inc(dec(dec(10)))
where dec means decrement, inc means increment. Here is what I got:
trait Interpreter[A]
case class V[A](a: A) extends Interpreter[A]
object Inc {
private[this] final val pattern = Pattern.compile("^inc\\((.*)\\)$")
def unapply(arg: String): Option[String] = {
val m = pattern.matcher(arg)
if(m.find()){
Some(m.group(1))
} else None
}
}
object Dec {
private[this] final val pattern = Pattern.compile("^dec\\((.*)\\)$")
def unapply(arg: String): Option[String] = {
val m = pattern.matcher(arg)
if(m.find()){
Some(m.group(1))
} else None
}
}
object Val {
def unapply(arg: String): Option[Int] =
if(arg.matches("^[0-9]+$")) Some(Integer.valueOf(arg))
else None
}
Now this is all I need to build AST. It currently looks as follows:
def buildAst(expression: String): Free[Interpreter, Int] =
expression match {
case Inc(arg) => inc(buildAst(arg))
case Dec(arg) => dec(buildAst(arg))
case Val(arg) => value(arg)
}
private def inc(i: Free[Interpreter, Int]) = i.map(_ + 1)
private def dec(d: Free[Interpreter, Int]) = d.map(_ - 1)
private def value(v: Int): Free[Interpreter, Int] = Free.liftF(V(v))
Now when testing the application:
object Test extends App{
val expression = "inc(dec(inc(inc(inc(dec(10))))))"
val naturalTransform = new (Interpreter ~> Id) {
override def apply[A](fa: Interpreter[A]): Id[A] = fa match {
case V(a) => a
}
}
println(buildAst(expression).foldMap(naturalTransform)) //prints 12
}
And it works pretty much fine (I'm not sure about if it is in scalaz style).
THE PROBLEM is the extractor objects Inc, Dec, Val feels like boilerplate code. Is there a way to reduce such a code duplication.
This will definitely become a problem if the number of functions supported gets larger.
Free monads are creating some boilerplate and that is a fact. However if you are willing to stick to some conventions, you could rewrite interpreter with Freasy Monad:
#free trait Interpreter {
type InterpreterF[A] = Free[InterpreterADT, A]
sealed trait InterpreterADT[A]
def inc(arg: InterpreterF[Int]): InterpreterF[Int]
def dec(arg: InterpreterF[Int]): InterpreterF[Int]
def value(arg: Int): InterpreterF[Int]
}
and that would generate all of case classes and matching on them. The interpreter becomes just a trait to implement.
However, you already have some logic within unapply - so you would have to split the parsing and executing logic:
import Interpreter.ops._
val incP = """^inc\\((.*)\\)$""".r
val decP = """^dec\\((.*)\\)$""".r
val valP = """^val\\((.*)\\)$""".r
def buildAst(expression: String): InterpreterF[Int] = expression match {
case incP(arg) => inc(buildAst(arg))
case decP(arg) => dec(buildAst(arg))
case valP(arg) => value(arg.toInt)
}
Then you could implement an actual interpreter:
val impureInterpreter = new Interpreter.Interp[Id] {
def inc(arg: Int): Int = arg+1
def dec(arg: Int): Int = arg-1
def value(arg: Int): Int = arg
}
and run it:
impureInterpreter.run(buildAst(expression))
I admit that this is more of a pseudocode than tested working solution, but it should give a general idea. Another library that uses similar idea is Freestyle but they use their own free monads implementation instead of relying on a cats/scalaz.
So, I would say it is possible to remove some boilerplate as long as you have no issue with splitting parsing and interpretation. Of course not all can be removed - you have to declare possible operations on your Interpreter algebra as well as you have to implement interpreter yourself.
Related
For Scala 3 macros, does anyone know of a way to find all functions with a given annotation?
For instance:
#fruit
def apple(): Int = ???
#fruit
def banana(): Int = ???
#fruit
def coconut(): Int = ???
#fruit
def durian(): Int = ???
def elephant(): Int = ???
#fruit
def fig(): Int = ???
I would want to find a list of apple, banana, coconut, durian, fig. They could be defined anywhere, but in my case they will all be in a single package.
This solution will extract all the definitions with some annotation from a given package. I will leverage also the compile-time reflection.
This solution will extract all the definitions with some annotations from a given package. I will also leverage the compile-time reflection.
So, To solve your problem, we need to divide it in:
methods gathering from a package;
filter only methods with a given annotation;
transform symbols in function application.
I suppose that you can pass the package and the annotation (and also the return type) as a type argument. So the macro signature is something like that:
inline def findAllFunction[P, A <: ConstantAnnotation, R]: List[() => R] =
${Implementation.myMacroImpl[P, A, R]()}
The first point is straightforward. we could extract all the methods defined as:
def methodsFromPackage(packageSymbol: Symbol): List[Symbol] =
packageSymbol.declaredTypes
.filter(_.isClassDef)
.flatMap(_.declaredMethods)
The second point is also quite easy. Symbol class has the method hasAnnotation that could be used in this case:
def methodsAnnotatatedWith(
methods: List[Symbol],
annotation: Symbol
): List[Symbol] =
methods.filter(_.hasAnnotation(annotation))
The last point is a little bit challenging. Here we should construct the method call. So we need to create the AST that correspond to the method call. Inspired by this example, we can call definition using Apply. Select and This serve to select the correct method that will be called:
def transformToFunctionApplication(methods: List[Symbol]): Expr[List[() => R]] =
val appliedDef = methods
.map(definition => Select(This(definition.owner), definition))
.map(select => Apply(select, List.empty))
.map(apply => '{ () => ${ apply.asExprOf[R] } })
Expr.ofList(appliedDef)
Here I used lamba call, if you want to return directly the value you should change the last two instructions:
def transformToFunctionApplication(methods: List[Symbol]): Expr[List[R]] =
val appliedDef = methods
.map(definition => Select(This(definition.owner), definition))
.map(select => Apply(select, List.empty))
.map(apply => apply.asExprOf[R])
Expr.ofList(appliedDef)
To sum up, the all methods could be defined as:
def myMacroImpl[P: Type, A: Type, R: Type]()(using
Quotes
): Expr[List[() => R]] = {
import quotes.reflect.*
val annotation = TypeRepr.of[A].typeSymbol
val moduleTarget = TypeRepr.of[P].typeSymbol
def methodsFromPackage(packageSymbol: Symbol): List[Symbol] =
packageSymbol.declaredTypes
.filter(_.isClassDef)
.flatMap(_.declaredMethods)
def methodsAnnotatatedWith(
methods: List[Symbol],
annotation: Symbol
): List[Symbol] =
methods.filter(_.hasAnnotation(annotation))
def transformToFunctionApplication(
methods: List[Symbol]
): Expr[List[() => R]] =
val appliedDef = methods
.map(definition => Select(This(definition.owner), definition))
.map(select => Apply(select, List.empty))
.map(apply => '{ () => ${ apply.asExprOf[R] } })
Expr.ofList(appliedDef)
val methods = methodsFromPackage(moduleTarget)
val annotatedMethod = methodsAnnotatatedWith(methods, annotation)
transformToFunctionApplication(annotatedMethod)
}
Finally, you can use the macro as:
package org.tests
import org.tests.Macros.fruit
package foo {
#fruit
def check(): Int = 10
#fruit
def other(): Int = 11
}
#main def hello: Unit =
println("Hello world!")
println(Macros.findAllFunction[org.tests.foo, fruit, Int].map(_.apply())) /// List(10, 11)
Scastie
I am trying to create a Macro to give me a list of val for a specific case class.
object CaseClass {
def valList[T]: List[String] = macro implValList[T]
def implValList[T](c: whitebox.Context): c.Expr[List[String]] = {
import c.universe._
val listApply = Select(reify(List).tree, TermName("apply"))
val vals = weakTypeOf[T].decls.collect {
case m: TermSymbol if m.isVal => q"${m.name}"
}
c.Expr[List[String]](Apply(listApply, vals.toList))
}
}
So given
case class AClass(
val a: String,
val b: Int
)
I want a list of CaseClass.valList[AClass] = List("a", "b")
Not an expert on macros, so take it with a grain of salt. But I tested it with Intellij.
First, to use weakTypeOf you need to take a WeakTypeTag as an implicit in your macro impl like this:
def implValList[T](c: whitebox.Context)(implicit wt: c.WeakTypeTag[T]) ...
Second, to create literals, you use this construct instead of your quasiquote, (which, I believe, actually does nothing):
Literal(Constant(m.name.toString))
Last, I recommend using this guard instead of isVal:
m.isCaseAccessor && m.isGetter
Which is properly checking for case class parameter and also being a getter (case class parameters are duplicated, one as isGetter, other one as isParam). The reason for this being that isVal names for case classes surprisingly produce a name ending in whitespace.
The final implementation that works for me is as follows:
object CaseClass {
def valList[T]: List[String] = macro implValList[T]
def implValList[T](c: whitebox.Context)(implicit wt: c.WeakTypeTag[T]): c.Expr[List[String]] = {
import c.universe._
val listApply = Select(reify(List).tree, TermName("apply"))
val vals = weakTypeOf[T].decls.collect {
case m: TermSymbol if m.isCaseAccessor && m.isGetter => Literal(Constant(m.name.toString))
}
c.Expr[List[String]](Apply(listApply, vals.toList))
}
}
As an alternative (because macros are somewhat of a pain to set up - you cannot use macro in the same subproject that defines it), and you don't need it very often, you might be able to get away with a shapeless one-liner:
import shapeless._
import shapeless.ops.record.Keys
case class Foo(a: Int, b: String)
Keys[the.`LabelledGeneric[Foo]`.Repr].apply().toList.map(_.name) // List("a", "b")
Suppose I've got a few case classes, e.g.:
case class C(c1: Int, c2: Double, c3: Option[String])
case class B(b: Int, cs: Seq[C])
case class A(a: String, bs: Seq[B])
Now I would like to generate a few instances of A with random values for tests.
I am looking for a generic way to do that. I can probably do it with runtime reflection but I prefer a compile-time solution.
def randomInstance[A](a: A): A = ???
How can I do it ? Can it be done with shapeless ?
The easiest way for you to do that would be using ScalaCheck. You do so by defining a Gen[A] for your instances:
import org.scalacheck.Gen
final case class C(c1: Int, c2: Double, c3: Option[String])
object C {
val cGen: Gen[C] = for {
c1 <- Gen.posNum[Int]
c2 <- Gen.posNum[Double]
c3 <- Gen.option(Gen.oneOf("foo", "bar", "hello"))
} yield C(c1, c2, c3)
}
And you consume it:
object F {
def main(args: Array[String]): Unit = {
val randomC: C = C.cGen.sample.get
}
}
On top of that, you can add scalacheck-shapeless which generates the Gen[A] for you, with completely random values (where you have no control over them).
You may also want to look into random-data-generator (thanks #Gabriele Petronella), which simplifies things even further. From the docs:
import com.danielasfregola.randomdatagenerator.RandomDataGenerator
object MyApp extends RandomDataGenerator {
case class Example(text: String, n: Int)
val example: Example = random[Example]
// Example(ਈ䈦㈾钜㔪旅ꪔ墛炝푰⡨䌆ᵅ퍧咪, 73967257)
}
This is also especially helpful in property based testing.
We've just moved away from scalacheck-shapeless and use Scala/Java reflection instead.
The main reasons are (1) scalacheck-shapeless uses Macros (slow compilation), (2) the API is a bit more verbose than my liking, and (3) the generated values are way too wild (e.g. generating strings with Japanese characters).
However, setting it up is a bit more involved. Here is a full working code that you can copy into your codebase:
import scala.reflect.api
import scala.reflect.api.{TypeCreator, Universe}
import scala.reflect.runtime.universe._
object Maker {
val mirror = runtimeMirror(getClass.getClassLoader)
var makerRunNumber = 1
def apply[T: TypeTag]: T = {
val method = typeOf[T].companion.decl(TermName("apply")).asMethod
val params = method.paramLists.head
val args = params.map { param =>
makerRunNumber += 1
param.info match {
case t if t <:< typeOf[Enumeration#Value] => chooseEnumValue(convert(t).asInstanceOf[TypeTag[_ <: Enumeration]])
case t if t =:= typeOf[Int] => makerRunNumber
case t if t =:= typeOf[Long] => makerRunNumber
case t if t =:= typeOf[Date] => new Date(Time.now.inMillis)
case t if t <:< typeOf[Option[_]] => None
case t if t =:= typeOf[String] && param.name.decodedName.toString.toLowerCase.contains("email") => s"random-$arbitrary#give.asia"
case t if t =:= typeOf[String] => s"arbitrary-$makerRunNumber"
case t if t =:= typeOf[Boolean] => false
case t if t <:< typeOf[Seq[_]] => List.empty
case t if t <:< typeOf[Map[_, _]] => Map.empty
// Add more special cases here.
case t if isCaseClass(t) => apply(convert(t))
case t => throw new Exception(s"Maker doesn't support generating $t")
}
}
val obj = mirror.reflectModule(typeOf[T].typeSymbol.companion.asModule).instance
mirror.reflect(obj).reflectMethod(method)(args:_*).asInstanceOf[T]
}
def chooseEnumValue[E <: Enumeration: TypeTag]: E#Value = {
val parentType = typeOf[E].asInstanceOf[TypeRef].pre
val valuesMethod = parentType.baseType(typeOf[Enumeration].typeSymbol).decl(TermName("values")).asMethod
val obj = mirror.reflectModule(parentType.termSymbol.asModule).instance
mirror.reflect(obj).reflectMethod(valuesMethod)().asInstanceOf[E#ValueSet].head
}
def convert(tpe: Type): TypeTag[_] = {
TypeTag.apply(
runtimeMirror(getClass.getClassLoader),
new TypeCreator {
override def apply[U <: Universe with Singleton](m: api.Mirror[U]) = {
tpe.asInstanceOf[U # Type]
}
}
)
}
def isCaseClass(t: Type) = {
t.companion.decls.exists(_.name.decodedName.toString == "apply") &&
t.decls.exists(_.name.decodedName.toString == "copy")
}
}
And, when you want to use it, you can call:
val user = Maker[User]
val user2 = Maker[User].copy(email = "someemail#email.com")
The code above generates arbitrary and unique values. They aren't exactly randomised. It's best for using in tests.
Read our full blog post here: https://give.engineering/2018/08/24/instantiate-case-class-with-arbitrary-value.html
We've started using Magnolia, which provides a faster type class derivation compared to shapeless for derivation of Arbitrary instances.
Here is the library to use, and here is an example (docs):
case class Inner(int: Int, str: String)
case class Outer(inner: Inner)
// ScalaCheck Arbitrary
import magnolify.scalacheck.auto._
import org.scalacheck._ // implicit instances for Arbitrary[Int], etc.
val arb: Arbitrary[Outer] = implicitly[Arbitrary[Outer]]
arb.arbitrary.sample
// = Some(Outer(Inter(12345, abcde)))
I am currently reading Hutton's and Meijer's paper on parsing combinators in Haskell http://www.cs.nott.ac.uk/~pszgmh/monparsing.pdf. For the sake of it I am trying to implement them in scala. I would like to construct something easy to code, extend and also simple and elegant. I have come up with two solutions for the following haskell code
/* Haskell Code */
type Parser a = String -> [(a,String)]
result :: a -> Parser a
result v = \inp -> [(v,inp)]
zero :: Parser a
zero = \inp -> []
item :: Parser Char
item = \inp -> case inp of
[] -> []
(x:xs) -> [(x,xs)]
/* Scala Code */
object Hutton1 {
type Parser[A] = String => List[(A, String)]
def Result[A](v: A): Parser[A] = str => List((v, str))
def Zero[A]: Parser[A] = str => List()
def Character: Parser[Char] = str => if (str.isEmpty) List() else List((str.head, str.tail))
}
object Hutton2 {
trait Parser[A] extends (String => List[(A, String)])
case class Result[A](v: A) extends Parser[A] {
def apply(str: String) = List((v, str))
}
case object Zero extends Parser[T forSome {type T}] {
def apply(str: String) = List()
}
case object Character extends Parser[Char] {
def apply(str: String) = if (str.isEmpty) List() else List((str.head, str.tail))
}
}
object Hutton extends App {
object T1 {
import Hutton1._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
object T2 {
import Hutton2._
def run = {
val r: List[(Int, String)] = Zero("test") ++ Result(5)("test")
println(r.map(x => x._1 + 1) == List(6))
println(Character("abc") == List(('a', "bc")))
}
}
T1.run
T2.run
}
Question 1
In Haskell, zero is a function value that can be used as it is, expessing all failed parsers whether they are of type Parser[Int] or Parser[String]. In scala we achieve the same by calling the function Zero (1st approach) but in this way I believe that I just generate a different function everytime Zero is called. Is this statement true? Is there a way to mitigate this?
Question 2
In the second approach, the Zero case object is extending Parser with the usage of existential types Parser[T forSome {type T}] . If I replace the type with Parser[_] I get the compile error
Error:(19, 28) class type required but Hutton2.Parser[_] found
case object Zero extends Parser[_] {
^
I thought these two expressions where equivalent. Is this the case?
Question 3
Which approach out of the two do you think that will yield better results in expressing the combinators in terms of elegance and simplicity?
I use scala 2.11.8
Note: I didn't compile it, but I know the problem and can propose two solutions.
The more Haskellish way would be to not use subtyping, but to define zero as a polymorphic value. In that style, I would propose to define parsers not as objects deriving from a function type, but as values of one case class:
final case class Parser[T](run: String => List[(T, String)])
def zero[T]: Parser[T] = Parser(...)
As shown by #Alec, yes, this will produce a new value every time, since a def is compiled to a method.
If you want to use subtyping, you need to make Parser covariant. Then you can give zero a bottom result type:
trait Parser[+A] extends (String => List[(A, String)])
case object Zero extends Parser[Nothing] {...}
These are in some way quite related; in system F_<:, which is the base of what Scala uses, the types _|_ (aka Nothing) and \/T <: Any. T behave the same (this hinted at in Types and Programming Languages, chapter 28). The two possibilities given here are a consequence of this fact.
With existentials I'm not so familiar with, but I think that while unbounded T forSome {type T} will behave like Nothing, Scala does not allow inhertance from an existential type.
Question 1
I think that you are right, and here is why: Zero1 below prints hello every time you use it. The solution, Zero2, involves using a val instead.
def Zero1[A]: Parser[A] = { println("hi"); str => List() }
val Zero2: Parser[Nothing] = str => List()
Question 2
No idea. I'm still just starting out with Scala. Hope someone answers this.
Question 3
The trait one will play better with Scala's for (since you can define custom flatMap and map), which turns out to be (somewhat) like Haskell's do. The following is all you need.
trait Parser[A] extends (String => List[(A, String)]) {
def flatMap[B](f: A => Parser[B]): Parser[B] = {
val p1 = this
new Parser[B] {
def apply(s1: String) = for {
(a,s2) <- p1(s1)
p2 = f(a)
(b,s3) <- p2(s2)
} yield (b,s3)
}
}
def map[B](f: A => B): Parser[B] = {
val p = this
new Parser[B] {
def apply(s1: String) = for ((a,s2) <- p(s1)) yield (f(a),s2)
}
}
}
Of course, to do anything interesting you need more parsers. I'll propose to you one simple parser combinator: Choice(p1: Parser[A], p2: Parser[A]): Parser[A] which tries both parsers. (And rewrite your existing parsers more to my style).
def choice[A](p1: Parser[A], p2: Parser[A]): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = { p1(s) ++ p2(s) }
}
def unit[A](x: A): Parser[A] = new Parser[A] {
def apply(s: String): List[(A,String)] = List((x,s))
}
val character: Parser[Char] = new Parser[Char] {
def apply(s: String): List[(Char,String)] = List((s.head,s.tail))
}
Then, you can write something like the following:
val parser: Parser[(Char,Char)] = for {
x <- choice(unit('x'),char)
y <- char
} yield (x,y)
And calling parser("xyz") gives you List((('x','x'),"yz"), (('x','y'),"z")).
It's been while Since I've started working on scala and I am wondering what kind of variable type is the best when I create a method which requires to return multiple data.
let's say If I have to make a method to get user info and it'll be called from many places.
def getUserParam(userId: String):Map[String,Any] = {
//do something
Map(
"isExist" -> true,
"userDataA" -> "String",
"userDataB" -> 1 // int
)
}
in this case, the result type is Map[String,Any] and since each param would be recognized as Any, You cannot pass the value to some other method requiring something spesifically.
def doSomething(foo: String){}
val foo = getUserParam("bar")
doSomething(foo("userDataA")) // type mismatch error
If I use Tuple, I can avoid that error, but I don't think it is easy to guess what each indexed number contains.
and of course there is a choice to use Case Class but once I use case class as a return type, I need to import the case class where ever I call the method.
What I want to ask is what is the best way to make a method returning more than 2 different variable type values.
Here are three options. Even though you might like the third option (using anonymous class) it's actually my least favorite. As you can see, it requires you to enable reflective calls (otherwise it throws a compilation warning). Scala will use reflection to achieve this which is not that great.
Personally, if there are only 2 values I use tuple. If there are more than two I will use a case class since it greatly improves code readability. The anonymous class option I knew it existed for a while, but I never used that it my code.
import java.util.Date
def returnTwoUsingTuple: (Date, String) = {
val date = new Date()
val str = "Hello world"
(date,str)
}
val tupleVer = returnTwoUsingTuple
println(tupleVer._1)
println(tupleVer._2)
case class Reply(date: Date, str: String)
def returnTwoUsingCaseClass: Reply = {
val date = new Date()
val str = "Hello world"
Reply(date,str)
}
val caseClassVer = returnTwoUsingCaseClass
println(caseClassVer.date)
println(caseClassVer.str)
import scala.language.reflectiveCalls
def returnTwoUsingAnonymousClass = {
val date = new Date()
val str = "Hello world"
new {
val getDate = date
val getStr = str
}
}
val anonClassVer = returnTwoUsingAnonymousClass
println(anonClassVer.getDate)
println(anonClassVer.getStr)
Sinse your logic with Map[String,Any] is more like for each key I have one of .. not for each key I have both ... more effective use in this case would be Either or even more effectively - scalaz.\/
scalaz.\/
import scalaz._
import scalaz.syntax.either._
def getUserParam(userId: String): Map[String, String \/ Int \/ Boolean] = {
//do something
Map(
"isExist" -> true.right,
"userDataA" -> "String".left.left,
"userDataB" -> 1.right.left
)
}
String \/ Int \/ Boolean is left-associatited to (String \/ Int) \/ Boolean
now you have
def doSomething(foo: String){}
unluckily it's the most complex case, if for example you had
def doSomethingB(foo: Boolean){}
you could've just
foo("userDataA").foreach(doSomethingB)
since the right value considered as correct so for String which is left to the left you could write
foo("userdata").swap.foreach(_.swap.foreach(doSomething))
Closed Family
Or you could craft you own simple type for large number of alternatives like
sealed trait Either3[+A, +B, +C] {
def ifFirst[T](action: A => T): Option[T] = None
def ifSecond[T](action: B => T): Option[T] = None
def ifThird[T](action: C => T): Option[T] = None
}
case class First[A](x: A) extends Either3[A, Nothing, Nothing] {
override def ifFirst[T](action: A => T): Option[T] = Some(action(x))
}
case class Second[A](x: A) extends Either3[Nothing, A, Nothing] {
override def ifSecond[T](action: A => T): Option[T] = Some(action(x))
}
case class Third[A](x: A) extends Either3[Nothing, Nothing, A] {
override def ifThird[T](action: A => T): Option[T] = Some(action(x))
}
now having
def getUserParam3(userId: String): Map[String, Either3[Boolean, String, Int]] = {
//do something
Map(
"isExist" -> First(true),
"userDataA" -> Second("String"),
"userDataB" -> Third(1)
)
}
val foo3 = getUserParam3("bar")
you can use your values as
foo3("userdata").ifSecond(doSomething)