I'm using Shapeless 2.0 and I'm trying to use HList to validate input — with as much of the checking as possible performed at compile time.
I have an HList spec that specifies what type of input I'm expecting (the types should be checked at compile time) and may also include a runtime check to be performed (e.g., to test if a number is even or odd).
Consider the following specification:
trait Pred[T] { def apply(t: T): Boolean }
val IsString = new Pred[String] { def apply(s: String) = true }
val IsOddNumber = new Pred[Int] { def apply(n: Int) = n % 2 != 0 }
val IsEvenNumber = new Pred[Int] { def apply(n: Int) = n % 2 == 0 }
val spec = IsEvenNumber :: IsString :: IsString :: IsOddNumber :: HNil
And various sample inputs:
val goodInput = 4 :: "foo" :: "" :: 5 :: HNil
val badInput = 4 :: "foo" :: "" :: 4 :: HNil
val malformedInput = 4 :: 5 :: "" :: 6 :: HNil
How would I make a function where I can effectively do:
input.zip(spec).forall{case (input, test) => test(input)}
So the following would happen:
f(spec, goodInput) // true
f(spec, badInput) // false
f(spec, malformedInput) // Does not compile
These answers by Travis Brown include most of what's needed:
https://stackoverflow.com/a/20452151/86485
https://stackoverflow.com/a/21005225/86485
but it took me a long time to find those answers, figure out that they were applicable to your problem, and work out the details of combining and applying them.
And I think your question adds value because it demonstrates how this can come up when solving a practical problem, namely validating input. I'll also try to add value below by showing a complete solution including demo code and tests.
Here's generic code for doing the checking:
object Checker {
import shapeless._, poly._, ops.hlist._
object check extends Poly1 {
implicit def apply[T] = at[(T, Pred[T])]{
case (t, pred) => pred(t)
}
}
def apply[L1 <: HList, L2 <: HList, N <: Nat, Z <: HList, M <: HList](input: L1, spec: L2)(
implicit zipper: Zip.Aux[L1 :: L2 :: HNil, Z],
mapper: Mapper.Aux[check.type, Z, M],
length1: Length.Aux[L1, N],
length2: Length.Aux[L2, N],
toList: ToList[M, Boolean]) =
input.zip(spec)
.map(check)
.toList
.forall(Predef.identity)
}
And here's the demo usage code:
object Frank {
import shapeless._, nat._
def main(args: Array[String]) {
val IsString = new Pred[String] { def apply(s: String) = true }
val IsOddNumber = new Pred[Int] { def apply(n: Int) = n % 2 != 0 }
val IsEvenNumber = new Pred[Int] { def apply(n: Int) = n % 2 == 0 }
val spec = IsEvenNumber :: IsString :: IsString :: IsOddNumber :: HNil
val goodInput = 4 :: "foo" :: "" :: 5 :: HNil
val badInput = 4 :: "foo" :: "" :: 4 :: HNil
val malformedInput1 = 4 :: 5 :: "" :: 6 :: HNil
val malformedInput2 = 4 :: "foo" :: "" :: HNil
val malformedInput3 = 4 :: "foo" :: "" :: 5 :: 6 :: HNil
println(Checker(goodInput, spec))
println(Checker(badInput, spec))
import shapeless.test.illTyped
illTyped("Checker(malformedInput1, spec)")
illTyped("Checker(malformedInput2, spec)")
illTyped("Checker(malformedInput3, spec)")
}
}
/*
results when run:
[info] Running Frank
true
false
*/
Note the use of illTyped to verify that code that should not compile, does not.
Some side notes:
I initially went down a long garden path with this, where I thought it would be important for the polymorphic function check to have a more specific type than Poly1, to represent that the return type in all cases is Boolean. So I kept trying to make it work with extends (Id ~>> Boolean). But it turns out not to matter whether the type system knows that the result type is the Boolean in every case. It's enough that the only case that we actually have has the right type. extends Poly1 is a marvelous thing.
Value-level zip traditionally allows unequal lengths and discards the extras. Miles followed suit in Shapeless's type-level zip, so we need a separate check for equal lengths.
It's a bit sad that the call site has to import nat._, otherwise the implicit instances for Length aren't found. One would prefer these details to be handled at the definition site. (A fix is pending.)
If I understand correctly, I can't use Mapped (a la https://stackoverflow.com/a/21005225/86485) to avoid the length check, because some of my checkers (e.g. IsString) have singleton types that are more specific than just e.g. Pred[String].
Travis points out that Pred could extend T => Boolean, making it possible to use ZipApply. I leave following this suggestion as an exercise for the reader :-)
Related
I have the following abstract class with the method :: and the inner class ListNode
abstract class SimpleList[+Elem] {
def head: Elem
def tail: SimpleList[Elem]
def isEmpty: Boolean
def ::[NewElem >: Elem] (newHead: NewElem): SimpleList[NewElem] = new ListNode(hd= newHead, tl= this)
private class ListNode[Elem](hd: Elem, tl: SimpleList[Elem]) extends SimpleList[Elem] {
override def head = hd
override def tail = tl
override def isEmpty = false
object SimpleList {
def main(args: Array[String]): Unit = {
val squareNumbers = 1 :: 4 :: 9 :: 16 :: Empty
}
}
The :: method uses 'this' to define a new ListNode.
As you can see in the main method, the :: method is invoked.
It seems like it creates a new ListNode from 1 and 4 etc.
However I dont understand what the 'this' keyword refernces in this case.
Does it reference the 4?
And what is a SimpleList in this defintion?
A custom data structure?
You have two questions and that is what is causing you confusion. The first thing is what does this means, well it means this object, the current instance where the method was called. Which makes sense, since prepending an element to a List is just create a new one where the passed element is the head and the current one is the tail.
Now the second question is how that works in the main, differently to most methods :: is called on the right-hand side and the argument passed is the left-hand side (this is specified on the language specs, every method that ends with colons : will behave like that).
So you have to see the code like this:
// The operations are applied in this order
1 :: (2 :: (3 :: (4 :: Empty)))
// And thus it is the same as
Empty.::(4).::(3).::(2).::(1)
// Which is easier to see like this
val res0 = Empty // Empty
val res1 = res0.::(4) // 4 :: Empty - (head = 4, tail / this = res0)
val res2 = res1.::(3) // 3 :: (4 :: Empty) - (head = 3, tail / this = res1)
val res3 = res2.::(2) // 2 :: (3 :: (4 :: Empty)) - (head = 2, tail / this = res2)
val res4 = res3.::(1) // 1 :: (2 :: (3 :: (4 :: Empty))) - (head = 1, tail / this = res3)
So you can see that when we create the final List we are creating the List whose head is 1 and its tail was 2 :: 3 :: 4 :: Empty because we called the :: method on that previous List.
For your final question, yes CustomList is a custom data structure, an abstract class, a new type, etc.
You should already understand that if you want to continue with more complex topics.
I have a polymorphic function which can turn lists into sets:
import shapeless.PolyDefns.~>
import shapeless._
val lists = List(1,2) :: List("A", "B") :: List(1.1, 2.2) :: HNil
object sss extends (List ~> Set) {
def apply[T](l:List[T]):Set[T] = {
l.toSet
}
}
lists.map(sss) // I want: Set(1,2) :: Set("A", "B") :: Set(1.1, 2.2) :: HNil
But what if I want to change the behavior of this function - I now want to add an extra argument which will specify which item in the input list should be put into the set. Here's an incorrect syntax - can you show me the correct way to do it?
object sss extends (List ~> Set) { // Compiler says no!
def apply[T](i:Int)(l:List[T]):Set[T] = {
l.slice(i,i+1).toSet
}
}
I think this is failing because the additional argument makes it no longer fit the signature of List ~> Set, so how can I overcome this?
There are a couple of workarounds for parametrizing a Poly, one of which is mentioned in the other answer, although the exact implementation there won't work. Instead you need to do this:
import shapeless._, shapeless.poly.~>
val lists = List(1, 2) :: List("A", "B") :: List(1.1, 2.2) :: HNil
class sss(i: Int) extends (List ~> Set) {
def apply[T](l: List[T]): Set[T] = l.slice(i, i+1).toSet
}
object sss1 extends sss(1)
lists.map(sss1)
…where the fact that sss1 is defined as an object (not a val) is necessary for the last line to compile.
That approach compiles, but it's not possible to use it in lots of contexts—e.g. you can't define your sss1 (or whatever) object in a method where the type of the hlist is generic.
Here's a slightly messier but more flexible workaround I've used before:
import shapeless._
val lists = List(1, 2) :: List("A", "B") :: List(1.1, 2.2) :: HNil
object sss extends Poly2 {
implicit def withI[T]: Case.Aux[List[T], Int, Set[T]] =
at((l, i) => l.slice(i, i + 1).toSet)
}
lists.zipWith(lists.mapConst(1))(sss)
// Set(2) :: Set(B) :: Set(2.2) :: HNil
Now you could actually write a method that took a generic L <: HList and the slice parameter i—you'd just need several implicit arguments to support the mapConst and zipWith applications.
Neither approach is very elegant, though, and I personally tend to avoid Poly most of the time—defining a custom type class is almost going to be cleaner, and in many cases required.
As you already pointed out, you cannot change the signature of the polymorphic function. But could create your function dynamically:
class sss(i: Int) extends (List ~> Set) {
def apply[T](l:List[T]): Set[T] = {
l.slice(i, i+1).toSet
}
}
val sss1 = new sss(1)
lists.map(sss1) // Set(2) :: Set(B) :: Set(2.2) :: HNil
I need to produce an extensible record given an HList of keys and a map of values, here's a MWE of what I'm trying to achieve (you can copy/paste this in any REPL with shapeless 2.0 available, in order to reproduce the issue)
import shapeless._; import syntax.singleton._; import record._
case class Foo[T](column: Symbol)
val cols = Foo[String]('column1) :: HNil
val values = Map("column1" -> "value1")
object toRecord extends Poly1 {
implicit def Foo[T] = at[Foo[T]] { foo =>
val k = foo.column.name
val v = values.get(k)
(k ->> v)
}
}
val r = cols.map(toRecord)
// r: shapeless.::[Option[String] with shapeless.record.KeyTag[k.type,Option[String]] forSome { val k: String },shapeless.HNil] = Some(value1) :: HNil
val value = r("column1")
// error: No field String("column1") in record shapeless.::[Option[String] with shapeless.record.KeyTag[k.type,Option[String]] forSome { val k: String },shapeless.HNil]
val value = r("column1")
If I try defining the record manually everything works as expected
val q = ("column1" ->> Some("value1")) :: HNil
// q: shapeless.::[Some[String] with shapeless.record.KeyTag[String("column1"),Some[String]],shapeless.HNil] = Some(value1) :: HNil
q("column1")
// Some[String] = Some(value1)
Clearly the difference is that in one case the KeyTag has type
KeyTag[String("column1"), Some[String]]
and in the (non-working) other
KeyTag[k.type,Option[String]] forSome { val k: String }
I sense the issue is with the string k not being statically known, but I have no clue on how to fix this.
Generally speaking, is there a way of dynamically generating an extensible record from a list of keys?
I fear the answer is to use a macro, but I'd be glad if another solution existed.
This isn't too bad if you can change your Foo definition a bit to allow it to keep track of the singleton type of the column key (note that I've removed the unused T type parameter):
import shapeless._; import syntax.singleton._; import record._
case class Foo[K <: Symbol](column: Witness.Aux[K])
val cols = Foo('column1) :: HNil
val values = Map("column1" -> "value1")
object toRecord extends Poly1 {
implicit def atFoo[K <: Symbol] = at[Foo[K]] { foo =>
field[K](values.get(foo.column.value.name))
}
}
val r = cols.map(toRecord)
And then:
scala> val value = r('column1)
value: Option[String] = Some(value1)
Note that I've changed your string key ("column1") to a symbol, since that's what we've put into the record.
I have the following definition on Scala
def seq(stms: Stm*): Stm = if (stms.isEmpty) EXP(CONST(0)) else stms reduce SEQ
I use it to write stms like:
ESEQ(
seq(MOVE(TEMP(r), CONST(1)),
genstm(t, f),
LABEL(f),
MOVE(TEMP(r), CONST(0)),
LABEL(t)),
TEMP(r))
But in some case i need that "the last element of the sequence to be a list" so, i write:
ESEQ (seq(
EXP(extenalCall("_newRecord", expressions.length)) ::
MOVE(rt, Frame.RV) ::
values:_*
), rt)
I whould like to be more homogeneous in how "seq" is called. I did method overloading:
def seq(stms: List[Stm]): Stm = if (stms.isEmpty) EXP(CONST(0)) else stms reduce SEQ
def seq(s1:Stm, l:List[Stm]) = seq(s1 :: l)
def seq(s1:Stm, s2:Stm, l:List[Stm]) = seq(s1 :: s2 :: l)
def seq(s1:Stm, s2:Stm, s3:Stm, l:List[Stm]) = seq(s1 :: s2 :: s3 :: l)
def seq(s1:Stm, s2:Stm, s3:Stm, s4:Stm, l:List[Stm]) = seq(s1 :: s2 :: s3 :: s4 :: l)
def seq(s1:Stm, s2:Stm, s3:Stm, s4:Stm, s5:Stm, l:List[Stm]) = seq(s1 :: s2 :: s3 :: s4 :: s5 :: l)
def seq(s:Stm*) = seq(s.toList)
In order to write the last snippet like this:
ESEQ (seq(
EXP(extenalCall("_newRecord", expressions.length)),
MOVE(rt, Frame.RV),
values
), rt)
There's a different approach for this?
The best idea that I can come up with is to define an implicit conversion from Stm to List[Stm], and then accept any number of List[Stm]:
implicit def stm2list(stm: Stm): List[Stm] = List(stm)
def seq(stms: List[Stm]*): Stm = {
val flat = stms.flatten
if (flat.isEmpty) EXP(CONST(0)) else stms reduce SEQ
}
But then it will also be valid for any of the first elements to be a List[Stm], not just the last one.
It also introduces a dangerous implicit conversion. You can protect it with a tiny class that will wrap the List[Stm] in a different type, then provide 2 implicit conversions:
class ListOfStmOrStm(val stms: List[Stm])
object ListOfStmOrStm {
implicit def fromStm(stm: Stm): ListOfStmOrStm = new ListOfStmOrStm(List(stm))
implicit def fromList(stms: List[Stm]): ListOfStmOrStm = new ListOfStmOrStm(stms)
}
def seq(stms: ListOfStmOrStm*): Stm = {
val flat = stms.flatMap(_.stms)
if (flat.isEmpty) EXP(CONST(0)) else stms reduce SEQ
}
Note: it is useless to make ListOfStmOrStm a value class, since it will have to be boxed to be stored in the Seq that comes with the * parameter.
You can use Seq() and ++ to keep on using commas:
ESEQ(seq(Seq(
x,
y,
z) ++ values: _*),
...)
Or if you can write seq() in such a way that
seq(xs: _*) ++ seq(ys: _*) == seq((xs ++ ys): _*)
then you could avoid writing Seq():
ESEQ(seq(
x,
y,
z) ++ seq(values: _*),
...)
Say I've got a function taking one argument
def fun(x: Int) = x
Based on that, I want to generate a new function with the same calling convention, but that'll apply some transformation to its arguments before delegating to the original function. For that, I could
def wrap_fun(f: (Int) => Int) = (x: Int) => f(x * 2)
wrap_fun(fun)(2) // 4
How might one go about doing the same thing, except to functions of any arity that only have the part of the arguments to apply the transformation to in common?
def fun1(x: Int, y: Int) = x
def fun2(x: Int, foo: Map[Int,Str], bar: Seq[Seq[Int]]) = x
wrap_fun(fun1)(2, 4) // 4
wrap_fun(fun2)(2, Map(), Seq()) // 4
How would a wrap_fun definition making the above invocations work look like?
This can be done in fairly straightforwardly using shapeless's facilities for abstracting over function arity,
import shapeless._
import HList._
import Functions._
def wrap_fun[F, T <: HList, R](f : F)
(implicit
hl : FnHListerAux[F, (Int :: T) => R],
unhl : FnUnHListerAux[(Int :: T) => R, F]) =
((x : Int :: T) => f.hlisted(x.head*2 :: x.tail)).unhlisted
val f1 = wrap_fun(fun _)
val f2 = wrap_fun(fun1 _)
val f3 = wrap_fun(fun2 _)
Sample REPL session,
scala> f1(2)
res0: Int = 4
scala> f2(2, 4)
res1: Int = 4
scala> f3(2, Map(), Seq())
res2: Int = 4
Note that you can't apply the wrapped function immediately (as in the question) rather than via an assigned val (as I've done above) because the explicit argument list of the wrapped function will be confused with the implicit argument list of wrap_fun. The closest we can get to the form in the question is to explicitly name the apply method as below,
scala> wrap_fun(fun _).apply(2)
res3: Int = 4
scala> wrap_fun(fun1 _).apply(2, 4)
res4: Int = 4
scala> wrap_fun(fun2 _).apply(2, Map(), Seq())
res5: Int = 4
Here the explicit mention of apply syntactically marks off the first application (of wrap_fun along with its implicit argument list) from the second application (of the transformed function with its explicit argument list).
As usual in Scala, there's yet another way to achieve what you want to do.
Here is a take based on currying of the first argument together with the compose of Function1:
def fun1(x : Int)(y : Int) = x
def fun2(x : Int)(foo : Map[Int, String], bar : Seq[Seq[Int]]) = x
def modify(x : Int) = 2*x
The resulting types as REPL shows you will be:
fun1: (x: Int)(y: Int)Int
fun2: (x: Int)(foo: Map[Int,String], bar: Seq[Seq[Int]])Int
modify: (x: Int)Int
And instead of wrapping the functions fun1 and fun2, you compose them, as technically, they are now both Function1 objects. This allows you to make calls like the following:
(fun1 _ compose modify)(2)(5)
(fun2 _ compose modify)(2)(Map(), Seq())
Both of which will return 4. Granted, the syntax is not that nice, given that you have to add the _ to distinguish fun1's application from the function object itself (on which you want to call the compose method in this case).
So Luigi's argument that it is impossible in general remains valid, but if you are free to curry your functions you can do it in this nice way.
Since functions taking different numbers of arguments are different, unrelated types, you cannot do this generically. trait Function1 [-T1, +R] extends AnyRef and nothing else. You will need a separate method for each arity.
While I voted for and agree with Luigi's answer–because, you know... he's right; Scala doesn't have direct, in-built support for such a thing–it's worth noting that what you're trying to do isn't impossible; it's just that it's a bit of a pain to pull off, and, often times, you're best off just implementing a separate method per desired arity.
That said, though... we can actually do this HLists. If you're interested in trying it out, naturally, you'll need to obtain an HList implementation. I recommend utilizing Miles Sabin's excellent shapeless project and its implementation of HLists. Anyway, here's an example of its use that accomplishes something akin to what you seem to be looking for:
import shapeless._
trait WrapperFunner[T] {
type Inputs <: HList
def wrapFun(inputs: Inputs) : T
}
class WrapsOne extends WrapperFunner[Int] {
type Inputs = Int :: HNil
def wrapFun(inputs: Inputs) : Int = {
inputs match {
case num :: HNil => num * 2
}
}
}
class WrapsThree extends WrapperFunner[String] {
type Inputs = Int :: Int :: String :: HNil
def wrapFun(inputs: Inputs) : String = {
inputs match {
case firstNum :: secondNum :: str :: HNil => str + (firstNum - secondNum)
}
}
}
object MyApp extends App {
val wo = new WrapsOne
println(wo.wrapFun(1 :: HNil))
println(wo.wrapFun(17 :: HNil))
//println(wo.wrapFun(18 :: 13 :: HNil)) // Would give type error
val wt = new WrapsThree
println(wt.wrapFun(5 :: 1 :: "your result is: " :: HNil))
val (first, second) = (60, 50)
println(wt.wrapFun(first :: second :: "%s minus %s is: ".format(first, second) :: HNil))
//println(wt.wrapFun(1 :: HNil)) // Would give type error
}
Running MyApp results in:
2
34
your result is: 4
60 minus 50 is: 10
Or, extended closer to your particular case:
import shapeless._
trait WrapperFunner[T] {
type Inputs <: HList
def wrapFun(inputs: Inputs) : T
}
trait WrapperFunnerBase extends WrapperFunner[Int] {
// Does not override `Inputs`
def wrapFun(inputs: Inputs) : Int = {
inputs match {
case (num: Int) :: remainder => num
}
}
}
class IgnoresNothing extends WrapperFunnerBase {
type Inputs = Int :: HNil
}
class IgnoresLastTwo extends WrapperFunnerBase {
type Inputs = Int :: Int :: String :: HNil
}
object MyApp extends App {
val in = new IgnoresNothing
println(in.wrapFun(1 :: HNil))
println(in.wrapFun(2 :: HNil))
//println(in.wrapFun(3 :: 4 :: HNil)) // Would give type error
val ilt = new IgnoresLastTwo
println(ilt.wrapFun(60 :: 13 :: "stupid string" :: HNil))
println(ilt.wrapFun(43 :: 7 :: "man, that string was stupid..." :: HNil))
//println(ilt.wrapFun(1 :: HNil)) // Would give type error
}
results in:
1
2
60
43