Related
I'm new to Scala and trying to understand the syntax the pattern matching constructs, specifically from examples in Unfiltered (http://unfiltered.databinder.net/Try+Unfiltered.html).
Here's a simple HTTP server that echos back Hello World! and 2 parts of the path if the path is 2 parts long:
package com.hello
import unfiltered.request.GET
import unfiltered.request.Path
import unfiltered.request.Seg
import unfiltered.response.ResponseString
object HelloWorld {
val sayhello = unfiltered.netty.cycle.Planify {
case GET(Path(Seg(p :: q :: Nil))) => {
ResponseString("Hello World! " + p + " " + q);
}
};
def main(args: Array[String]) {
unfiltered.netty.Http(10000).plan(sayhello).run();
}
}
Also for reference the source code for the Path, Seg, and GET/Method objects:
package unfiltered.request
object Path {
def unapply[T](req: HttpRequest[T]) = Some(req.uri.split('?')(0))
def apply[T](req: HttpRequest[T]) = req.uri.split('?')(0)
}
object Seg {
def unapply(path: String): Option[List[String]] = path.split("/").toList match {
case "" :: rest => Some(rest) // skip a leading slash
case all => Some(all)
}
}
class Method(method: String) {
def unapply[T](req: HttpRequest[T]) =
if (req.method.equalsIgnoreCase(method)) Some(req)
else None
}
object GET extends Method("GET")
I was able to break down how most of it works, but this line leaves me baffled:
case GET(Path(Seg(p :: q :: Nil))) => {
I understand the purpose of the code, but not how it gets applied. I'm very interested in learning the ins and outs of Scala rather than simply implementing an HTTP server with it, so I've been digging into this for a couple hours. I understand that it has something to do with extractors and the unapply method on the GET, Path, and Seg objects, I also knows that when I debug it hits unapply in GET before Path and Path before Seg.
I don't understand the following things:
Why can't I write GET.unapply(req), but I can write GET(req) or GET() and it will match any HTTP GET?
Why or how does the compiler know what values get passed to each extractor's unapply method? It seems that it will just chain them together unless one of them returns a None instead of an Some?
How does it bind the variables p and q? It knows they are Strings, it must infer that from the return type of Seg.unapply, but I don't understand the mechanism that assigns p the value of the first part of the list and q the value of the second part of the list.
Is there a way to rewrite it to make it more clear what's happening? When I first looked at this example, I was confused by the line
val sayhello = unfiltered.netty.cycle.Planify {, I dug around and rewrote it and found out that it was implicitly creating a PartialFunction and passing it to Planify.apply.
One way to understand it is to rewrite this expression the way that it gets rewritten by the Scala compiler.
unfiltered.netty.cycle.Planify expects a PartialFunction[HttpRequest[ReceivedMessage], ResponseFunction[NHttpResponse]], that is, a function that may or may not match the argument. If there's no match in either of the case statements, the request gets ignored. If there is a match -- which also has to pass all of the extractors -- the response will be returned.
Each case statement gets an instance of HttpRequest[ReceivedMessage]. Then, it applies it with left associativity through a series of unapply methods for each of the matchers:
// The request passed to us is HttpRequest[ReceivedMessage]
// GET.unapply only returns Some if the method is GET
GET.unapply(request) flatMap { getRequest =>
// this separates the path from the query
Path.unapply(getRequest) flatMap { path =>
// splits the path by "/"
Seg.unapply(path) flatMap { listOfParams =>
// Calls to unapply don't end here - now we build an
// instance of :: class, which
// since a :: b is the same as ::(a, b)
::.unapply(::(listOfParams.head, listOfParams.tail)) flatMap { case (p, restOfP) =>
::.unapply(::(restOfP.head, Nil)) map { case (q, _) =>
ResponseString("Hello World! " + p + " " + q)
}
}
}
}
}
Hopefully, this gives you an idea of how the matching works behind the scenes. I'm not entirely sure if I got the :: bit right - comments are welcome.
I've obviously done a very poor job of explaining what I'm looking for in my original post so let's try this one more time. What I'm trying to accomplish is the ability to pass a sequence of items, extract one or more of the items, and then pass the REMAINDER of the sequence on to another extractor. Note that by sequence I mean sequence (not necessarily a List). My previous examples used list as the sequence and I gave some examples of extraction using cons (::), but I could just as well pass an Array as my sequence.
I thought I knew how pattern matching and extraction worked but I could be wrong so to avoid any more basic comments and links to how to do pattern matching sites here's my understanding:
If I want to return a single item from my extractor I would define an unapply method. This method takes whatever type I chose as input (the type could be a sequence...) and returns a single optional item (the return type could itself be a sequence). The return must be wrapped in Some if I want a match or None if I don't. Here is an example that takes a sequence as input and returns the same sequence wrapped in Some but only if it contains all Strings. I could very well just return the sequence wrapped in Some and not do anything else, but this seems to cause confusion for people. The key is if it is wrapped in Some then it will match and if it is None it will not. Just to be more clear, the match will also not happen unless the input also matches my unapply methods input type. Here is my example:
object Test {
// In my original post I just returned the Seq itself just to verify I
// had matched but many people commented they didn't understand what I
// was trying to do so I've made it a bit more complicated (e.g. match
// only if the sequence is a sequence of Strings). Hopefully I don't
// screw this up and introduce a bug :)
def unapply[A](xs: Seq[A]): Option[Seq[String]] =
if (xs forall { _.isInstanceOf[String] })
Some(xs.asInstanceOf[Seq[String]])
else
None
}
Using List as an example, I can now perform the following:
// This works
def test1(xs: List[_]) = xs match {
case (s: String) :: Test(rest) =>
println("s = " + s + ", rest = " + rest)
case _ =>
println("no match")
}
test1(List("foo", "bar", "baz")) // "s = foo, rest = List(bar, baz)"
My test1 function takes List as input and extracts the head and tail using cons via the constructor pattern (e.g. ::(s, rest)). It then uses type ascription (: String) to make sure the head (s) is a String. The tail contains List("bar", "baz"). This is a List which means it is also a Seq (sequence). It is then passed as input to my Test extractor which verifies that both "bar" and "baz" are strings and returns the List wrapped in Some. Since Some is returned it is considered a match (although in my original post where I inadvertently mixed up unapplySeq with unapply this didn't work as expected, but that aside...). This is NOT what I'm looking for. This was only an example to show that Test does in fact extract a Seq as input as expected.
Now, here's where I caused mass confusion last time when I inadvertently used unapplySeq instead of unapply in my write up. After much confusion trying to understand the comments that were posted I finally picked up on the mistake. Many thanks to Dan for pointing me in the right direction...
But just be avoid any more confusion, let me clarify my understanding of unapplySeq. Like unapply, unapplySeq takes in whatever argument I choose as input, but instead of returning a single element it returns a sequence of elements. Each item in this sequence can then be used for additional pattern matching. Again, to make a match happen the input type must match and my returned sequence must be wrapped in Some and not be None. When extracting over the sequence of items returned from unapplySeq, you can use _* to match any remaining items not yet matched.
Ok, so my extractor takes a sequence as input and returns a sequence (as a single item) in return. Since I only want to return a single item as a match I need to use unapply NOT unapplySeq. Even though in my case I'm returning a Seq, I don't want unapplySeq because I don't want to do more pattern matching on the items in the Seq. I just want to return the items as a Seq on its own to then be passed to the body of my case match. This sounds confusing, but to those that understand unapply vs unapplySeq I hope it isn't.
So here is what I WANT to do. I want to take something that returns a sequence (e.g. List or Array) and I want to extract a few items from this sequence and then extract the REMAINDER of the items (e.g. _*) as a sequence. Let's call it the remainder sequence. I want to then pass the remainder sequence as input to my extractor. My extractor will then return the remaining items as a single Seq if it matches my criteria. Just to be 100% clear. The List (or Array, etc) will have its unapplySeq extractor called to create the sequence of items. I will extract a one or more of these items and then pass what is left as a sequence to my Test extractor which will use unapply (NOT unapplySeq) to return the remainder. If you are confused by this, then please don't comment...
Here are my tests:
// Doesn't compile. Is there a syntax for this?
def test2(xs: Seq[_]) = xs match {
// Variations tried:
// Test(rest) # _* - doesn't compile (this one seems reasonable to me)
// Test(rest # _*) - doesn't compile (would compile if Test had
// unapplySeq, but in that case would bind List's
// second element to Test as a Seq and then bind
// rest to that Seq (if all strings) - not what I'm
// looking for...). I though that this might work
// since Scala knows Test has no unapplySeq only
// unapply so # _* can be tied to the List not Test
// rest # Test(_*) - doesn't compile (didn't expect to)
case List(s: String, Test(rest) # _*) =>
println("s = " + s + " rest = " + rest)
case _ =>
println("no match")
}
// This works, but messy
def test3(xs: List[_]) = xs match {
case List(s: String, rest # _*) if (
rest match { case Test(rest) => true; case _ => false }
) =>
println("s = " + s + " rest = " + rest)
case _ =>
println("no match")
}
I created test3 based on comments from Julian (thanks Julian..). Some have commented that test3 does what I want so they are confused what I'm looking for. Yes, it accomplishes what I want to accomplish, but I'm not satisfied with it. Daniel's example also works (thanks Daniel), but I'm also not satisfied with having to create another extractor to split things and then do embedded extractions. These solutions seem too much work in order to accomplish something that seems fairly straight forward to me. What I WANT is to make test2 work or know that it can't be done this way. Is the error given because the syntax is wrong? I know that rest # _* will return a Seq, that can be verified here:
def test4(xs: List[_]) = xs match {
case List(s: String, rest # _*) =>
println(rest.getClass) // scala.collection.immutable.$colon$colon
case _ =>
println("no match")
}
It returns cons (::) which is a List which is a Seq. So how can I pass the _* Seq on to my extractor and have is return bound to the variable rest?
Note that I've also tried passing varargs to my unapply constructor (e.g. unapply(xs: A*)...) but that won't match either.
So, I hope it is clear now when I say I want to extract the remainder of a sequence in pattern matching. I'm not sure how else I can word it.
Based on the great feedback from Daniel I'm hoping he is going to have an answer for me :)
I'd like to extract the first item and pass the remainder on to another extractor.
OK. Your test1 does that, exactly. first_item :: Extractor(the_rest). The weird behavior you're seeing comes from your Test extractor. As you already had the answer to your stated question, and as expected behavior from your Test strikes you as a problem with test1, it seems that what you really want is some help with extractors.
So, please read Extractor Objects, from docs.scala-lang.org, and Pattern Matching in Scala (pdf). Although that PDF has an example of unapplySeq, and suggests where you'd want to use it, here are some extra examples:
object Sorted {
def unapply(xs: Seq[Int]) =
if (xs == xs.sortWith(_ < _)) Some(xs) else None
}
object SortedSeq {
def unapplySeq(xs: Seq[Int]) =
if (xs == xs.sortWith(_ < _)) Some(xs) else None
}
Interactively:
scala> List(1,2,3,4) match { case Sorted(xs) => Some(xs); case _ => None }
res0: Option[Seq[Int]] = Some(List(1, 2, 3, 4))
scala> List(4,1,2,3) match { case Sorted(xs) => Some(xs); case _ => None }
res1: Option[Seq[Int]] = None
scala> List(4,1,2,3) match { case first :: Sorted(rest) => Some(first, rest); case _ => None }
res2: Option[(Int, Seq[Int])] = Some((4,List(1, 2, 3)))
scala> List(1,2,3,4) match { case SortedSeq(a,b,c,d) => (a,b,c,d) }
res3: (Int, Int, Int, Int) = (1,2,3,4)
scala> List(4,1,2,3) match { case _ :: SortedSeq(a, b, _*) => (a,b) }
res4: (Int, Int) = (1,2)
scala> List(1,2,3,4) match { case SortedSeq(a, rest # _*) => (a, rest) }
res5: (Int, Seq[Int]) = (1,List(2, 3, 4))
Or maybe -- I only have the faint suspicion of this, you haven't said as much -- you don't want extractor help, but actually you want a terse way to express something like
scala> List(1,2,3,4) match { case 1 :: xs if (xs match { case Sorted(_) => true; case _ => false }) => xs }
res6: List[Int] = List(2, 3, 4)
Erlang has a feature like this (although, without these crazy extractors):
example(L=[1|_]) -> examine(L).
, which pattern-matches the same argument twice - to L and also to [1|_]. In Erlang both sides of the = are full-fledged patterns and could be anything, and you can add a third or more patterns with more =. Scala seems to only support the L=[1|_] form, having a variable and then a full pattern.
scala> List(4,1,2,3) match { case xs # _ :: Sorted(_) => xs }
collection.immutable.::[Int] = List(4, 1, 2, 3)
Well, the easiest way is this:
case (s: String) :: Test(rest # _*) =>
If you need this to work on general Seq, you can just define an extractor to split head from tail:
object Split {
def unapply[T](xs: Seq[T]): Option[(T, Seq[T])] = if (xs.nonEmpty) Some(xs.head -> xs.tail) else None
}
And then use it like
case Split(s: String, Test(rest # _*)) =>
Also note that if you had defined unapply instead of unapplySeq, then # _* would not be required on the pattern matched by Test.
:: is an extractor. For how it works (from a random googling), see, for example, here.
def test1(xs: List[_]) = xs match {
case s :: rest =>
println("s = " + s + " rest = " + rest)
case _ =>
println("no match")
}
scala> test1(List("a", "b", "c"))
s = a rest = List(b, c)
I think this is what you wanted?
Messing around with this, it seems that the issue has something to do with unapplySeq.
object Test {
def unapply[A](xs: List[A]): Option[List[A]] = Some(xs)
}
def test1(xs: List[_]) = xs match {
case (s: String) :: Test(s2 :: rest) =>
println("s = " + s + " rest = " + rest)
case _ =>
println("no match")
}
test1(List("foo", "bar", "baz"))
produces the output:
s = foo rest = List(baz)
I'm havng trouble googling up docs on the difference between unapply and unapplySeq.
Is it there an equivalent of Pythons repr function in scala?
Ie a function which you can give any scala object an it will produce a string representation of the object which is valid scala code.
eg:
val l = List(Map(1 -> "a"))
print(repr(l))
Would produce
List(Map(1 -> "a"))
There is mostly only the toString method on every object. (Inherited from Java.) This may or may not result in a parseable representation. In most generic cases it probably won’t; there is no real convention for this as there is in Python but some of the collection classes at least try to. (As long as they are not infinite.)
The point where it breaks down is of course already reached when Strings are involved
"some string".toString == "some string"
however, for a proper representation, one would need
repr("some string") == "\"some string\""
As far as I know there is no such thing in Scala. Some of the serialisation libraries might be of some help for this, though.
Based on the logic at Java equivalent of Python repr()?, I wrote this little function:
object Util {
def repr(s: String): String = {
if (s == null) "null"
else s.toList.map {
case '\0' => "\\0"
case '\t' => "\\t"
case '\n' => "\\n"
case '\r' => "\\r"
case '\"' => "\\\""
case '\\' => "\\\\"
case ch if (' ' <= ch && ch <= '\u007e') => ch.toString
case ch => {
val hex = Integer.toHexString(ch.toInt)
"\\u%s%s".format("0" * (4 - hex.length), hex)
}
}.mkString("\"", "", "\"")
}
}
I've tried it with a few values and it seems to work, though I'm pretty sure sticking in a Unicode character above U+FFFF would cause problems.
If you deal with case classes, you can mix in the following trait StringMaker, so that calling toString on such case classes will work even if their arguments are strings:
trait StringMaker {
override def toString = {
this.getClass.getName + "(" +
this.getClass.getDeclaredFields.map{
field =>
field.setAccessible(true)
val name = field.getName
val value = field.get(this)
value match {
case s: String => "\"" + value + "\"" //Or Util.repr(value) see the other answer
case _ => value.toString
}
}
.reduceLeft{_+", "+_} +
")"
}
}
trait Expression
case class EString(value: String, i: Int) extends Expression with StringMaker
case class EStringBad(value: String, i: Int) extends Expression //w/o StringMaker
val c_good = EString("641", 151)
val c_bad = EStringBad("641", 151)
will result in:
c_good: EString = EString("641", 151)
c_bad: EStringBad = EStringBad(641,151)
So you can parse back the firsst expression, but not the first one.
No, there is no such feature in Scala.
I found myself writing something like this quite often:
a match {
case `b` => // do stuff
case _ => // do nothing
}
Is there a shorter way to check if some value matches a pattern? I mean, in this case I could just write if (a == b) // do stuff, but what if the pattern is more complex? Like when matching against a list or any pattern of arbitrary complexity. I'd like to be able to write something like this:
if (a matches b) // do stuff
I'm relatively new to Scala, so please pardon, if I'm missing something big :)
This is exactly why I wrote these functions, which are apparently impressively obscure since nobody has mentioned them.
scala> import PartialFunction._
import PartialFunction._
scala> cond("abc") { case "def" => true }
res0: Boolean = false
scala> condOpt("abc") { case x if x.length == 3 => x + x }
res1: Option[java.lang.String] = Some(abcabc)
scala> condOpt("abc") { case x if x.length == 4 => x + x }
res2: Option[java.lang.String] = None
The match operator in Scala is most powerful when used in functional style. This means, rather than "doing something" in the case statements, you would return a useful value. Here is an example for an imperative style:
var value:Int = 23
val command:String = ... // we get this from somewhere
command match {
case "duplicate" => value = value * 2
case "negate" => value = -value
case "increment" => value = value + 1
// etc.
case _ => // do nothing
}
println("Result: " + value)
It is very understandable that the "do nothing" above hurts a little, because it seems superflous. However, this is due to the fact that the above is written in imperative style. While constructs like these may sometimes be necessary, in many cases you can refactor your code to functional style:
val value:Int = 23
val command:String = ... // we get this from somewhere
val result:Int = command match {
case "duplicate" => value * 2
case "negate" => -value
case "increment" => value + 1
// etc.
case _ => value
}
println("Result: " + result)
In this case, you use the whole match statement as a value that you can, for example, assign to a variable. And it is also much more obvious that the match statement must return a value in any case; if the last case would be missing, the compiler could not just make something up.
It is a question of taste, but some developers consider this style to be more transparent and easier to handle in more real-world examples. I would bet that the inventors of the Scala programming language had a more functional use in mind for match, and indeed the if statement makes more sense if you only need to decide whether or not a certain action needs to be taken. (On the other hand, you can also use if in the functional way, because it also has a return value...)
This might help:
class Matches(m: Any) {
def matches[R](f: PartialFunction[Any, R]) { if (f.isDefinedAt(m)) f(m) }
}
implicit def any2matches(m: Any) = new Matches(m)
scala> 'c' matches { case x: Int => println("Int") }
scala> 2 matches { case x: Int => println("Int") }
Int
Now, some explanation on the general nature of the problem.
Where may a match happen?
There are three places where pattern matching might happen: val, case and for. The rules for them are:
// throws an exception if it fails
val pattern = value
// filters for pattern, but pattern cannot be "identifier: Type",
// though that can be replaced by "id1 # (id2: Type)" for the same effect
for (pattern <- object providing map/flatMap/filter/withFilter/foreach) ...
// throws an exception if none of the cases match
value match { case ... => ... }
There is, however, another situation where case might appear, which is function and partial function literals. For example:
val f: Any => Unit = { case i: Int => println(i) }
val pf: PartialFunction[Any, Unit] = { case i: Int => println(i) }
Both functions and partial functions will throw an exception if called with an argument that doesn't match any of the case statements. However, partial functions also provide a method called isDefinedAt which can test whether a match can be made or not, as well as a method called lift, which will turn a PartialFunction[T, R] into a Function[T, Option[R]], which means non-matching values will result in None instead of throwing an exception.
What is a match?
A match is a combination of many different tests:
// assign anything to x
case x
// only accepts values of type X
case x: X
// only accepts values matches by pattern
case x # pattern
// only accepts a value equal to the value X (upper case here makes a difference)
case X
// only accepts a value equal to the value of x
case `x`
// only accept a tuple of the same arity
case (x, y, ..., z)
// only accepts if extractor(value) returns true of Some(Seq()) (some empty sequence)
case extractor()
// only accepts if extractor(value) returns Some something
case extractor(x)
// only accepts if extractor(value) returns Some Seq or Tuple of the same arity
case extractor(x, y, ..., z)
// only accepts if extractor(value) returns Some Tuple2 or Some Seq with arity 2
case x extractor y
// accepts if any of the patterns is accepted (patterns may not contain assignable identifiers)
case x | y | ... | z
Now, extractors are the methods unapply or unapplySeq, the first returning Boolean or Option[T], and the second returning Option[Seq[T]], where None means no match is made, and Some(result) will try to match result as described above.
So there are all kinds of syntactic alternatives here, which just aren't possible without the use of one of the three constructions where pattern matches may happen. You may able to emulate some of the features, like value equality and extractors, but not all of them.
Patterns can also be used in for expressions. Your code sample
a match {
case b => // do stuff
case _ => // do nothing
}
can then be expressed as
for(b <- Some(a)) //do stuff
The trick is to wrap a to make it a valid enumerator. E.g. List(a) would also work, but I think Some(a) is closest to your intended meaning.
The best I can come up with is this:
def matches[A](a:A)(f:PartialFunction[A, Unit]) = f.isDefinedAt(a)
if (matches(a){case ... =>}) {
//do stuff
}
This won't win you any style points though.
Kim's answer can be “improved” to better match your requirement:
class AnyWrapper[A](wrapped: A) {
def matches(f: PartialFunction[A, Unit]) = f.isDefinedAt(wrapped)
}
implicit def any2wrapper[A](wrapped: A) = new AnyWrapper(wrapped)
then:
val a = "a" :: Nil
if (a matches { case "a" :: Nil => }) {
println("match")
}
I wouldn't do it, however. The => }) { sequence is really ugly here, and the whole code looks much less clear than a normal match. Plus, you get the compile-time overhead of looking up the implicit conversion, and the run-time overhead of wrapping the match in a PartialFunction (not counting the conflicts you could get with other, already defined matches methods, like the one in String).
To look a little bit better (and be less verbose), you could add this def to AnyWrapper:
def ifMatch(f: PartialFunction[A, Unit]): Unit = if (f.isDefinedAt(wrapped)) f(wrapped)
and use it like this:
a ifMatch { case "a" :: Nil => println("match") }
which saves you your case _ => line, but requires double braces if you want a block instead of a single statement... Not so nice.
Note that this construct is not really in the spirit of functional programming, as it can only be used to execute something that has side effects. We can't easily use it to return a value (therefore the Unit return value), as the function is partial — we'd need a default value, or we could return an Option instance. But here again, we would probably unwrap it with a match, so we'd gain nothing.
Frankly, you're better off getting used to seeing and using those match frequently, and moving away from this kind of imperative-style constructs (following Madoc's nice explanation).
Does scala have an equivalent to scheme's cond?
I guess you're looking for match (or just simply if/else if/else).
case class Paired(x: Int, y: Int)
def foo(x: Any) = x match {
case string : String => println("Got a string")
case num : Int if num < 100 => println("Number less than 100")
case Paired(x,y) => println("Got x and y: " + x + ", " + y)
case unknown => println("??: " + unknown)
}
The first two case statements show type based pattern matching. The third shows the use of an Extractor to break data down into constituent parts and to assign those parts to variables. The third shows a variable pattern match which will match anything. Not shown is the _ case:
case _ => println("what")
Which like the variable pattern match, matches anything, but does not bind the matched object to a variable.
The case class at the top is Scala shorthand for creating an extractor as well as the class itself.
Of course, neither match nor if does exactly the same thing as cond. One possibility is to do like this:
object Cond {
def apply(clauses: Iterable[(()=>Boolean, ()=>Any)]): Any = {
clauses find (_._1()) map (_._2()) getOrElse ()
}
}
This object accepts something Iterable where each item is a pair of a function returning Boolean and a function returning Any. It tries to find an item whose first function returns true, stops looking if it finds one, calls the second function on a found item and returns the result of that function (or () if none was found).
Examples:
val clauses = Seq(
({()=>false}, {()=>println("foo")}),
({()=>true}, {()=>println("bar")})
)
Cond(clauses)
def checkYear(year: Int) = {
Cond(Seq(
({()=>year % 400 == 0}, {()=>42}),
({()=>true}, {()=>{c:Char => (c.toString * 3)}})
))
}
ETA: Yes, I know it is ugly, but it works.
The most straightforward translation is to use pattern guards, although it requires some boilerplate. Pattern guards only work in a case pattern, and case only works in a match (unless we're writing a PartialFunction).
We can satisfy these conditions by matching a unit value against trivial cases:
;; Scheme
(cond
(foo bar)
(baz quux)
(t mydefault))
// Scala
() match {
case _ if foo => bar
case _ if baz => quux
case _ => mydefault
}