Below is an implementation of a function that returns the lexographically next permutation. This is useful in one of the Euler problems.
It's written to work on Strings (which I needed for that). However, it should work on any indexed sequence of comparable values. I've tried generalising it by changing the two occurrences of String to IndexedSeq[Char], but this gets an error:
euler-lib.scala:26: error: type mismatch;
found : IndexedSeq[Char]
required: String
((n.slice(pivot+1, successor):+ n(pivot)) + n.drop(successor+1)).reverse
^
Why has the type inferencer inferred String there? I don't seem to have done any operation that requires a String?
And can I make it more general still by having IndexedSeq["something-comparable"]? I've not been able to make this work.
// return the lexographically next permutation to the one passed as a parameter
// pseudo-code from an article on StackOverflow
def nextPermutation(n:String):String = {
// 1. scan the array from right-to-left
//1.1. if the current element is less than its right-hand neighbor,
// call the current element the pivot,
// and stop scanning
// (We scan left-to-right and return the last such).
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
//1.2. if the left end is reached without finding a pivot,
// reverse the array and return
// (the permutation was the lexicographically last, so its time to start over)
if (pivot < 0) return n.reverse
//2. scan the array from right-to-left again,
// to find the rightmost element larger than the pivot
// (call that one the successor)
val successor = n.lastIndexWhere{_ > n(pivot)}
//3. swap the pivot and the successor, and
//4. reverse the portion of the array to the right of where the pivot was found
return (n.take(pivot) :+ n(successor)) +
((n.slice(pivot+1, successor):+ n(pivot)) + n.drop(successor+1)).reverse
}
The method + in IndexedSeq is used to produce a new sequence containing one additional given element but you want to produce one containing an additional sequence. The method for this is ++ thus your last line must look like this:
(n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
You are seeing this strange compiler message about a String being expected because +'s signature does not match and thus an explicit conversion used for String concatenation kicks in (this conversion is there because it lets you write something like List(8) + " Test").
EDIT: Generalization over sequence types of ordered elements:
As I said in the comments, generalization over sequences is a bit more complicated. In addition to your element type A you will need another type CC[X] <: SeqLike[X,CC[X]] that represents the sequence. Normally C <: SeqLike[A,C] would be sufficient but the type inferencer does not like that one (you would always need to pass the types of A and C when calling that method).
If you just change your signature that way the compiler will complain that it requires an implicit CanBuildFrom[CC[A],A,CC[A]] parameter as that one is needed e.g. by the reverse method. That parameter is used to build one sequence type from another one - just search the site to see some examples of how it is used by the collections API.
The final result would look like this:
import collection.SeqLike
import collection.generic.CanBuildFrom
def nextPermutation[A, CC[X] <: SeqLike[X,CC[X]]](n: CC[A])(
implicit ord: Ordering[A], bf: CanBuildFrom[CC[A],A,CC[A]]): CC[A] = {
import ord._
// call toSeq to avoid having to require an implicit CanBuildFrom for (A,A)
val pivot = n.toSeq.zip(n.tail.toSeq).lastIndexWhere{
case (first, second) => first < second
}
if (pivot < 0) {
n.reverse
}
else {
val successor = n.lastIndexWhere{_ > n(pivot)}
(n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
}
This way you get a Vector[Int] if you passed one to the method and a List[Double] if you passed that to the method. So what about Strings? Those are not actual sequences but they can be implicitly converted into a Seq[Char]. It is possible alter the definition of that method expect some type that can be implicitly converted into a Seq[A] but then again type inference would not work reliably - or at least I could not make it work reliably. As a simple workaround you could define an additional method for Strings:
def nextPermutation(s: String): String =
nextPermutation[Char,Seq](s.toSeq).mkString
Little tip here:
n(pivot)) + n.drop(successor+1)
^
When you get a type mismatch error, and the ^ points to the first parenthesis of the last argument list (ie, it would point to the second ( in x.foldLeft(y)(z)), that means the value returned by that method has the wrong type.
Or, in this case, n.drop(sucessor+1) has type IndexedSeq[Char], but the + method expects a String.
Another little tip: the only things that accept + are the numeric classes and String. If you try to add things and get an error, most likely it is Scala thinking you are using + to add Strings. For example:
true + true // expected String, got Boolean error
"true" + true // works, the second true is converted to String
true + "true" // works, the first true is converted to String
So, avoid + unless you are working with numbers or strings.
So, about making that general...
def nextPermutation[A <% Ordered[A]](n: IndexedSeq[A]): IndexedSeq[A] = {
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
if (pivot < 0) return n.reverse
val successor = n.lastIndexWhere{_ > n(pivot)}
return (n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
The easy part is just declaring IndexedSeq. But you have to parameterize on A, and there must be a way to order A so that you can compare the elements (<% means there's an implicit conversion from A to an Ordered[A] available). Another way to declare it would be like this:
def nextPermutation[A : Ordering](n: IndexedSeq[A]): IndexedSeq[A] = {
val ordering = implicitly[Ordering[A]]; import ordering._
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
if (pivot < 0) return n.reverse
val successor = n.lastIndexWhere{_ > n(pivot)}
return (n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
Here, A : Ordering means there is an implicit Ordering[A] available, which is then obtained and imported into scope, so that it can offer implicit conversions to make < work. The difference between an Ordered[A] and an Ordering[A] can be found on other questions.
Problem 24 had me stumped for a while:
println("0123456789".permutations.drop(1000000 - 1).next);
The code compiles correclty for me in Scala 2.8.0. Which version of Scala are you using ?
scala> nextPermutation("12354")
res0: String = 12435
Related
I written the code below for finding even numbers and the number just before it in a RDD object. In this I first converted that to a List and tried to use my own function to find the even numbers and the numbers just before them. The following is my code. In this I have made an empty list in which I am trying to append the numbers one by one.
object EvenandOdd
{
def mydef(nums:Iterator[Int]):Iterator[Int]=
{
val mylist=nums.toList
val len= mylist.size
var elist=List()
var i:Int=0
var flag=0
while(flag!=1)
{
if(mylist(i)%2==0)
{
elist.++=List(mylist(i))
elist.++=List(mylist(i-1))
}
if(i==len-1)
{
flag=1
}
i=i+1
}
}
def main(args:Array[String])
{
val myrdd=sc.parallelize(List(1,2,3,4,5,6,7,8,9,10),2)
val myx=myrdd.mapPartitions(mydef)
myx.collect
}
}
I am not able to execute this command in Scala shell as well as in Eclipse and not able to figure out the error as I am just a beginner to Scala.
The following are the errors I got in Scala Shell.
<console>:35: error: value ++= is not a member of List[Nothing]
elist.++=List(mylist(i))
^
<console>:36: error: value ++= is not a member of List[Nothing]
elist.++=List(mylist(i-1))
^
<console>:31: error: type mismatch;
found : Unit
required: Iterator[Int]
while(flag!=1)
^
Your code looks too complicated and not functional. Also, it introduce potential problems with memory: you take Iterator as param and return Iterator as output. So, knowing that Iterator itself could be lazy and has under the hood huge amount of data, materializing it inside method with list could cause OOM. So your task is to get as much data from initial iterator as it it enough to answer two methods for new Iterator: hasNext and next
For example (based on your implementation, which outputs duplicates in case of sequence of even numbers) it could be:
def mydef(nums:Iterator[Int]): Iterator[Int] = {
var before: Option[Int] = None
val helperIterator = new Iterator[(Option[Int], Int)] {
override def hasNext: Boolean = nums.hasNext
override def next(): (Option[Int], Int) = {
val result = (before, nums.next())
before = Some(result._2)
result
}
}
helperIterator.withFilter(_._2 % 2 == 0).flatMap{
case (None, next) => Iterator(next)
case (Some(prev), next) => Iterator(prev, next)
}
}
Here you have two iterators. One helper, which just prepare data, providing previous element for each next. And next on - resulting, based on helper, which filter only even for sequence elements (second in pair), and output both when required (or just one, if first element in sequence is even)
For initial code
Additionally to answer of #pedrorijo91, in initial code you do did not also return anything (suppose you wanted to convert elist to Iterator)
It will be easier if you use a functional coding style rather than an iterative coding style. In functional style the basic operation is straightforward.
Given a list of numbers, the following code will find all the even numbers and the values that precede them:
nums.sliding(2,1).filter(_(1) % 2 == 0)
The sliding operation creates a list containing all possible pairs of adjacent values in the original list.
The filter operation takes only those pairs where the second value is even.
The result is an Iterator[List[Int]] where each List[Int] has two elements. You should be able to use this in your RDD framework.
It's marked part of the developer API, so there's no guarantee it'll stick around, but the RDDFunctions object actually defines sliding for RDDs. You will have to make sure it sees elements in the order you want.
But this becomes something like
rdd.sliding(2).filter(x => x(1) % 2 == 0) # pairs of (preceding number, even number)
for the first 2 errors:
there's no ++= operator on Lists. You will have to do list = list ++ element
I have this two functions
def pattern(s: String): Option[Pattern] =
try {
Some(Pattern.compile(s))
} catch {
case e: PatternSyntaxException => None
}
and
def mkMatcher(pat: String): Option[String => Boolean] =
pattern(pat) map (p => (s: String) => p.matcher(s).matches)
Map is the higher-order function that applies a given function to each element of a list.
Now I am not getting that how map is working here as per above statement.
Map is the higher-order function that applies a given function to each element of a list.
This is an uncommonly restrictive definition of map.
At any rate, it works because it was defined by someone who did not hold to that.
For example, that someone wrote something akin to
sealed trait Option[+A] {
def map[B](f: A => B): Option[B] = this match {
case Some(value) => Some(f(value))
case None => None
}
}
as part of the standard library. This makes map applicable to Option[A]
It was defined because it makes sense to map many kinds of data structures not just lists.
Mapping is a transformation applied to the elements held by the data structure.
It applies a function to each element.
Option[A] can be thought of as a trivial sequence. It either has zero or one elements. To map it means to apply the function on its element if it has one.
Now it may not make much sense to use this facility all of the time, but there are cases where it is useful.
For example, it is one of a few distinct methods that, when present enable enable For Expressions to operate on a type. Option[A] can be used in for expressions which can be convenient.
For example
val option: Option[Int] = Some(2)
val squared: Option[Int] = for {
n <- option
if n % 2 == 0
} yield n * n
Interestingly, this implies that filter is also defined on Option[A].
If you just have a simple value it may well be clearer to use a less general construct.
Map is working the same way that it does with other collections types like List and Vector. It applies your function to the contents of the collection, potentially changing the type but keeping the collection type the same.
In many cases you can treat an Option just like a collection with either 0 or 1 elements. You can do a lot of the same operations on Option that you can on other collections.
You can modify the value
var opt = Option(1)
opt.map(_ + 3)
opt.map(_ * math.Pi)
opt.filter(_ == 1)
opt.collect({case i if i > 0 => i.toString })
opt.foreach(println)
and you can test the value
opt.contains(3)
opt.forall(_ > 0)
opt.exists(_ > 0)
opt.isEmpty
Using these methods you rarely need to use a match statement to unpick an Option.
I'm confused over when a select is appropriate and when an apply is. I thought apply was a function or method application, thus
scala> val expr = u reify { Random.nextInt }
expr: reflect.runtime.universe.Expr[Int] = Expr[Int](Random.nextInt())
scala> u showRaw expr.tree
res0: String = Apply(Select(Ident(scala.util.Random), newTermName("nextInt")), List())
Is what I expect; an application of nextInt with an empty parameter list.
Also,
scala> val expr = u reify { Random.shuffle(List("sammy", "snake")) }
expr: reflect.runtime.universe.Expr[List[String]] = Expr[List[String]](Random.shuffle(List.apply("sammy", "snake"))(List.canBuildFrom))
scala> u showRaw expr.tree
res2: String = Apply(Apply(Select(Ident(scala.util.Random), newTermName("shuffle")), List(Apply(Select(Ident(scala.collection.immutable.List), newTermName("apply")), List(Literal(Constant("sammy")), Literal(Constant("snake")))))), List(Select(Ident(scala.collection.immutable.List), newTermName("canBuildFrom"))))
Is again what I expected - an application of the shuffle method, with a list of strings.
However,
scala> val expr = u reify { Random.shuffle(List("sammy", "snake")).head }
expr: reflect.runtime.universe.Expr[String] = Expr[String](Random.shuffle(List.apply("sammy", "snake"))(List.canBuildFrom).head)
scala> u showRaw expr.tree
res1: String = Select(Apply(Apply(Select(Ident(scala.util.Random), newTermName("shuffle")), List(Apply(Select(Ident(scala.collection.immutable.List), newTermName("apply")), List(Literal(Constant("sammy")), Literal(Constant("snake")))))), List(Select(Ident(scala.collection.immutable.List), newTermName("canBuildFrom")))), newTermName("head"))
Is a Select on head. I'm confused because I would have thought that that outer node would be an application of head. Why is it a select when head is a method?
Is there a web page that details the difference between Select and Apply because I cannot find one.
The meaning of Select and Apply is purely syntactical.
Select(prefix, newTermName("member")) means prefix.member
Apply(prefix, List(arg1, arg2, ...)) means prefix(arg1, arg2, ...)
So techically, Random.nextInt is a Select, but the Scala typechecker wrapped it in an additional Apply because Random.nextInt is defined as a function that takes single, empty parameter list. In other words, this is a desugaring performed by the typechecker.
This isn't the case with List's head method. If you look at its definition, it takes no parameter lists:
def head: A
And that's why the typechecker does not wrap it in an Apply tree.
Summarizing - there is difference between methods that take single, empty parameter list and methods that take no parameter lists. As a syntactic sugar, Scala allows to call methods with single, empty parameter list as if they take no parameter lists (without any parens), but the typechecker will still add the Apply in such cases.
In this case, trees precisely reproduce syntactic structure of Scala programs. Select corresponds to member selection, e.g. to foo.bar, whereas Apply stands for an application of an argument list, e.g. baz(1, 2, 3).
If we have an empty arglist method in Scala, e.g. def x() = ??? (or Random.nextInt()), then its invocation can be written both as a Select (corresponding to something.x) and an Apply(Select(...), Nil) (corresponding to something.x()), because both are valid Scala expressions.
If we have a nullary method in Scala, e.g. def y = ??? (or List.head), then its invocation can only be written as a Select (which corresponds to somethingElse.y). If we write Apply(Select(...), Nil), that would mean somethingElse.y(), which would stand for applying an empty argument list to the result of calling method y on somethingElse.
I've searched for a half-hour, and still cannot figure it out.
In SIP: Modularizing Language Features there are a number of features which will require explicit "enabling" in Scala 2.10 (import language.feature).
Amongst them there is postfixOps, to which I just cannot find a reference anywhere. What exactly does this feature allow?
It allows you to use operator syntax in postfix position. For example
List(1,2,3) tail
rather than
List(1,2,3).tail
In this harmless example it is not a problem, but it can lead to ambiguities. This will not compile:
val appender:List[Int] => List[Int] = List(1,2,3) ::: //add ; here
List(3,4,5).foreach {println}
And the error message is not very helpful:
value ::: is not a member of Unit
It tries to call the ::: method on the result of the foreach call, which is of type Unit. This is likely not what the programmer intended. To get the correct result, you need to insert a semicolon after the first line.
The simplest answer ever:
Dropping dot from methods without parameters is DEPRECATED!
List(1,2,3) reverse //is bad style and will lead to unpredicted behaviour
List(1,2,3) map(_*2) reverse //bad too, because reverse can take first method call from the next line (details below)
OK to drop dot in methods that take one parameter of higher order function like map, filter, count and be safe!
Also, purely functional methods like zip.
List(1,2,3) map(_*2) filter(_>2)
(List(1,2,3) map(_*2)).reverse //safe and good
List(1,3,5) zip List(2,4,6)
Long answer WHY
case class MyBool(x: Boolean) {
def !!! = MyBool(!x) //postfix
def or(other: MyBool): MyBool = if(x) other else this //infix
def justMethod0() = this //method with empty parameters
def justMethod2(a: MyBool, b: MyBool) = this //method with two or more
override def toString = if(x) "true" else "false"
}
1) Postfix operator - is actually a method call with no parameters (a!==a.!) and without brackets. (considered not safe and deprecated)
val b1 = MyBool(false) !!!
List(1,2,3) head
2) Postfix operator is method, that should end the line, or else it will be treated as infix.
val b1 = MyBool(true) no! no! //ERROR
//is actually parsed like
val b2 = MyBool(true).no!(no!) //(no!) is unknown identifier
//as bad as
Vector(1,2,3) toList map(_*2) //ERROR
3) Infix operator is method with one parameter, that can be called without dot and parentheses. Only for purely functional methods
val c1 = MyBool(true) or b1 or MyBool(true)
val c2 = MyBool(true).or(b1).or(MyBool(true))
c1 == c2
4) Method with one or more parameters will chain without dot if you call it with parameters. def a(), def a(x), def a(x,y)
But you should do this only for methods that use higher order function as parameter!
val d1 = MyBool(true) justMethod2(b1, c1) or b1 justMethod0() justMethod2(c1, b1)
//yes, it works, but it may be confusing idea
val d2 = MyBool(true).justMethod2(b1,c1).or(b1).justMethod0().justMethod2(c1, b1)
d1 == d2
//looks familiar? This is where it should be used:
List(1,2,3) filter(_>1) map(_*2)
Sample warnings:
warning: there were 1 deprecation warning(s); re-run with -deprecation
for details warning: postfix operator tail should be enabled by making
the implicit value scala.language.postfixOps visible. This can be
achieved by adding the import clause 'import
scala.language.postfixOps' or by setting the compiler option
-language:postfixOps. See the Scala docs for value scala.language.postfixOps for a discussion why the feature should be
explicitly enabled.
It refers to the ability to call a nullary (with no arg list or empty arg list) method as a postfix operator:
By example:
case class MyBool(value: Boolean) {
def negated = new MyBool(!value)
}
val b1 = MyBool( true )
val b2 = b1 negated // Same as b1.negated
See: http://www.scala-lang.org/node/118
I've tried this line of code
def **[A <% Numeric[A]](l:List[A],m:List[A])=l.zip(m).map({t=>t._1*t._2})
However on compilation, I get this error
error: value * is not a member of type parameter A
def **[A <% Numeric[A]](l:List[A],m:List[A])=l.zip(m).map({t=>t._1*t._2})
When I look at the source for the Numeric trait, I see a * op defined.
What am I doing wrong?
The instance of Numeric is not a number itself, but it is an object that offers operations to do the arithmetic. For example, an object num of type Numeric[Int] can add two integers like this: num.plus(3, 5) The result of this operation is the integer 7.
For integers, this is very trivial. However, for all basic numerical types, there is one implicit instance of Numeric available. And if you define your own numeric types, you can provide one.
Therefore, you should leave the bounds for A open and add an implicit parameter of type Numeric[A], with which you do the calculations. Like this:
def **[A](l:List[A],m:List[A])(implicit num:Numeric[A])=l.zip(m).map({t=>num.times(t._1, t._2)})
Of course, num.times(a,b) looks less elegant than a*b. In most of the cases, one can live with that. However, you can wrap the value a in an object of type Ops that supports operators, like this:
// given are: num:Numeric[A], a:A and b:A
val a_ops = num.mkNumericOps(a)
val product = a_ops * b
Since the method mkNumericOps is declared implicit, you can also import it and use it implicitly:
// given are: num:Numeric[A], a:A and b:A
import num._
val product = a * b
You can also solve this with a context bound. Using the context method from this answer, you can write:
def **[A : Numeric](l:List[A],m:List[A]) =
l zip m map { t => context[A]().times(t._1, t._2) }
or
def **[A : Numeric](l:List[A],m:List[A]) = {
val num = context[A]()
import num._
l zip m map { t => t._1 * t._2 }
}