In language like python and ruby to ask the language what index-related methods its string class supports (which methods’ names contain the word “index”) you can do
“”.methods.sort.grep /index/i
And in java
List results = new ArrayList();
Method[] methods = String.class.getMethods();
for (int i = 0; i < methods.length; i++) {
Method m = methods[i];
if (m.getName().toLowerCase().indexOf(“index”) != -1) {
results.add(m.getName());
}
}
String[] names = (String[]) results.toArray();
Arrays.sort(names);
return names;
How would you do the same thing in Scala?
Curious that no one tried a more direct translation:
""
.getClass.getMethods.map(_.getName) // methods
.sorted // sort
.filter(_ matches "(?i).*index.*") // grep /index/i
So, some random thoughts.
The difference between "methods" and the hoops above is striking, but no one ever said reflection was Java's strength.
I'm hiding something about sorted above: it actually takes an implicit parameter of type Ordering. If I wanted to sort the methods themselves instead of their names, I'd have to provide it.
A grep is actually a combination of filter and matches. It's made a bit more complex because of Java's decision to match whole strings even when ^ and $ are not specified. I think it would some sense to have a grep method on Regex, which took Traversable as parameters, but...
So, here's what we could do about it:
implicit def toMethods(obj: AnyRef) = new {
def methods = obj.getClass.getMethods.map(_.getName)
}
implicit def toGrep[T <% Traversable[String]](coll: T) = new {
def grep(pattern: String) = coll filter (pattern.r.findFirstIn(_) != None)
def grep(pattern: String, flags: String) = {
val regex = ("(?"+flags+")"+pattern).r
coll filter (regex.findFirstIn(_) != None)
}
}
And now this is possible:
"".methods.sorted grep ("index", "i")
You can use the scala REPL prompt. To find list the member methods of a string object, for instance, type "". and then press the TAB key (that's an empty string - or even a non-empty one, if you like, followed by a dot and then press TAB). The REPL will list for you all member methods.
This applies to other variable types as well.
More or less the same way:
val names = classOf[String].getMethods.toSeq.
filter(_.getName.toLowerCase().indexOf(“index”) != -1).
map(_.getName).
sort(((e1, e2) => (e1 compareTo e2) < 0))
But all on one line.
To make it more readable,
val names = for(val method <- classOf[String].getMethods.toSeq
if(method.getName.toLowerCase().indexOf("index") != -1))
yield { method.getName }
val sorted = names.sort(((e1, e2) => (e1 compareTo e2) < 0))
This is as far as I got:
"".getClass.getMethods.map(_.getName).filter( _.indexOf("in")>=0)
It's strange Scala array doesn't have sort method.
edit
It would end up like.
"".getClass.getMethods.map(_.getName).toList.sort(_<_).filter(_.indexOf("index")>=0)
Now, wait a minute.
I concede Java is verbose compared to Ruby for instance.
But that piece of code shouldn't have been so verbose in first place.
Here's the equivalent :
Collection<String> mds = new TreeSet<String>();
for( Method m : "".getClass().getMethods()) {
if( m.getName().matches(".*index.*")){ mds.add( m.getName() ); }
}
Which has almost the same number of characters as the marked as correct, Scala version
Just using the Java code direct will get you most of the way there, as Scala classes are still JVM ones. You could port the code to Scala pretty easily as well, though, for fun/practice/ease of use in REPL.
Related
I written the code below for finding even numbers and the number just before it in a RDD object. In this I first converted that to a List and tried to use my own function to find the even numbers and the numbers just before them. The following is my code. In this I have made an empty list in which I am trying to append the numbers one by one.
object EvenandOdd
{
def mydef(nums:Iterator[Int]):Iterator[Int]=
{
val mylist=nums.toList
val len= mylist.size
var elist=List()
var i:Int=0
var flag=0
while(flag!=1)
{
if(mylist(i)%2==0)
{
elist.++=List(mylist(i))
elist.++=List(mylist(i-1))
}
if(i==len-1)
{
flag=1
}
i=i+1
}
}
def main(args:Array[String])
{
val myrdd=sc.parallelize(List(1,2,3,4,5,6,7,8,9,10),2)
val myx=myrdd.mapPartitions(mydef)
myx.collect
}
}
I am not able to execute this command in Scala shell as well as in Eclipse and not able to figure out the error as I am just a beginner to Scala.
The following are the errors I got in Scala Shell.
<console>:35: error: value ++= is not a member of List[Nothing]
elist.++=List(mylist(i))
^
<console>:36: error: value ++= is not a member of List[Nothing]
elist.++=List(mylist(i-1))
^
<console>:31: error: type mismatch;
found : Unit
required: Iterator[Int]
while(flag!=1)
^
Your code looks too complicated and not functional. Also, it introduce potential problems with memory: you take Iterator as param and return Iterator as output. So, knowing that Iterator itself could be lazy and has under the hood huge amount of data, materializing it inside method with list could cause OOM. So your task is to get as much data from initial iterator as it it enough to answer two methods for new Iterator: hasNext and next
For example (based on your implementation, which outputs duplicates in case of sequence of even numbers) it could be:
def mydef(nums:Iterator[Int]): Iterator[Int] = {
var before: Option[Int] = None
val helperIterator = new Iterator[(Option[Int], Int)] {
override def hasNext: Boolean = nums.hasNext
override def next(): (Option[Int], Int) = {
val result = (before, nums.next())
before = Some(result._2)
result
}
}
helperIterator.withFilter(_._2 % 2 == 0).flatMap{
case (None, next) => Iterator(next)
case (Some(prev), next) => Iterator(prev, next)
}
}
Here you have two iterators. One helper, which just prepare data, providing previous element for each next. And next on - resulting, based on helper, which filter only even for sequence elements (second in pair), and output both when required (or just one, if first element in sequence is even)
For initial code
Additionally to answer of #pedrorijo91, in initial code you do did not also return anything (suppose you wanted to convert elist to Iterator)
It will be easier if you use a functional coding style rather than an iterative coding style. In functional style the basic operation is straightforward.
Given a list of numbers, the following code will find all the even numbers and the values that precede them:
nums.sliding(2,1).filter(_(1) % 2 == 0)
The sliding operation creates a list containing all possible pairs of adjacent values in the original list.
The filter operation takes only those pairs where the second value is even.
The result is an Iterator[List[Int]] where each List[Int] has two elements. You should be able to use this in your RDD framework.
It's marked part of the developer API, so there's no guarantee it'll stick around, but the RDDFunctions object actually defines sliding for RDDs. You will have to make sure it sees elements in the order you want.
But this becomes something like
rdd.sliding(2).filter(x => x(1) % 2 == 0) # pairs of (preceding number, even number)
for the first 2 errors:
there's no ++= operator on Lists. You will have to do list = list ++ element
I am Scala beginner and started going through "Programming in Scala". I came across the following code.
for (arg <- args)
println(arg)
In the book it say's arg is of type val and not var. Why is it so.
As per my understanding for each iteration the value of arg is changed, as it holds new value each time it is looped.
Following are my questions
Why is arg val and not var ?
As per Scala if it is val we cannot change the value (Since it is final in Scala)
Also will it create multiple objects for each iteration and how are
the objects destroyed after the loop ?
Also i know the difference between var and val in scala. I have gone across this link. What is the difference between a var and val definition in Scala?
In Scala, a for comprehension is syntax sugar over foreach, map, flatMap and filter combinators. When you write:
for (arg <- args)
The compiler re-writes to:
args.foreach { }
Since foreach introduces a fresh value for each element, you don't actually reassign arg to anything:
args.foreach { arg => // }
For example, if we take a look at the implementation of List.foreach:
#inline final override def foreach[U](f: A => U) {
var these = this
while (!these.isEmpty) {
f(these.head)
these = these.tail
}
}
You see that these is a var which is being re-assigned as you iterate the list, so internally the Scala collection library keeps a mutable reference to the list, but from the outside each value of type A exposed to you as a caller via f(these.head) is immutable, so there isn't actually a need to reassign a reference to a value like in a C style for-loop.
In Scala, you can do
val l = List(1, 2, 3)
l.filter(_ > 2) // returns a List[Int]
val s = Set("hello", "world")
s.map(_.length) // returns a Set[Int]
The question is: why is this useful?
Scala collections are probably the only existing collection framework that does this. Scala community seems to agree that this functionality is needed. Yet, noone seems to miss this functionality in the other languages. Example C# (modified naming to match Scala's):
var l = new List<int> { 1, 2, 3 }
l.filter(i => i > 2) // always returns Iterable[Int]
l.filter(i => i > 2).toList // if I want a List, no problem
l.filter(i => i > 2).toSet // or I want a Set
In .NET, I always get back an Iterable and it is up to me what I want to do with it. (This also makes .NET collections very simple) .
The Scala example with Set forces me to make a Set of lengths out of a Set of string. But what if I just want to iterate over the lengths, or construct a List of lengths, or keep the Iterable to filter it later. Constructing a Set right away seems pointless. (EDIT: collection.view provides the simpler .NET functionality, nice)
I am sure you will show me examples where the .NET approach is absolutely wrong or kills performance, but I just can't see any (using .NET for years).
Not a full answer to your question, but Scala never forces you to use one collection type over another. You're free to write code like this:
import collection._
import immutable._
val s = Set("hello", "world")
val l: Vector[Int] = s.map(_.length)(breakOut)
Read more about breakOut in Daniel Sobral's detailed answer to another question.
If you want your map or filter to be evaluated lazily, use this:
s.view.map(_.length)
This whole behavior makes it easy to integrate your new collection classes and inherit all the powerful capabilities of the standard collection with no code duplication, all of this ensuring that YourSpecialCollection#filter returns an instance of YourSpecialCollection; that YourSpecialCollection#map returns an instance of YourSpecialCollection if it supports the type being mapped to, or a built-in fallback collection if it doesn't (like what happens of you call map on a BitSet). Surely, a C# iterator has no .toMySpecialCollection method.
See also: “Integrating new sets and maps” in The Architecture of Scala Collections.
Scala follows the "uniform return type principle" assuring that you always end up with the appropriate return type, instead of loosing that information like in C#.
The reason C# does it this was is that their type system is not good enough to provide these assurances without overriding the whole implementation of every method in every single subclass. Scala solves this with the usage of Higher Kinded Types.
Why Scala has the only collection framework doing this? Because it is harder than most people think it is, especially when things like Strings and Arrays which are no "real" collections should be integrated as well:
// This stays a String:
scala> "Foobar".map(identity)
res27: String = Foobar
// But this falls back to the "nearest" appropriate type:
scala> "Foobar".map(_.toInt)
res29: scala.collection.immutable.IndexedSeq[Int] = Vector(70, 111, 111, 98, 97, 114)
If you have a Set, and an operation on it returns an Iterable while its runtime type is still a Set, then you're losing important informations about its behavior, and the access to set-specific methods.
BTW: There are other languages behaving similar, like Haskell, which influenced Scala a lot. The Haskell version of map would look like this translated to Scala (without implicitmagic):
//the functor type class
trait Functor[C[_]] {
def fmap[A,B](f: A => B, coll: C[A]) : C[B]
}
//an instance
object ListFunctor extends Functor[List] {
def fmap[A,B](f: A => B, list: List[A]) : List[B] = list.map(f)
}
//usage
val list = ListFunctor.fmap((x:Int) => x*x, List(1,2,3))
And I think the Haskell community values this feature as well :-)
It is a matter of consistency. Things are what they are, and return things like them. You can depend on it.
The difference you make here is one of strictness. A strict method is immediately evaluated, while a non-strict method is only evaluated as needed. This has consequences. Take this simple example:
def print5(it: Iterable[Int]) = {
var flag = true
it.filter(_ => flag).foreach { i =>
flag = i < 5
println(i)
}
}
Test it with these two collections:
print5(List.range(1, 10))
print5(Stream.range(1, 10))
Here, List is strict, so its methods are strict. Conversely, Stream is non-strict, so its methods are non-strict.
So this isn't really related to Iterable at all -- after all, both List and Stream are Iterable. Changing the collection return type can cause all sort of problems -- at the very least, it would make the task of keeping a persistent data structure harder.
On the other hand, there are advantages to delaying certain operations, even on a strict collection. Here are some ways of doing it:
// Get an iterator explicitly, if it's going to be used only once
def print5I(it: Iterable[Int]) = {
var flag = true
it.iterator.filter(_ => flag).foreach { i =>
flag = i < 5
println(i)
}
}
// Get a Stream explicitly, if the result will be reused
def print5S(it: Iterable[Int]) = {
var flag = true
it.toStream.filter(_ => flag).foreach { i =>
flag = i < 5
println(i)
}
}
// Use a view, which provides non-strictness for some methods
def print5V(it: Iterable[Int]) = {
var flag = true
it.view.filter(_ => flag).foreach { i =>
flag = i < 5
println(i)
}
}
// Use withFilter, which is explicitly designed to be used as a non-strict filter
def print5W(it: Iterable[Int]) = {
var flag = true
it.withFilter(_ => flag).foreach { i =>
flag = i < 5
println(i)
}
}
Below is an implementation of a function that returns the lexographically next permutation. This is useful in one of the Euler problems.
It's written to work on Strings (which I needed for that). However, it should work on any indexed sequence of comparable values. I've tried generalising it by changing the two occurrences of String to IndexedSeq[Char], but this gets an error:
euler-lib.scala:26: error: type mismatch;
found : IndexedSeq[Char]
required: String
((n.slice(pivot+1, successor):+ n(pivot)) + n.drop(successor+1)).reverse
^
Why has the type inferencer inferred String there? I don't seem to have done any operation that requires a String?
And can I make it more general still by having IndexedSeq["something-comparable"]? I've not been able to make this work.
// return the lexographically next permutation to the one passed as a parameter
// pseudo-code from an article on StackOverflow
def nextPermutation(n:String):String = {
// 1. scan the array from right-to-left
//1.1. if the current element is less than its right-hand neighbor,
// call the current element the pivot,
// and stop scanning
// (We scan left-to-right and return the last such).
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
//1.2. if the left end is reached without finding a pivot,
// reverse the array and return
// (the permutation was the lexicographically last, so its time to start over)
if (pivot < 0) return n.reverse
//2. scan the array from right-to-left again,
// to find the rightmost element larger than the pivot
// (call that one the successor)
val successor = n.lastIndexWhere{_ > n(pivot)}
//3. swap the pivot and the successor, and
//4. reverse the portion of the array to the right of where the pivot was found
return (n.take(pivot) :+ n(successor)) +
((n.slice(pivot+1, successor):+ n(pivot)) + n.drop(successor+1)).reverse
}
The method + in IndexedSeq is used to produce a new sequence containing one additional given element but you want to produce one containing an additional sequence. The method for this is ++ thus your last line must look like this:
(n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
You are seeing this strange compiler message about a String being expected because +'s signature does not match and thus an explicit conversion used for String concatenation kicks in (this conversion is there because it lets you write something like List(8) + " Test").
EDIT: Generalization over sequence types of ordered elements:
As I said in the comments, generalization over sequences is a bit more complicated. In addition to your element type A you will need another type CC[X] <: SeqLike[X,CC[X]] that represents the sequence. Normally C <: SeqLike[A,C] would be sufficient but the type inferencer does not like that one (you would always need to pass the types of A and C when calling that method).
If you just change your signature that way the compiler will complain that it requires an implicit CanBuildFrom[CC[A],A,CC[A]] parameter as that one is needed e.g. by the reverse method. That parameter is used to build one sequence type from another one - just search the site to see some examples of how it is used by the collections API.
The final result would look like this:
import collection.SeqLike
import collection.generic.CanBuildFrom
def nextPermutation[A, CC[X] <: SeqLike[X,CC[X]]](n: CC[A])(
implicit ord: Ordering[A], bf: CanBuildFrom[CC[A],A,CC[A]]): CC[A] = {
import ord._
// call toSeq to avoid having to require an implicit CanBuildFrom for (A,A)
val pivot = n.toSeq.zip(n.tail.toSeq).lastIndexWhere{
case (first, second) => first < second
}
if (pivot < 0) {
n.reverse
}
else {
val successor = n.lastIndexWhere{_ > n(pivot)}
(n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
}
This way you get a Vector[Int] if you passed one to the method and a List[Double] if you passed that to the method. So what about Strings? Those are not actual sequences but they can be implicitly converted into a Seq[Char]. It is possible alter the definition of that method expect some type that can be implicitly converted into a Seq[A] but then again type inference would not work reliably - or at least I could not make it work reliably. As a simple workaround you could define an additional method for Strings:
def nextPermutation(s: String): String =
nextPermutation[Char,Seq](s.toSeq).mkString
Little tip here:
n(pivot)) + n.drop(successor+1)
^
When you get a type mismatch error, and the ^ points to the first parenthesis of the last argument list (ie, it would point to the second ( in x.foldLeft(y)(z)), that means the value returned by that method has the wrong type.
Or, in this case, n.drop(sucessor+1) has type IndexedSeq[Char], but the + method expects a String.
Another little tip: the only things that accept + are the numeric classes and String. If you try to add things and get an error, most likely it is Scala thinking you are using + to add Strings. For example:
true + true // expected String, got Boolean error
"true" + true // works, the second true is converted to String
true + "true" // works, the first true is converted to String
So, avoid + unless you are working with numbers or strings.
So, about making that general...
def nextPermutation[A <% Ordered[A]](n: IndexedSeq[A]): IndexedSeq[A] = {
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
if (pivot < 0) return n.reverse
val successor = n.lastIndexWhere{_ > n(pivot)}
return (n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
The easy part is just declaring IndexedSeq. But you have to parameterize on A, and there must be a way to order A so that you can compare the elements (<% means there's an implicit conversion from A to an Ordered[A] available). Another way to declare it would be like this:
def nextPermutation[A : Ordering](n: IndexedSeq[A]): IndexedSeq[A] = {
val ordering = implicitly[Ordering[A]]; import ordering._
val pivot = n.zip(n.tail).lastIndexWhere{ case (first, second) => first < second }
if (pivot < 0) return n.reverse
val successor = n.lastIndexWhere{_ > n(pivot)}
return (n.take(pivot) :+ n(successor)) ++
((n.slice(pivot+1, successor):+ n(pivot)) ++ n.drop(successor+1)).reverse
}
Here, A : Ordering means there is an implicit Ordering[A] available, which is then obtained and imported into scope, so that it can offer implicit conversions to make < work. The difference between an Ordered[A] and an Ordering[A] can be found on other questions.
Problem 24 had me stumped for a while:
println("0123456789".permutations.drop(1000000 - 1).next);
The code compiles correclty for me in Scala 2.8.0. Which version of Scala are you using ?
scala> nextPermutation("12354")
res0: String = 12435
I have to translate the following code from Java to Scala:
EDIT: added if-statements in the source (forgot them in first version)
for (Iterator<ExceptionQueuedEvent> i = getUnhandledExceptionQueuedEvents().iterator(); i.hasNext();)
{
if (someCondition) {
ExceptionQueuedEvent event = i.next();
try {
//do something
} finally {
i.remove();
}
}
}
I'm using the JavaConversions library to wrap the Iterable. But as i'm not using the original Iterator, i don't know how to remove the current element correctly from the collection the same way as i did in Java:
import scala.collection.JavaConversions._
(...)
for (val event <- events) {
if (someCondition) {
try {
// do something
} finally {
// how can i remove the current event from events?
// the underlying type of events is java.lang.Iterable[javax.faces.event.ExceptionQueuedEvent]
}
}
}
Can someone help me?
I guess it's easy, but i'm still kinda new to Scala and don't understand what's going on when Scala wraps something of Java.
When you use JavaConversions to wrap Java collections, you just get an object that adapts the Java collection to the appropriate Scala trait (interface). In Java, you might see the same thing (for example, you could imagine a adapter class that implements the Iterator interface and wraps an Enumeration.) The only difference is that in Scala you can add the 'implicit' modifier to a declaration to tell the compiler to automatically insert calls to that method if it will make the code compile.
As for your specific use case, Iterators in Scala intentionally omit the remove() method for a number of reasons. The conversion from scala.collection.Iterator to java.util.Iterator promises to unwrap a j.u.Iterator if possible, so I suppose you could rely on that to access the remove() method. However, if you are iterating over the entire collection and removing everything, why not just do your work in a foreach loop and then clear the collection or replace it with an empty one after you finish?
Does this suggest how do accomplish what you want?
scala> val l1 = List("How", "do", "I", "love", "you")
l1: List[java.lang.String] = List(How, do, I, love, you)
scala> val evens = for ( w <- l1; if w.length % 2 == 0 ) yield { printf("even: %s%n", w); w }
even: do
even: love
evens: List[java.lang.String] = List(do, love)
Basically, you get your Scala Iterable or Iterator using the appropriate implicit conversion from JavaConversions, use a for comprehension that includes the condition on which elements you want to process and collect the results. Use exception handling as necessary.
Thanks for all the help. So i had to do without using JavaConversions. But it still looks nice&scalafied ;)**
This is my final code, which seems to work:
val eventsIterator = events.iterator
for (eventsIterator.hasNext) {
if (someCondition) {
try {
// do something
} finally {
eventsIterator.remove
}
}
}