For example I have following Scala list, I want get a sublist until there is a requirement can be met.
val list = Seq(1,2,3,4,5,5,4,1,2,5)
The requirement is the number is 5, so I want the result as:
Seq(1,2,3,4)
Currently I use Scala collection's indexWhere and splitAt to return:
list.splitAt(list.indexWhere(x => x == 5))
(Seq(1,2,3,4), Seq(5,5,4,1,2,5))
I am not sure there are more better ways to achieve the same with better Scala collection's method I didn't realise?
You can use takeWhile:
scala> val list = Seq(1,2,3,4,5,5,4,1,2,5)
list: Seq[Int] = List(1, 2, 3, 4, 5, 5, 4, 1, 2, 5)
scala> list.takeWhile(_ != 5)
res30: Seq[Int] = List(1, 2, 3, 4)
Use span like this,
val (l,r) = list.span(_ != 5)
which delivers
l: List(1, 2, 3, 4)
r: List(5, 5, 4, 1, 2, 5)
Alternatively, you can write
val l = list.span(_ != 5)._1
to access only the first element of the resulting tuple.
This bisects the list at the first element that does not hold the condition.
Related
I have a list of elements List(1,2,3,4,5,6) which I hope to get a few elements from it to form a new List to List(2,4,5,6).
How should I go about it? Thanks!
scala collections can be mapped or filtered. In your case you simply can filter with the function you want.
eg. in scala REPL.
filter elements which are greater than or equals to 2.
scala> List(1,2,3,4,5,6).filter(_>=2)
res3: List[Int] = List(2, 3, 4, 5, 6)
or to filter all elements which are not 1 and 3,
scala> List(1,2,3,4,5,6).filter(element => (element!=1 && element!=3))
res6: List[Int] = List(2, 4, 5, 6)
Also read
https://twitter.github.io/scala_school/collections.html#filter
http://alvinalexander.com/scala/how-to-use-filter-method-scala-collections-cookbook
I have a query about intersection between two RDDs.
My first RDD has a list of elements like this:
A = List(1,2,3,4), List(4,5,6), List(8,3,1),List(1,6,8,9,2)
And the second RDD is like this:
B = (1,2,3,4,5,6,8,9)
(I could store B in memory as a Set but not the first one.)
I would like to do an intersection of each element in A with B
List(1,2,3,4).intersect((1,2,3,4,5,6,8,9))
List(4,5,6).intersect((1,2,3,4,5,6,8,9))
List(8,3,1).intersect((1,2,3,4,5,6,8,9))
List(1,6,8,9,2).intersect((1,2,3,4,5,6,8,9))
How can I do this in Scala?
val result = rdd.map( x => x.intersect(B))
Both B and rdd have to have the same type (in this case List[Int]). Also, note that if B is big but fits in memory, you would want to probably broadcast it as documented here.
scala> val B = List(1,2,3,4,5,6,8,9)
B: List[Int] = List(1, 2, 3, 4, 5, 6, 8, 9)
scala> val rdd = sc.parallelize(Seq(List(1,2,3,4), List(4,5,6), List(8,3,1),List(1,6,8,9,2)))
rdd: org.apache.spark.rdd.RDD[List[Int]] = ParallelCollectionRDD[0] at parallelize at <console>:21
scala> rdd.map( x => x.intersect(B)).collect.mkString("\n")
res3: String =
List(1, 2, 3, 4)
List(4, 5, 6)
List(8, 3, 1)
List(1, 6, 8, 9, 2)
Working with Lists in Scala I would like a simple way to get all elements but the last element. Is there a complementary method for .last similar to .head/.tail complement?
I'd rather not dirty up code with something like:
val x: List[String] = List("abc", "def", "ghi")
val allButLast: List[String] = x.reverse.tail.reverse
// List(abc, def)
Thanks.
init selects all elements but the last one.
List API for init.
scala> List(1,2,3,4,5)
res0: List[Int] = List(1, 2, 3, 4, 5)
scala> res0.init
res1: List[Int] = List(1, 2, 3, 4)
The 4 related methods here are head, tail, init, and last.
head and last get the first and final member, whereas
tail and init exclude the first and final members.
scala> val list = (0 to 10).toList
list: List[Int] = List(0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
scala> list.head
res0: Int = 0
scala> list.tail
res1: List[Int] = List(1, 2, 3, 4, 5, 6, 7, 8, 9, 10)
scala> list.init
res2: List[Int] = List(0, 1, 2, 3, 4, 5, 6, 7, 8, 9)
scala> list.last
res3: Int = 10
You should also take care, because all 4 of them are unsafe on the empty list and will throw exceptions.
These methods are defined on GenTraversableLike, which List implements.
That's init.
link to Scaladoc: http://www.scala-lang.org/api/2.11.5/index.html#scala.collection.immutable.List#init:Repr
def init: List[A]
Selects all elements except the last.
Also, note that it's defined in GenTraversableLike, so pretty much any Scala collection has this method.
For dropping off any number of items from the end of a list consider dropRight,
val xs = (1 to 5).toList
xs.dropRight(1)
List(1, 2, 3, 4)
xs.dropRight(2)
List(1, 2, 3)
xs.dropRight(10)
List()
What I want to do is sort List objects in Scala, not sort the elements in the list. For example If I have two lists of Ints:
val l1 = List(1, 2, 3, 7)
val l2 = List(1, 2, 3, 4, 10)
I want to be able to put them in order where l1 > l2.
I have created a case class that does what I need it to but the problem is that when I use it none of my other methods work. Do I need to implement all the other methods in the class i.e. flatten, sortWith etc.?
My class code looks like this:
class ItemSet(itemSet: List[Int]) extends Ordered[ItemSet] {
val iSet: List[Int] = itemSet
def compare(that: ItemSet) = {
val thisSize = this.iSet.size
val thatSize = that.iSet.size
val hint = List(thisSize, thatSize).min
var result = 0
var loop = 0
val ths = this.iSet.toArray
val tht = that.iSet.toArray
while (loop < hint && result == 0) {
result = ths(loop).compare(tht(loop))
loop += 1
}
if (loop == hint && result == 0 && thisSize != thatSize) {
thisSize.compare(thatSize)
} else
result
}
}
Now if I create an Array of ItemSets I can sort it:
val is1 = new ItemSet(List(1, 2, 5, 8))
val is2 = new ItemSet(List(1, 2, 5, 6))
val is3 = new ItemSet(List(1, 2, 3, 7, 10))
Array(is1, is2, is3).sorted.foreach(i => println(i.iSet))
scala> List(1, 2, 3, 7, 10)
List(1, 2, 5, 6)
List(1, 2, 5, 8)
The two methods that are giving me problems are:
def itemFrequencies(transDB: Array[ItemSet]): Map[Int, Int] = transDB.flatten.groupBy(x => x).mapValues(_.size)
The error I get is:
Expression of type Map[Nothing, Int] doesn't conform to expected type Map[Int, Int]
And for this one:
def sortListAscFreq(transDB: Array[ItemSet], itemFreq: Map[Int, Int]): Array[List[Int]] = {
for (l <- transDB) yield
l.sortWith(itemFreq(_) < itemFreq(_))
}
I get:
Cannot resolve symbol sortWith.
Is there a way I can just extend List[Int] so that I can sort a collection of lists without loosing the functionality of other methods?
The standard library provides a lexicographic ordering for collections of ordered things. You can put it into scope and you're done:
scala> import scala.math.Ordering.Implicits._
import scala.math.Ordering.Implicits._
scala> val is1 = List(1, 2, 5, 8)
is1: List[Int] = List(1, 2, 5, 8)
scala> val is2 = List(1, 2, 5, 6)
is2: List[Int] = List(1, 2, 5, 6)
scala> val is3 = List(1, 2, 3, 7, 10)
is3: List[Int] = List(1, 2, 3, 7, 10)
scala> Array(is1, is2, is3).sorted foreach println
List(1, 2, 3, 7, 10)
List(1, 2, 5, 6)
List(1, 2, 5, 8)
The Ordering type class is often more convenient than Ordered in Scala—it allows you to specify how some existing type should be ordered without having to change its code or create a proxy class that extends Ordered[Whatever], which as you've seen can get messy very quickly.
How do you replace an element by index with an immutable List.
E.g.
val list = 1 :: 2 ::3 :: 4 :: List()
list.replace(2, 5)
If you want to replace index 2, then
list.updated(2,5) // Gives 1 :: 2 :: 5 :: 4 :: Nil
If you want to find every place where there's a 2 and put a 5 in instead,
list.map { case 2 => 5; case x => x } // 1 :: 5 :: 3 :: 4 :: Nil
In both cases, you're not really "replacing", you're returning a new list that has a different element(s) at that (those) position(s).
In addition to what has been said before, you can use patch function that replaces sub-sequences of a sequence:
scala> val list = List(1, 2, 3, 4)
list: List[Int] = List(1, 2, 3, 4)
scala> list.patch(2, Seq(5), 1) // replaces one element of the initial sequence
res0: List[Int] = List(1, 2, 5, 4)
scala> list.patch(2, Seq(5), 2) // replaces two elements of the initial sequence
res1: List[Int] = List(1, 2, 5)
scala> list.patch(2, Seq(5), 0) // adds a new element
res2: List[Int] = List(1, 2, 5, 3, 4)
You can use list.updated(2,5) (which is a method on Seq).
It's probably better to use a scala.collection.immutable.Vector for this purpose, becuase updates on Vector take (I think) constant time.
You can use map to generate a new list , like this :
# list
res20: List[Int] = List(1, 2, 3, 4, 4, 5, 4)
# list.map(e => if(e==4) 0 else e)
res21: List[Int] = List(1, 2, 3, 0, 0, 5, 0)
It can also be achieved using patch function as
scala> var l = List(11,20,24,31,35)
l: List[Int] = List(11, 20, 24, 31, 35)
scala> l.patch(2,List(27),1)
res35: List[Int] = List(11, 20, 27, 31, 35)
where 2 is the position where we are looking to add the value, List(27) is the value we are adding to the list and 1 is the number of elements to be replaced from the original list.
If you do a lot of such replacements, it is better to use a muttable class or Array.
following is a simple example of String replacement in scala List, you can do similar for other types of data
scala> val original: List[String] = List("a","b")
original: List[String] = List(a, b)
scala> val replace = original.map(x => if(x.equals("a")) "c" else x)
replace: List[String] = List(c, b)