Appending element to list in Scala - scala

val indices: List[Int] = List()
val featValues: List[Double] = List()
for (f <- feat) {
val q = f.split(':')
if (q.length == 2) {
println(q.mkString("\n")) // works fine, displays info
indices :+ (q(0).toInt)
featValues :+ (q(1).toDouble)
}
}
println(indices.mkString("\n") + indices.length) // prints nothing and 0?
indices and featValues are not being filled. I'm at a loss here.

You cannot append anything to an immutable data structure such as List stored in a val (immutable named slot).
What your code is doing is creating a new list every time with one element appended, and then throwing it away (by not doing anything with it) — the :+ method on lists does not modify the list in place (even when it's a mutable list such as ArrayBuffer) but always returns a new list.
In order to achieve what you want, the quickest way (as opposed to the right way) is either to use a var (typically preferred):
var xs = List.empty[Int]
xs :+= 123 // same as `xs = xs :+ 123`
or a val containing a mutable collection:
import scala.collection.mutable.ArrayBuffer
val buf = ArrayBuffer.empty[Int]
buf += 123
However, if you really want to make your code idiomatic, you should instead just use a functional approach:
val indiciesAndFeatVals = feat.map { f =>
val Array(q0, q1) = f.split(':') // pattern matching in action
(q0.toInt, q1.toDouble)
}
which will give you a sequence of pairs, which you can then unzip to 2 separate collections:
val (indicies, featVals) = indiciesAndFeatVals.unzip
This approach will avoid the use of any mutable data structures as well as vars (i.e. mutable slots).

Related

appending elements to list of list in scala

i have created a empty scala mutable list
import scala.collection.mutable.ListBuffer
val list_of_list : List[List[String]] = List.empty
i want to append elements to it as below
filtered_df.collect.map(
r => {
val val_list = List(r(0).toString,r(4).toString,r(5).toString)
list_of_list += val_list
}
)
error that i am getting is
Error:(113, 26) value += is not a member of List[List[String]]
Expression does not convert to assignment because receiver is not assignable.
list_of_list += val_list
Can someone help
Your declaration seems wrong:
val list_of_list : List[List[String]] = List.empty
means that you've declared scala.collection.immutable.List whose operations return a new list without changing the current.
To fix the error you need to change the outer List type to ListBuffer that you imported above the declaration as follows:
val list_of_list : ListBuffer[List[String]] = ListBuffer.empty
Also it looks like you don't to use map here unless you want to modify your data collected from DataFrame, so you can change it to foreach:
filtered_df.collect.foreach {
r => {
val val_list = List(r(0).toString,r(4).toString,r(5).toString)
list_of_list += val_list
}
}
Furthermore you can make it in a functional way without resorting to ListBuffer, by using immutable List and foldRight as follows:
val list_of_list: List[List[String]] =
filtered_df.collect.toList
.foldRight(List.empty[List[String]])((r, acc) => List(r(0).toString,r(4).toString,r(5).toString) :: acc)
toList is used to achieve a stack safety when calling foldRight, because it's not stack safe for Arrays
More info about foldLeft and foldRight
You have to change that val list_of_list to var list_of_list. That alone would not be enough as you also have to change the type of list_of_list into a mutable alternative.

working with RDDS and collections in Scala

I have the below function that takes an array of means of type Likes (type Likes=Int)
and an RDD of numbers of type Likes (likesVector). For each number in the likesVector RDD, it computes the distance from each mean in means array and chooses the mean which has the least distance (val distance = (mean-number).abs). While I expect a result of type Map[Likes,Array[Likes]], I get an empty map. Map[Likes,Array[Likes]] represents (mean->Array of number-nearest numbers).
What is the best way to achieve this? I suspect it has a lot to do with the mutability of Scala collections.
def assignDataPoints(means:Array[Likes],likesVector:RDD[Likes]): Map[Likes,Array[Likes]] ={
var likes_Mean = IntMap(1->1)
var likes_mean_final = mutable.Map.empty[Likes,Array[Likes]]
likesVector.map(dataPoint => {
means.foldLeft(Array.empty[Likes])( (accumulator, mean)=> {
val dist= computeDistance(dataPoint,mean)
val nearestMean = if (dist < accumulator(0)) {
accumulator(0)=dist
accumulator(0)
} else{
accumulator(0)
}
val b= IntMap(nearestMean.toInt -> dataPoint)
println("b:"+ b)
likes_mean_final ++ likes_Mean.intersectionWith(b,(_, av, bv: Likes) => Array(av, bv))
accumulator
})})
likes_mean_final.toMap
}
The reason for your empty map is that you are using ++ operation here in the line:
likes_mean_final ++ likes_Mean.intersectionWith(b,(_, av, bv: Likes) => Array(av, bv))
++ operation returns a new map hence we are creating a new object and not changes the current value
The operation for mutating the current mutable map is ++=. So you should use:
likes_mean_final ++= likes_Mean.intersectionWith(b,(_, av, bv: Likes) => Array(av, bv))
Also no need to use var in your context as you are not reassigning values to the same references.
But you should not use mutability in Scala.

Appending to a list in Scala

I have no clue why Scala decided to make this such a chore, but I simply want to add an item to a list
var previousIds: List[String] = List()
I have tried the following:
previousIds ::: List(dataListItem.id)
previousIds :: List(dataListItem.id)
previousIds :+ List(dataListItem.id)
previousIds +: List(dataListItem.id)
previousIds :+ dataListItem.id
previousIds +: dataListItem.id
In every one of these instances, the line will run but the list still will contain 0 items
When I try to add a new list:
val list = List[String](dataListItem.id)
previousIds += list
I get an error that list needs to be a string. When I add a string
previousIds += dataListItem.id
I get an error that it needs to be a list
For some reason, the only thing that will work is the following:
previousIds :::= List[String](dataListItem.id)
which seems really excessive, since, adding to a list should be a trivial option. I have no idea why nothing else works though.
How do you add an item to a list in scala (that already exists) without having to make a new list like I am doing?
Next code should help you for a start.
My assumption you are dealing with mutable collections:
val buf = scala.collection.mutable.ListBuffer.empty[String]
buf += "test"
buf.toList
In case you are dealing with immutable collections next approach would help:
val previousIds = List[String]("A", "TestB")
val newList = previousIds :: List("TestB")
Please refer to documentation for mode details:
http://www.scala-lang.org/api/current/scala/collection/immutable/List.html
Use MutableList
scala> var a = scala.collection.mutable.MutableList[String]()
a: scala.collection.mutable.MutableList[String] = MutableList()
scala> a += "s"
res0: scala.collection.mutable.MutableList[String] = MutableList(s)
scala> a :+= "s"
scala> a
res1: scala.collection.mutable.MutableList[String] = MutableList(s, s)

scala append to a mutable LinkedList

Please check this
import scala.collection.mutable.LinkedList
var l = new LinkedList[String]
l append LinkedList("abc", "asd")
println(l)
// prints
// LinkedList()
but
import scala.collection.mutable.LinkedList
var l = new LinkedList[String]
l = LinkedList("x")
l append LinkedList("abc", "asd")
println(l)
// prints
// LinkedList(x, abc, asd)
Why does the second code snippet works but the first one doesnt? This is on Scala 2.10
The documentation says If this is empty then it does nothing and returns that. Otherwise, appends that to this.. That is exactly, what you observed. If you really need a mutable list, I would suggest you to use scala.collection.mutable.ListBuffer instead, with it you can do
val lb = new ListBuffer[Int]
scala> lb += 1
res14: lb.type = ListBuffer(1)
scala> lb
res15: scala.collection.mutable.ListBuffer[Int] = ListBuffer(1)
scala> lb ++= Seq(1,2,3)
res17: lb.type = ListBuffer(1, 1, 2, 3, 1, 2, 3)
scala> lb
res18: scala.collection.mutable.ListBuffer[Int] = ListBuffer(1, 1, 2, 3, 1, 2, 3)
As I understand it is related to First/Last (Nil) element in the list (if list is empty Nil is first and last element at the same time).
LinkedList (still) follows "primitive charm" strategy. So it does not try to add/append new data to/after Nil, to have possible result like this: {Nil, newElement}. (After all Nil should be last element)
Of course it could check if list is empty then put addingList to the beginning and Nil to the end. But this would be "too smart", I guess.
But, anyway append() returns "expecting" result Like this:
val addingList = new LinkedList[String]("a", "b")
val result = emptyList append addingList
result = {"a", "b"}. In this case it returns 'addingList' itself, and/but does not change initial list.
If we try to assign newElement to the next ref:
emptyList.next = LinkedList("whatever")
As result we would have emtyList changed like this:
LinkedList(null, whatever)
I.e. it creates fist element as null, since we have used next() assigning new/next element to it. So it moves Nil to the end, because first element which is null, has next reference to new element we added (addingElelement).
Because
"the "emptyList" is also the "head" link"
and head in our case head is Nil, but Nill can not have next, so it has to create new first element (which is has null value) with next() referece to our new addingElelement.
Personally I find it "too much primitive" and not "so much elegant". But it depends, I guess.
Task oriented story:
For my initial task (why I start thinking about this 'strange' list behaviour [even though it's mutable]) -- I wanted to use mutable list for a class/object called Dictionary which would keep Words in it (dictionary by default has not any words). And I would have methods like addWord(wod:String) for adding new words. For now my implementation will be changed (I'm not going to use this LinkedList, but rather MutableList. It seems it is more mutable than previous one):
object Dictionary {
val words = new mutable.MutableList[Word]();
def addWord(word: Word): Unit = {
words += word;
}
}
But possible implementation could be like this:
object Dictionary {
var words = new mutable.LinkedList[Word]();
def addWord(word: Word): Unit = {
if (words.isEmpty) {
words = words append( mutable.LinkedList[Word](word) ) // rely on append result
} else {
words append( mutable.LinkedList[Word](word) )
}
}
}
But then I have to use var instead of val, and I should transform every new Word to LinkedList, and my logic became more complicated.

Can't append to scala's mutable LinkedList?

I'm looking at the API and the :+ method returns a new LinkedList. The append method will only allow the appending of another linked list. The += method needs a var to work. Why would anyone ever need these if the LinkedList is mutable? What craziness is this?
If I had something like this in Java
final LinkedList myList = new LinkedList<String>();
mylist.add("balh");
How do I achieve the same thing in Scala?
If append can only take a LinkedList then why not use
mylist append LinkedList("something")
or
mylist append LinkedList(otherContainer: _*)
There is a reason for allowing only other LinkedLists in append, I think, because this guarantees the following:
l1 = LinkedList(1, 2, 3)
l2 = LinkedList(4)
l3 = LinkedList(5)
l1 append l2
// l1 == LinkedList(1, 2, 3, 4)
// l2 == LinkedList(4)
l2 append l3
// l1 == LinkedList(1, 2, 3, 4, 5)
// l2 == LinkedList(4, 5)
// l3 == LinkedList(5)
You can use a Buffer to build your values and convert it in the data structure using mapResult.
//Create a buffer which will build a linked list
val buf = new ArrayBuffer[String] mapResult { xs => LinkedList( xs:_* ) }
//You can append elements with +=, it is overriden to allow its use on a val
buf += "Something"
buf += "else"
//At the end you get your list
val lst = buf.result
// lst == LinkedList(Something, else)
Mutability is on the actual elements within the list and not on the structure that the list has (persistence). This is also stated in the scaladoc.
If returning a new List each time is not what you are after, and you cannot use var, you could always use the Java LinkedList.
val mylist = new java.util.LinkedList[String]
mylist add "something"
I'd stick with the Scala lists, if at all possible.
First, please pay more attention to the docs:
This class implements single linked lists where both the head (elem)
and the tail (next) are mutable.
So what it is giving you is mutable head and tail. These operations are represented through three methods:
append and insert change tail, so they receive a LinkedList as argument
update change the head of an element.
If you want a class that can grow, look at the classes extending Growable.