I am new to scala and I am practicing it with k-means algorithm following the tutorial from k-means
I am confused by this part of this tutorial:
var newCentroids = pointsGroup.mapValues(ps => average(ps)).collectAsMap()
This causes a type mismatch error because function average needs a Seq, while we give it an Iterable. How can I fix this? What caused this error?
Well Seq is a sub-type of Iterable but not vice-versa, so it is not possible to convert these types in the type systems.
There is an explicit conversion available by writing average(ps.toSeq). This conversion will iterate the Iterable and collect the items into a Seq.
We could easily replace Seq with Iterable in provided solution for average function:
def average(ps: Iterable[Vector]) : Vector = {
val numVectors = ps.size
var out = new Vector(ps.head.elements)
ps foreach ( out += _)
out / numVectors
}
Or even in constant space:
def average(ps: Iterable[Vector]): Vector = {
val numVectors = ps.size
val vSize = ps.head.elements.length
def element(index: Int): Double = ps.map(_(index)).sum / numVectors
new Vector(0 until vSize map element toArray)
}
Related
New to Scala, I want to try to rewrite some code in flatMap by calling a function instead of writing the whole process inside "()".
The original code is like:
val longForm = summary.flatMap(row => {
/*This is the code I want to replace with a function*/
val metric = row.getString(0)
(1 until row.size).map{i=>
(metric,schema(i).name,row.getString(i).toDouble)
})
}/*End of function*/)
The function I wrote is:
def tfunc(line:Row):List[Any] ={
val metric = line.getString(0)
var res = List[Any]
for (i<- 1 to line.size){
/*Save each iteration result as a List[tuple], then append to the res List.*/
val tup = (metric,schema(i).name,line.getString(i).toDouble)
val tempList = List(tup)
res = res :: tempList
}
res
}
The function did not passed compilation with the following error:
error: missing argument list for method apply in object List
Unapplied methods are only converted to functions when a function type is expected.
You can make this conversion explicit by writing apply _ or apply(_) instead of apply.
var res = List[Any]
What is wrong with this function?
And for flatMap, is it the write way to return the result as a List?
You haven't explained why you want to replace that code block. Is there a particular goal you're after? There are many, many, different ways that block could be rewritten. How can we know which would be better at meeting you requirements?
Here's one approach.
def tfunc(line :Row) :List[(String,String,Double)] ={
val metric = line.getString(0)
List.tabulate(line.tail.length){ idx =>
(metric, schema(idx+1).name, line.getString(idx+1).toDouble)
}
}
I'm looking for a way to convert a Vector[(Future[TypeA], TypeB)] to a Future[Vector[(TypeA, TypeB)]].
I'm aware of the conversion of a collection of futures to a future of a collection using Future.sequence(...) but cannot find out a way to manage the step from the tuple with a future to a future of tuple.
So I'm looking for something that implements the desired functionality of the dummy extractFutureFromTuple in the following.
val vectorOfTuples: Vector[(Future[TypeA], TypeB)] = ...
val vectorOfFutures: Vector[Future[(TypeA, TypeB)]] = vectorOfTuples.map(_.extractFutureFromTuple)
val futureVector: Future[Vector[(TypeA, TypeB)]] = Future.sequence(vectorOfFutures)
Note that you can do this with a single call to Future.traverse:
val input: Vector[(Future[Int], Long)] = ???
val output: Future[Vector[(Int, Long)]] = Future.traverse(input) {
case (f, v) => f.map(_ -> v)
}
I have a java.util.Map[String, MyObject] and want to create a Scala List[MyNewObject] consisting of alle entries of the map with some special values.
I found a way but, well, this is really ugly:
val result = ListBuffer[MyNewObject]()
myJavaUtilMap.forEach
(
(es: Entry[String, MyObject]) =>
{ result += MyNewObject(es.getKey(), ey.getValue().getMyParameter); println("Aa")}
)
How can I get rid of the println("Aa")? Just deleting does not help because foreach needs a Consumer but the += operation yields a list....
Is there a more elegant way to convert the java.util.Map to a List[MyNewObject]?
Scala has conversions that give you all the nice methods of the Scala collection API on Java collections:
import collection.JavaConversions._
val result = myJavaUtilMap.map{
case (k,v) => MyNewObject(k, v.getMyParameter)
}.toList
By the way: to define a function which returns Unit, you can explicitly specify the return type:
val f = (x: Int) => x: Unit
I would like to write a Matrix class in Scala from that I can instantiate objects like this:
val m1 = new Matrix( (1.,2.,3.),(4.,5.,6.),(7.,8.,9.) )
val m2 = new Matrix( (1.,2.,3.),(4.,5.,6.) )
val m3 = new Matrix( (1.,2.),(3.,4.) )
val m4 = new Matrix( (1.,2.),(3.,4.),(5.,6.) )
I have tried this:
class Matrix(a: (List[Double])* ) { }
but then I get a type mismatch because the matrix rows are not of type List[Double].
Further it would be nice to just have to type Integers (1,2,3) instead of (1.,2.,3.) but still get a double matrix.
How to solve this?
Thanks!
Malte
(1.0, 2.0) is a Tuple2[Double, Double] not a List[Double]. Similarly (1.0, 2.0, 3.0) is a Tuple3[Double, Double, Double].
If you need to handle a fixed number of cardinality, the simplest solution in plain vanilla scala would be to have
class Matrix2(rows: Tuple2[Double, Double]*)
class Matrix3(rows: Tuple3[Double, Double, Double]*)
and so on.
Since there exist an implicit conversion from Int to Double, you can pass a tuple of ints and it will be automatically converted.
new Matrix2((1, 2), (3, 4))
If you instead need to abstract over the row cardinality, enforcing an NxM using types, you would have to resort to some more complex solutions, perhaps using the shapeless library.
Or you can use an actual list, but you cannot restrict the cardinality, i.e. you cannot ensure that all rows have the same length (again, in vanilla scala, shapeless can help)
class Matrix(rows: List[Double]*)
new Matrix(List(1, 2), List(3, 4))
Finally, the 1. literal syntax is deprecated since scala 2.10 and removed in scala 2.11. Use 1.0 instead.
If you need support for very large matrices, consider using an existing implementation like Breeze. Breeze has a DenseMatrix which probably meets your requirements. For performance reasons, Breeze offloads more complex operations into native code.
Getting Matrix algebra right is a difficult exercise and unless you are specifically implementing matrix to learn/assignment, better to go with proven libraries.
Edited based on comment below:
You can consider the following design.
class Row(n : Int*)
class Matrix(rows: Row*) {...}
Usage:
val m = new Matrix(Row(1, 2, 3), Row(2,3,4))
You need to validate that all Rows are of the length and reject the input if they are not.
I have hacked a solution in an - I think - a bit unscala-ish way
class Matrix(values: Object*) { // values is of type WrappedArray[Object]
var arr : Array[Double] = null
val rows : Integer = values.size
var cols : Integer = 0
var _arrIndex = 0
for(row <- values) {
// if Tuple (Tuple extends Product)
if(row.isInstanceOf[Product]) {
var colcount = row.asInstanceOf[Product].productIterator.length
assert(colcount > 0, "empty row")
assert(cols == 0 || colcount == cols, "varying number of columns")
cols = colcount
if(arr == null) {
arr = new Array[Double](rows*cols)
}
for(el <- row.asInstanceOf[Product].productIterator) {
var x : Double = 0.0
if(el.isInstanceOf[Integer]) {
x = el.asInstanceOf[Integer].toDouble
}
else {
assert(el.isInstanceOf[Double], "unknown element type")
x = el.asInstanceOf[Double]
}
arr(_arrIndex) = x
_arrIndex = _arrIndex + 1
}
}
}
}
works like
object ScalaMatrix extends App {
val m1 = new Matrix((1.0,2.0,3.0),(5,4,5))
val m2 = new Matrix((9,8),(7,6))
println(m1.toString())
println(m2.toString())
}
I don't really like it. What do you think about it?
I am writing my code in Scala and need to have a loop over a vector of points in an image but getting type mismatch error!. I can understand why I have the error but I don't know how to solve it. here is my code:
val output= new Mat (image.rows, image.cols,CV_8UC3,new Scalar(0, 0, 0))
val it = Iterator(vect1)
var vect3=new Array[Byte](3)
vect3(0)=0
vect3(1)=255.toByte
vect3(2)=0
var e= new Point(0,0)
while(it.hasNext){
e = it.next();
output.put(e.x.toInt,e.y.toInt,vect3)
}
and I am getting this error:
...
type mismatch;
found : scala.collection.mutable.ArrayBuffer[org.opencv.core.Point]
required: org.opencv.core.Point[
e = it.next()
By doing val it = Iterator(vect1), you are creating an iterator that iterates on vect1 itself, and not on vect1's elements. Thankfully, you don't need to create an iterator for that, because it already exists :
val vect1 = ArrayBuffer(1, 2, 3)
// vect1: scala.collection.mutable.ArrayBuffer[Int] = ArrayBuffer(1, 2, 3)
val it = vect1.iterator
// it: Iterator[Int] = non-empty iterator
while (it.hasNext) {
println(it.next)
}
// 1
// 2
// 3
// res0: Unit = ()
Note that, according to the Scala API documentation, ArrayBuffer inherits from IterableLike. IterableLike basically means that a collection is iterable, so it makes sense that it defines a method which returns an iterator.
By the way, you can also avoid directly accessing the iterator entirely, using either the foreach method or a for comprehension, because IterableLike also defines the foreachmethod:
// foreach
vect1.foreach(p => output.put(p.x.toInt, p.y.toInt, vect3))
// for comprehension
for (p <- vect1) {
output.put(p.x.toInt, p.y.toInt, vect3)
}
Using the foreach method or the for comprehension is strictly equivalent: the compiler translates for comprehensions to one or more method calls; in this case, a call to foreach.