Scala spire is giving the following result. As per my understanding goes it must give List((0.0,0.1],[3.0,5.0)). Why such result?
scala> val x = Interval.openLower(0.0,0.1)
x: spire.math.Interval[Double] = (0.0, 0.1]
scala> val y = Interval.openUpper(3.0,5.0)
y: spire.math.Interval[Double] = [3.0, 5.0)
scala> x.union(y)
res0: spire.math.Interval[Double] = (0.0, 5.0)
And also
val S = Interval.open(1.0, 4.5)
val A = Interval.open(1.0, 3.0)
val B = Interval.open(2.0, 4.0)
val C = Interval.openUpper(3.0, 4.5)
println(S \ (A ∩ B))
val list = (S \ A).union(S \ B)
println(list)
The result is
List((1.0, 2.0], [3.0, 4.5))
List([3.0, 4.5), (1.0, 2.0], [4.0, 4.5))
How shall i unify the lower result to upper so that both will be equal.
I ran into the same issue and found out that Spire's IntervalSeq gets the job done.
// ammonite script intervals.sc
import $ivy.`org.typelevel::spire:0.17.0-M1`
import $ivy.`org.typelevel::spire-extras:0.17.0-M1`
import spire.math.Interval
import spire.math.extras.interval.IntervalSeq
import spire.implicits._
val S = IntervalSeq(Interval.open(1.0, 4.5))
val A = IntervalSeq(Interval.open(1.0, 3.0))
val B = IntervalSeq(Interval.open(2.0, 4.0))
val C = IntervalSeq(Interval.openUpper(3.0, 4.5))
val r1 = (S ^ (A & B))
println("r1=>" + r1.intervals.toList)
val r2 = ((S ^ A) | (S ^ B))
println("r2=>" + r2.intervals.toList)
Running this using the Ammonite REPL results in the following output:
r1=>List((1.0, 2.0], [3.0, 4.5))
r2=>List((1.0, 2.0], [3.0, 4.5))
Related
I am working with Spark RDD. I need to append/concatenate two RDDs of type Set.
scala> var ek: RDD[Set[Int]] = sc.parallelize(Seq(Set(7)))
ek: org.apache.spark.rdd.RDD[Set[Int]] = ParallelCollectionRDD[31] at parallelize at <console>:32
scala> val vi: RDD[Set[Int]] = sc.parallelize(Seq(Set(3,5)))
vi: org.apache.spark.rdd.RDD[Set[Int]] = ParallelCollectionRDD[32] at parallelize at <console>:32
scala> val z = vi.union(ek)
z: org.apache.spark.rdd.RDD[Set[Int]] = UnionRDD[34] at union at <console>:36
scala> z.collect
res15: Array[Set[Int]] = Array(Set(3, 5), Set(7))
scala> val t = visited++ek
t: org.apache.spark.rdd.RDD[Set[Int]] = UnionRDD[40] at $plus$plus at <console>:36
scala> t.collect
res30: Array[Set[Int]] = Array(Set(3, 5), Set(7))
I have tried using two operators, union and ++. However, it does not produce the expected result.
Array(Set(3, 5), Set(7))
The expected result should be like this:
scala> val u = Set(3,5)
u: scala.collection.immutable.Set[Int] = Set(3, 5)
scala> val o = Set(7)
o: scala.collection.immutable.Set[Int] = Set(7)
scala> u.union(o)
res28: scala.collection.immutable.Set[Int] = Set(3, 5, 7)
Can anybody give me direction how to do it
You are applying the union on a list (seq) of sets that is why the elements are the complete sets and not their elements. Try using:
var ek = sc.parallelize(Set(7).toSeq)
val vi = sc.parallelize(Set(3,5).toSeq)
val z = vi.union(ek)
i have existing variables:
scala> a
res69: Double = 5.0
scala> b
res70: Double = 10.0
scala> c
res71: Double = 15.0
There is a list containing variable names as string like:
scala> val variableList = List("a","b","c")
variableList: List[String] = List(a, b, c)
How to get values of variables in this list. I am expecting output as:
List(5.0, 10.0, 15.0)
if the scope of question is limited to getting values of terms defined in scala REPL, following works:
> val a = 5.0
> val b = 10.0
> val c = 15.0
> val variableList = List("a", "b", "c")
> variableList.map(v => $intp.valueOfTerm(v).getOrElse("Undefined: " + v))
// List[AnyRef] = List(5.0, 10.0, 15.0)
$intp is the REPL's interpreter.IMain object.
I am struggling with some very basic spark code. I would like to define a matrix x with 2 columns. This is what I have tried:
scala> val s = breeze.linalg.linspace(-3,3,5)
s: breeze.linalg.DenseVector[Double] = DenseVector(-3.0, -1.5, 0.0, 1.5, 3.0) // in this case I want s to be both column 1 and column 2 of x
scala> val ss = s.toArray ++ s.toArray
ss: Array[Double] = Array(-3.0, -1.5, 0.0, 1.5, 3.0, -3.0, -1.5, 0.0, 1.5, 3.0)
scala> import org.apache.spark.mllib.linalg.distributed.RowMatrix
import org.apache.spark.mllib.linalg.distributed.RowMatrix
scala> val mat = new RowMatrix(ss, 5, 2)
<console>:17: error: type mismatch;
found : Array[Double]
required: org.apache.spark.rdd.RDD[org.apache.spark.mllib.linalg.Vector]
val mat = new RowMatrix(ss, 5, 2)
I do not understand how I can get the right transformation in order to pass the values to the distributed matrix ^
EDIT:
Maybe I have been able to solve:
scala> val s = breeze.linalg.linspace(-3,3,5)
s: breeze.linalg.DenseVector[Double] = DenseVector(-3.0, -1.5, 0.0, 1.5, 3.0)
scala> val ss = s.to
toArray toDenseMatrix toDenseVector toScalaVector toString
toVector
scala> val ss = s.toArray ++ s.toArray
ss: Array[Double] = Array(-3.0, -1.5, 0.0, 1.5, 3.0, -3.0, -1.5, 0.0, 1.5, 3.0)
scala> val x = new breeze.linalg.Dense
DenseMatrix DenseVector
scala> val x = new breeze.linalg.DenseMatrix(5, 2, ss)
x: breeze.linalg.DenseMatrix[Double] =
-3.0 -3.0
-1.5 -1.5
0.0 0.0
1.5 1.5
3.0 3.0
scala> val xDist = sc.parallelize(x.toArray)
xDist: org.apache.spark.rdd.RDD[Double] = ParallelCollectionRDD[0] at parallelize at <console>:18
Something like this. This typechecks, but for some reason won't run in my Scala worksheet.
import org.apache.spark.mllib.linalg._
import org.apache.spark.mllib.linalg.distributed._
import org.apache.spark.SparkContext
import org.apache.spark.SparkContext._
import org.apache.spark.SparkConf
import org.apache.spark.rdd.RDD
val conf = new SparkConf().setAppName("spark-scratch").setMaster("local")
val sc= new SparkContext(conf)
// the values for the column in each row
val col = List(-3.0, -1.5, 0.0, 1.5, 3.0) ;
// make two rows of the column values, transpose it,
// make Vectors of the result
val t = List(col,col).transpose.map(r=>Vectors.dense(r.toArray))
// make an RDD from the resultant sequence of Vectors, and
// make a RowMatrix from that.
val rm = new RowMatrix(sc.makeRDD(t));
val A = 3
val (A) = (3)
Both correct. But:
val (A,B) = (2,3)
can't be compiled:
scala> val (A,B) = (2,3)
<console>:7: error: not found: value A
val (A,B) = (2,3)
^
<console>:7: error: not found: value B
val (A,B) = (2,3)
^
Why?
In the second code snippet, it using pattern matching to do assessment.
It is translated to the follow code:
val Tuple(A, B) = Tuple2(2,3)
When Scala is doing pattern matching, variable starts with a upper case in the pattern is considered as an constant value (or singleton Object), so val (a, b) = (2, 3) works but not val (A, B) = (2, 3).
BTW, your first code snippet does not using pattern matching, it's just an ordinary variable assignment.
If you using Tuple1 explicitly, it will have same error.
scala> val Tuple1(Z) = Tuple1(3)
<console>:7: error: not found: value Z
val Tuple1(Z) = Tuple1(3)
Here is some interesting example:
scala> val A = 10
A: Int = 10
scala> val B = 20
B: Int = 20
scala> val (A, x) = (10, 20)
x: Int = 20
scala> val (A, x) = (10, 30)
x: Int = 30
scala> val (A, x) = (20, 20)
scala.MatchError: (20,20) (of class scala.Tuple2$mcII$sp)
at .<init>(<console>:9)
at .<clinit>(<console>)
I'm trying to figure out how to use StateT to combine two State state transformers based on a comment on my Scalaz state monad examples answer.
It seems I'm very close but I got an issue when trying to apply sequence.
import scalaz._
import Scalaz._
import java.util.Random
val die = state[Random, Int](r => (r, r.nextInt(6) + 1))
val twoDice = for (d1 <- die; d2 <- die) yield (d1, d2)
def freqSum(dice: (Int, Int)) = state[Map[Int,Int], Int]{ freq =>
val s = dice._1 + dice._2
val tuple = s -> (freq.getOrElse(s, 0) + 1)
(freq + tuple, s)
}
type StateMap[x] = State[Map[Int,Int], x]
val diceAndFreqSum = stateT[StateMap, Random, Int]{ random =>
val (newRandom, dice) = twoDice apply random
for (sum <- freqSum(dice)) yield (newRandom, sum)
}
So I got as far as having a StateT[StateMap, Random, Int] that I can unwrap with initial random and empty map states:
val (freq, sum) = diceAndFreqSum ! new Random(1L) apply Map[Int,Int]()
// freq: Map[Int,Int] = Map(9 -> 1)
// sum: Int = 9
Now I'd like to generate a list of those StateT and use sequence so that I can call list.sequence ! new Random(1L) apply Map[Int,Int](). But when trying this I get:
type StT[x] = StateT[StateMap, Random, x]
val data: List[StT[Int]] = List.fill(10)(diceAndFreqSum)
data.sequence[StT, Int]
//error: could not find implicit value for parameter n: scalaz.Applicative[StT]
data.sequence[StT, Int]
^
Any idea? I can use some help for the last stretch - assuming it's possible.
Ah looking at the scalaz Monad source, I noticed there was an implicit def StateTMonad that confirms that StateT[M, A, x] is a monad for type parameter x. Also monads are applicatives, which was confirmed by looking at the definition of the Monad trait and by poking in the REPL:
scala> implicitly[Monad[StT] <:< Applicative[StT]]
res1: <:<[scalaz.Monad[StT],scalaz.Applicative[StT]] = <function1>
scala> implicitly[Monad[StT]]
res2: scalaz.Monad[StT] = scalaz.MonadLow$$anon$1#1cce278
So this gave me the idea of defining an implicit Applicative[StT] to help the compiler:
type StT[x] = StateT[StateMap, Random, x]
implicit val applicativeStT: Applicative[StT] = implicitly[Monad[StT]]
That did the trick:
val data: List[StT[Int]] = List.fill(10)(diceAndFreqSum)
val (frequencies, sums) =
data.sequence[StT, Int] ! new Random(1L) apply Map[Int,Int]()
// frequencies: Map[Int,Int] = Map(10 -> 1, 6 -> 3, 9 -> 1, 7 -> 1, 8 -> 2, 4 -> 2)
// sums: List[Int] = List(9, 6, 8, 8, 10, 4, 6, 6, 4, 7)