In this code where I'm attempting to xor the corresponding characters of two strings :
val s1 = "1c0111001f010100061a024b53535009181c";
val s2 = "686974207468652062756c6c277320657965";
val base64p1 = Base64.getEncoder().encodeToString(new BigInteger(s1, 16).toByteArray())
val base64p2 = Base64.getEncoder().encodeToString(new BigInteger(s2, 16).toByteArray())
val zs : IndexedSeq[(Char, Char)] = base64p1.zip(base64p2);
val xor = zs.foldLeft(List[Char]())((a: List[Char] , b: (Char, Char)) => ((Char)((b._1 ^ b._2))) :: a)
produces error :
Char.type does not take parameters
[error] val xor = zs.foldLeft(List[Char]())((a: List[Char] , b: (Char, Char)) => ((Char)((b._1 ^ b._2))) :: a)
How to xor the corresponding string char values and add them to List ?
What you're doing is can be simplified.
val xor = base64p1.zip(base64p2).map{case (a,b) => (a^b).toChar}.reverse
The result of the XOR op (^) is an Int. Just add .toChar to change it to a Char value.
But it looks like what you really want to do is XOR two large hex values that are represented as strings, and then return the result as a string. To do that all you need is...
val (v1, v2) = (BigInt(s1, 16), BigInt(s2, 16))
f"${v1 ^ v2}%x" // res0: String = 746865206b696420646f6e277420706c6179
You use java casting syntax. In scalla you cast like var.asInstanceOf[Type].
Should be (b._1 ^ b._2).asInstanceOf[Char].
Related
I have an RDD
res8: (String,(String,String)) = (CUST1#NODE1#16,(callduration,1))
I need to transform this as
(String,(String,String,String)) = (CUST1#NODE1,(16,callduration,1))
Please show me the spark scala code for this transformation.
You can split on the last # with regex #(?=[^#]*$). In case you are not familiar with regex, ?= is a look ahead syntax, [^#] is a character class that matches any character except for # and * is quantifier which means 0 or more, $ matches the end of string, so combined together, [^#]*$ matches a pattern from the end of string which doesn't contain any #, which makes #(?=[^#]*$) matches the last # in the string;
After the first element is split by the last #, you can reconstruct the tuple with split elements.
Given a tuple:
val t = ("CUST1#NODE1#16",("callduration","1"))
// t: (String, (String, String)) = (CUST1#NODE1#16,(callduration,1))
t match {
case (x, y) => {
val Array(l, r) = x.split("#(?=[^#]*$)")
(l, (r, y._1, y._2))
}
}
// res8: (String, (String, String, String)) = (CUST1#NODE1,(16,callduration,1))
Given a rdd, the procedure is similar, you can map the above transformation on rdd:
val rdd = sc.parallelize(Seq(t))
rdd.map(_ match {
case (x, y) => {
val Array(l, r) = x.split("#(?=[^#]*$)")
(l, (r, y._1, y._2))
}}).collect
// res11: Array[(String, (String, String, String))] = Array((CUST1#NODE1,(16,callduration,1)))
Normally if I am reading a line of input from stdin that looks like "100 200" I can store them both as ints with this line:
val Array(a, b) = readLine.split(" ").map(_.toInt);
But what if a is an integer and b is a string?
You cannot store values of different types into an homogeneous container such as an Array.
If you know the cardinality, just store them in separate variables.
val input = "100 foo"
val Array(a, b) = input.split(" ")
val p1 = scala.util.Try(a.toInt)
val p2 = b
I used scala.util.Try because toInt may fail and throw an exception
val readLine = "100 foo"
val (int, str) = readLine.split(" ") match {case Array(i, s) => (i.toInt, s)}
This seems not logical for me:
scala> val a = Map((1, "111"), (2, "222"))
a: scala.collection.immutable.Map[Int,String] = Map(1 -> 111, 2 -> 222)
scala> val b = a.map((key, value) => value)
<console>:8: error: wrong number of parameters; expected = 1
val b = a.map((key, value) => value)
^
scala> val c = a.map(x => x._2)
c: scala.collection.immutable.Iterable[String] = List(111, 222)
I know that I can say val d = a.map({ case(key, value) => value })
But why isn't it possible to say a.map((key, value) => value) ? There is only one argument there of type Tuple2[Int, String] or Pair of Int, String. What's the difference between a.map((key, value) => value) and a.map(x => x._2) ?
UPDATE:
val myTuple2 = (1, 2) -- this is one variable, correct?
for ( (k, v) <- a ) yield v -- (k, v) is also only one variable, correct?
map((key, value) => value) -- 2 variables. weird.
So how do I specify a variable of type Tuple2 (or any other type) in map without using case?
UPDATE2:
What's wrong with that?
Map((1, "111"), (2, "222")).map( ((x,y):Tuple2[Int, String]) => y) -- wrong
Map((1, "111"), (2, "222")).map( ((x):Tuple2[Int, String]) => x._2) -- ok
Okay, you still not convinced. In cases like this it is pretty reasonable to fallback to the source of the truth (well, kinda): The Holy Specification (aka, Scala Language Specification).
So, in anonymous function parameters are treated on individual basis, not as a whole tuple band (and it is pretty smart, otherwise, how would you call the anonymous function with 2, ... n parameters?).
At the same time
val x = (1, 2)
is a single item of type Tiple2[Int,Int] (if you're interested you may find corresponding section of spec as well).
for ( (k, v) <- a ) yield v
In this case you have one variable unpacked to two variables. It is similar to
val x = (1, 2) // one variable -- tuple
val (y,z) = x // two integer variables unpacked from one
Some call this destructuring assignment and this is a particular case of pattern matching. And you've already provided another example of pattern matching in action:
a.map({ case(key, value) => value })
Which we can read as map accepts a function produced by a partial function literal, which enables use of pattern matching.
You're basically asking this same questions:
Scala - can a lambda parameter match a tuple?
You've already listed most of the options they listed there, including the accepted answer of using a PartialFunction.
However, since you're using your lambda in a map function, you could use a for comprehension instead:
for ( (k, v) <- a ) yield v
Alternatively, you can use the Function2.tupled method to fix your lambda's type:
scala> val a = Map((1, "111"), (2, "222"))
a: scala.collection.immutable.Map[Int,String] = Map(1 -> 111, 2 -> 222)
scala> a.map( ((k:Int,v:String) => v).tupled )
res1: scala.collection.immutable.Iterable[String] = List(111, 222)
To answer your question in your thread with om-nom-nom above, look at this output:
scala> ( (x:Int,y:String) => y ).getClass.getSuperclass
res0: Class[?0] forSome { type ?0 >: ?0; type ?0 <: (Int, String) => String } = class scala.runtime.AbstractFunction2
Notice that the superclass of the anonymous function (x:Int,y:String) => y is Function2[Int, String, String], not Function1[(Int, String), String].
You can use pattern matching (or partial function, in this instance this is the same), notice angular brackets:
val b = a.map{ case (key, value) => value }
So this might not be the best way to tackle it but my initial thought was a for expression.
Say I have a List like
List(List('a','b','c'),List('d','e','f'),List('h','i','j'))
I would like to find the row and column for a character, say 'e'.
def findChar(letter: Char, list: List[List[Char]]): (Int, Int) =
for {
r <- (0 until list.length)
c <- (0 until list(r).length)
if list(r)(c) == letter
} yield (r, c)
If there is a more elegant way I'm all ears but I would also like to understand what's wrong with this. Specifically the error the compiler gives me here is
type mismatch; found : scala.collection.immutable.IndexedSeq[(Int, Int)] required: (Int, Int)
on the line assigning to r. It seems to be complaining that my iterator doesn't match the return type but I don't quite understand why this is or what to do about it ...
In the signature of findChar you are telling the compiler that it returns (Int, Int). However, the result of your for expression (as inferred by Scala) is IndexedSeq[(Int, Int)] as the error message indicates. The reason is that (r, c) after yield is produced for every "iteration" in the for expression (i.e., you are generating a sequence of results, not just a single result).
EDIT: As for findChar, you could do:
def findChar(letter: Char, list: List[List[Char]]) = {
val r = list.indexWhere(_ contains letter)
val c = list(r).indexOf(letter)
(r, c)
}
It is not the most efficient solution, but relatively short.
EDIT: Or reuse your original idea:
def findAll(letter: Char, list: List[List[Char]]) =
for {
r <- 0 until list.length
c <- 0 until list(r).length
if list(r)(c) == letter
} yield (r, c)
def findChar(c: Char, xs: List[List[Char]]) = findAll(c, xs).head
In both cases, be aware that an exception occurs if the searched letter is not contained in the input list.
EDIT: Or you write a recursive function yourself, like:
def findPos[A](c: A, list: List[List[A]]) = {
def aux(i: Int, xss: List[List[A]]) : Option[(Int, Int)] = xss match {
case Nil => None
case xs :: xss =>
val j = xs indexOf c
if (j < 0) aux(i + 1, xss)
else Some((i, j))
}
aux(0, list)
}
where aux is a (locally defined) auxiliary function that does the actual recursion (and remembers in which sublist we are, the index i). In this implementation a result of None indicates that the searched element was not there, whereas a successful result might return something like Some((1, 1)).
For your other ear, the question duplicates
How to capture inner matched value in indexWhere vector expression?
scala> List(List('a','b','c'),List('d','e','f'),List('h','i','j'))
res0: List[List[Char]] = List(List(a, b, c), List(d, e, f), List(h, i, j))
scala> .map(_ indexOf 'e').zipWithIndex.find(_._1 > -1)
res1: Option[(Int, Int)] = Some((1,1))
I have a Map[String, String] and want to concatenate the values to a single string.
I can see how to do this using a List...
scala> val l = List("te", "st", "ing", "123")
l: List[java.lang.String] = List(te, st, ing, 123)
scala> l.reduceLeft[String](_+_)
res8: String = testing123
fold* or reduce* seem to be the right approach I just can't get the syntax right for a Map.
Folds on a map work the same way they would on a list of pairs. You can't use reduce because then the result type would have to be the same as the element type (i.e. a pair), but you want a string. So you use foldLeft with the empty string as the neutral element. You also can't just use _+_ because then you'd try to add a pair to a string. You have to instead use a function that adds the accumulated string, the first value of the pair and the second value of the pair. So you get this:
scala> val m = Map("la" -> "la", "foo" -> "bar")
m: scala.collection.immutable.Map[java.lang.String,java.lang.String] = Map(la -> la, foo -> bar)
scala> m.foldLeft("")( (acc, kv) => acc + kv._1 + kv._2)
res14: java.lang.String = lalafoobar
Explanation of the first argument to fold:
As you know the function (acc, kv) => acc + kv._1 + kv._2 gets two arguments: the second is the key-value pair currently being processed. The first is the result accumulated so far. However what is the value of acc when the first pair is processed (and no result has been accumulated yet)? When you use reduce the first value of acc will be the first pair in the list (and the first value of kv will be the second pair in the list). However this does not work if you want the type of the result to be different than the element types. So instead of reduce we use fold where we pass the first value of acc as the first argument to foldLeft.
In short: the first argument to foldLeft says what the starting value of acc should be.
As Tom pointed out, you should keep in mind that maps don't necessarily maintain insertion order (Map2 and co. do, but hashmaps do not), so the string may list the elements in a different order than the one in which you inserted them.
The question has been answered already, but I'd like to point out that there are easier ways to produce those strings, if that's all you want. Like this:
scala> val l = List("te", "st", "ing", "123")
l: List[java.lang.String] = List(te, st, ing, 123)
scala> l.mkString
res0: String = testing123
scala> val m = Map(1 -> "abc", 2 -> "def", 3 -> "ghi")
m: scala.collection.immutable.Map[Int,java.lang.String] = Map((1,abc), (2,def), (3,ghi))
scala> m.values.mkString
res1: String = abcdefghi