I assigned the variable already, but I still get the error. I think that is not a typo.
val inputJPG = input.filter(context => context.contains("jpg")).collect
inputJPG.take(10)
------------------------------------------------------
scala> inputJPG.take(10)
<console>:20: error: not found: value inputJPG
inputJPG.take(10)
Just remove collect. (except you have a reason that is not clear by your question)
val input = Seq("none","image.jpg")
val inputJPG = input.filter(context => context.contains("jpg"))
inputJPG.take(10) // -> List(image.jpg)
collect is used when you want to filter and map in one step. See the API
Related
I created an RDD containing (key, value) pair by reading the file.I want to read the value of a specified key and assign it to a string.
RDD like this:
scala> minValue.foreach(println)
(http://subdom0003.example.com,100B)
(http://subdom0001.example.com,333B)
(http://subdom0002.example.com,8704B)
First,i tried to use val a = minValue("http://subdom0003.example.com")
it didn't work.
I tried to get the value with get, but it didn't work.
scala> val a = minValue.get("http://subdom0003.example.com").get
<console>:25: error: value get is not a member of org.apache.spark.rdd.RDD[(String, String)]
val a = minValue.get("http://subdom0003.example.com").get
Then I wonder if I should first map() and then get().But it is still unsuccessful
scala> val a = minValue.map(_.get("http://subdom0003.example.com").get)
<console>:25: error: value get is not a member of (String, String)
val a = minValue.map(_.get("http://subdom0003.example.com").get)
The result I want is that if I get a key, I want to store its value in a variable.for example
if key = "http://subdom0003.example.com"
then val minString = 100B
I think the best thing you can do is to filter your RDD by your key, and then get the first result with first() method. like this:
minValue.filter(_._1 == "your key").first();
which in your case, you should have:
val minString = minValue.filter(_._1 == "http://subdom0003.example.com").first()._2
I have this code:
1. var data = sc.textFile("test3.tsv")
2. var satir = data.map(line=> ((line.split("\t")(1),line.split("\t")(2)),(1,1)))
3. satir.reduce(((a,b),(c,k)) => k + k)
First and second works properly. What I want is by reducing (a,b), specify last '1'
For example, like this:
((a,b),(1,1))
But when I compile third one I get this error:
<console>:29: error: type mismatch;
found : (Int, Int)
required: String
satir.reduce({ case ((a,b),(k,o)) =>o+o})
What should I do?
When you reduce a value, the output value type must be the same as the input value type, you can use a folding method instead, because you can return another type with him.
scala.io.Source.fromFile("test3.tsv")
.getLines
.toList
.map { line =>
val value = line.split("\t")
((value(0), value(1)), (1,1))
}
.foldLeft(0)((response, tuple) => tuple._2._2 + tuple._2._2)
If you care to understand the theory behind this:
Fold explanation
A tutorial on the universality and
expressiveness of fold
A Translation from Attribute Grammars to
Catamorphisms
I am writing a parser in which I have the following function:
def lastop:(Either[RDD[(Int,Array[Float])], Float], Either[RDD[(Int,Array[Float])], Float]) => RDD[(Int,Array[Float])] = add
In which "add" is a function to perform addition. Then I want to use it in my program like the following line:
terms.foreach(t =>
t match { case nums ~ op => lastop = op; stack = reduce(stack ++ nums, op)}
I am getting the following error:
[error] /home/mahsa/calculator/temp/ScalaParser.scala:183: reassignment to val
[error] t match { case nums ~ op => lastop = op; stack = reduce(stack ++ nums, op)}
[error] ^
Can't figure how to solve this error!
You want to store a changing reference to the function you want to invoke. If you are storing and reassigning something, that implies you need a var, not a val or a def. Try declaring lastop like:
var lastop:(Either[RDD[(Int,Array[Float])], Float], Either[RDD[(Int,Array[Float])], Float]) => RDD[(Int,Array[Float])] = add
Note that you will still need to invoke lastop like a function, since retrieving the var's value will return a function. It's a subtle but significant difference.
Using Scala "2.10.4", I have a implicit definition like this:
implicit class MyImplicits(val s: S) {
def ==>(relation: W):Option[List[S]] = {
getRelation(s,relation)
}
}
when I want to use it, following works fine:
import MyImplicits
val list1 = s ==>(w)
val value = list1.get
But when I write this I get error:
import MyImplicits
val value = s ==>(w).get
Error:(56, 67) value get is not a member of MyImplicits
val value = s ==>(w).get
^
What is the reason for this error and is there anyway to solve it?
That's because it applies get to (w) rather than to the whole expression.
Try this:
val value = (s ==>(w)).get
As Ashalynd already explained, the period has a higher precedence than the ==> operator. You can get around it with parenthesis or you could use the get as an postfix operator:
val value = s ==> w get
I am trying to set an empty value (as a first choice) in a lift select element:
SHtml.select(("", "") :: (MyObject.findAll(By(MyObject.creator, User.currentUser.open_!.id))), ...
However it gives me this error:
error: type mismatch;
found: List[(String, java.lang.Object)]
required: Seq[(String, String)]
Any ideas?
Thanks!
Try breaking it up into a few steps.
val choices:Seq[(String,String)] = ("","") :: (all the rest of your options here)
scala> ("","")::List(("c","d"), ("d","e"))
res8: List[(java.lang.String, java.lang.String)] = List((,), (c,d), (d,e))
Then use that choices var as input to the SHtml.select method. There's nothing preventing this approach from working.
Example of I82Much has problems with compiling and shows errors.
I have modified his answer and have no problems. Tested.
define variable:
val choices = List(("",""), ("S", "Storage"))
and then use it
"strType" -> SHtml.select(choices, Empty, storage.strType(_), "class" -> "smallInputSelect"),