I am using the Scopt library
Sample code:
val parser = new OptionParser[parser1]{
opt[String]("custId").required().text("customer id").action((x,p)=>p.copy(custId = x))
opt[String]("myid").optional().text("my id").action((x,p)=>p.copy(myid = x))
}
parser.parse(args,parser1()) match {
case Some(a) =>
---code---
case _ =>
somefunction(Map(custId -> "unknown))
}
Now, here instead of mapping custId as unknown I want to map it to the given input in case it is provided. i.e. In case the input args are:
--custId 12 --someInvalidArg hf23
then I don't want to map custId as unknown since it is provided in arguments even though the parsing fails
Basically, in case _ i want to check if custId is given. Map it to the value if its given and map it to "unknown" if it is not given.
How can i achieve this
Related
I have this code:
1. var data = sc.textFile("test3.tsv")
2. var satir = data.map(line=> ((line.split("\t")(1),line.split("\t")(2)),(1,1)))
3. satir.reduce(((a,b),(c,k)) => k + k)
First and second works properly. What I want is by reducing (a,b), specify last '1'
For example, like this:
((a,b),(1,1))
But when I compile third one I get this error:
<console>:29: error: type mismatch;
found : (Int, Int)
required: String
satir.reduce({ case ((a,b),(k,o)) =>o+o})
What should I do?
When you reduce a value, the output value type must be the same as the input value type, you can use a folding method instead, because you can return another type with him.
scala.io.Source.fromFile("test3.tsv")
.getLines
.toList
.map { line =>
val value = line.split("\t")
((value(0), value(1)), (1,1))
}
.foldLeft(0)((response, tuple) => tuple._2._2 + tuple._2._2)
If you care to understand the theory behind this:
Fold explanation
A tutorial on the universality and
expressiveness of fold
A Translation from Attribute Grammars to
Catamorphisms
I have the following input string:
"0.3215,Some(0.5123)"
I would like to retrieve the tuple (0.3215,Some(0.5123)) with: (BigDecimal,Option[BigDecimal]).
Here is one of the thing I tried so far:
"\\d+\\.\\d+,Some\\(\\d+\\.\\d+".r findFirstIn iData match {
case None => Map[BigDecimal, Option[BigDecimal]]()
case Some(s) => {
val oO = s.split(",Some\\(")
BigDecimal.valueOf(oO(0).toDouble) -> Option[BigDecimal](BigDecimal.valueOf(lSTmp2(1).toDouble))
}
}
Using a Map and transforming it into a tuple.
When I try directly the tuple I get an Equals or an Object.
Must miss something here...
Your code has several issues, but the big one seems to be that the case None side of the match returns a Map but the Some(s) side returns a Tuple2. Map and Tuple2 unify to their lowest-common-supertype, Equals, which is what you're seeing.
I think this is what you're trying to achieve?
val Pattern = "(\\d+\\.\\d+),Some\\((\\d+\\.\\d+)\\)".r
val s = "0.3215,Some(0.5123)"
s match {
case Pattern(a,b) => Map(BigDecimal(a) -> Some(BigDecimal(b)))
case _ => Map[BigDecimal, Option[BigDecimal]]()
}
// Map[BigDecimal,Option[BigDecimal]] = Map(0.3215 -> Some(0.5123))
Trying to create multiple dataframes in a single foreach, using spark, as below
I get values delivery and click out of row.getAs("type"), when I try to print them.
val check = eachrec.foreach(recrd => recrd.map(row => {
row.getAs("type") match {
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
case "click" => val click_data = delivery(row.get(0).toString,row.get(1).toString)
case _ => "not sure if this impacts"
}})
)
but getting below error:
Error:(41, 14) type mismatch; found : String("delivery") required: Nothing
case "delivery" => val delivery_data = delivery(row.get(0).toString,row.get(1).toString)
^
My plan is to create dataframe using todf() once I create these individual delivery objects referenced by delivery_data and click_data by:
delivery_data.toDF() and click_data.toDF().
Please provide any clue regarding the error above (in match case).
How can I create two df's using todf() in val check?
val declarations make your first 2 cases return type to be unit, but in the third case you return a String
for instance, here the z type was inferred by the compiler, Unit:
def x = {
val z: Unit = 3 match {
case 2 => val a = 2
case _ => val b = 3
}
}
I think you need to cast this match clause to String.
row.getAs("type").toString
I'm new to Scala and I'm trying to convert code of the form
val series: ListBuffer[Seq[Seq[Any]]] = ListBuffer[Seq[Seq[Any]]]()
points.foreach(point => {
if( conditionA )
series += doA(...) // returns a ListBuffer[Seq[Any]]
else
series += doB(...) // returns a ListBuffer[Seq[Any]]
})
to use .map(). I'm thinking its something like:
val series: ListBuffer[Seq[Seq[Any]]] = points.map(point => {
case conditionA => doA(...)
case _ => doB(...)
})
but this doesn't compile because (I believe) the mapped sequences get appended as a single Seq[Any] instead of Seq[Seq[Any]], so I get the error
Expression of type Seq[Any] doesn't conform to expected type ListBuffer[Seq[Seq[Any]]]
Any ideas? Is there something wrong with syntax?
Let me suppose a few things, you have some function def doA(arg1: A): ListBuffer[Seq[Any]] such that you ultimately want to arrive at a List[Seq[Any]] as the final result type after mapping this function over your collection. Then what you want is flatMap instead of map:
val series = points flatMap{
case point if conditionA(point) => doA(point) result ()
case point => doB(point) result ()
}
The reason I make such a supposition is that the only reason you'd ever want to use a ListBuffer[A] in the general form is to create a List[A] through some side-effecting expression. Hence, you ultimately want a List[Seq[A]] as your final output.
Consider the following from the Scala interpreter:
scala> JSON.parseFull("""{"name":"jack","greeting":"hello world"}""")
res6: Option[Any] = Some(Map(name -> jack, greeting -> hello world))
Why is the Map returned in Some() thing? And how do I work with it?
I want to put the values in an xml template:
<test>
<name>name goes here</name>
<greeting>greeting goes here</greeting>
</test>
What is the Scala way of getting my map out of Some(thing) and getting those values in the xml?
You should probably use something like this:
res6 collect { case x: Map[String, String] => renderXml(x) }
Where:
def renderXml(m: Map[String, String]) =
<test><name>{m.get("name") getOrElse ""}</name></test>
The collect method on Option[A] takes a PartialFunction[A, B] and is a combination of filter (by a predicate) and map (by a function). That is:
opt collect pf
opt filter (a => pf isDefinedAt a) map (a => pf(a))
Are both equivalent. When you have an optional value, you should use map, flatMap, filter, collect etc to transform the option in your program, avoiding extracting the option's contents either via a pattern-match or via the get method. You should never, ever use Option.get - it is the canonical sign that you are doing it wrong. Pattern-matching should be avoided because it represents a fork in your program and hence adds to cyclomatic complexity - the only time you might wish to do this might be for performance
Actually you have the issue that the result of the parseJSON method is an Option[Any] (the reason is that it is an Option, presumably, is that the parsing may not succeed and Option is a more graceful way of handling null than, well, null).
But the issue with my code above is that the case x: Map[String, String] cannot be checked at runtime due to type erasure (i.e. scala can check that the option contains a Map but not that the Map's type parameters are both String. The code will get you an unchecked warning.
An Option is returned because parseFull has different possible return values depending on the input, or it may fail to parse the input at all (giving None). So, aside from an optional Map which associates keys with values, an optional List can be returned as well if the JSON string denoted an array.
Example:
scala> import scala.util.parsing.json.JSON._
import scala.util.parsing.json.JSON._
scala> parseFull("""{"name":"jack"}""")
res4: Option[Any] = Some(Map(name -> jack))
scala> parseFull("""[ 100, 200, 300 ]""")
res6: Option[Any] = Some(List(100.0, 200.0, 300.0))
You might need pattern matching in order to achieve what you want, like so:
scala> parseFull("""{"name":"jack","greeting":"hello world"}""") match {
| case Some(m) => Console println ("Got a map: " + m)
| case _ =>
| }
Got a map: Map(name -> jack, greeting -> hello world)
Now, if you want to generate XML output, you can use the above to iterate over the key/value pairs:
import scala.xml.XML
parseFull("""{"name":"jack","greeting":"hello world"}""") match {
case Some(m: Map[_,_]) =>
<test>
{
m map { case (k,v) =>
XML.loadString("<%s>%s</%s>".format(k,v,k))
}
}
</test>
case _ =>
}
parseFull returns an Option because the string may not be valid JSON (in which case it will return None instead of Some).
The usual way to get the value out of a Some is to pattern match against it like this:
result match {
case Some(map) =>
doSomethingWith(map)
case None =>
handleTheError()
}
If you're certain the input will always be valid and so you don't need to handle the case of invalid input, you can use the get method on the Option, which will throw an exception when called on None.
You have two separate problems.
It's typed as Any.
Your data is inside an Option and a Map.
Let's suppose we have the data:
val x: Option[Any] = Some(Map("name" -> "jack", "greeting" -> "hi"))
and suppose that we want to return the appropriate XML if there is something to return, but not otherwise. Then we can use collect to gather those parts that we know how to deal with:
val y = x collect {
case m: Map[_,_] => m collect {
case (key: String, value: String) => key -> value
}
}
(note how we've taken each entry in the map apart to make sure it maps a string to a string--we wouldn't know how to proceed otherwise. We get:
y: Option[scala.collection.immutable.Map[String,String]] =
Some(Map(name -> jack, greeting -> hi))
Okay, that's better! Now if you know which fields you want in your XML, you can ask for them:
val z = for (m <- y; name <- m.get("name"); greet <- m.get("greeting")) yield {
<test><name>{name}</name><greeting>{greet}</greeting></test>
}
which in this (successful) case produces
z: Option[scala.xml.Elem] =
Some(<test><name>jack</name><greeting>hi</greeting></test>)
and in an unsuccessful case would produce None.
If you instead want to wrap whatever you happen to find in your map in the form <key>value</key>, it's a bit more work because Scala doesn't have a good abstraction for tags:
val z = for (m <- y) yield <test>{ m.map { case (tag, text) => xml.Elem(null, tag, xml.Null, xml.TopScope, xml.Text(text)) }}</test>
which again produces
z: Option[scala.xml.Elem] =
Some(<test><name>jack</name><greeting>hi</greeting></test>)
(You can use get to get the contents of an Option, but it will throw an exception if the Option is empty (i.e. None).)