I am working on a scala application. I have a string as follows:
val str = abc,def,xyz
I want to split this string and access splitted parts separately like abc , def and xyx. My code is as follows
val splittedString = str.split(',')
to access each part of this splitted string I am trying something like this splittedString._1, splittedString._2, splittedString._3. But intellij is giving me error stating "cannot resolve symbol _1" and same error for part 2 and 3 as well. How can I access each element of splitted string?
The method split is defined over Strings to return an Array[String].
What you can do is access by (zero-based) index, splittedString(0) being the first item.
Alternatively, if you know the length of resulting array you want to obtain, you can convert it to a tuple and access with the accessor methods you were referring to:
val tuple =
str.split(",") match {
case Array(a, b, c) => (a, b, c)
case _ => throw new IllegalArgumentException
}
tuple._1 will now contain abc in your example.
Related
I have a List of “rules” tuples as follows:
val rules = List[(A, List[B])], where A and B are two separate case-classes
For the purposes of my use-case, I need to convert this to an Option[(A, List[B])]. The A case class contains a property id which is of type Option[String], based on which the tuple is returned.
I have written a function def findRule(entryId: Option[String]), from which I intend to return the tuple (A, List[B]) whose A.id = entryId as an Option. So far, I have written the following snippet of code:
def findRule(entryId: Option[String]) = {
for {
ruleId <- rules.flatMap(_._1.id) // ruleId is a String
id <- entryId // entryId is an Option[String]
} yield {
rules.find(_ => ruleId.equalsIgnoreCase(id)) // returns List[Option[(A, List[B])]
}
}
This snippet returns a List[Option[(A, List[B])] but I can’t figure out how to retrieve just the Option from it. Using .head() isn’t an option, since the rules list may be empty. Please help as I am new to Scala.
Example (the real examples are too large to post here, so please consider this representative example):
val rules = [(A = {id=1, ….}, B = [{B1}, {B2}, {B3}, …]), (A={id=2, ….}, B = [{B10}, {B11}, {B12}, …]), …. ] (B is not really important here, I need to find the tuple based on the id element of case-class A)
Now, suppose entryId = Some(1)
After the findRule() function, this would currently look like:
[Some((A = {id=1, …}, B = [{B1}, {B2}, {B3}, …]))]
I want to return:
Some((A = {id=1, …}, B = [{B1}, {B2}, {B3}, …])) , ie, the Option within the List returned (currently) from findRule()
From your question, it sounds like you're trying to pick a single item from your list based on some conditional, which means you'll probably want to start with rules.find. The problem then becomes how to express the predicate function that you pass to find.
From my read of your question, the conditional is
The id on the A part of the tuple needs to match the entryId that was passed in elsewhere
def findRule(entryId: Option[String]) =
rules.find { case (a, listOfB) => entryId.contains(a.id) }
The case expression is just nice syntax for dealing with the tuple. I could have also done
rules.find { tup => entryId.contains(tup._1.id) }
The contains method on Option roughly does "if I'm a Some, see if my value equals the argument; if I'm a None, just return false and ignore the argument". It works as a comparison between the Option[String] you have for entryId and the plain String you have for A's id.
Your first attempt didn't work because rules.flatMap got you a List, and using that after the <- in the for-comprehension means another flatMap, which keeps things as a List.
A variant that I personally prefer is
def findRule(entryId: Option[String]): Option[(A, List[B])] = {
entryId.flatMap { id =>
rules.find { case (a, _) => a.id == id }
}
}
In the case where entryId is None, it just immediately returns None. If you start with rules.find, then it will iterate through all of the rules and check each one even when entryId is None. It's unlikely to make any observable performance difference if the list of rules is small, but I also find it more intuitive to understand.
I'm very new to Scala and trying to use it as an interface to Spark. I'm running into a problem making a generic CSV to DataFrame function. For example, I've got a CSV with about 50 fields, the first of which are task, name, and id. I can get the following to work:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case Array(task, name, id, _*) => Row(task, name, id)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
However, I'd rather not have to hard code the number of fields needed to create a spark Row. I tried this:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case Array(args # _*) => Row(args)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
But it just creates a Row object with a single element. How can I make it expand the args in Row(args) so that if I have an array of N elements I'll get a Row with N elements?
Change your input to be variable length by adding _*:
Row(args:_*)
This is what Row accepts per its apply signature.
In fact, you don't even need to do anything other than pass this in to the Row as it is already of the right sequence type.
reader.readAll().map(Row(_:_*))
This should do the trick:
val reader = new CSVReader(new StringReader(txt))
reader.readAll().map(_ match {
case a: Array[String] => Row(a:_*)
case unexpectedArrayForm =>
throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})
Edited to correct omission of Array type
I'm writing methods to convert Set[Tuple2[String, String]] to String and vice versa.
I'm saving the string value as v1,v2#v3,v4#v5,v6
In order to fill the Set I'm splitting the string by ',' and in order to extract the values I'm trying to split each value by '#' but i receive
type mismatch: found: x.type (with underlying type Array[String]
The code I tried using is
val x = overwriters.split("#")
for(tuple <- x) {
tuple.split(",")
}
The returned type of split is an array of String so it is not clear to me why i cannot split each member of the returned array
tuple.split(",") returns an array of two elements. You need to convert it to a tuple.
val overwriters ="v1,v2#v3,v4#v5,v6"
val x = overwriters.split("#").toSet
for(tuple <- x) yield {
val t = tuple.split(",")
(t(0),t(1))
}
overwrites.split("#").map(_.split(",")).map(x=> (x(0),x(1))).toSet
This will achieve the same in little more idiomatic way.
My Query is, read input from a file and convert data lines of the file to List[Map[Int,String]] using scala. Here I give a dataset as the input. My code is,
def id3(attrs: Attributes,
examples: List[Example],
label: Symbol
) : Node = {
level = level+1
// if all the examples have the same label, return a new node with that label
if(examples.forall( x => x(label) == examples(0)(label))){
new Leaf(examples(0)(label))
} else {
for(a <- attrs.keySet-label){ //except label, take all attrs
("Information gain for %s is %f".format(a,
informationGain(a,attrs,examples,label)))
}
// find the best splitting attribute - this is an argmax on a function over the list
var bestAttr:Symbol = argmax(attrs.keySet-label, (x:Symbol) =>
informationGain(x,attrs,examples,label))
// now we produce a new branch, which splits on that node, and recurse down the nodes.
var branch = new Branch(bestAttr)
for(v <- attrs(bestAttr)){
val subset = examples.filter(x=> x(bestAttr)==v)
if(subset.size == 0){
// println(levstr+"Tiny subset!")
// zero subset, we replace with a leaf labelled with the most common label in
// the examples
val m = examples.map(_(label))
val mostCommonLabel = m.toSet.map((x:Symbol) => (x,m.count(_==x))).maxBy(_._2)._1
branch.add(v,new Leaf(mostCommonLabel))
}
else {
// println(levstr+"Branch on %s=%s!".format(bestAttr,v))
branch.add(v,id3(attrs,subset,label))
}
}
level = level-1
branch
}
}
}
object samplet {
def main(args: Array[String]){
var attrs: sample.Attributes = Map()
attrs += ('0 -> Set('abc,'nbv,'zxc))
attrs += ('1 -> Set('def,'ftr,'tyh))
attrs += ('2 -> Set('ghi,'azxc))
attrs += ('3 -> Set('jkl,'fds))
attrs += ('4 -> Set('mno,'nbh))
val examples: List[sample.Example] = List(
Map(
'0 -> 'abc,
'1 -> 'def,
'2 -> 'ghi,
'3 'jkl,
'4 -> 'mno
),
........................
)
// obviously we can't use the label as an attribute, that would be silly!
val label = 'play
println(sample.try(attrs,examples,label).getStr(0))
}
}
But How I change this code to - accepting input from a .csv file?
I suggest you use Java's io / nio standard library to read your CSV file. I think there is no relevant drawback in doing so.
But the first question we need to answer is where to read the file in the code? The parsed input seems to replace the value of examples. This fact also hints us what type the parsed CSV input must have, namely List[Map[Symbol, Symbol]]. So let us declare a new class
class InputFromCsvLoader(charset: Charset = Charset.defaultCharset()) {
def getInput(file: Path): List[Map[Symbol, Symbol]] = ???
}
Note that the Charset is only needed if we must distinguish between differently encoded CSV-files.
Okay, so how do we implement the method? It should do the following:
Create an appropriate input reader
Read all lines
Split each line at the comma-separator
Transform each substring into the symbol it represents
Build a map from from the list of symbols, using the attributes as key
Create and return the list of maps
Or expressed in code:
class InputFromCsvLoader(charset: Charset = Charset.defaultCharset()) {
val Attributes = List('outlook, 'temperature, 'humidity, 'wind, 'play)
val Separator = ","
/** Get the desired input from the CSV file. Does not perform any checks, i.e., there are no guarantees on what happens if the input is malformed. */
def getInput(file: Path): List[Map[Symbol, Symbol]] = {
val reader = Files.newBufferedReader(file, charset)
/* Read the whole file and discard the first line */
inputWithHeader(reader).tail
}
/** Reads all lines in the CSV file using [[java.io.BufferedReader]] There are many ways to do this and this is probably not the prettiest. */
private def inputWithHeader(reader: BufferedReader): List[Map[Symbol, Symbol]] = {
(JavaConversions.asScalaIterator(reader.lines().iterator()) foldLeft Nil.asInstanceOf[List[Map[Symbol, Symbol]]]){
(accumulator, nextLine) =>
parseLine(nextLine) :: accumulator
}.reverse
}
/** Parse an entry. Does not verify the input: If there are less attributes than columns or vice versa, zip creates a list of the size of the shorter list */
private def parseLine(line: String): Map[Symbol, Symbol] = (Attributes zip (line split Separator map parseSymbol)).toMap
/** Create a symbol from a String... we could also check whether the string represents a valid symbol */
private def parseSymbol(symbolAsString: String): Symbol = Symbol(symbolAsString)
}
Caveat: Expecting only valid input, we are certain that the individual symbol representations do not contain the comma-separation character. If this cannot be assumed, then the code as is would fail to split certain valid input strings.
To use this new code, we could change the main-method as follows:
def main(args: Array[String]){
val csvInputFile: Option[Path] = args.headOption map (p => Paths get p)
val examples = (csvInputFile map new InputFromCsvLoader().getInput).getOrElse(exampleInput)
// ... your code
Here, examples uses the value exampleInput, which is the current, hardcoded value of examples if no input argument is specified.
Important: In the code, all error handling has been omitted for convenience. In most cases, errors can occur when reading from files and user input must be considered invalid, so sadly, error handling at the boundaries of your program is usally not optional.
Side-notes:
Try not to use null in your code. Returning Option[T] is a better option than returning null, because it makes "nullness" explicit and provides static safety thanks to the type-system.
The return-keyword is not required in Scala, as the last value of a method is always returned. You can still use the keyword if you find the code more readable or if you want to break in the middle of your method (which is usually a bad idea).
Prefer val over var, because immutable values are much easier to understand than mutable values.
The code will fail with the provided CSV string, because it contains the symbols TRUE and FALSE which are not legal according to your programs logic (they should be true and false instead).
Add all information to your error-messages. Your error message only tells me what that a value for the attribute 'wind is bad, but it does not tell me what the actual value is.
Read a csv file ,
val datalines = Source.fromFile(filepath).getLines()
So this datalines contains all the lines from the csv file.
Next, convert each line into a Map[Int,String]
val datamap = datalines.map{ line =>
line.split(",").zipWithIndex.map{ case (word, idx) => idx -> word}.toMap
}
Here, we split each line with ",". Then construct a map with key as column number and value as each word after the split.
Next, If we want List[Map[Int,String]],
val datamap = datalines.map{ line =>
line.split(",").zipWithIndex.map{ case (word, idx) => idx -> word}.toMap
}.toList
Consider the following from the Scala interpreter:
scala> JSON.parseFull("""{"name":"jack","greeting":"hello world"}""")
res6: Option[Any] = Some(Map(name -> jack, greeting -> hello world))
Why is the Map returned in Some() thing? And how do I work with it?
I want to put the values in an xml template:
<test>
<name>name goes here</name>
<greeting>greeting goes here</greeting>
</test>
What is the Scala way of getting my map out of Some(thing) and getting those values in the xml?
You should probably use something like this:
res6 collect { case x: Map[String, String] => renderXml(x) }
Where:
def renderXml(m: Map[String, String]) =
<test><name>{m.get("name") getOrElse ""}</name></test>
The collect method on Option[A] takes a PartialFunction[A, B] and is a combination of filter (by a predicate) and map (by a function). That is:
opt collect pf
opt filter (a => pf isDefinedAt a) map (a => pf(a))
Are both equivalent. When you have an optional value, you should use map, flatMap, filter, collect etc to transform the option in your program, avoiding extracting the option's contents either via a pattern-match or via the get method. You should never, ever use Option.get - it is the canonical sign that you are doing it wrong. Pattern-matching should be avoided because it represents a fork in your program and hence adds to cyclomatic complexity - the only time you might wish to do this might be for performance
Actually you have the issue that the result of the parseJSON method is an Option[Any] (the reason is that it is an Option, presumably, is that the parsing may not succeed and Option is a more graceful way of handling null than, well, null).
But the issue with my code above is that the case x: Map[String, String] cannot be checked at runtime due to type erasure (i.e. scala can check that the option contains a Map but not that the Map's type parameters are both String. The code will get you an unchecked warning.
An Option is returned because parseFull has different possible return values depending on the input, or it may fail to parse the input at all (giving None). So, aside from an optional Map which associates keys with values, an optional List can be returned as well if the JSON string denoted an array.
Example:
scala> import scala.util.parsing.json.JSON._
import scala.util.parsing.json.JSON._
scala> parseFull("""{"name":"jack"}""")
res4: Option[Any] = Some(Map(name -> jack))
scala> parseFull("""[ 100, 200, 300 ]""")
res6: Option[Any] = Some(List(100.0, 200.0, 300.0))
You might need pattern matching in order to achieve what you want, like so:
scala> parseFull("""{"name":"jack","greeting":"hello world"}""") match {
| case Some(m) => Console println ("Got a map: " + m)
| case _ =>
| }
Got a map: Map(name -> jack, greeting -> hello world)
Now, if you want to generate XML output, you can use the above to iterate over the key/value pairs:
import scala.xml.XML
parseFull("""{"name":"jack","greeting":"hello world"}""") match {
case Some(m: Map[_,_]) =>
<test>
{
m map { case (k,v) =>
XML.loadString("<%s>%s</%s>".format(k,v,k))
}
}
</test>
case _ =>
}
parseFull returns an Option because the string may not be valid JSON (in which case it will return None instead of Some).
The usual way to get the value out of a Some is to pattern match against it like this:
result match {
case Some(map) =>
doSomethingWith(map)
case None =>
handleTheError()
}
If you're certain the input will always be valid and so you don't need to handle the case of invalid input, you can use the get method on the Option, which will throw an exception when called on None.
You have two separate problems.
It's typed as Any.
Your data is inside an Option and a Map.
Let's suppose we have the data:
val x: Option[Any] = Some(Map("name" -> "jack", "greeting" -> "hi"))
and suppose that we want to return the appropriate XML if there is something to return, but not otherwise. Then we can use collect to gather those parts that we know how to deal with:
val y = x collect {
case m: Map[_,_] => m collect {
case (key: String, value: String) => key -> value
}
}
(note how we've taken each entry in the map apart to make sure it maps a string to a string--we wouldn't know how to proceed otherwise. We get:
y: Option[scala.collection.immutable.Map[String,String]] =
Some(Map(name -> jack, greeting -> hi))
Okay, that's better! Now if you know which fields you want in your XML, you can ask for them:
val z = for (m <- y; name <- m.get("name"); greet <- m.get("greeting")) yield {
<test><name>{name}</name><greeting>{greet}</greeting></test>
}
which in this (successful) case produces
z: Option[scala.xml.Elem] =
Some(<test><name>jack</name><greeting>hi</greeting></test>)
and in an unsuccessful case would produce None.
If you instead want to wrap whatever you happen to find in your map in the form <key>value</key>, it's a bit more work because Scala doesn't have a good abstraction for tags:
val z = for (m <- y) yield <test>{ m.map { case (tag, text) => xml.Elem(null, tag, xml.Null, xml.TopScope, xml.Text(text)) }}</test>
which again produces
z: Option[scala.xml.Elem] =
Some(<test><name>jack</name><greeting>hi</greeting></test>)
(You can use get to get the contents of an Option, but it will throw an exception if the Option is empty (i.e. None).)