I have a csv file of countries and a CountryData case class
Example data from file:
Denmark, Europe, 1.23, 7.89
Australia, Australia, 8.88, 9.99
Brazil, South America, 7.77,3.33
case class CountryData(country: String, region: String, population: Double, economy: Double)
I can read in the file and split, etc to get
(List(Denmark, Europe, 1.23, 7.89)
(List(Australia, Australia, 8.88, 9.99)
(List(Brazil, South America, 7.77,3.33)
How can I now populate a CountryData case class for each list item?
I've tried:
for (line <- Source.getLines.drop(1)) {
val splitInput = line.split(",", -1).map(_.trim).toList
val country = splitInput(0)
val region = splitInput(1)
val population = splitInput(2)
val economy = splitInput(3)
val dataList: List[CountryData]=List(CountryData(country,region,population,economy))
But that doesn't work because it's not reading the val, it sees it as a string 'country' or 'region'.
It is not clear where exactly is your issue. Is it about Double vs String or about List being inside the loop. Still something like this will probably work
case class CountryData(country: String, region: String, population: Double, economy: Double)
object CountryDataMain extends App {
val src = "\nDenmark, Europe, 1.23, 7.89\nAustralia, Australia, 8.88, 9.99\nBrazil, South America, 7.77,3.33"
val list = Source.fromString(src).getLines.drop(1).map(line => {
val splitInput = line.split(",", -1).map(_.trim).toList
val country = splitInput(0)
val region = splitInput(1)
val population = splitInput(2)
val economy = splitInput(3)
CountryData(country, region, population.toDouble, economy.toDouble)
}).toList
println(list)
}
I would use scala case matching: i.e.
def doubleOrNone(str: Double): Option[Double] = {
Try {
Some(str.toDouble) //Not sure of exact name of method, should be quick to find
} catch {
case t: Throwable => None
}
}
def parseCountryLine(line: String): Vector[CountryData] = {
lines.split(",").toVector match {
case Vector(countryName, region, population, economy) => CountryData(countryName, region, doubleOrNone(population), doubleOrNone(economy))//I would suggest having them as options because you may not have data for all countries
case s => println(s"Error parsing line:\n$s");
}
}
Related
I am kind of stuck with this, and I know this is a bloody simple question :(
I have a case class such as:
case class Students(firstName: String, lastName: String, hobby: String)
I need to return a new list but change the value of hobby based on Student name. For example:
val classToday = List(Students("John","Smith","Nothing"))
Say if student name is John I want to change the hobby to Soccer so the resulting list should be:
List(Students("John","Smith","Soccer")
I think this can be done via map? I have tried:
classToday.map(x => if (x.firstName == "John") "Soccer" else x)
This will just replace firstName with Soccer which I do not want, I tried setting the "True" condition to x.hobby == "Soccer" but that does not work.
I think there is a simple solution to this :(
The lambda function in map has to return a Students value again, not just "Soccer". For example, if you had to replace everyone's hobbies with "Soccer", this is not right:
classToday.map(x => "Soccer")
What you want is the copy function:
classToday.map(x => x.copy(hobby = "Soccer"))
Or for the original task:
classToday.map(x => if (x.firstName == "John") x.copy(hobby = "Soccer") else x)
You can use pattern-matching syntax to pretty up this type of transition.
val newList = classToday.map{
case s#Students("John",_,_) => s.copy(hobby = "Soccer")
case s => s
}
I suggest to make it more generic, you can create a map of names to hobbies:
For example:
val firstNameToHobby = Map("John" -> "Soccer", "Brad" -> "Basketball")
And use it as follows:
case class Students(firstName: String, lastName: String, hobby: String)
val classToday = List(Students("John","Smith","Nothing"), Students("Brad","Smith","Nothing"))
val result = classToday.map(student => student.copy(hobby = firstNameToHobby.getOrElse(student.firstName, "Nothing")))
// result = List(Students(John,Smith,Soccer), Students(Brad,Smith,Basketball))
It would be better if you can create a mapping between the firstName of the student with Hobby, then you can use it like this:
scala> val hobbies = Map("John" -> "Soccer", "Messi" -> "Soccer", "Williams" -> "Cricket")
hobbies: scala.collection.immutable.Map[String,String] = Map(John -> Soccer, Messi -> Soccer, Williams -> Cricket)
scala> case class Student(firstName: String, lastName: String, hobby: String)
defined class Student
scala> val students = List(Student("John", "Smith", "Nothing"), Student("Williams", "Lopez", "Nothing"), Student("Thomas", "Anderson", "Nothing"))
students: List[Student] = List(Student(John,Smith,Nothing), Student(Williams,Lopez,Nothing), Student(Thomas,Anderson,Nothing))
scala> students.map(student => student.copy(hobby = hobbies.getOrElse(student.firstName, "Nothing")))
res2: List[Student] = List(Student(John,Smith,Soccer), Student(Williams,Lopez,Cricket), Student(Thomas,Anderson,Nothing))
I have a quite odd problem to solve, I have a String, a custom Type and a Map of Maps.
The string needs to have a few values replaced based on mapping between a value in custom type (which is a key in the map of maps).
This is the current structure:
case class Students(favSubject: String)
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val studentInfo: List[Students] = List(Students("English"))
val data: String = "John is the favourite hobby"
I tried the following:
mapping.foldLeft(data){ case (outputString, (studentName, favSubject)) => outputString.replace(studentName, favSubject.getOrElse(studentInfo.map(x => x.favSubject).toString, "")) }
What I need to get is:
"Soccer is the favourite hobby"
What I get is:
" is the favourite hobby"
So looks like I am getting the map of maps traversal right but the getOrElse part is having issues.
What I would do, would be to first change the structure of mappings so it makes more sense for the problem.
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val mapping2 =
mapping.iterator.flatMap {
case (student, map) => map.iterator.map {
case (info, value) => (info, student, value)
}
}.toList
.groupBy(_._1)
.view
.mapValues { group =>
group.iterator.map {
case (_, student, value) => student -> value
}.toList
}.toMap
// mapping2: Map[String, List[(String, String)]] = Map("English" -> List(("John", "Soccer")))
Then I would just traverse the students informativo, making all the necessary replacements.
final case class StudentInfo(favSubject: String)
val studentsInformation: List[StudentInfo] = List(StudentInfo("English"))
val data: String = "John is the favourite hobby"
val result =
studentsInformation.foldLeft(data) { (acc, info) =>
mapping2
.getOrElse(key = info.favSubject, default = List.empty)
.foldLeft(acc) { (acc2, tuple) =>
val (key, replace) = tuple
acc2.replace(key, replace)
}
}
// result: String = "Soccer is the favourite hobby"
When you map() a List, you get a List back. It's toString has a format "List(el1,el2,...)". Surely you cannot use it as a key for your sub-map, you would want just el1.
Here is a version of the working code. It might not be a solution you are looking for(!), just a solution to your question:
case class Students(favSubject: String)
val mapping: Map[String, Map[String, String]] = Map("John" -> Map("English" -> "Soccer"))
val studentInfo: List[Students] = List(Students("English"))
val data: String = "John is the favourite hobby"
val res = mapping.foldLeft(data) {
case (outputString, (studentName, favSubjectDict)) =>
outputString.replace(
studentName,
favSubjectDict.getOrElse(studentInfo.map(x => x.favSubject).head, "?")
)
}
println(s"$res") //prints "Soccer is the favourite hobby"
val notMatchingSubject = studentInfo.map(x => x.favSubject).toString
println(s"Problem in previous code: '$notMatchingSubject' !== 'English'")
Try it here: https://scastie.scala-lang.org/flQNRrUQSXWPxSTXOPPFgA
The issue
It is a bit unclear why StudentInfo is a List in this form... If I guessed, it was designed to be a list of StudentInfo containing both, name and favSubject and you would need to search it by name to find favSubject. But it is just a guess.
I went with the simplest working solution, to get a .head (first element) of the sequence from the map. Which will always be "English" even if you add more Studends to the list.
I am trying to parse a text file. My input file looks like this:
ID: 12343-7888
Name: Mary, Bob, Jason, Jeff, Suzy
Harry, Steve
Larry, George
City: New York, Portland, Dallas, Kansas City
Tampa, Bend
Expected output would:
“12343-7888”
“Mary, Bob, Jason, Jeff, Suzy, Harry, Steve, Larry, George”
“New York, Portland, Dallas, Kansas City, Tampa, Bend"
Note the “Name” and "City" fields have new lines or returns in them. I have this code below, but it is not working. The second line of code places each character in a line. Plus, I am having troubles only grabbing the data from the field, like only returning the actual names, where the “Name: “ is not part of the results. Also, looking to put quotes around each return field.
Can you help fix up my problems?
val lines = Source.fromFile("/filesdata/logfile.text").getLines().toList
val record = lines.dropWhile(line => !line.startsWith("Name: ")).takeWhile(line => !line.startsWith("Address: ")).flatMap(_.split(",")).map(_.trim()).filter(_.nonEmpty).mkString(", ")
val final results record.map(s => "\"" + s + "\"").mkString(",\n")
How can I get my results that I am looking for?
SHORT ANSWER
A two-liner that produces a string that looks exactly as you specified:
println(lines.map{line => if(line.trim.matches("[a-zA-Z]+:.*"))
("\"\n\"" + line.split(":")(1).trim) else (", " + line.trim)}.mkString.drop(2) + "\"")
LONG ANSWER
Why try to solve something in one line, if you can achieve the same thing in 94?
(That's the exact opposite of the usual slogan when working with Scala collections, but the input was sufficiently messy that I found it worthwhile to actually write out some of the intermediate steps. Maybe that's just because I've bought a nice new keyboard recently...)
val input = """ID: 12343-7888
Name: Mary, Bob, Jason, Jeff, Suzy
Harry, Steve
Larry, George
City: New York, Portland, Dallas, Kansas City
Tampa, Bend
ID: 567865-676
Name: Alex, Bob
Chris, Dave
Evan, Frank
Gary
City: Los Angeles, St. Petersburg
Washington D.C., Phoenix
"""
case class Entry(id: String, names: List[String], cities: List[String])
def parseMessyInput(input: String): List[Entry] = {
// just a first rought approximation of the structure of the input
sealed trait MessyInputLine { def content: String }
case class IdLine(content: String) extends MessyInputLine
case class NameLine(content: String) extends MessyInputLine
case class UnlabeledLine(content: String) extends MessyInputLine
case class CityLine(content: String) extends MessyInputLine
val lines = input.split("\n").toList
// a helper function for checking whether a line starts with a label
def tryParseLabeledLine
(label: String, line: String)
(cons: String => MessyInputLine)
: Option[MessyInputLine] = {
if (line.startsWith(label + ":")) {
Some(cons(line.drop(label.size + 1)))
} else {
None
}
}
val messyLines: List[MessyInputLine] = for (line <- lines) yield {
(
tryParseLabeledLine("Name", line){NameLine(_)} orElse
tryParseLabeledLine("City", line){CityLine(_)} orElse
tryParseLabeledLine("ID", line){IdLine(_)}
).getOrElse(UnlabeledLine(line))
}
/** Combines the content of the first line with the content
* of all unlabeled lines, until the next labeled line or
* the end of the list is hit. Returns the content of
* the first few lines and the list of the remaining lines.
*/
def readUntilNextLabel(messyLines: List[MessyInputLine])
: (List[String], List[MessyInputLine]) = {
messyLines match {
case Nil => (Nil, Nil)
case h :: t => {
val (unlabeled, rest) = t.span {
case UnlabeledLine(_) => true
case _ => false
}
(h.content :: unlabeled.map(_.content), rest)
}
}
}
/** Glues multiple lines to entries */
def combineToEntries(messyLines: List[MessyInputLine]): List[Entry] = {
if (messyLines.isEmpty) Nil
else {
val (idContent, namesCitiesRest) = readUntilNextLabel(messyLines)
val (namesContent, citiesRest) = readUntilNextLabel(namesCitiesRest)
val (citiesContent, rest) = readUntilNextLabel(citiesRest)
val id = idContent.head.trim
val names = namesContent.map(_.split(",").map(_.trim).toList).flatten
val cities = citiesContent.map(_.split(",").map(_.trim).toList).flatten
Entry(id, names, cities) :: combineToEntries(rest)
}
}
// invoke recursive function on the entire input
combineToEntries(messyLines)
}
// how to use
val entries = parseMessyInput(input)
// output
for (Entry(id, names, cities) <- entries) {
println(id)
println(names.mkString(", "))
println(cities.mkString(", "))
}
Output:
12343-7888
Mary, Bob, Jason, Jeff, Suzy, Harry, Steve, Larry, George
New York, Portland, Dallas, Kansas City, Tampa, Bend
567865-676
Alex, Bob, Chris, Dave, Evan, Frank, Gary
Los Angeles, St. Petersburg, Washington D.C., Phoenix
You probably could write it down in one line, sooner or later. But if you write dumb code consisting of many simple intermediate steps, you don't have to think that hard, and there are no obstacles large enough to get stuck.
My input file is below . It contains some purchase details for each Customer.
Input:
100,Surender,2015-01-23,PHONE,20000
100,Surender,2015-01-24,LAPTOP,25000
101,Ajay,2015-02-21,LAPTOP,40000
101,Ajay,2015-03-10,MUSIC_SYSTEM,50000
102,Vikram,2015-07-20,WATCH,60000
My requirement is I would like to find out the latest Purchase details for each Customer .
So the expected output is
Expected OutPut:
List(101,Ajay,2015-03-10,MUSIC_SYSTEM,50000)
List(100,Surender,2015-01-24,LAPTOP,25000)
List(102,Vikram,2015-07-20,WATCH,60000)
I tried the below code and it is giving me the expected output..
But this below logic is somewhat similar to java .
My Scala code :
package pack1
import scala.io.Source
import scala.collection.mutable.ListBuffer
object LatestObj {
def main(args:Array[String])=
{
var maxDate ="0001-01-01"
var actualData:List[String] =List()
var resultData:ListBuffer[String] = ListBuffer()
val myList=Source.fromFile("D:\\Scala_inputfiles\\records.txt").getLines().toList;
val myGrped = myList.groupBy { x => x.substring(0,3) }
//println(myGrped)
for(mappedIterator <- myGrped)
{
// println(mappedIterator._2)
actualData =mappedIterator._2
maxDate=findMaxDate(actualData)
println( actualData.filter { x => x.contains(maxDate) })
}
}
def findMaxDate( mytempList:List[String]):String =
{
var maxDate ="0001-01-01"
for(m <- mytempList)
{
var transDate= m.split(",")(2)
if(transDate > maxDate)
{
maxDate =transDate
}
}
return maxDate
}
}
Could some one help me on trying the same approach in a simpler way using scala?
Or The above code is the only way to achieve that logic?
Even simpler version, also using a case class with coincidentally the same name. Doesn't remove bad records like Tzach's, though, and I leave everything as String.
case class Record(id: String, name: String, dateString: String, item: String, count: String)
myList.map { line =>
val Array(id, name, dateString, item, count) = line.split(",")
Record(id, name, dateString, item, count)
}
.groupBy(_.id)
.map(_._2.maxBy(_.dateString))
.toList
Here's a simple version using groupBy and reduce, plus using a convenient case class to elegantly represent records:
case class Record(id: Int, username: String, date: Date, product: String, cost: Double)
val dateFormat: SimpleDateFormat = new SimpleDateFormat("yyyy-MM-dd")
val stringList = Source.fromFile("./records.txt").getLines().toList
// split by comma and parse into case class - while REMOVING bad records
val records = stringList.map(_.split(",")).collect {
case Array(id, username, date, product, cost) => Record(id.toInt, username, dateFormat.parse(date), product, cost.toDouble)
}
// group by key, and reduce each group to latest record
val result = records.groupBy(_.id).map { _._2.reduce {
(r1: Record, r2: Record) => if (r1.date.after(r2.date)) r1 else r2
}}
result.foreach(println)
// prints:
// Record(101,Ajay,Tue Mar 10 00:00:00 IST 2015,MUSIC_SYSTEM,50000.0)
// Record(100,Surender,Sat Jan 24 00:00:00 IST 2015,LAPTOP,25000.0)
// Record(102,Vikram,Mon Jul 20 00:00:00 IDT 2015,WATCH,60000.0)
Note that this implementation does not make any use of mutable variables or collections, which often simplifies the code significantly, and is considered more idiomatic for functional languages like Scala.
I'm a Scala beginner and this piece of code makes me struggle.
Is there a way to do pattern matching to make sure everything i pass to Data is of the correct type? As you can see i have quite strange datatypes...
class Data (
val recipient: String,
val templateText: String,
val templateHtml: String,
val blockMaps: Map[String,List[Map[String,String]]],
templateMap: Map[String,String]
)
...
val dataParsed = JSON.parseFull(message)
dataParsed match {
case dataParsed: Map[String, Any] => {
def e(s: String) = dataParsed get s
val templateText = e("template-text")
val templateHtml = e("template-html")
val recipient = e("email")
val templateMap = e("data")
val blockMaps = e("blkdata")
val dependencies = new Data(recipient, templateText, templateHtml, blockMaps, templateMap)
Core.inject ! dependencies
}
...
I guess your problem is you want to be able to patten match the map that you get from parseFull(), but Map doesn't have an unapply.
So you could pattern match every single value, providing a default if it is not of the correct type:
val templateText: Option[String] = e("template-text") match {
case s: String => Some(s)
case _ => None
}
Or temporarily put all the data into some structure that can be pattern matched:
val data = (e("template-text"), e("template-html"), e("email"), e("data"),
e("blkdata"))
val dependencies: Option[Data] = data match {
case (templateText: String,
templateHtml: String,
blockMaps: Map[String,List[Map[String,String]]],
templateMap: Map[String,String]) =>
Some(new Data(recipient, templateText, templateHtml, blockMaps, templateMap))
case _ => None
}