I am parsing an XML document and store its data in various other structured document formats. In this XML document, the elements reference other elements, such as:
<myCar id="12" name="Porsche XYZ" ...>
<connected refId="3" />
</myCar>
...
<myCar id="3" name="Audi XYZ" ...>
...
</myCar>
Here, refId maps to id. When creating the myCar instance with the id 12, I cannot reference to myCar with id 3, because it has not yet been parsed.
Obviously, the easy solution would be to parse the document twice (instantiate references in the second run, after all elements have been parsed and created). However, for performance reasons I only want to parse the document once. Thus, I thought I could just store the relevant reference data in a case class and build a list of instances that is passed from one method to another, in order to process it after having parsed the entire document.
However, my problem is that the logic for creating the references varies to a great extent. So, I cannot use something like this:
case class Ref (a: String, b: String)
val refs: List[Ref] = List.empty
// 1. fill the list with references during parsing
// 2. after parsing the document, process all references in the list
I think what I need is to move all my reference creation logic to separate methods, and then when parsing the document maintain a list with "pointers" to these methods including the appropriate parameters. In this way, I could just iterate through the list after parsing the entire document, and call every method with the correct parameters.
How can this be achieved?
I'm not 100% sure what you're asking but I think you are asking for a way to link the connected cars to the base case class such that after parsing the XML you have a list of cars and for any given car you can access the refId attribute (materialized as a full car object) from the connected tag.
Here is a simple approach:
given:
val xml = <root>
<myCar id="12" name="Porsche XYZ">
<connected refId="3" />
</myCar>
<myCar id="3" name="Audi XYZ">
</myCar>
</root>
First we'll make a case class to model a myCar:
case class Car(
id: String,
name: String,
connectedId: Option[String]
)
Then we parse the XML into Car instances. I'm going to parse it into a Map[String, Car] where the key is Car.id:
val result = (xml \ "myCar").foldLeft(Map.empty[String, Car]) {
case (acc, next) =>
val id: String = (next \# "id")
val name = (next \# "name")
val connectionStr = (next \ "connected" \# "refId")
val connection = Option.unless(connectionStr.isEmpty)(connectionStr)
val car = Car(
id,
name,
connection
)
acc + (id -> car)
}
Next we need a way to turn connectedId into an actual car. I did this by adding a method to Car changing the case class to:
case class Car(
id: String,
name: String,
connectedId: Option[String]
) {
def getConnected(cars: Map[String, Car]): Option[Car] = {
connectedId.flatMap { id =>
cars.get(id)
}
}
}
This method (getConnected) takes the Map produced in the previous step.
Get the list of cars with:
result.values // Iterable(Car(12,Porsche XYZ,Some(3)), Car(3,Audi XYZ,None))
Get the connected car for the first car in the list:
result.values.head.getConnected(result) // Some(Car(3,Audi XYZ,None))
If you want to "fill in" the connected cars add a field to hold the connected car (pass None in the initial foldLeft above):
case class Car(
id: String,
name: String,
connectedId: Option[String],
connected: Option[Car],
) {
def getConnected(cars: Map[String, Car]): Option[Car] = {
connectedId.flatMap { id =>
cars.get(id)
}
}
}
Then just map over the list, adding the connections:
result.values
.map { car =>
val connectedCar = car.connectedId.flatMap { id =>
car.getConnected(result)
}
car.copy(connected = connectedCar)
}
This produces:
List(Car(12,Porsche XYZ,Some(3),Some(Car(3,Audi XYZ,None,None))), Car(3,Audi XYZ,None,None))
This does not recursively fill in the connected cars. You'd have to switch to either make this recursive somehow or use a var in Car to track the connected car and modify references instead of using .copy to accomplish that. I haven't thought about this too much though.
Full working code here: https://scastie.scala-lang.org/YuhNdszQROKTaNExMchaCg
Related
i have a problem. I'm having trouble creating a method that takes a user's name as input and manages how many times a user is named. Basically I have this map as data:
private var players: Seq[GamePlayer] = _
and game player:
case class GamePlayer(override val id: String, username: String, override val actorRef: ActorRef) extends Player
from this map I have to create a method that, taken as input a name of a user, creates a two-dimensional array Array [String] [Int] in which a name is associated with the number of times that user is named. Any ideas about it?
I managed to do this but I just don't know how to create and manage a two-dimensional array at scale.
sorry but I'm new to scala and I'm only starting to understand things now ^^"
Thanks
private def manageVote(username: String): Unit = {
//var matrixOfVotes = Array.ofDim[String][Int](this.numberOfPlayers,2)
var numberOfVotes = this.numberOfPlayers
var votes = new Array[Int](this.numberOfPlayers)
isEmpty(username) match {
case true => numberOfVotes=numberOfVotes-1
case false => numberOfVotes=numberOfVotes-1
votes.add(players.indexOf(username))
}
}
and isEmpty:
private def isEmpty(x: String) = Option(x).forall(_.isEmpty)
First of all, there is no such a thing in Java/Scala Array's as you tried to describe Array [String] [Int] - two dimensional array as name stands is an array of arrays, like: Array[Array[Int]].
What you want is Map[String, Int]. If you would like to, how many times Player with specific name appears in Seq[GamePlayer] you can use just groupBy operation, like:
players.groupBy(_.username).map {
case(username, players) => username -> players.size
}
I have two lists:
val generalInfo = List[GeneralInfo]
val countInfo = List[CountInfo]
case class GeneralInfo(id: String, source: String, languages: Array[String], var count: BigDecimal)
case class CountInfo(id: String, count: BigDecimal)
Every GeneralInfo object is initialized with count = 0;
I need to fill in the count variable in GeneralInfo object, with the count value from CountInfo object, when the id of them are the same. (not all the id's in GeneralInfo list are in CountInfo list)
I am quite new to Scala, is there any way to do it elegantly without a use of dictionary?
If you know that there's a one-to-one relation, you can go through every CountInfo, find the corresponding GeneralInfo object, and set the count for that GeneralInfo.
countInfo.foreach(c => generalInfo.find(_.id == c.id).map(_.count = c.count))
If one id can be shared by many GeneralInfo objects, use filter instead of find:
countInfo.foreach(c =>
generalInfo.filter(_.id == c.id).foreach(_.count = c.count)
)
You can also do it the other way:
generalInfo.foreach(g => countInfo.find(_.id == g.id).map(c => g.count = c.count))
Demo in Scastie
I'm new to Scala and attempting to do some data analysis.
I have a CSV files with a few headers - lets say item no., item type, month, items sold.
I have made an Item class with the fields of the headers.
I split the CSV into a list with each iteration of the list being a row of the CSV file being represented by the Item class.
I am attempting to make a method that will create maps based off of the parameter I send in. For example if I want to group the items sold by month, or by item type. However I am struggling to send the Item.field into a method.
F.e what I am attempting is something like:
makemaps(Item.month);
makemaps(Item.itemtype);
def makemaps(Item.field):
if (item.field==Item.month){}
else (if item.field==Item.itemType){}
However my logic for this appears to be wrong. Any ideas?
def makeMap[T](items: Iterable[Item])(extractKey: Item => T): Map[T, Iterable[Item]] =
items.groupBy(extractKey)
So given this example Item class:
case class Item(month: String, itemType: String, quantity: Int, description: String)
You could have (I believe the type ascriptions are mandatory):
val byMonth = makeMap[String](items)(_.month)
val byType = makeMap[String](items)(_.itemType)
val byQuantity = makeMap[Int](items)(_.quantity)
val byDescription = makeMap[String](items)(_.description)
Note that _.month, for instance, creates a function taking an Item which results in the String contained in the month field (simplifying a little).
You could, if so inclined, save the functions used for extracting keys in the companion object:
object Item {
val month: Item => String = _.month
val itemType: Item => String = _.itemType
val quantity: Item => Int = _.quantity
val description: Item => String = _.description
// Allows us to determine if using a predefined extractor or using an ad hoc one
val extractors: Set[Item => Any] = Set(month, itemType, quantity, description)
}
Then you can pass those around like so:
val byMonth = makeMap[String](items)(Item.month)
The only real change semantically is that you explicitly avoid possible extra construction of lambdas at runtime, at the cost of having the lambdas stick around in memory the whole time. A fringe benefit is that you might be able to cache the maps by extractor if you're sure that the source Items never change: for lambdas, equality is reference equality. This might be particularly useful if you have some class representing the collection of Items as opposed to just using a standard collection, like so:
object Items {
def makeMap[T](items: Iterable[Item])(extractKey: Item => T): Map[T,
Iterable[Item]] =
items.groupBy(extractKey)
}
class Items(val underlying: immutable.Seq[Item]) {
def makeMap[T](extractKey: Item => T): Map[T, Iterable[Item]] =
if (Item.extractors.contains(extractKey)) {
if (extractKey == Item.month) groupedByMonth.asInstanceOf[Map[T, Iterable[Item]]]
else if (extractKey == Item.itemType) groupedByItemType.asInstanceOf[Map[T, Iterable[Item]]]
else if (extractKey == Item.quantity) groupedByQuantity.asInstanceOf[Map[T, Iterable[Item]]]
else if (extractKey == Item.description) groupedByDescription.asInstanceOf[Map[T, Iterable[Item]]]
else throw new AssertionError("Shouldn't happen!")
} else {
Items.makeMap(underlying)(extractKey)
}
lazy val groupedByMonth = Items.makeMap[String](underlying)(Item.month)
lazy val groupedByItemType = Items.makeMap[String](underlying)(Item.itemType)
lazy val groupedByQuantity = Items.makeMap[Int](underlying)(Item.quantity)
lazy val groupedByDescription = Items.makeMap[String](underlying)(Item.description)
}
(that is almost certainly a personal record for asInstanceOfs in a small block of code... I'm not sure if I should be proud or ashamed of this snippet)
I was trying Active-Slick and was able to execute active slick example https://github.com/reactivemaster/active-slick-example
But i am not sure how to manage associations using Active-slick. Please provide example.
Also i tried to achieve it using below method but not sure is it good way of doing and is it still eligible to be called as active record pattern.
BookService.scala
val book= Book(None,"Harry Potter")
val action = for {
id <- bookDao.insert(acc)
y<-authorDao.insert(new Author(None,id,"J.K.Rowling"))
}yield y
db.run(action.transactionally
We use UUIDs for the ID column and they are generated in the Scala code, not by the database. I don't know how this will work with your "active record pattern" but it is nice because you can associate objects all you want before having to talk to the database. I also prefer this typed Id[T] in favour of the individual types like BookId and AuthorId.
case class Id[+T](value: String) extends MappedTo[String]
case object Id {
def generate[T]: Id[T] = Id[T](java.util.UUID.randomUUID().toString)
}
case class Author(authorId: Id[Author], name: String)
case class Book(authorId: Id[Book], title: String, authorId: Id[Author])
val newAuthor = Author(Id.generate, "JK Rowling")
val newBook = Book(Id.generate, "Harry Potter", newAuthor.id)
// do other stuff?
val action = for {
_ <- authorDao.insert(newAuthor)
_ <- bookDao.insert(newBook)
} yield 1
db.run(action.transactionally)
Hope this helps.
Using Monocle I can define a Lens to read a case class member without issue,
val md5Lens = GenLens[Message](_.md5)
This can used to compare the value of md5 between two objects and fail with an error message that includes the field name when the values differ.
Is there a way to produce a user-friendly string from the Lens alone that identifies the field being read by the lens? I want to avoid providing the field name explicitly
val md5LensAndName = (GenLens[Message](_.md5), "md5")
If there is a solution that also works with lenses with more than one component then even better. For me it would be good even if the solution only worked to a depth of one.
This is fundamentally impossible. Conceptually, lens is nothing more than a pair of functions: one to get a value from object and one to obtain new object using a given value. That functions can be implemented by the means of accessing the source object's fields or not. In fact, even GenLens macro can use a chain field accessors like _.field1.field2 to generate composite lenses to the fields of nested objects. That can be confusing at first, but this feature have its uses. For example, you can decouple the format of data storage and representation:
import monocle._
case class Person private(value: String) {
import Person._
private def replace(
array: Array[String], index: Int, item: String
): Array[String] = {
val copy = Array.ofDim[String](array.length)
array.copyToArray(copy)
copy(index) = item
copy
}
def replaceItem(index: Int, item: String): Person = {
val array = value.split(delimiter)
val newArray = replace(array, index, item)
val newValue = newArray.mkString(delimiter)
Person(newValue)
}
def getItem(index: Int): String = {
val array = value.split(delimiter)
array(index)
}
}
object Person {
private val delimiter: String = ";"
val nameIndex: Int = 0
val cityIndex: Int = 1
def apply(name: String, address: String): Person =
Person(Array(name, address).mkString(delimiter))
}
val name: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.nameIndex)
)(
name => person => person.replaceItem(Person.nameIndex, name)
)
val city: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.cityIndex)
)(
city => person => person.replaceItem(Person.cityIndex, city)
)
val person = Person("John", "London")
val personAfterMove = city.set("New York")(person)
println(name.get(personAfterMove)) // John
println(city.get(personAfterMove)) // New York
While not very performant, that example illustrates the idea: Person class don't have city or address fields, but by wrapping data extractor and a string rebuild function into Lens, we can pretend it have them. For more complex objects, lens composition works as usual: inner lens just operates on extracted object, relying on outer one to pack it back.