I am using Scala 2.12 and have required libraries downloaded via build.sbt.
I have my DB output in below format.
Basically, it is like per valuation date and book, there can be multiple currency data.
I have group by on book (majorly), which will have list of Pnl data based on currency.
Just the rough representation:
{ Bookid: 1234,
BookName: EQUITY,
PnlBreakdown: [currency: cad, actual_pnl_local: 100, actual_pnl_cde: 100], [currency: usd, actual_pnl_local: 100, actual_pnl_cde: 130]
}
Basically. Key will be book and value will be list of pnl data.
I have a case class defined as below:
case class PnlData(valuation_date: Option[String], currency: Option[String],pnl_status: Option[String],actual_pnl_local: Option[String] ,actual_pnl_cde: Option[String], actual_pnl_local_me_adj: Option[String] ,actual_pnl_cde_me_adj: Option[String] ) {
override def toString():String= {
s"valuation_date=$valuation_date,currency=$currency,pnl_status=$pnl_status,actual_pnl_local=$actual_pnl_local,actual_pnl_cde=$actual_pnl_cde,actual_pnl_local_me_adj=$actual_pnl_local_me_adj,actual_pnl_cde_me_adj=$actual_pnl_cde_me_adj"
}
}
case class BookLevelDaily(book_id: Option[String], book: Option[String], pnlBreakdown: List[SaPnlData]){
override def toString():String= {
s"book_id=$book_id,book=$book,pnl=$pnlBreakdown"
}
}
Basically, my final object is of type BookLevelDaily.
How do I translate the DB output (above) to my BookLevelDaily object?
I can convert the entire result to the list, but further how should I do groupBy?
val list: List[BookLevelDaily] =
sql"""
|SELECT QUERY TO GET ABOVE RESULTSET
""".stripMargin.map(rs =>
BookLevelDaily(
valuation_date = rs.stringOpt("valuation_date"),
book_id = rs.stringOpt("book_id"),
book = rs.stringOpt("book"),
currency= rs.stringOpt("currency"),
pnl_status= rs.stringOpt("pnl_status"),
actual_pnl_local= rs.stringOpt("actual_pnl_local"),
actual_pnl_cde= rs.stringOpt("actual_pnl_cde"),
actual_pnl_local_me_adj= rs.stringOpt("actual_pnl_local_me_adj"),
actual_pnl_cde_me_adj= rs.stringOpt("actual_pnl_cde_me_adj")
)
).list().apply()
Firstly above is not of type BookLevelDaily. So how to iterate or group by to separate Pnl level data and map it to key (book).
If I understand it correctly, it seems to be a one-to-many relationship (one: book_level_daily, many: pnl_breakdown). If so, check the following documentation.
http://scalikejdbc.org/documentation/one-to-x.html
Related
I am parsing an XML document and store its data in various other structured document formats. In this XML document, the elements reference other elements, such as:
<myCar id="12" name="Porsche XYZ" ...>
<connected refId="3" />
</myCar>
...
<myCar id="3" name="Audi XYZ" ...>
...
</myCar>
Here, refId maps to id. When creating the myCar instance with the id 12, I cannot reference to myCar with id 3, because it has not yet been parsed.
Obviously, the easy solution would be to parse the document twice (instantiate references in the second run, after all elements have been parsed and created). However, for performance reasons I only want to parse the document once. Thus, I thought I could just store the relevant reference data in a case class and build a list of instances that is passed from one method to another, in order to process it after having parsed the entire document.
However, my problem is that the logic for creating the references varies to a great extent. So, I cannot use something like this:
case class Ref (a: String, b: String)
val refs: List[Ref] = List.empty
// 1. fill the list with references during parsing
// 2. after parsing the document, process all references in the list
I think what I need is to move all my reference creation logic to separate methods, and then when parsing the document maintain a list with "pointers" to these methods including the appropriate parameters. In this way, I could just iterate through the list after parsing the entire document, and call every method with the correct parameters.
How can this be achieved?
I'm not 100% sure what you're asking but I think you are asking for a way to link the connected cars to the base case class such that after parsing the XML you have a list of cars and for any given car you can access the refId attribute (materialized as a full car object) from the connected tag.
Here is a simple approach:
given:
val xml = <root>
<myCar id="12" name="Porsche XYZ">
<connected refId="3" />
</myCar>
<myCar id="3" name="Audi XYZ">
</myCar>
</root>
First we'll make a case class to model a myCar:
case class Car(
id: String,
name: String,
connectedId: Option[String]
)
Then we parse the XML into Car instances. I'm going to parse it into a Map[String, Car] where the key is Car.id:
val result = (xml \ "myCar").foldLeft(Map.empty[String, Car]) {
case (acc, next) =>
val id: String = (next \# "id")
val name = (next \# "name")
val connectionStr = (next \ "connected" \# "refId")
val connection = Option.unless(connectionStr.isEmpty)(connectionStr)
val car = Car(
id,
name,
connection
)
acc + (id -> car)
}
Next we need a way to turn connectedId into an actual car. I did this by adding a method to Car changing the case class to:
case class Car(
id: String,
name: String,
connectedId: Option[String]
) {
def getConnected(cars: Map[String, Car]): Option[Car] = {
connectedId.flatMap { id =>
cars.get(id)
}
}
}
This method (getConnected) takes the Map produced in the previous step.
Get the list of cars with:
result.values // Iterable(Car(12,Porsche XYZ,Some(3)), Car(3,Audi XYZ,None))
Get the connected car for the first car in the list:
result.values.head.getConnected(result) // Some(Car(3,Audi XYZ,None))
If you want to "fill in" the connected cars add a field to hold the connected car (pass None in the initial foldLeft above):
case class Car(
id: String,
name: String,
connectedId: Option[String],
connected: Option[Car],
) {
def getConnected(cars: Map[String, Car]): Option[Car] = {
connectedId.flatMap { id =>
cars.get(id)
}
}
}
Then just map over the list, adding the connections:
result.values
.map { car =>
val connectedCar = car.connectedId.flatMap { id =>
car.getConnected(result)
}
car.copy(connected = connectedCar)
}
This produces:
List(Car(12,Porsche XYZ,Some(3),Some(Car(3,Audi XYZ,None,None))), Car(3,Audi XYZ,None,None))
This does not recursively fill in the connected cars. You'd have to switch to either make this recursive somehow or use a var in Car to track the connected car and modify references instead of using .copy to accomplish that. I haven't thought about this too much though.
Full working code here: https://scastie.scala-lang.org/YuhNdszQROKTaNExMchaCg
Given this case class:
case class Categories(fruit: String, amount: Double, mappedTo: String)
I have a list containing the following:
List(
Categories("Others",22.38394964594807,"Others"),
Categories("Others",77.6160503540519,"Others")
)
I want to combine two elements in the list by summing up their amount if they are in the same category, so that the end result in this case would be:
List(Categories("Others",99.99999999999997,"Others"))
How can I do that?
Since groupMapReduce was introduced in Scala 2.13, I'll try to provide another approch to Martinjn's great answer.
Assuming we have:
case class Categories(Fruit: String, amount: Double, mappedTo: String)
val categories = List(
Categories("Apple",22.38394964594807,"Others"),
Categories("Apple",77.6160503540519,"Others")
)
If you want to aggregate by both mappedTo and Fruit
val result = categories.groupBy(c => (c.Fruit, c.mappedTo)).map {
case ((fruit, mappedTo), categories) => Categories(fruit, categories.map(_.amount).sum, mappedTo)
}
Code run can be found at Scastie.
If you want to aggregate only by mappedTo, and choose a random Fruit, you can do:
val result = categories.groupBy(c => c.mappedTo).map {
case (mappedTo, categories) => Categories(categories.head.Fruit, categories.map(_.amount).sum, mappedTo)
}
Code run can be found at Scastie
You want to group your list entries by category, and reduce them to a single value. There is groupMapReduce for that, which groups entries, and then maps the group (you don't need this) and reduces the group to a single value.
given
case class Category(category: String, amount: Double)
if you have a val myList: List[Category], then you want to group on Category#category, and reduce them by merging the members, summing up the amount.
that gives
myList.groupMapReduce(_.category) //group
(identity) //map. We don't need to map, so we use the identity mapping
{
case (Category(name, amount1), Category(_, amount2)) =>
Category(name, amount1 + amount2) }
} //reduce, combine each elements by taking the name, and summing the amojunts
In theory just a groupReduce would have been enough, but that doesn't exist, so we're stuck with the identity here.
I was trying Active-Slick and was able to execute active slick example https://github.com/reactivemaster/active-slick-example
But i am not sure how to manage associations using Active-slick. Please provide example.
Also i tried to achieve it using below method but not sure is it good way of doing and is it still eligible to be called as active record pattern.
BookService.scala
val book= Book(None,"Harry Potter")
val action = for {
id <- bookDao.insert(acc)
y<-authorDao.insert(new Author(None,id,"J.K.Rowling"))
}yield y
db.run(action.transactionally
We use UUIDs for the ID column and they are generated in the Scala code, not by the database. I don't know how this will work with your "active record pattern" but it is nice because you can associate objects all you want before having to talk to the database. I also prefer this typed Id[T] in favour of the individual types like BookId and AuthorId.
case class Id[+T](value: String) extends MappedTo[String]
case object Id {
def generate[T]: Id[T] = Id[T](java.util.UUID.randomUUID().toString)
}
case class Author(authorId: Id[Author], name: String)
case class Book(authorId: Id[Book], title: String, authorId: Id[Author])
val newAuthor = Author(Id.generate, "JK Rowling")
val newBook = Book(Id.generate, "Harry Potter", newAuthor.id)
// do other stuff?
val action = for {
_ <- authorDao.insert(newAuthor)
_ <- bookDao.insert(newBook)
} yield 1
db.run(action.transactionally)
Hope this helps.
Using Monocle I can define a Lens to read a case class member without issue,
val md5Lens = GenLens[Message](_.md5)
This can used to compare the value of md5 between two objects and fail with an error message that includes the field name when the values differ.
Is there a way to produce a user-friendly string from the Lens alone that identifies the field being read by the lens? I want to avoid providing the field name explicitly
val md5LensAndName = (GenLens[Message](_.md5), "md5")
If there is a solution that also works with lenses with more than one component then even better. For me it would be good even if the solution only worked to a depth of one.
This is fundamentally impossible. Conceptually, lens is nothing more than a pair of functions: one to get a value from object and one to obtain new object using a given value. That functions can be implemented by the means of accessing the source object's fields or not. In fact, even GenLens macro can use a chain field accessors like _.field1.field2 to generate composite lenses to the fields of nested objects. That can be confusing at first, but this feature have its uses. For example, you can decouple the format of data storage and representation:
import monocle._
case class Person private(value: String) {
import Person._
private def replace(
array: Array[String], index: Int, item: String
): Array[String] = {
val copy = Array.ofDim[String](array.length)
array.copyToArray(copy)
copy(index) = item
copy
}
def replaceItem(index: Int, item: String): Person = {
val array = value.split(delimiter)
val newArray = replace(array, index, item)
val newValue = newArray.mkString(delimiter)
Person(newValue)
}
def getItem(index: Int): String = {
val array = value.split(delimiter)
array(index)
}
}
object Person {
private val delimiter: String = ";"
val nameIndex: Int = 0
val cityIndex: Int = 1
def apply(name: String, address: String): Person =
Person(Array(name, address).mkString(delimiter))
}
val name: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.nameIndex)
)(
name => person => person.replaceItem(Person.nameIndex, name)
)
val city: Lens[Person, String] =
Lens[Person, String](
_.getItem(Person.cityIndex)
)(
city => person => person.replaceItem(Person.cityIndex, city)
)
val person = Person("John", "London")
val personAfterMove = city.set("New York")(person)
println(name.get(personAfterMove)) // John
println(city.get(personAfterMove)) // New York
While not very performant, that example illustrates the idea: Person class don't have city or address fields, but by wrapping data extractor and a string rebuild function into Lens, we can pretend it have them. For more complex objects, lens composition works as usual: inner lens just operates on extracted object, relying on outer one to pack it back.
So, the problem is in the title, but here are the details.
I have two case classes:
case class JourneyGroup(id: Option[Int] = None,
key: UUID,
name: String,
data: Option[JsValue],
accountId: Int,
createdAt: DateTime = DateTime.now,
createdById: Int,
updatedAt: Option[DateTime] = None,
updatedById: Option[Int] = None,
deletedAt: Option[DateTime] = None,
deletedById: Option[Int] = None)
and
case class JourneyGroupApi(id: Option[Int] = None,
key: UUID,
name: String,
data: Option[JsValue],
accountId: Int,
createdAt: DateTime = DateTime.now,
createdById: Int,
updatedAt: Option[DateTime] = None,
updatedById: Option[Int] = None,
deletedAt: Option[DateTime] = None,
deletedById: Option[Int] = None,
parties: Seq[Party] = Seq.empty[Party])
Background: the reason for having these two separate classes is the fact that slick does not support collections, and I do need collections of related objects that I build manually. Bottom line, I could not make it work with a single class.
What I need is an easy way to convert from one to another.
At this point, to unblock myself, I created a manual conversion:
def toJourneyGroupApi(parties: Seq[Party]): JourneyGroupApi = JourneyGroupApi(
id = id,
key = key,
name = name,
data = data,
accountId = accountId,
createdAt = createdAt,
createdById = createdById,
updatedAt = updatedAt,
updatedById = updatedById,
deletedAt = deletedAt,
deletedById = deletedById,
parties = parties
)
Which is working, but extremely ugly and requires a lot of maintenance.
One thing that I tried doing is:
convert the source object to tuple
Add an element to that tuple using shapeless
and build a target object from resulting tuple
import shapeless._
import syntax.std.tuple._
val groupApi = (JourneyGroup.unapply(group).get :+ Seq.empty[Party])(JourneyGroupApi.tupled)
But, this thing is claiming, that the result of :+ is not tuple, even though in console:
Party.unapply(p).get :+ Seq.empty[Participant]
res0: (Option[Int], model.Parties.Type.Value, Int, Int, org.joda.time.DateTime, Int, Option[org.joda.time.DateTime], Option[Int], Option[org.joda.time.DateTime], Option[Int], Seq[model.Participant]) = (None,client,123,234,2016-11-12T03:55:24.006-08:00,987,None,None,None,None,List())
What am I doing wrong? Maybe there is another way of achieving this.
Could you consider Composition?
case class JourneyGroup(
...
)
case class JourneyGroupApi(
journeyGroup: JourneyGroup=JourneyGroup(),
parties: Seq[Party] = Seq()
)
Converting a journeyGroup would just be something like JourneyGroupApi(journeyGroup,parties) and "converting" a journeyGroupApi would be a matter of accessing journeyGroupApi.journeyGroup. You could perhaps come up with names that worked better for this case. Not sure if this approach would fit the rest of your code. In particular referencing journeyGroup attributes in a journeyGroupApi will be one extra level, e.g. journeyGroupApi.journeyGroup.accountId. (This could potentially be mitigated by "shortcut" definitions on journeyGroupApi like lazy val accountId = journeyGroup.accountId.)
Inheritance might also be an approach to consider with a base case class of JourneyGroup then a normal class (not case class) that extends it with parties as the extra attribute. This option is discussed further in this SO thread.