Scala case class equality when reading from Database - scala

I'm having trouble with checking for equality on a case class that I've mapped to an apache tinkerpop graph, but I want to be able to check for equality after I query the graph.
#label("apartment")
case class Apartment(#id id: Option[Int], address: String, zip: Zip, rooms: Rooms, size: Size, price: Price, link: String, active: Boolean = true, created: Date = new Date()) {}
val ApartmentAddress = Key[String]("address")
Await.result(result, Duration.Inf).foreach(apt => {
val dbResult = graph.V.hasLabel[Apartment].has(ApartmentAddress, apt.address).head().toCC[Apartment]
println(dbResult == apt) //always false :(
})
My problem is that when I've created the object it has no id, and the time stamp on it is obviously different. I read that if you add a second parameter list, it is excluded from equals, so I changed it:
#label("apartment")
case class Apartment(address: String, zip: Zip, rooms: Rooms, size: Size, price: Price, link: String, active: Boolean = true)(#id implicit val id: Option[Int] = None, implicit val created: Date = new Date()) {}
val ApartmentAddress = Key[String]("address")
Await.result(result, Duration.Inf).foreach(apt => {
val dbResult = graph.V.hasLabel[Apartment].has(ApartmentAddress, apt.address).head().toCC[Apartment]
println(dbResult == apt) //true! yay! :D
})
I can now check for equality using ==, but the value from the database loses its id, and the "created" value gets reset. And, one other frustrating thing, they always need to be created with extra parenthesis at the end:
Apartment(address, zip, rooms, size, price, link)()
Is there a way to achieve this functionality without overloading equals? Or make the value from the database maintain the original values using this approach?

It seems in your case, you just need it only for one time comparison, so I would not play with equals and just modified value on comparison
case class Apartment(
#id id: Option[Int] = None,
address: String,
zip: Zip,
rooms: Rooms,
size: Size,
price: Price,
link: String,
active: Boolean = true,
created: Date = new Date(0)) {
}
println(dbResult.copy(id = None, created = new Date(0)) == apt) //true! yay! :D
or add extra function to the class
case class Apartment(
#id id: Option[Int] = None,
address: String,
zip: Zip,
rooms: Rooms,
size: Size,
price: Price,
link: String,
active: Boolean = true,
created: Date = new Date(0)) {
def equalsIgnoreIdAndCreated(other: Apartment) = {
this.equals(other.copy(id = id, created = created))
}
}
println(dbResult.equalsIgnoreIdAndCreated(apt))
You can look at good explanation for case classes in
http://www.alessandrolacava.com/blog/scala-case-classes-in-depth/ and reasons why you should not override equals from automatically generated, otherwise just override equals.

Related

Convert spark scala dataset of one type to another

I have a dataset with following case class type:
case class AddressRawData(
addressId: String,
customerId: String,
address: String
)
I want to convert it to:
case class AddressData(
addressId: String,
customerId: String,
address: String,
number: Option[Int], //i.e. it is optional
road: Option[String],
city: Option[String],
country: Option[String]
)
Using a parser function:
def addressParser(unparsedAddress: Seq[AddressData]): Seq[AddressData] = {
unparsedAddress.map(address => {
val split = address.address.split(", ")
address.copy(
number = Some(split(0).toInt),
road = Some(split(1)),
city = Some(split(2)),
country = Some(split(3))
)
}
)
}
I am new to scala and spark. Could anyone please let me know how can this be done?
You were on the right path! There are multiple ways of doing this of course. But as you're already on the way by making some case classes, and you've started making a parsing function an elegant solution is by using the Dataset's map function. From the docs, this map function signature is the following:
def map[U](func: (T) ⇒ U)(implicit arg0: Encoder[U]): Dataset[U]
Where T is the starting type (AddressRawData in your case) and U is the type you want to get to (AddressData in your case). So the input of this map function is a function that transforms a AddressRawData to a AddressData. That could perfectly be the addressParser you've started making!
Now, your current addressParser has the following signature:
def addressParser(unparsedAddress: Seq[AddressData]): Seq[AddressData]
In order to be able to feed it to that map function, we need to make this signature:
def newAddressParser(unparsedAddress: AddressRawData): AddressData
Knowing all of this, we can work further! An example would be the following:
import spark.implicits._
import scala.util.Try
// Your case classes
case class AddressRawData(addressId: String, customerId: String, address: String)
case class AddressData(
addressId: String,
customerId: String,
address: String,
number: Option[Int],
road: Option[String],
city: Option[String],
country: Option[String]
)
// Your addressParser function, adapted to be able to feed into the Dataset.map
// function
def addressParser(rawAddress: AddressRawData): AddressData = {
val addressArray = rawAddress.address.split(", ")
AddressData(
rawAddress.addressId,
rawAddress.customerId,
rawAddress.address,
Try(addressArray(0).toInt).toOption,
Try(addressArray(1)).toOption,
Try(addressArray(2)).toOption,
Try(addressArray(3)).toOption
)
}
// Creating a sample dataset
val rawDS = Seq(
AddressRawData("1", "1", "20, my super road, beautifulCity, someCountry"),
AddressRawData("1", "1", "badFormat, some road, cityButNoCountry")
).toDS
val parsedDS = rawDS.map(addressParser)
parsedDS.show
+---------+----------+--------------------+------+-------------+----------------+-----------+
|addressId|customerId| address|number| road| city| country|
+---------+----------+--------------------+------+-------------+----------------+-----------+
| 1| 1|20, my super road...| 20|my super road| beautifulCity|someCountry|
| 1| 1|badFormat, some r...| null| some road|cityButNoCountry| null|
+---------+----------+--------------------+------+-------------+----------------+-----------+
As you see, thanks to the fact that you had already foreseen that parsing can go wrong, it was easily possible to use scala.util.Try to try and get the pieces of that raw address and add some robustness in there (the second line contains some null values where it could not parse the address string.
Hope this helps!

Scala Option and Some mismatch

I want to parse province to case class, it throws mismatch
scala.MatchError: Some(USA) (of class scala.Some)
val result = EntityUtils.toString(entity,"UTF-8")
val address = JsonParser.parse(result).extract[Address]
val value.province = Option(address.province)
val value.city = Option(address.city)
case class Access(
device: String,
deviceType: String,
os: String,
event: String,
net: String,
channel: String,
uid: String,
nu: Int,
ip: String,
time: Long,
version: String,
province: Option[String],
city: Option[String],
product: Option[Product]
)
This:
val value.province = Option(address.province)
val value.city = Option(address.city)
doesn't do what you think it does. It tries to treat value.province and value.city as extractors (which don't match the type, thus scala.MatchError exception). It doesn't mutate value as I believe you intended (because value apparently doesn't have such setters).
Since value is (apparently) Access case class, it is immutable and you can only obtain an updated copy:
val value2 = value.copy(
province = Option(address.province),
city = Option(address.city)
)
Assuming the starting point:
val province: Option[String] = ???
You can get the string with simple pattern matching:
province match {
case Some(stringValue) => JsonParser.parse(stringValue).extract[Province] //use parser to go from string to a case class
case None => .. //maybe provide a default value, depends on your context
}
Note: Without knowing what extract[T] returns it's hard to recommend a follow-up

How to apply a filter with FP/scala

I have a class that has some Option values, and I need to apply all the values that come as Some in the class to a list of objects.
Example:
class Thing(name: String, age: Int)
class Filter(name: Option[String], age: Option[Int], size: Option[Int])
val list: List[Thing] = functionThatReturnsAListOfThings(param)
val filter: Filter = functionThatReturnsAFilter(otherParam)
list.filter{ thing =>
if filter.name.isDefined {
thing.name.equals(filter.name.get)
}
if filter.age.isDefined {
thing.age == filter.age.get
}
}.take{
if filter.size.isDefined filter.size.get
else list.size
}
How can I apply the filter to the list properly with FP?
First off we need to make a small change so that the constructor arguments are public members.
class Thing(val name: String, val age: Int)
class Filter(val name : Option[String]
,val age : Option[Int]
,val size : Option[Int])
Next, it's not clear, from your example code, what should happen when filter.name and filter.age are both None. I'll assume that None means true, i.e. not filtered out.
list.filter { thing =>
filter.name.fold(true)(_ == thing.name) &&
filter.age.fold(true)(_ == thing.age)
}.take(filter.size.getOrElse(Int.MaxValue))
Note that take(Int.MaxValue) is a bit more efficient than take(list.size).

scala assign value to object

I have the following code:
def getContentComponents: Action[AnyContent] = Action.async {
contentComponentDTO.list().map(contentComponentsFuture =>
contentComponentsFuture.foreach(contentComponentFuture =>
contentComponentFuture.typeOf match {
case 5 =>
contentComponentDTO.getContentComponentText(contentComponentFuture.id.get).map(
text => contentComponentFuture.text = text.text
)
}
)
Ok(Json.toJson(contentComponentsFuture))
)
and get this error message while assigning a value:
Is there a way to solve this issue?
I thought about creating a copy but that would mean that I have do lots of other stuff later. It would be much easier for me if I could reassign the value.
thanks
Unfortunately this is not possible as contentComponentFuture.text is an immutable property so you cannot change its value like that.
The only way to "change" the values of the job object is to actually create a totally new instance and discard the old one. This is standard practice when dealing with immutable objects.
Sorry for the bad news.
Just found the solution ...
In model I have to define the Attributes as var
case class ContentComponentModel(
id: Option[Int] = None,
typeOf: Int,
name: String,
version: String,
reusable: Option[Boolean],
createdat: Date,
updatedat: Date,
deleted: Boolean,
processStepId: Int,
var text: Option[String],

Converting one case class to another that is similar with additional parameter in Scala

So, the problem is in the title, but here are the details.
I have two case classes:
case class JourneyGroup(id: Option[Int] = None,
key: UUID,
name: String,
data: Option[JsValue],
accountId: Int,
createdAt: DateTime = DateTime.now,
createdById: Int,
updatedAt: Option[DateTime] = None,
updatedById: Option[Int] = None,
deletedAt: Option[DateTime] = None,
deletedById: Option[Int] = None)
and
case class JourneyGroupApi(id: Option[Int] = None,
key: UUID,
name: String,
data: Option[JsValue],
accountId: Int,
createdAt: DateTime = DateTime.now,
createdById: Int,
updatedAt: Option[DateTime] = None,
updatedById: Option[Int] = None,
deletedAt: Option[DateTime] = None,
deletedById: Option[Int] = None,
parties: Seq[Party] = Seq.empty[Party])
Background: the reason for having these two separate classes is the fact that slick does not support collections, and I do need collections of related objects that I build manually. Bottom line, I could not make it work with a single class.
What I need is an easy way to convert from one to another.
At this point, to unblock myself, I created a manual conversion:
def toJourneyGroupApi(parties: Seq[Party]): JourneyGroupApi = JourneyGroupApi(
id = id,
key = key,
name = name,
data = data,
accountId = accountId,
createdAt = createdAt,
createdById = createdById,
updatedAt = updatedAt,
updatedById = updatedById,
deletedAt = deletedAt,
deletedById = deletedById,
parties = parties
)
Which is working, but extremely ugly and requires a lot of maintenance.
One thing that I tried doing is:
convert the source object to tuple
Add an element to that tuple using shapeless
and build a target object from resulting tuple
import shapeless._
import syntax.std.tuple._
val groupApi = (JourneyGroup.unapply(group).get :+ Seq.empty[Party])(JourneyGroupApi.tupled)
But, this thing is claiming, that the result of :+ is not tuple, even though in console:
Party.unapply(p).get :+ Seq.empty[Participant]
res0: (Option[Int], model.Parties.Type.Value, Int, Int, org.joda.time.DateTime, Int, Option[org.joda.time.DateTime], Option[Int], Option[org.joda.time.DateTime], Option[Int], Seq[model.Participant]) = (None,client,123,234,2016-11-12T03:55:24.006-08:00,987,None,None,None,None,List())
What am I doing wrong? Maybe there is another way of achieving this.
Could you consider Composition?
case class JourneyGroup(
...
)
case class JourneyGroupApi(
journeyGroup: JourneyGroup=JourneyGroup(),
parties: Seq[Party] = Seq()
)
Converting a journeyGroup would just be something like JourneyGroupApi(journeyGroup,parties) and "converting" a journeyGroupApi would be a matter of accessing journeyGroupApi.journeyGroup. You could perhaps come up with names that worked better for this case. Not sure if this approach would fit the rest of your code. In particular referencing journeyGroup attributes in a journeyGroupApi will be one extra level, e.g. journeyGroupApi.journeyGroup.accountId. (This could potentially be mitigated by "shortcut" definitions on journeyGroupApi like lazy val accountId = journeyGroup.accountId.)
Inheritance might also be an approach to consider with a base case class of JourneyGroup then a normal class (not case class) that extends it with parties as the extra attribute. This option is discussed further in this SO thread.