Scala pattern matching on generic Map - scala

Whats the best way to handle generics and erasure when doing pattern matching in Scala (a Map in my case). I am looking for a proper implementation without compiler warnings. I have a function that I want to return Map[Int, Seq[String]] from. Currently the code looks like:
def teams: Map[Int, Seq[String]] = {
val dateam = new scala.collection.mutable.HashMap[Int, Seq[String]]
// data.attributes is Map[String, Object] returned from JSON parsing (jackson-module-scala)
val teamz = data.attributes.get("team_players")
if (teamz.isDefined) {
val x = teamz.get
try {
x match {
case m: mutable.Map[_, _] => {
m.foreach( kv => {
kv._1 match {
case teamId: String => {
kv._2 match {
case team: Seq[_] => {
val tid: Int = teamId.toInt
dateam.put(tid, team.map(s => s.toString))
}
}
}
}
})
}
}
} catch {
case e: Exception => {
logger.error("Unable to convert the team_players (%s) attribute.".format(x), e)
}
}
dateam
} else {
logger.warn("Missing team_players attribute in: %s".format(data.attributes))
}
dateam.toMap
}

Use a Scala library to handle it. There are some based on Jackson (Play's ScalaJson, for instance -- see this article on using it stand-alone), as well as libraries not based on Jackson (of which my preferred is Argonaut, though you could also go with Spray-Json).
These libraries, and others, solve this problem. Doing it by hand is awkward and prone to errors, so don't do it.

It could be reasonable to use for comprehension (with some built in pattern matching). Also we could take into account that Map is a list of tuples, in our case of (String, Object) type. As well we will ignore for this example probable exceptions, so:
import scala.collection.mutable.HashMap
def convert(json: Map[String, Object]): HashMap[Int, Seq[String]] = {
val converted = for {
(id: String, description: Seq[Any]) <- json
} yield (id.toInt, description.map(_.toString))
HashMap[Int, Seq[String]](converted.toSeq: _*)
}
So, our for comprehension taking into account only tuples with (String, Seq[Any]) type, then combines converted String to Int and Seq[Any] to Seq[String]. And makes Map to be mutable.

Related

Using Scala groupBy(), from method fetchUniqueCodesForARootCode(). I want to get a map from rootCodes to lists of uniqueCodes

I want to to return Future[Map[String, List[String]]] from fetchUniqueCodesForARootCode method
import scala.concurrent._
import ExecutionContext.Implicits.global
case class DiagnosisCode(rootCode: String, uniqueCode: String, description: Option[String] = None)
object Database {
private val data: List[DiagnosisCode] = List(
DiagnosisCode("A00", "A001", Some("Cholera due to Vibrio cholerae")),
DiagnosisCode("A00", "A009", Some("Cholera, unspecified")),
DiagnosisCode("A08", "A080", Some("Rotaviral enteritis")),
DiagnosisCode("A08", "A083", Some("Other viral enteritis"))
)
def getAllUniqueCodes: Future[List[String]] = Future {
Database.data.map(_.uniqueCode)
}
def fetchDiagnosisForUniqueCode(uniqueCode: String): Future[Option[DiagnosisCode]] = Future {
Database.data.find(_.uniqueCode.equalsIgnoreCase(uniqueCode))
}
}
getAllUniqueCodes returns all unique codes from data List.
fetchDiagnosisForUniqueCode returns DiagnosisCode when uniqueCode matches.
From fetchDiagnosisForUniqueCodes, I am returningFuture[List[DiagnosisCode]] using getAllUniqueCodes() and fetchDiagnosisForUniqueCode(uniqueCode).*
def fetchDiagnosisForUniqueCodes: Future[List[DiagnosisCode]] = {
val xa: Future[List[Future[DiagnosisCode]]] = Database.getAllUniqueCodes.map { (xs:
List[String]) =>
xs.map { (uq: String) =>
Database.fetchDiagnosisForUniqueCode(uq)
}
}.map(n =>
n.map(y=>
y.map(_.get)))
}
xa.flatMap {
listOfFuture =>
Future.sequence(listOfFuture)
}}
Now, def fetchUniqueCodesForARootCode should return Future[Map[String, List[DiagnosisCode]]] using fetchDiagnosisForUniqueCodes and groupBy
Here is the method
def fetchUniqueCodesForARootCode: Future[Map[String, List[String]]] = {
fetchDiagnosisForUniqueCodes.map { x =>
x.groupBy(x => (x.rootCode, x.uniqueCode))
}
}
Need to get the below result from fetchUniqueCodesForARootCode:-
A00 -> List(A001, A009), H26 -> List(H26001, H26002), B15 -> List(B150, B159), H26 -> List(H26001, H26002)
It's hard to decode from the question description, what the problem is. But if I understood correctly, you want to get a map from rootCodes to lists of uniqueCodes.
The groupBy method takes a function that for every element returns its key. So first you have to group by the rootCodes and then you have to use map to get the correct values.
groupBy definition: https://dotty.epfl.ch/api/scala/collection/IterableOps.html#groupBy-f68
scastie: https://scastie.scala-lang.org/KacperFKorban/PL1X3joNT3qNOTm6OQ3VUQ

How to make only few datatype which is not related to each other acceptable by generics

There is a trait which works perfectly. However, I would like to refactor the part related to generic [T] in order to limit the data type which could be accepted by generic [T] (I need only Option[JsValue] , JsValue , StringEnumEntry , String ). Is it possible to solve this problem through shapeless coproduct? Maybe there are other solutions?
trait ParameterBinders extends Log {
def jsonBinder[T](json: T, jsonType: java.lang.String = "json"): ParameterBinderWithValue = {
val jsonObject = new PGobject()
jsonObject.setType(jsonType)
json match {
case json: Option[JsValue] =>
jsonObject.setValue(json.map(Json.stringify).orNull)
case json: JsValue =>
jsonObject.setValue(Json.stringify(json))
case json: StringEnumEntry =>
jsonObject.setValue(json.value)
case json: String =>
jsonObject.setValue(json)
case _ =>
logger.error("unexpected data type ")
}
if (jsonType == "JSONSCHEMATYPE" || jsonType == "SYSPROPERTYTYPE") {
ParameterBinder(this, (ps, i) => {
ps.setObject(i, jsonObject)
})
} else {
ParameterBinder(json, (ps, i) => {
ps.setObject(i, jsonObject)
})
}
}
}
The easiest way is to use an ADT as described in the link of the first comment.
If you don't want to change the types that are accepted in jsonBinder then you can solve the problem by using a typeclass.
e.g.
trait JsonBindValue[T] {
def value(t: T): String
}
you would then have to provide instances for your accepted datatypes
object JsonBindValue {
implicit val OptJsBinder = new JsonBindValue[Option[JsValue]] {
def value(t: Option[JsValue]): String = {
t.map(Json.stringify).orNull
}
}
... more instances here
}
finally your function would look like this:
def jsonBinder[T : JsonBindValue](json: T, jsonType: java.lang.String = "json"): ParameterBinderWithValue = {
val binder = implicitly[JsonBindValue[T]]
jsonObject.setType(jsonType)
jsonObject.setValue(binder.value(json))
...
}
if you call the function without a implicit instance in scope you will get a compile time error.

Reading a file path from property and then reading the file idiomatic Scala

I want to read in the path of a file from configureation and then read the file in an idomatic Scala way. This is the code I have so far:
val key: Option[String] = {
val publicKeyPath: Option[String] = conf.getString("bestnet.publicKeyFile")
publicKeyPath match {
case Some(path) => {
Future {
val source = fromFile(s"./$path")
val key: String = source.getLines.toIterable.drop(1).dropRight(1).mkString
source.close()
key
} onComplete {
case Success(key) => Success(key)
case Failure(t) => None
}
}
case None => None
}
}
However this is not working since Im getting the error Expression of type Unit does not conform to Option[String]
What am I getting wrong and is my approach idiomatic Scala or should it be done in some other way?
If you want to return the contents as String there is no need to use a Future. E.g. the following would do:
val key: Option[String] = {
val publicKeyPath: Option[String] = conf.getString("bestnet.publicKeyFile")
publicKeyPath match {
case Some(path) =>
val source = fromFile(s"./$path")
val key: String = source.getLines.toIterable.drop(1).dropRight(1).mkString
source.close()
Some(key)
case None =>
None
}
}
The pattern of transforming the value of a Some(_) can be done more idiomatic using the higher-level function map, i.e.:
val key: Option[String] = {
val publicKeyPath = conf.getString("bestnet.publicKeyFile")
publicKeyPath.map(path => {
val source = fromFile(s"./$path")
val key = source.getLines.toIterable.drop(1).dropRight(1).mkString
source.close()
key
})
}
A more idiomatic way to do resource management (i.e. closing the Source) is by using the "Loan Pattern". For example:
def using[A](r: Resource)(f: Resource => A): A = try {
    f(r)
} finally {
r.dispose()
}
val key: Option[String] = {
val publicKeyPath = conf.getString("bestnet.publicKeyFile")
publicKeyPath.map(path =>
using(fromFile(s"./$path"))(source =>
source.getLines.toIterable.drop(1).dropRight(1).mkString
)
)
}
Scala is a flexible language and it is not uncommon to define such an abstraction in user-land (whereas in Java, the using abstraction is a language feature).
If you need non-blocking parallel code you should return a Future[String] instead of an Option[String]. This complicates the automatic resource management, since code is executed at a different time. Anyway, this should give you some pointers for improving your code.

Scala Future[A] and Future[Option[B]] composition

I have an app that manages Items. When the client queries an item by some info, the app first tries to find an existing item in the db with the info. If there isn't one, the app would
Check if info is valid. This is an expensive operation (much more so than a db lookup), so the app only performs this when there isn't an existing item in the db.
If info is valid, insert a new Item into the db with info.
There are two more classes, ItemDao and ItemService:
object ItemDao {
def findByInfo(info: Info): Future[Option[Item]] = ...
// This DOES NOT validate info; it assumes info is valid
def insertIfNotExists(info: Info): Future[Item] = ...
}
object ItemService {
// Very expensive
def isValidInfo(info: Info): Future[Boolean] = ...
// Ugly
def findByInfo(info: Info): Future[Option[Item]] = {
ItemDao.findByInfo(info) flatMap { maybeItem =>
if (maybeItem.isDefined)
Future.successful(maybeItem)
else
isValidInfo(info) flatMap {
if (_) ItemDao.insertIfNotExists(info) map (Some(_))
else Future.successful(None)
}
}
}
}
The ItemService.findByInfo(info: Info) method is pretty ugly. I've been trying to clean it up for a while, but it's difficult since there are three types involved (Future[Boolean], Future[Item], and Future[Option[Item]]). I've tried to use scalaz's OptionT to clean it up but the non-optional Futures make it not very easy either.
Any ideas on a more elegant implementation?
To expand on my comment.
Since you've already indicated a willingness to go down the route of monad transformers, this should do what you want. There is unfortunately quite a bit of line noise due to Scala's less than stellar typechecking here, but hopefully you find it elegant enough.
import scalaz._
import Scalaz._
object ItemDao {
def findByInfo(info: Info): Future[Option[Item]] = ???
// This DOES NOT validate info; it assumes info is valid
def insertIfNotExists(info: Info): Future[Item] = ???
}
object ItemService {
// Very expensive
def isValidInfo(info: Info): Future[Boolean] = ???
def findByInfo(info: Info): Future[Option[Item]] = {
lazy val nullFuture = OptionT(Future.successful(none[Item]))
lazy val insert = ItemDao.insertIfNotExists(info).liftM[OptionT]
lazy val validation =
isValidInfo(info)
.liftM[OptionT]
.ifM(insert, nullFuture)
val maybeItem = OptionT(ItemDao.findByInfo(info))
val result = maybeItem <+> validation
result.run
}
}
Two comments about the code:
We are using the OptionT monad transformer here to capture the Future[Option[_]] stuff and anything that just lives inside Future[_] we're liftMing up to our OptionT[Future, _] monad.
<+> is an operation provided by MonadPlus. In a nutshell, as the name suggests, MonadPlus captures the intuition that often times monads have an intuitive way of being combined (e.g. List(1, 2, 3) <+> List(4, 5, 6) = List(1, 2, 3, 4, 5, 6)). Here we're using it to short-circuit when findByInfo returns Some(item) rather than the usual behavior to short-circuit on None (this is roughly analogous to List(item) <+> List() = List(item)).
Other small note, if you actually wanted to go down the monad transformers route, often times you end up building everything in your monad transformer (e.g. ItemDao.findByInfo would return an OptionT[Future, Item]) so that you don't have extraneous OptionT.apply calls and then .run everything at the end.
You don't need scalaz for this. Just break your flatMap into two steps:
first, find and validate, then insert if necessary. Something like this:
ItemDao.findByInfo(info).flatMap {
case None => isValidInfo(info).map(None -> _)
case x => Future.successful(x -> true)
}.flatMap {
case (_, true) => ItemDao.insertIfNotExists(info).map(Some(_))
case (x, _) => Future.successful(x)
}
Doesn't look too bad, does it? If you don't mind running validation in parallel with retrieval (marginally more expensive resource-vise, but likely faster on average), you could further simplify it like this:
ItemDao
.findByInfo(info)
.zip(isValidInfo(info))
.flatMap {
case (None, true) => ItemDao.insertIfNotExists(info).map(Some(_))
case (x, _) => x
}
Also, what does insertIfNotExists return if the item does exist? If it returned the existing item, things could be even simpler:
isValidInfo(info)
.filter(identity)
.flatMap { _ => ItemDao.insertIfNotExists(info) }
.map { item => Some(item) }
.recover { case _: NoSuchElementException => None }
If you are comfortable with path-dependent type and higher-kinded type, something like the following can be an elegant solution:
type Const[A] = A
sealed trait Request {
type F[_]
type A
type FA = F[A]
def query(client: Client): Future[FA]
}
case class FindByInfo(info: Info) extends Request {
type F[x] = Option[x]
type A = Item
def query(client: Client): Future[Option[Item]] = ???
}
case class CheckIfValidInfo(info: Info) extends Request {
type F[x] = Const[x]
type A = Boolean
def query(client: Client): Future[Boolean] = ???
}
class DB {
private val dbClient: Client = ???
def exec(request: Request): request.FA = request.query(dbClient)
}
What this does is basically to abstract over both the wrapper type (eg. Option[_]) as well as inner type. For types without a wrapper type, we use Const[_] type which is basically an identity type.
In scala, many problems alike this can be solved elegantly using Algebraic Data Type and its advanced type system (i.e path-dependent type & higher-kinded type). Note that now we have single point of entry exec(request: Request) for executing db requests instead of something like DAO.

How do you write a json4s CustomSerializer that handles collections

I have a class that I am trying to deserialize using the json4s CustomSerializer functionality. I need to do this due to the inability of json4s to deserialize mutable collections.
This is the basic structure of the class I want to deserialize (don't worry about why the class is structured like this):
case class FeatureValue(timestamp:Double)
object FeatureValue{
implicit def ordering[F <: FeatureValue] = new Ordering[F] {
override def compare(a: F, b: F): Int = {
a.timestamp.compareTo(b.timestamp)
}
}
}
class Point {
val features = new HashMap[String, SortedSet[FeatureValue]]
def add(name:String, value:FeatureValue):Unit = {
val oldValue:SortedSet[FeatureValue] = features.getOrElseUpdate(name, SortedSet[FeatureValue]())
oldValue += value
}
}
Json4s serializes this just fine. A serialized instance might look like the following:
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
I've tried writing a custom deserializer, but I don't know how to deal with the list tails. In a normal matcher you can just call your own function recursively, but in this case the function is anonymous and being called through the json4s API. I cannot find any examples that deal with this and I can't figure it out.
Currently I can match only a single hash key, and a single FeatureValue instance in its value. Here is the CustomSerializer as it stands:
import org.json4s.{FieldSerializer, DefaultFormats, Extraction, CustomSerializer}
import org.json4s.JsonAST._
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(JField("features", JObject(Nil)) :: Nil) => new Point
case JObject(List(("features", JObject(List(
(feature:String, JArray(List(JObject(List(("timestamp",JDouble(ts)))))))))
))) => {
val point = new Point
point.add(feature, FeatureValue(ts))
point
}
},
{
// don't need to customize this, it works fine
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
If I try to change to using the :: separated list format, so far I have gotten compiler errors. Even if I didn't get compiler errors, I am not sure what I would do with them.
You can get the list of json features in your pattern match and then map over this list to get the Features and their codes.
class PointSerializer extends CustomSerializer[Point](format => (
{
case JObject(List(("features", JObject(featuresJson)))) =>
val features = featuresJson.flatMap {
case (code:String, JArray(timestamps)) =>
timestamps.map { case JObject(List(("timestamp",JDouble(ts)))) =>
code -> FeatureValue(ts)
}
}
val point = new Point
features.foreach((point.add _).tupled)
point
}, {
case x: Point => Extraction.decompose(x)(DefaultFormats + FieldSerializer[Point]())
}
))
Which deserializes your json as follows :
import org.json4s.native.Serialization.{read, write}
implicit val formats = Serialization.formats(NoTypeHints) + new PointSerializer
val json = """
{"features":
{
"CODE0":[{"timestamp":4.8828914447482E8}],
"CODE1":[{"timestamp":4.8828914541333E8}],
"CODE2":[{"timestamp":4.8828915127325E8},{"timestamp":4.8828910097466E8}]
}
}
"""
val point0 = read[Point]("""{"features": {}}""")
val point1 = read[Point](json)
point0.features // Map()
point1.features
// Map(
// CODE0 -> TreeSet(FeatureValue(4.8828914447482E8)),
// CODE2 -> TreeSet(FeatureValue(4.8828910097466E8), FeatureValue(4.8828915127325E8)),
// CODE1 -> TreeSet(FeatureValue(4.8828914541333E8))
// )