The following function takes in keys and the events to sort. Keys can be a maximum of size 4, for now.
def sortNow(keys: List[(String, String), events:List[java.util.HashMap[String, Any]]): Any = {
keys.length match {
case 1 => events.sortBy(event => {
val values = getValues(keys, event)
values match {
case List(a: String) => a.asInstanceOf[String]
}
})
case 2 => events.sortBy(event => {
val values = getValues(keys, event)
values match {
case List(a, b) => (a.asInstanceOf[String], b.asInstanceOf[String])
}
})
case 3 => events.sortBy(event => {
val values = getValues(keys, event)
values match {
case List(a, b, c) => (a.asInstanceOf[String], b.asInstanceOf[String], c.asInstanceOf[String])
}
})(Ordering[(String, String, String)].reverse)
case 4 => events.sortBy(event => {
val values = getValues(keys, event)
values match {
case List(a, b, c, d) => (a.asInstanceOf[String], b.asInstanceOf[String], c.asInstanceOf[String], d.asInstanceOf[String])
}
})
case default => throw new NotImplementedException
}
}
I have the following function definition:
private val checkSapHealth: IO[ServerHealth] =
BlazeClientBuilder[IO](global).resource.use { client =>
discovery
.senderAddr
.flatMap { addr =>
client
.get(addr.url |+| addr.health) { res =>
res.status match {
case Ok =>
IO(ServerOnline)
case _ =>
IO(ServerOffline)
}
}
.timeout(2.second)
.recover {
case _: Throwable => ServerOffline
}
}
}
I would like to replace the concrete type IO through F[_] to make it more abstract. The problem here is the line:
IO(ServerOnline)
The question is how to make it to
F(ServerOnline)
Try to use cats.effect.Sync
https://typelevel.org/cats-effect/typeclasses/sync.html
So basically using Sync[IO].delay is equivalent to using IO.apply.
private def checkSapHealth[F[_]: Sync]: F[ServerHealth] = ...
use ConcurrentEffect and add an implicit Applicative of F, thus giving you the option to lift values into the F context
private def checkSapHealth[F[_] : ConcurrentEffect](implicit A: Applicative[F]): F[ServerHealth] =
BlazeClientBuilder[F](global).resource.use { client =>
discovery
.senderAddr
.flatMap { addr =>
client
.get(addr.url |+| addr.health) { res =>
res.status match {
case Ok =>
A.pure(ServerOnline)
case _ =>
A.pure(ServerOffline)
}
}
.timeout(2.second)
.recover {
case _: Throwable => ServerOffline
}
}
}
If I have a java.util.Map[String,Java.util.Map] for the first call, correct overloaded method is called: toScala(java.util.Map[_,_]). But in the mapValues invocation, function toScala(other:Any) is called. How to avoid this?
I would be calling toScala using a java object like Map, List, Int, String. The map may contain another java.util.Map/java.util.List/ String/Int as one of its values. Similarly the list may also contain another java.util.List/ String/ Int as its members
private def toScala(map:java.util.Map[_,_]) : Any = {
map match {
case null => Map.empty
case _ => map.asScala.mapValues(toScala).toMap
}
}
private def toScala(list:java.util.List[_]) : Any = {
list match {
case null => List.empty
case _ => list.asScala.map(toScala).toList
}
}
private def toScala(other:Any) : Any = {
other match {
case null => None
case _ => other
}
}
Maybe this helps you:
private def toScala(v: Any): Any = vmatch {
case null => throw new NullPointerException("null value encountered while converting Java to Scala")
case m: java.util.Map[_, _] => m.asScala.mapValues(toScala).toMap
case m: java.util.List[_] => m.asScala.map(toScala).toMap
case m: java.util.Set[_] => m.asScala.map(toScala).toMap
case m => m
}
Or:
private def toScalaOption(v: Any): Any = vmatch {
case null => None
case m: java.util.Map[_, _] => m.asScala.mapValues(toScala).toMap
case m: java.util.List[_] => m.asScala.map(toScala).toMap
case m: java.util.Set[_] => m.asScala.map(toScala).toMap
case m => Some(m)
}
Cheers
I need to convert the datatypes of columns in dataframe and catch all data type conversion failures. I have tried the below option but it throws "Task not serializable".
var errorListBuffer = new ListBuffer[Map[String, String]]()
df.map(r => {
val value = r.getAs(columnName).toString
val index = r.fieldIndex(columnName)
Try {
val cleanValue = value match {
case n if r.isNullAt(index) => null
case x => x.trim
}
new_type match {
case "date" => new SimpleDateFormat("yyyy-MM-dd").format(new SimpleDateFormat(dateFormat).parse(cleanValue))
case "datetime" => new SimpleDateFormat("yyyy-MM-dd hh:mm:ss").format(new SimpleDateFormat(dateFormat).parse(cleanValue))
case "string" => toLower match {
case "1" => cleanValue.toLowerCase
case _ => cleanValue
}
case _ => cleanValue
}
} match {
case Success(v) => org.apache.spark.sql.Row.fromSeq(r.toSeq ++ v)
case Failure(e) => errorListBuffer += Map(
LOADER_COLUMN_NAME -> columnName,
LOADER_LEVEL -> "ERROR",
LOADER_ERROR_MESSAGE -> e.getMessage,
LOADER_RECORD_UUID -> r.getAs(LOADER_UUID).toString)
org.apache.spark.sql.Row.fromSeq(r.toSeq ++ null)
}
})
var dfnew = sqlContext.createDataFrame(df, schema)
Please let me know how I can resolve this.
I try to simplify validation process for serving responses for HTTP requests in Spray (I use Slick for database access). Currently, I check in one query if I should go further to the following query or not (return error). This end up with nested pattern matching. Every validation case can return different error so I can not use any flatMap.
class LocationDao {
val db = DbProvider.db
// Database tables
val devices = Devices.devices
val locations = Locations.locations
val programs = Programs.programs
val accessTokens = AccessTokens.accessTokens
def loginDevice(deviceSerialNumber: String, login: String, password: String): Either[Error, LocationResponse] = {
try {
db withSession { implicit session =>
val deviceRowOption = devices.filter(d => d.serialNumber === deviceSerialNumber).map(d => (d.id, d.currentLocationId.?, d.serialNumber.?)).firstOption
deviceRowOption match {
case Some(deviceRow) => {
val locationRowOption = locations.filter(l => l.id === deviceRow._2.getOrElse(0L) && l.login === login && l.password === password).firstOption
locationRowOption match {
case Some(locationRow) => {
val programRowOption = programs.filter(p => p.id === locationRow.programId).firstOption
programRowOption match {
case Some(programRow) => {
val program = Program(programRow.name, programRow.logo, programRow.moneyLevel, programRow.pointsForLevel,
programRow.description, programRow.rules, programRow.dailyCustomerScansLimit)
val locationData = LocationData(program)
val locationResponse = LocationResponse("access_token", System.currentTimeMillis(), locationData)
Right(locationResponse)
}
case None => Left(ProgramNotExistError)
}
}
case None => Left(IncorrectLoginOrPasswordError)
}
}
case None => Left(DeviceNotExistError)
}
}
} catch {
case ex: SQLException =>
Left(DatabaseError)
}
}
}
What is a good way to simplify this? Maybe there is other approach..
Generally, you can use a for-comprehension to chain together a lot of the monadic structures you have here (including Try, Option, and Either) without nesting. For example:
for {
val1 <- Try("123".toInt)
} yield for {
val2 <- Some(val1).map(_ * 2)
val3 = Some(val2 - 55)
val4 <- val3
} yield val4 * 2
In your style, this might otherwise have looked like:
Try("123".toInt) match {
case Success(val1) => {
val val2 = Some(val1).map(_ * 2)
val2 match {
case Some(val2value) => {
val val3 = Some(val2value - 55)
val3 match {
case Some(val4) => Some(val4)
case None => None
}
}
case None => None
}
case f:Failure => None
}
}
You can define a helper method for Eiterising your control flow.
The benefit of doing this is that you will have great control and flexibility in your flow.
def eitherMe[ I, T ]( eitherIn: Either[ Error, Option[ I ] ],
err: () => Error,
block: ( I ) => Either[ Error, Option[ T ] ]
): Either[ Error, Option[ T ] ] = {
eitherIn match {
case Right( oi ) => oi match {
case Some( i ) => block( i )
case None => Left( err() )
}
case Left( e ) => Left( e )
}
}
def loginDevice(deviceSerialNumber: String, login: String, password: String): Either[Error, LocationResponse] = {
try {
db withSession { implicit session =>
val deviceRowOption = devices.filter(d => d.serialNumber === deviceSerialNumber).map(d => (d.id, d.currentLocationId.?, d.serialNumber.?)).firstOption
val locationRowEither = eitherMe(
Right( deviceRowOption ),
() => { DeviceNotExistError },
deviceRow => {
val locationRowOption = locations.filter(l => l.id === deviceRow._2.getOrElse(0L) && l.login === login && l.password === password).firstOption
Right( locationRowOption )
}
)
val programRowEither = eitherMe(
locationRowEither,
() => { IncorrectLoginOrPasswordError },
locationRow => {
val programRowOption = programs.filter(p => p.id === locationRow.programId).firstOption
Right( programRowOption )
}
)
val locationResponseEither = eitherMe(
programRowEither,
() => { ProgramNotExistError },
programRow => {
val program = Program(programRow.name, programRow.logo, programRow.moneyLevel, programRow.pointsForLevel,
programRow.description, programRow.rules, programRow.dailyCustomerScansLimit)
val locationData = LocationData(program)
val locationResponse = LocationResponse("access_token", System.currentTimeMillis(), locationData)
Right(locationResponse)
}
)
locationResponseEither
}
} catch {
case ex: SQLException =>
Left(DatabaseError)
}
}
For me, when I sometime can't avoid nested complexity, I would extract out section of the code that make sense together and make it into a new method and give it a meaningful name. This will both document the code and make it more readable, and reduce the complexity inside each individual method. And usually, once I have done that, I am able to see the flow better and might be able to refactor it to make more sense (after first writing the tests to cover the behavior I want).
e.g. for your code, you can do something like this:
class LocationDao {
val db = DbProvider.db
// Database tables
val devices = Devices.devices
val locations = Locations.locations
val programs = Programs.programs
val accessTokens = AccessTokens.accessTokens
def loginDevice(deviceSerialNumber: String, login: String, password: String): Either[Error, LocationResponse] = {
try {
db withSession { implicit session =>
checkDeviceRowOption(deviceSerialNumber, login, password)
}
} catch {
case ex: SQLException =>
Left(DatabaseError)
}
}
def checkDeviceRowOption(deviceSerialNumber: String, login: String, password: String): Either[Error, LocationResponse] = {
val deviceRowOption = devices.filter(d => d.serialNumber === deviceSerialNumber).map(d => (d.id, d.currentLocationId.?, d.serialNumber.?)).firstOption
deviceRowOption match {
case Some(deviceRow) => {
val locationRowOption = locations.filter(l => l.id === deviceRow._2.getOrElse(0L) && l.login === login && l.password === password).firstOption
locationRowOption match {
case Some(locationRow) => { checkProgramRowOption(locationRow) }
case None => Left(IncorrectLoginOrPasswordError)
}
}
case None => Left(DeviceNotExistError)
}
}
def checkProgramRowOption(locationRow: LocationRowType): Either[Error, LocationResponse] = {
val programRowOption = programs.filter(p => p.id === locationRow.programId).firstOption
programRowOption match {
case Some(programRow) => {
val program = Program(programRow.name, programRow.logo, programRow.moneyLevel, programRow.pointsForLevel,
programRow.description, programRow.rules, programRow.dailyCustomerScansLimit)
val locationData = LocationData(program)
val locationResponse = LocationResponse("access_token", System.currentTimeMillis(), locationData)
Right(locationResponse)
}
case None => Left(ProgramNotExistError)
}
}
}
Note that this is just an illustration and probably won't compile because I don't have your lib, but you should be able to tweak the code for it to compile.