I have function like this
def jsonFlatten(obj: JsValue, coun: JsObject = Json.obj(), pref: String = ""): JsObject = {
var con = coun
var kel = ""
var el: JsValue = Json.obj()
obj match {
case JsObject(_) =>
var ob = obj.validate[JsObject].get
ob.value.foreach({
case (k, v) =>
if (pref != "") {
kel = pref + "." + k
} else {
kel = k
}
el = v
el match {
case JsObject(_) =>
con = jsonFlatten(el, con.++(con), kel)
case JsArray(_) =>
con = jsonFlatten(el, con.++(con), kel)
case _ =>
con = con.+(kel -> el)
}
})
case _ =>
}
return con
}
That function called on another iteration within thread and I want to optimize this function because it takes lot time (around 300ms) and I have to call lot of that function in iteration.
Is there any suggestion how to optimize that function with concurrency or effective recursion ?
Thanks
as
Related
Trying to execute a function in a given time frame, but if computation fails by TimeOut get a partial result instead of an empty exception.
The attached code solves it.
The timedRun function is from Computation with time limit
Any better approach?.
package ga
object Ga extends App {
//this is the ugly...
var bestResult = "best result";
try {
val result = timedRun(150)(bestEffort())
} catch {
case e: Exception =>
print ("timed at = ")
}
println(bestResult)
//dummy function
def bestEffort(): String = {
var res = 0
for (i <- 0 until 100000) {
res = i
bestResult = s" $res"
}
" " + res
}
//This is the elegant part from stackoverflow gruenewa
#throws(classOf[java.util.concurrent.TimeoutException])
def timedRun[F](timeout: Long)(f: => F): F = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val task = new FutureTask(new Callable[F]() {
def call() = f
})
new Thread(task).start()
task.get(timeout, TimeUnit.MILLISECONDS)
}
}
I would introduce a small intermediate class for more explicitly communicating the partial results between threads. That way you don't have to modify non-local state in any surprising ways. Then you can also just catch the exception within the timedRun method:
class Result[A](var result: A)
val result = timedRun(150)("best result")(bestEffort)
println(result)
//dummy function
def bestEffort(r: Result[String]): Unit = {
var res = 0
for (i <- 0 until 100000) {
res = i
r.result = s" $res"
}
r.result = " " + res
}
def timedRun[A](timeout: Long)(initial: A)(f: Result[A] => _): A = {
import java.util.concurrent.{ Callable, FutureTask, TimeUnit }
val result = new Result(initial)
val task = new FutureTask(new Callable[A]() {
def call() = { f(result); result.result }
})
new Thread(task).start()
try {
task.get(timeout, TimeUnit.MILLISECONDS)
} catch {
case e: java.util.concurrent.TimeoutException => result.result
}
}
It's admittedly a bit awkward since you don't usually have the "return value" of a function passed in as a parameter. But I think it's the least-radical modification of your code that makes sense. You could also consider modeling your computation as something that returns a Stream or Iterator of partial results, and then essentially do .takeWhile(notTimedOut).last. But how feasible that is really depends on the actual computation.
First, you need to use one of the solution to recover after the future timed out which are unfortunately not built-in in Scala:
See: Scala Futures - built in timeout?
For example:
def withTimeout[T](fut:Future[T])(implicit ec:ExecutionContext, after:Duration) = {
val prom = Promise[T]()
val timeout = TimeoutScheduler.scheduleTimeout(prom, after)
val combinedFut = Future.firstCompletedOf(List(fut, prom.future))
fut onComplete{case result => timeout.cancel()}
combinedFut
}
Then it is easy:
var bestResult = "best result"
val expensiveFunction = Future {
var res = 0
for (i <- 0 until 10000) {
Thread.sleep(10)
res = i
bestResult = s" $res"
}
" " + res
}
val timeoutFuture = withTimeout(expensiveFunction) recover {
case _: TimeoutException => bestResult
}
println(Await.result(timeoutFuture, 1 seconds))
I am quite new with functional programming in scala and play-framework
I have an API request that need to add a chapter to a comic book authored by that particular user, thus I have to verify that the comic exist and belongs to that user. What complicates is the Future, Option etc and I can't make sure that the last command is executed last. Indeed, my problem occurs because the last command is executed before all my db access is completed
Here are the code, hope it is clear
def addchapter(): Action[AnyContent] = Action.async { implicit request =>
var hashMap: HashMap[String, JsValue] = HashMap()
var apiResult: ApiResult = ApiResult(
ReturnCode.COMIC_ADDCHAPTER.id,
ReturnCode.COMIC_ADDCHAPTER.toString(),
ReturnResult.RESULT_ERROR.toString(),
"init"
)
var fHashMap: Future[HashMap[String, JsValue]] = Future {
hashMap
}
try {
val oReq = request.body.asJson
val jsReq = oReq.getOrElse(JsString("null"))
val sessionid = (jsReq \
"sessionid").getOrElse(JsString("0")).as[String]
val comicid = (jsReq \
"comicid").getOrElse(JsString("comicid")).as[String]
val foUser = userRepo.getUserBySessionId(sessionid)
LogManager.DebugLog(this, "add chapter: " + sessionid + " => " +
comicid)
fHashMap = foUser.flatMap( oUser => {
oUser match {
case Some(user) => {
val foComic = comicRepo.getComicByComicId(comicid)
fHashMap = foComic.flatMap( oComic => {
oComic match {
case Some(comic) => {
LogManager.DebugLog(this, "about to add chapter")
val fTup = comicRepo.addChapterToComic(comic)
fHashMap = fTup.map( tuple => {
val wr = tuple._1
val mc = tuple._2
apiResult = ApiResult(
ReturnCode.COMIC_ADDCHAPTER.id,
ReturnCode.COMIC_ADDCHAPTER.toString(),
ReturnResult.RESULT_ERROR.toString(),
"successfully added chapter!"
)
val payloadcomic =
PayloadComicFactory.createWithComic(mc)
hashMap("apiresult") = Json.toJson(apiResult)
hashMap += "comic" -> Json.toJson(payloadcomic)
LogManager.DebugLog(this, "successfully added
chapter!")
hashMap
})
// return
fHashMap
}
case None => {
apiResult = ApiResult(
ReturnCode.COMIC_ADDCHAPTER.id,
ReturnCode.COMIC_ADDCHAPTER.toString(),
ReturnResult.RESULT_ERROR.toString(),
"comic not found"
)
hashMap("apiresult") = Json.toJson(apiResult)
Future { hashMap }
}
}
})
Future { hashMap }
}
case None => {
apiResult = ApiResult(
ReturnCode.COMIC_ADDCHAPTER.id,
ReturnCode.COMIC_ADDCHAPTER.toString(),
ReturnResult.RESULT_ERROR.toString(),
"unauthorized to add chapter to this comic"
)
hashMap("apiresult") = Json.toJson(apiResult)
Future { hashMap }
}
}
})
// I cannot put the return here, it is compile error saying that the return value is a Unit
// fHashMap.map( hashmap => {
// Ok(Json.toJson(hashmap))
// })
} catch {
case t: Throwable => {
LogManager.DebugException(this, "ex: ", t)
// I cannot put it here, it is compile error saying that the return value is Unit
// fHashMap.map( hashmap => {
// Ok(Json.toJson(hashmap))
// })
fHashMap
}
}
// I have to return here, but "return of the request" will be executed first before all my db access is completed, thus the ApiResult is still returning wrong state
LogManager.DebugLog(this, "return of the request")
fHashMap.map( hashmap => {
Ok(Json.toJson(hashmap))
})
}
I see codes like this:
trait LoginInfoRepoImpl extends LoginInfoRepo {
def loginInfoRepository = new LoginInfoRepository {
private val loginInfoTable = TableQuery[LoginInfoTable]
// return a LoginID
def save(userLoginInfo: userLoginInfo): Future[userLoginID] = Future {
val newRecord = DB.withSession { implicit session =>
loginInfoTable.filter(
l => l.userID === userLoginInfo.userID &&
(l.deviceID === userLoginInfo.deviceID || (l.deviceID.isEmpty && userLoginInfo.userID.isEmpty))).list.headOption.fold {
val newSubID = loginInfoTable.filter(l => l.userID === userLoginInfo.userID).sortBy(_.subID.desc).take(1).map(_.subID).list.headOption.getOrElse(0) + 1
(loginInfoTable returning loginInfoTable) += LoginInfoRecord(userLoginInfo.userID, newSubID, userLoginInfo.deviceID, userLoginInfo.userAgent, getCurrentTime)
} { l =>
// to do : update time
val q = for (l <- loginInfoTable if l.userID === userLoginInfo.userID && ((l.deviceID === userLoginInfo.deviceID)
|| (l.deviceID.isEmpty && userLoginInfo.userID.isEmpty)))
yield l.lastLoginTime
q.updateReturning(loginInfoTable.map(identity), getCurrentTime).head
}
}
userLoginID(newRecord.userID, newRecord.subID.toString)
}
}
}
This looks a little monstrous for me. I found many things are crowded into one line. Also, I found l.deviceID is of type Column[Option[String]], while userLoginInfo.deviceID is of typeOption[String], they don't equal if both of them is None. Thus a l.device.isEmpty looks necessary..
Does anyone have suggestions about how to refactor these codes? Thanks!
First of all: Better indentation and breaking up long lines.
Slick-specific: pull queries out of the withSession block and reuse them for shared logic. Use firstOption instead of .list.headOption.
Side-note: Since you are generating IDs based on old ones, you may want to use a transaction or use a more efficient means provided by your DB.
trait LoginInfoRepoImpl extends LoginInfoRepo {
def loginInfoRepository = new LoginInfoRepository {
private val loginInfoTable = TableQuery[LoginInfoTable]
// return a LoginID
def save(userLoginInfo: userLoginInfo): Future[userLoginID] = Future {
val userLoginInfoQuery = loginInfoTable.filter(l => l.userID === userLoginInfo.userID)
val deviceLoginInfoQuery = userLoginInfoQuery.filter(
l =>
l.deviceID === userLoginInfo.deviceID ||
(
l.deviceID.isEmpty && userLoginInfo.userID.isEmpty
)
)
val subIdQuery = userLoginInfoQuery.sortBy(_.subID.desc).map(_.subID)
val newRecord = DB.withTransaction{ implicit session =>
deviceLoginInfoQuery
.firstOption
.fold {
val newSubID = subIdQuery.firstOption.getOrElse(0) + 1
val newLoginInfo = LoginInfoRecord(userLoginInfo.userID, newSubID, userLoginInfo.deviceID, userLoginInfo.userAgent, getCurrentTime)
(loginInfoTable returning loginInfoTable) += newLoginInfo
}( _ =>
// to do : update time
deviceLoginInfoQuery
.map(_.lastLoginTime)
.updateReturning(loginInfoTable.map(identity), getCurrentTime)
.head
)
}
userLoginID(newRecord.userID, newRecord.subID.toString)
}
}
}
I am doing Exercises from Learning Concurrent Programming in Scala.
For an exercise question in code comment.
Program prints proper output of HTML contents for proper URL and timeout sufficiently enough.
Program prints "Error occured" for proper URL and low timeout.
However for invalid URL "Error occured" is not printed. What is the problem with the code below?
/*
* Implement a command-line program that asks the user to input a URL of some website,
* and displays the HTML of that website. Between the time that the user hits ENTER and
* the time that the HTML is retrieved, the program should repetitively print a . to the
* standard output every 50 milliseconds, with a two seconds timeout. Use only futures
* and promises, and avoid the synchronization primitives from the previous chapters.
* You may reuse the timeout method defined in this chapter.
*/
object Excersices extends App {
val timer = new Timer()
def timeout(t: Long = 1000): Future[Unit] = {
val p = Promise[Unit]
val timer = new Timer(true)
timer.schedule(new TimerTask() {
override def run() = {
p success ()
timer cancel()
}
}, t)
p future
}
def printDot = println(".")
val taskOfPrintingDot = new TimerTask {
override def run() = printDot
}
println("Enter a URL")
val lines = io.Source.stdin.getLines()
val url = if (lines hasNext) Some(lines next) else None
timer.schedule(taskOfPrintingDot, 0L, 50.millisecond.toMillis)
val timeOut2Sec = timeout(2.second.toMillis)
val htmlContents = Future {
url map { x =>
blocking {
Source fromURL (x) mkString
}
}
}
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents)) map { x =>
timer cancel ()
x match {
case Some(x) =>
println(x)
case _ =>
println("Error occured")
}
}
Thread sleep 5000
}
As #Gábor Bakos said exception produces Failure which doesn't handled by map:
val fut = Future { Some(Source fromURL ("hhhttp://google.com")) }
scala> fut map { x => println(x) } //nothing printed
res12: scala.concurrent.Future[Unit] = scala.concurrent.impl.Promise$DefaultPromise#5e025724
To process failure - use recover method :
scala> fut recover { case failure => None } map { x => println(x) }
None
res13: scala.concurrent.Future[Unit] = scala.concurrent.impl.Promise$DefaultPromise#578afc83
In your context it's something like:
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents)) recover {case x => println("Error:" + x); None} map { x => ...}
The Complete Code after using recover as advised by #dk14:
object Exercises extends App {
val timer = new Timer()
def timeout(t: Long = 1000): Future[Unit] = {
val p = Promise[Unit]
val timer = new Timer(true)
timer.schedule(new TimerTask() {
override def run() = {
p success ()
timer cancel ()
}
}, t)
p future
}
def printDot = println(".")
val taskOfPrintingDot = new TimerTask {
override def run() = {
printDot
}
}
println("Enter a URL")
val lines = io.Source.stdin.getLines()
val url = if (lines hasNext) Some(lines next) else None
timer.schedule(taskOfPrintingDot, 0L, 50.millisecond.toMillis)
val timeOut2Sec = timeout(2.second.toMillis)
val htmlContents = Future {
url map { x =>
blocking {
Source fromURL (x) mkString
}
}
}
Future.firstCompletedOf(Seq(timeOut2Sec, htmlContents))
.recover { case x => println("Error:" + x); None }
.map { x =>
timer cancel ()
x match {
case Some(x) =>
println(x)
case _ =>
println("Timeout occurred")
}
}
Thread sleep 5000
}
I'm new to scala and functional programming. I'm trying out the usual beginner applications and scripts(Obviously using a bit of over-technology)
Anyways I have this code for a calculator that takes arguments and a switch to dictate the operation to use on the arguments.
object Main {
def main(args: Array[String]): Unit = {
var calc = calculate( "" , _:List[Int])
var values:List[Int] = List()
if(args.size < 1) println("No arguments supplied") else{
args collect {_ match{
case arg if arg.contains("-") => {
if(values.size>0){
calc(values)
values = List()}
calc = calculate( arg , _:List[Int])
}
case arg => {
try{
val value=arg.toInt
values = values ::: List(value)
}catch{case e:NumberFormatException=>println("\nError:Invalid argument:\""+arg+"\"\nCannot convert to Integer.\n")}
}
}
}
calc(values)
}
}
def sum(values:List[Int]) { println("The sum is:"+(values.foldLeft(0)((sum,value) => sum + value))) }
def subtract(values:List[Int]) {
val initial:Int = values(0)
var total:Int = 0
for(i <- 1 until values.size){
total = total + values(i)
}
val diff:Int = initial - total
println("The difference is:"+diff)
}
def calculate(operation:String,values:List[Int]) = operation match {
case "-sum" => sum(values)
case "-diff" => subtract(values)
case _ => println("Default operation \"Sum\" will be applied");sum(values)
}
}
Some points that id like to find if theres a better way to do is like removing the try catch statement.
A better way to compose this application would be very welcome.
How about this one?
object Main extends App {
require(args.size > 0, "Please, supply more arguments")
#annotation.tailrec
def parseArguments(arguments: List[String], operation: String, values: List[Int]() = Nil) {
if(values.nonEmpty) calculate(operation, values)
arguments match {
case op::unprocessed if op.startsWith("-") => parseArguments(unprocessed, op)
case maybeNumber::unprocessed => {
val newValues = try {
maybeNumber.toInt::values
} catch {
case _: NumberFormatException =>
println("\nError:Invalid argument:\""+maybeNumber+"\"\nCannot convert to Integer.\n")
values
}
parseArguments(unprocessed, operation, newValues)
}
case Nil => //done processing, exiting
}
}
parseArguments(args.toList, "")
def diff(values:List[Int]) = {
val initial::tail = values
val total = tail.sum
initial - total
}
def calculate(operation:String, values:List[Int]) = operation match {
case "-sum" => println("The sum is " + values.sum)
case "-diff" => println("The difference is: " + diff(values))
case _ =>
println("""Default operation "Sum" will be applied""")
sum(values)
}
}