slick returning a dictionaries object from database - scala

I am trying to select from a few tables and put their results in an Object. I want to do this because those are dictionaries and I want all of them at startup.
This is what I have right now:
Controller:
def getDictionaries = Action.async { implicit request =>
Future.successful(Ok(Json.toJson(dictionaryService.getDictionaries)))
}
DictionaryService:
override def getDictionaries : Dictionaries = {
val currencies: Future[Seq[Currency]] = dictionaryDao.getCurrencies
val propertyTypes: Future[Seq[PropertyType]] = dictionaryDao.getPropertyTypes
Dictionaries(
currencies.result(Duration(10L, TimeUnit.SECONDS)),
propertyTypes.result(Duration(10L, TimeUnit.SECONDS))
)
}
DictionaryDAO:
override def getCurrencies: Future[Seq[Currency]] = {
db.run(slickCurrencies.result)
}
... getPropertyTypes ...
Dictionaries case class and companion object
case class Dictionaries (currencies: Seq[Currency], propertyTypes: Seq[PropertyType])
object Dictionaries {
implicit val jsonFormat = Json.format[Dictionaries]
}
I am not very proud of currencies.result(Duration(10L, TimeUnit.SECONDS)) but I am not sure what I should retrieve from this function so that I can easily transform it to JSON. Also this line of code is still not working because the compiler is telling me to use Await object instead.
Also for PropertyType I need to do the same thing as for currencies.
What is the best way to obtain the desired result?
I am in the learning phase so I don't get much of this.
LATER EDIT: the flow is: request -> getDictionaries -> dictionaryService.getDictionaries -> dictionaryDAO.getCurrencies&PropertyTypes
It seems to me that I need to get them in a synchronized way.
The purpose of this is to not create a request for each type of dictionary. If I have 10 dictionaries, I want to get all of them in one request.
Later EDIT 2
This is my working example which does not look very well:
Controller:
def getDictionaries = Action.async { implicit request =>
dictionaryService.getDictionaries.map {
dictionaries => Ok(Json.toJson(dictionaries))
}
}
DictionaryService:
override def getDictionaries : Future[Dictionaries] = {
dictionaryDao.getCurrencies.flatMap { currencies =>
dictionaryDao.getPropertyTypes.flatMap { propertyTypes =>
Future.successful(Dictionaries(
currencies,
propertyTypes
))
}
}
}

Your method getDictionaries should return Future[Dictionaries]:
override def getDictionaries : Future[Dictionaries] = {
val futureCurrencies = dictionaryDao.getCurrencies
futureCurrencies map { currencies => // currencies is here Seq[Currency]
Dictionaries(
currencies,
List(PropertyType(1, "x"))
)
}
And your controller method:
def getDictionaries = Action.async { implicit request =>
val futureDictionaries = dictionaryService.getDictionaries
futureDictionaries map { dictionaries =>
Ok(Json.toJson(dictionaries))
}
}
EXPLANATION: You have to use Futures all the way bottom up if you want your actions to be really asynchronous. You'll have to learn how to compose them with map, flatMap and for comprehensions. I didn't find any gentle and short introduction for that, you'll have to search it by yourself. Basically, when you map over Future, you are transforming it's(successful) result, and thus getting another Future. With flatMap you can have some ordering of futures, like: when f1 finishes, start f2 etc...

Related

How to access computed data from other scala file?

I need to access this data mapData from Calculation.scala. The way I call the function in Final.scala is as seen below.
When I see the output of fetch_data() or print it I see Future(< not completed >) and result is empty. I do not know how to wait until all data is downloaded and then access mapData? MAy I know how to do it? I am new to scala. In C++ I am aware of callbacks and handling is easy there. But in scala I a using Future, Await or OnComplete, but not clear how to do it.
Final.Scala
object finalComputation {
val calculationInfo = new Calculaton()
calclulationInfo.fetch_data()
val result = calculationInfo.getMapData()
def main(args: Array[String]): Unit = {
........
}
}
Calculation.scala
class Calculation {
var mapData = Map.empty[String, String]
def createMapData(metricItem: ActualMetrics) = {
mapData += (metricItem._1 -> metricItem._2)
}
def getMapData() = {
mapData
}
def fetch_data() = {
val totalData: Future[Done] =
querApi
.getData()
.map { data =>
(data)
}
}
}
Await.result(totalData, Duration.Inf).runForeach(unit => {
createMapData(parse.From(totalData))
})
}
}
Well, by starter don't mix concurrency with mutability.
Second, don't create imperative APIs that require some specific call order.
Third, Future is just a fancy wrapper over callbacks.
And fourth, don't initiate computations before the main
// file: Calculation.scala
class Calculation(queryApi: Api) {
def fetchData(): Future[Map[String, String]] =
querApi.getData().map { data =>
data.view.map { metric =>
val ActualMetrics(key, value) = parse(metric)
key -> value
}.toMap
}
}
// file: Main.scala
object Main {
def main(args: Array[String]): Unit = {
val calculation = new Calculation(...)
val dataF = calculation.fetchData()
val result = dataF.map { data =>
// Here you can process the fetched data
// It may be flatMap or foreach, instead of map;
// depending on what you want to do, check the Scaladoc
}
// I don't use future, but I think here you need to do a final await of result.
// In order to avoid the program to finish before the async computation.
}
}
I had to assume some types, but I hope this gives you a general idea of what to do.
My advice, pick any Scala course / book / tutorial and properly learn the language.

Evaluating a Future Boolean asynchronously using Scala and Play

I have a method that returns a Future[Boolean] in a Play controller and i want to evaluate that using async but i can't seem to get it to compile.
The following will work:
def health = Action {
logger.info("Endpoint method: health")
val isHealthy = healthCheckService.checkDynamo()
val b: Boolean = Await.result(isHealthy, scala.concurrent.duration.Duration(5, "seconds"))
Ok(Json.toJson(HealthCheckResponse(b.toString)))
}
But i don't think i want that Await in there. So i'm trying things like this with no success:
def health =
Action.async {
Future {
logger.info("Endpoint method: health")
healthCheckService.checkDynamo() match {
case Future.successful(true) => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
val r = healthCheckService.checkDynamo() match {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
}
I can't even get those to compile to test them out.
Any suggestions?
Try this:
def health = Action.async {
healthCheckService.checkDynamo().map {
case true => Ok(Json.toJson("false"))
case false => Ok(Json.toJson("true"))
}
}
Let Play handle the awaiting for you under the hood. That is, Action.async accepts a Future, which checkDynamo() already returns. All you have to do is map it to the appropriate result.
With Futures you have to use combinators like map and flatMap to express the final value. For example:
Action.async {
healthCheckService.checkDynamo()
.map { result => // boolean
HealthCheckResponse(result.toString)
}
.map(Json.toJson(_))
.map(Ok(_))
}
(You can merge maps above to one map and construct the final Ok value there; it is more or less a matter of taste)
If you have, say, two async calls which you want to execute and return a result based on their results, you can use flatMap, which could be easily expressed using a for comprehension:
Action.async {
for {
result1 <- someService.someCall()
result2 <- anotherService.anotherCall(result1.someProperty)
finalResult = SomeFinalResultType(result1, result2)
} yield Ok(Json.toJson(finalResult))
}
If you are not familiar with futures, you might want to read some tutorial which explains their nature, how to combine them and how to get useful results from them, like this one: http://hello-scala.com/920-scala-futures.html

Passing a list of functions to a method to be executed later with data supplied

I am trying to build a multi level Validator object that is fairly generic. The idea being you have levels of validations if Level 1 passes then you do Level 2, etc. but I am struggling with one specific area: creating a function call but not executing it until a later point.
Data:
case class FooData(alpha: String, beta: String) extends AllData
case class BarData(gamma: Int, delta: Int) extends AllData
ValidationError:
case class ValidationError(code: String, message: String)
Validator:
object Validator {
def validate(validations: List[List[Validation]]): List[ValidationError] = {
validations match {
case head :: nil => // Execute the functions and get the results back
// Recursively work down the levels (below syntax may be incorrect)
case head :: tail => validate(head) ... // if no errors then validate(tail) etc.
...
}
}
}
Sample Validator:
object CorrectNameFormatValidator extends Validation {
def validate(str: String): Seq[ValidationError] = {
...
}
}
How I wish to use it:
object App {
def main(args: Array[String]): Unit = {
val fooData = FooData(alpha = "first", beta = "second")
val levelOneValidations = List(
CorrectNameFormatValidator(fooData.alpha),
CorrectNameFormatValidator(fooData.beta),
SomeOtherValidator(fooData.beta)
)
// I don't want these to execute as function calls here
val levelTwoValidations = List(
SomeLevelTwoValidator (fooData.alpha),
SomeLevelTwoValidator(fooData.beta),
SomeOtherLevelValidator(fooData.beta),
SomeOtherLevelValidator(fooData.alpha)
)
val validationLevels = List(levelOneValidations, levelTwoValidations)
Validator.validate(validationLevels)
}
}
Am I doing something really convoluted when I don't need to be or am I just missing a component?
Essentially I want to define when a function will be called and with which parameters but I don't want the call to happen until I say within the Validator. Is this something that's possible?
You can use lazy val or def when defining levelOneValidation, levelTwoValidations and validationLevel:
object App {
def main(args: Array[String]): Unit = {
val fooData = FooData(alpha = "first", beta = "second")
lazy val levelOneValidations = List(
CorrectNameFormatValidator(fooData.alpha),
CorrectNameFormatValidator(fooData.beta),
SomeOtherValidator(fooData.beta)
)
// I don't want these to execute as function calls here
lazy val levelTwoValidations = List(
SomeLevelTwoValidator (fooData.alpha),
SomeLevelTwoValidator(fooData.beta),
SomeOtherLevelValidator(fooData.beta),
SomeOtherLevelValidator(fooData.alpha)
)
lazy val validationLevels = List(levelOneValidations, levelTwoValidations)
Validator.validate(validationLevels)
}
}
You also need to change validate method to get the validations ByName and not
ByValue using : => :
object Validator {
def validate(validations: => List[List[Validation]]): List[ValidationError] = {
validations match {
case head :: nil => // Execute the functions and get the results back
// Recursively work down the levels (below syntax may be incorrect)
case head :: tail => validate(head) ... // if no errors then validate(tail) etc.
...
}
}
}
Anyway, I think you can implement it differently by just using some OOP design patterns, like Chain of Responsibility.

Scala: Cool way to manage sequential execution of Futures?

I'm trying to write a data module in Scala.
While loading entire data in parallel, some data depends on other data, so execution sequence has to be managed in efficient way.
For example in code, I keep a map with name of data and manifest
val dataManifestMap = Map(
"foo" -> manifest[String],
"bar" -> manifest[Int],
"baz" -> manifest[Int],
"foobar" -> manifest[Set[String]], // need to be executed after "foo" and "bar" is ready
"foobarbaz" -> manifest[String], // need to be executed after "foobar" and "baz" is ready
)
These data will be stored in a mutable hash map
private var dataStorage = new mutable.HashMap[String, Future[Any]]()
There are some code that will load data
def loadAllData(): Future[Unit] = {
Future.join(
(dataManifestMap map {
case (data, m) => loadData(data, m) } // function has all the string matching and loading stuff
).toSeq
)
}
def loadData[T](data: String, m: Manifest[T]): Future[Unit] = {
val d = data match {
case "foo" => Future.value("foo")
case "bar" => Future.value(3)
case "foobar" => // do something with dataStorage("foo") and dataStorage("bar")
... // and so forth (in a real example it would be much more complicated for sure)
}
d flatMap {
dVal => { this.synchronized { dataStorage(data) = dVal }; Future.value(Unit) }
}
}
This way, I cannot make sure "foobar" is loaded when "foo" and "bar" is ready, and so forth.
How can I manage this in a "cool" way, since I might have hundreds of different data?
It would be "awesome" if I could have some kind of data structure that has all the info about something has to be loaded after something, and sequential execution can be handled by flatMap in a neat way.
Thanks for the help in advance.
All things being equal, I'd tend to use for comprehensions. For example:
def findBucket: Future[Bucket[Empty]] = ???
def fillBucket(bucket: Bucket[Empty]): Future[Bucket[Water]] = ???
def extinguishOvenFire(waterBucket: Bucket[Water]): Future[Oven] = ???
def makeBread(oven: Oven): Future[Bread] = ???
def makeSoup(oven: Oven): Future[Soup] = ???
def eatSoup(soup: Soup, bread: Bread): Unit = ???
def doLunch = {
for (bucket <- findBucket;
filledBucket <- fillBucket(bucket);
oven <- extinguishOvenFire(filledBucket);
soupFuture = makeSoup(oven);
breadFuture = makeBread(oven);
soup <- soupFuture;
bread <- breadFuture) {
eatSoup(soup, bread)
}
}
This chains futures together, and calls the relevant methods once dependencies are satisfied. Note that we use = in the for comprehension to allow us to start two Futures at the same time. As it stands, doLunch returns Unit, but if you replace the last few lines with:
// ..snip..
bread <- breadFuture) yield {
eatSoup(soup, bread)
oven
}
}
Then it will return Future[Oven] - which might be useful if you want to use the oven for something else after lunch.
As for your code, my first though would be that you should consider Spray cache, as it looks like it might fit your requirements. If not, my next thought would be to replace the Stringly typed interface you've currently got and go with something based on typed method calls:
private def save[T](key: String)(value: Future[T]) = this.synchronized {
dataStorage(key) = value
value
}
def loadFoo = save("foo"){Future("foo")}
def loadBar = save("bar"){Future(3)}
def loadFooBar = save("foobar"){
for (foo <- loadFoo;
bar <- loadBar) yield foo + bar // Or whatever
}
def loadBaz = save("baz"){Future(200L)}
def loadAll = {
val topLevelFutures = Seq(loadFooBar, loadBaz)
// Use standard library function to combine futures
Future.fold(topLevelFutures)(())((u,f) => ())
}
// I don't consider this method necessary, but if you've got a legacy API to support...
def loadData[T](key: String)(implicit manifest: Manifest[T]) = {
val future = key match {
case "foo" => loadFoo
case "bar" => loadBar
case "foobar" => loadFooBar
case "baz" => loadBaz
case "all" => loadAll
}
future.mapTo[T]
}

Concurrent.patchPanel not sending data from multiple enumerators - only sends from last enumerator added

I'm hoping someone else has used patchPanel to combine multiple enumerators together going down to a client over a websocket. The issue i'm running into is that the patchPanel is only sending the data feed from the last enumerator added into it.
I followed the example from; http://lambdaz.blogspot.ca/2012/12/play-21-multiplexing-enumerators-into.html which is the only reference I've been able to find regarding patchPanel.
Versions; play! 2.1.1 (using Java 1.7.0_11 and Scala 2.10.0)
The web socket method;
def monitorStream = WebSocket.async[JsValue] { request =>
val promiseIn = promise[Iteratee[JsValue, Unit]]
val out = Concurrent.patchPanel[JsValue] { patcher =>
val in = Iteratee.foreach[JsValue] { json =>
val event:Option[String] = (json \ "event").asOpt[String]
val systemId = (json \ "systemId").as[Long]
event.getOrElse("") match {
case "join" =>
val physicalSystem = SystemIdHandler.getById(systemId)
val monitorOut = (MonitorStreamActor.joinMonitor(physicalSystem))
monitorOut map { enum =>
val success = patcher.patchIn(enum)
}
}
}.mapDone { _ => Logger.info("Disconnected") }
promiseIn.success(in)
}
future(Iteratee.flatten(promiseIn.future),out)
}
The MonitorStreamActor call;
def joinMonitor(physicalSystem: PhysicalSystem):
scala.concurrent.Future[Enumerator[JsValue]]
= {
val monitorActor = ActorBase.akkaSystem.actorFor("/user/system-" + physicalSystem.name +"/stream")
(monitorActor ? MonitorJoin()).map {
case MonitorConnected(enumerator) =>
enumerator
}
}
The enumerator is returned fine, and the data fed into it is coming from a timer calling the actor. Actor definition, the timer hits the UpdatedTranStates case;
class MonitorStreamActor() extends Actor {
val (monitorEnumerator, monitorChannel) = Concurrent.broadcast[JsValue]
import play.api.Play.current
def receive = {
case MonitorJoin() => {
Logger.debug ("Actor monitor join")
sender ! MonitorConnected(monitorEnumerator)
}
case UpdatedTranStates(systemName,tranStates) => {
//println("Got updated Tran States")
val json = Json.toJson(tranStates.map(m => Map("State" -> m._1, "Count" -> m._2) ))
//println("Pushing updates to monitorChannel")
sendUpdateToClients(systemName, "states", json)
}
def sendUpdateToClients(systemName:String, updateType:String, json:JsValue) {
monitorChannel.push(Json.toJson(
Map(
"dataType"->Json.toJson(updateType),
"systemName" -> Json.toJson(systemName),
"data"->json)))
}
}
}
I've poked around for a while on this and haven't found a reason why only the last enumerator that is added into the patchPanel has the data sent. the API docs are not of much help, it really sounds like all you have to do is call patchIn and it should combine all enumerators to an iteratee, but that doesn't seem to be the case.
The PatchPanel by design replaces the current enumerator with the new one provided by the patchIn method.
In order to have multiple Enumerators combined together you need to use interleave or andThen methods to combine enumerators together. Interleave is preferred for this case as it will take events from each Enumerator as they are available, vs emptying one then moving to the next (as with andThen operator).
ie, in monitorStream;
val monitorOut = (MonitorStreamActor.joinMonitor(physicalSystem))
monitorOut map { enum =>
mappedEnums += ((physicalSystem.name, enum))
patcher.patchIn(Enumerator.interleave[JsValue]( mappedEnums.values.toSeq))
}
patcher is the patch panel, and mappedEnums is a HashMap[String,Enumerator[JsValue]] - re-add the patcher each time the Enumerators change (add or delete) - it works, not sure if it's the best way, but it'll do for now :)