How to access computed data from other scala file? - scala

I need to access this data mapData from Calculation.scala. The way I call the function in Final.scala is as seen below.
When I see the output of fetch_data() or print it I see Future(< not completed >) and result is empty. I do not know how to wait until all data is downloaded and then access mapData? MAy I know how to do it? I am new to scala. In C++ I am aware of callbacks and handling is easy there. But in scala I a using Future, Await or OnComplete, but not clear how to do it.
Final.Scala
object finalComputation {
val calculationInfo = new Calculaton()
calclulationInfo.fetch_data()
val result = calculationInfo.getMapData()
def main(args: Array[String]): Unit = {
........
}
}
Calculation.scala
class Calculation {
var mapData = Map.empty[String, String]
def createMapData(metricItem: ActualMetrics) = {
mapData += (metricItem._1 -> metricItem._2)
}
def getMapData() = {
mapData
}
def fetch_data() = {
val totalData: Future[Done] =
querApi
.getData()
.map { data =>
(data)
}
}
}
Await.result(totalData, Duration.Inf).runForeach(unit => {
createMapData(parse.From(totalData))
})
}
}

Well, by starter don't mix concurrency with mutability.
Second, don't create imperative APIs that require some specific call order.
Third, Future is just a fancy wrapper over callbacks.
And fourth, don't initiate computations before the main
// file: Calculation.scala
class Calculation(queryApi: Api) {
def fetchData(): Future[Map[String, String]] =
querApi.getData().map { data =>
data.view.map { metric =>
val ActualMetrics(key, value) = parse(metric)
key -> value
}.toMap
}
}
// file: Main.scala
object Main {
def main(args: Array[String]): Unit = {
val calculation = new Calculation(...)
val dataF = calculation.fetchData()
val result = dataF.map { data =>
// Here you can process the fetched data
// It may be flatMap or foreach, instead of map;
// depending on what you want to do, check the Scaladoc
}
// I don't use future, but I think here you need to do a final await of result.
// In order to avoid the program to finish before the async computation.
}
}
I had to assume some types, but I hope this gives you a general idea of what to do.
My advice, pick any Scala course / book / tutorial and properly learn the language.

Related

scala - Remove on Read

Could anyone suggest an efficient and succinct way of realizing the following feature of "remove on read"? That is, when the attribute _events is read, it's cleared.
case class Event(nanos: Long)
case class History() {
private val _events = ArrayBuffer[Event]()
def add(event: Event): Unit = _events += event
def events: List[Event] = {
val builder = ArrayBuffer[Event]()
builder ++= _events
_events.clear()
builder.toList
}
}
Assuming you are not doing concurrency. You don't need to create a new ArrayBuffer. toList creates a new immutable List which will sustain when you clear. Just do
def events: List[Event] = {
val res = _events.toList
_events.clear()
res
}
You can use ConcurrentLinkedQueue of Java. The following solution supports concurrency, in a way that if you add objects, while retrieving the history, they will be added to the history, and to the result of the current call. However, they are read once in this solution. If you need to stop adding while retrieving the history, we can think about a different solution. Consider having the History class as follows:
case class History() {
private val _events = new ConcurrentLinkedQueue[Event]()
def add(event: Event): Unit = _events.add(event)
def events: List[Event] = {
if (_events.isEmpty) {
List()
} else {
_events.poll() +: events
}
}
}
Then, when running the following code:
def main(args: Array[String]): Unit = {
val h = History()
List(
Event(10),
Event(20),
Event(30)
).foreach(h.add)
println(s"Events are: ${h.events.mkString(", ")}")
println(s"Events are: ${h.events.mkString(", ")}")
}
The output is:
Events are: Event(10), Event(20), Event(30)
Events are:
Process finished with exit code 0

slick returning a dictionaries object from database

I am trying to select from a few tables and put their results in an Object. I want to do this because those are dictionaries and I want all of them at startup.
This is what I have right now:
Controller:
def getDictionaries = Action.async { implicit request =>
Future.successful(Ok(Json.toJson(dictionaryService.getDictionaries)))
}
DictionaryService:
override def getDictionaries : Dictionaries = {
val currencies: Future[Seq[Currency]] = dictionaryDao.getCurrencies
val propertyTypes: Future[Seq[PropertyType]] = dictionaryDao.getPropertyTypes
Dictionaries(
currencies.result(Duration(10L, TimeUnit.SECONDS)),
propertyTypes.result(Duration(10L, TimeUnit.SECONDS))
)
}
DictionaryDAO:
override def getCurrencies: Future[Seq[Currency]] = {
db.run(slickCurrencies.result)
}
... getPropertyTypes ...
Dictionaries case class and companion object
case class Dictionaries (currencies: Seq[Currency], propertyTypes: Seq[PropertyType])
object Dictionaries {
implicit val jsonFormat = Json.format[Dictionaries]
}
I am not very proud of currencies.result(Duration(10L, TimeUnit.SECONDS)) but I am not sure what I should retrieve from this function so that I can easily transform it to JSON. Also this line of code is still not working because the compiler is telling me to use Await object instead.
Also for PropertyType I need to do the same thing as for currencies.
What is the best way to obtain the desired result?
I am in the learning phase so I don't get much of this.
LATER EDIT: the flow is: request -> getDictionaries -> dictionaryService.getDictionaries -> dictionaryDAO.getCurrencies&PropertyTypes
It seems to me that I need to get them in a synchronized way.
The purpose of this is to not create a request for each type of dictionary. If I have 10 dictionaries, I want to get all of them in one request.
Later EDIT 2
This is my working example which does not look very well:
Controller:
def getDictionaries = Action.async { implicit request =>
dictionaryService.getDictionaries.map {
dictionaries => Ok(Json.toJson(dictionaries))
}
}
DictionaryService:
override def getDictionaries : Future[Dictionaries] = {
dictionaryDao.getCurrencies.flatMap { currencies =>
dictionaryDao.getPropertyTypes.flatMap { propertyTypes =>
Future.successful(Dictionaries(
currencies,
propertyTypes
))
}
}
}
Your method getDictionaries should return Future[Dictionaries]:
override def getDictionaries : Future[Dictionaries] = {
val futureCurrencies = dictionaryDao.getCurrencies
futureCurrencies map { currencies => // currencies is here Seq[Currency]
Dictionaries(
currencies,
List(PropertyType(1, "x"))
)
}
And your controller method:
def getDictionaries = Action.async { implicit request =>
val futureDictionaries = dictionaryService.getDictionaries
futureDictionaries map { dictionaries =>
Ok(Json.toJson(dictionaries))
}
}
EXPLANATION: You have to use Futures all the way bottom up if you want your actions to be really asynchronous. You'll have to learn how to compose them with map, flatMap and for comprehensions. I didn't find any gentle and short introduction for that, you'll have to search it by yourself. Basically, when you map over Future, you are transforming it's(successful) result, and thus getting another Future. With flatMap you can have some ordering of futures, like: when f1 finishes, start f2 etc...

cache using functional callbacks/ proxy pattern implementation scala

How to implement cache using functional programming
A few days ago I came across callbacks and proxy pattern implementation using scala.
This code should only apply inner function if the value is not in the map.
But every time map is reinitialized and values are gone (which seems obivous.
How to use same cache again and again between different function calls
class Aggregator{
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
val cache = HashMap[Int, Int]()
(t:Int) => {
if (!cache.contains(t)) {
println("Evaluating..."+t)
val r = function.apply(t);
cache.put(t,r)
r
}
else
{
cache.get(t).get;
}
}
}
def memoizedDoubler = memoize( (key:Int) => {
println("Evaluating...")
key*2
})
}
object Aggregator {
def main( args: Array[String] ) {
val agg = new Aggregator()
agg.memoizedDoubler(2)
agg.memoizedDoubler(2)// It should not evaluate again but does
agg.memoizedDoubler(3)
agg.memoizedDoubler(3)// It should not evaluate again but does
}
I see what you're trying to do here, the reason it's not working is that every time you call memoizedDoubler it's first calling memorize. You need to declare memoizedDoubler as a val instead of def if you want it to only call memoize once.
val memoizedDoubler = memoize( (key:Int) => {
println("Evaluating...")
key*2
})
This answer has a good explanation on the difference between def and val. https://stackoverflow.com/a/12856386/37309
Aren't you declaring a new Map per invocation ?
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
val cache = HashMap[Int, Int]()
rather than specifying one per instance of Aggregator ?
e.g.
class Aggregator{
private val cache = HashMap[Int, Int]()
def memoize(function: Function[Int, Int] ):Function[Int,Int] = {
To answer your question:
How to implement cache using functional programming
In functional programming there is no concept of mutable state. If you want to change something (like cache), you need to return updated cache instance along with the result and use it for the next call.
Here is modification of your code that follows that approach. function to calculate values and cache is incorporated into Aggregator. When memoize is called, it returns tuple, that contains calculation result (possibly taken from cache) and new Aggregator that should be used for the next call.
class Aggregator(function: Function[Int, Int], cache:Map[Int, Int] = Map.empty) {
def memoize:Int => (Int, Aggregator) = {
t:Int =>
cache.get(t).map {
res =>
(res, Aggregator.this)
}.getOrElse {
val res = function(t)
(res, new Aggregator(function, cache + (t -> res)))
}
}
}
object Aggregator {
def memoizedDoubler = new Aggregator((key:Int) => {
println("Evaluating..." + key)
key*2
})
def main(args: Array[String]) {
val (res, doubler1) = memoizedDoubler.memoize(2)
val (res1, doubler2) = doubler1.memoize(2)
val (res2, doubler3) = doubler2.memoize(3)
val (res3, doubler4) = doubler3.memoize(3)
}
}
This prints:
Evaluating...2
Evaluating...3

Slick - What if database does not contain result

I am trying to build a simple RESTful service that performs CRUD operations on a database and returns JSON. I have a service adhering to an API like this
GET mydomain.com/predictions/some%20string
I use a DAO which contains the following method that I have created to retrieve the associated prediction:
def getPrediction(rawText: String): Prediction = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.head
val f = db.run(header)
f.onComplete{case pred => pred}
throw new Exception("Oops")
}
However, this can't be right, so I started reading about Option. I changed my code accordingly:
def getPrediction(rawText: String): Option[Prediction] = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.headOption
val f = db.run(header)
f.onSuccess{case pred => pred}
None
}
This still doesn't feel quite right. What is the best way to invoke these filters, return the results, and handle any uncertainty?
I think the best way to rewrite your code is like this:
def getPrediction(rawText: String): Future[Option[Prediction]] = {
db.run(users.filter(_.rawText === rawText).result.headOption)
}
In other words, return a Future instead of the plain result. This way, the database actions will execute asynchronously, which is the preferred way for both Play and Akka.
The client code will then work with the Future. Per instance, a Play action would be like:
def prediction = Action.async {
predictionDao.getPrediction("some string").map { pred =>
Ok(views.html.predictions.show(pred))
}.recover {
case ex =>
logger.error(ex)
BadRequest()
}
}

Scala: Cool way to manage sequential execution of Futures?

I'm trying to write a data module in Scala.
While loading entire data in parallel, some data depends on other data, so execution sequence has to be managed in efficient way.
For example in code, I keep a map with name of data and manifest
val dataManifestMap = Map(
"foo" -> manifest[String],
"bar" -> manifest[Int],
"baz" -> manifest[Int],
"foobar" -> manifest[Set[String]], // need to be executed after "foo" and "bar" is ready
"foobarbaz" -> manifest[String], // need to be executed after "foobar" and "baz" is ready
)
These data will be stored in a mutable hash map
private var dataStorage = new mutable.HashMap[String, Future[Any]]()
There are some code that will load data
def loadAllData(): Future[Unit] = {
Future.join(
(dataManifestMap map {
case (data, m) => loadData(data, m) } // function has all the string matching and loading stuff
).toSeq
)
}
def loadData[T](data: String, m: Manifest[T]): Future[Unit] = {
val d = data match {
case "foo" => Future.value("foo")
case "bar" => Future.value(3)
case "foobar" => // do something with dataStorage("foo") and dataStorage("bar")
... // and so forth (in a real example it would be much more complicated for sure)
}
d flatMap {
dVal => { this.synchronized { dataStorage(data) = dVal }; Future.value(Unit) }
}
}
This way, I cannot make sure "foobar" is loaded when "foo" and "bar" is ready, and so forth.
How can I manage this in a "cool" way, since I might have hundreds of different data?
It would be "awesome" if I could have some kind of data structure that has all the info about something has to be loaded after something, and sequential execution can be handled by flatMap in a neat way.
Thanks for the help in advance.
All things being equal, I'd tend to use for comprehensions. For example:
def findBucket: Future[Bucket[Empty]] = ???
def fillBucket(bucket: Bucket[Empty]): Future[Bucket[Water]] = ???
def extinguishOvenFire(waterBucket: Bucket[Water]): Future[Oven] = ???
def makeBread(oven: Oven): Future[Bread] = ???
def makeSoup(oven: Oven): Future[Soup] = ???
def eatSoup(soup: Soup, bread: Bread): Unit = ???
def doLunch = {
for (bucket <- findBucket;
filledBucket <- fillBucket(bucket);
oven <- extinguishOvenFire(filledBucket);
soupFuture = makeSoup(oven);
breadFuture = makeBread(oven);
soup <- soupFuture;
bread <- breadFuture) {
eatSoup(soup, bread)
}
}
This chains futures together, and calls the relevant methods once dependencies are satisfied. Note that we use = in the for comprehension to allow us to start two Futures at the same time. As it stands, doLunch returns Unit, but if you replace the last few lines with:
// ..snip..
bread <- breadFuture) yield {
eatSoup(soup, bread)
oven
}
}
Then it will return Future[Oven] - which might be useful if you want to use the oven for something else after lunch.
As for your code, my first though would be that you should consider Spray cache, as it looks like it might fit your requirements. If not, my next thought would be to replace the Stringly typed interface you've currently got and go with something based on typed method calls:
private def save[T](key: String)(value: Future[T]) = this.synchronized {
dataStorage(key) = value
value
}
def loadFoo = save("foo"){Future("foo")}
def loadBar = save("bar"){Future(3)}
def loadFooBar = save("foobar"){
for (foo <- loadFoo;
bar <- loadBar) yield foo + bar // Or whatever
}
def loadBaz = save("baz"){Future(200L)}
def loadAll = {
val topLevelFutures = Seq(loadFooBar, loadBaz)
// Use standard library function to combine futures
Future.fold(topLevelFutures)(())((u,f) => ())
}
// I don't consider this method necessary, but if you've got a legacy API to support...
def loadData[T](key: String)(implicit manifest: Manifest[T]) = {
val future = key match {
case "foo" => loadFoo
case "bar" => loadBar
case "foobar" => loadFooBar
case "baz" => loadBaz
case "all" => loadAll
}
future.mapTo[T]
}