I have model repository class with byId and save metdhods
def byID(id:Long) = db.run{ query.filter(_.id === id).result }.map(_.headOption)
def save(model:User) = db.run{ query.filter(_.id===model.id).update(model) }
Now I want to use both these methods, first load user, then change something, and then save user, like this
userRepository.byID(5L).map{_.map{user =>
val newuser = user.copy(name = "John")
userRepository.save(newuser)
}}
How can I do it in one transaction?
I think that slick 3 doesn't support having a transaction that spans over different futures and when you call db.run you are passing in a DBIO[SomeType] in and getting a Future out. The good news is that you can instead compose your DBIOs structuring your API in a slightly different way:
def byID(id:Long) = query.filter(_.id === id).result }.map(_.headOption)
def save(model:User) = query.filter(_.id===model.id).update(model)
And then:
db.run(userRepository.byID(5L).flatMap { users =>
DBIO.sequence(users.map { user =>
userRepository.save(user.copy(name = "John"))
})
}.transactionally)
Related
I have a method in which there are multiple calls to db. As I have not implemented any concurrent processing, a 2nd db call has to wait until the 1st db call gets completed, 3rd has to wait until the 2nd gets completed and so on.
All db calls are independent of each other. I want to make this in such a way that all DB calls run concurrently.
I am new to Akka framework.
Can someone please help me with small sample or references would help. Application is developed in Scala Lang.
There are three primary ways that you could achieve concurrency for the given example needs.
Futures
For the particular use case that is asked about in the question I would recommend Futures before any akka construct.
Suppose we are given the database calls as functions:
type Data = ???
val dbcall1 : () => Data = ???
val dbcall2 : () => Data = ???
val dbcall3 : () => Data = ???
Concurrency can be easily applied, and then the results can be collected, using Futures:
val f1 = Future { dbcall1() }
val f2 = Future { dbcall2() }
val f3 = Future { dbcall3() }
for {
v1 <- f1
v2 <- f2
v3 <- f3
} {
println(s"All data collected: ${v1}, ${v2}, ${v3}")
}
Akka Streams
There is a similar stack answer which demonstrates how to use the akka-stream library to do concurrent db querying.
Akka Actors
It is also possible to write an Actor to do the querying:
object MakeQuery
class DBActor(dbCall : () => Data) extends Actor {
override def receive = {
case _ : MakeQuery => sender ! dbCall()
}
}
val dbcall1ActorRef = system.actorOf(Props(classOf[DBActor], dbcall1))
However, in this use case Actors are less helpful because you still need to collect all of the data together.
You can either use the same technique as the "Futures" section:
val f1 : Future[Data] = (dbcall1ActorRef ? MakeQuery).mapTo[Data]
for {
v1 <- f1
...
Or, you would have to wire the Actors together by hand through the constructor and handle all of the callback logic for waiting on the other Actor:
class WaitingDBActor(dbCall : () => Data, previousActor : ActorRef) {
override def receive = {
case _ : MakeQuery => previousActor forward MakeQuery
case previousData : Data => sender ! (dbCall(), previousData)
}
}
If you want to querying database, you should use something like slick which is a modern database query and access library for Scala.
quick example of slick:
case class User(id: Option[Int], first: String, last: String)
class Users(tag: Tag) extends Table[User](tag, "users") {
def id = column[Int]("id", O.PrimaryKey, O.AutoInc)
def first = column[String]("first")
def last = column[String]("last")
def * = (id.?, first, last) <> (User.tupled, User.unapply)
}
val users = TableQuery[Users]
then your need to create configuration for your db:
mydb = {
dataSourceClass = "org.postgresql.ds.PGSimpleDataSource"
properties = {
databaseName = "mydb"
user = "myuser"
password = "secret"
}
numThreads = 10
}
and in your code you load configuration:
val db = Database.forConfig("mydb")
then run your query with db.run method which gives you future as result, for example you can get all rows by calling method result
val allRows: Future[Seq[User]] = db.run(users.result)
this query run without blocking current thread.
If you have task which take long time to execute or calling to another service, you should use futures.
Example of that is simple HTTP call to external service. you can find example in here
If you have task which take long time to execute and for doing so, you have to keep mutable states, in this case the best option is using Akka Actors which encapsulate your state inside an actor which solve problem of concurrency and thread safety as simple as possible.Example of suck tasks are:
import akka.actor.Actor
import scala.concurrent.Future
case class RegisterEndpoint(endpoint: String)
case class NewUpdate(update: String)
class UpdateConsumer extends Actor {
val endpoints = scala.collection.mutable.Set.empty[String]
override def receive: Receive = {
case RegisterEndpoint(endpoint) =>
endpoints += endpoint
case NewUpdate(update) =>
endpoints.foreach { endpoint =>
deliverUpdate(endpoint, update)
}
}
def deliverUpdate(endpoint: String, update: String): Future[Unit] = {
Future.successful(Unit)
}
}
If you want to process huge amount of live data, or websocket connection, processing CSV file which is growing over time, ... or etc, the best option is Akka stream. For example reading data from kafka topic using Alpakka:Alpakka kafka connector
I have a method that makes an API call and returns a Future[Seq[Post]], with each post having a sequential id. The API call returns 200 posts per call. In order to get a complete sequence of posts, I submit the smallest id (as maxId) in subsequent calls. Each Post object also contains a User object that contains a field specifying the total number of posts made by the user. What's the best way to accomplish getting all of a user's posts in a non-blocking manner?
Thus far, I have the following blocking code:
def completeTimeline(userId: Long) = {
def getTimeline(maxId = Option[Long] = None): Seq[Post] = {
val timeline: Future[Seq[Post]] = userPosts(userId) // calls method that returns Future[Seq[Post]]
Await.result(timeline, 5.seconds)
}
def recurseTimeline(timeline: Seq[Post]): Seq[Post] = {
if (timeline.length < timeline.head.user.map(user => user.post_count).getOrElse(0)) {
val maxId = timeline.min(Ordering.by((p: Post) => p.id)).id - 1 // API call with maxId is inclusive, subtract 1 to avoid duplicates
val nextTimeline = getTimeline(maxId = Some(maxId))
val newTimeline = timeline ++ nextTimeline
recurseTimeline(newTimeline)
} else timeline
}
recurseTimeline(getTimeline())
}
Something like this:
def completeTimeline(userId: Long): Future[Seq[Post]] = {
def getTimeline(maxId: Option[Long] = None): Future[Seq[Post]] = {
val timeline: Future[Seq[Post]] = userPosts(userId) // calls method that returns Future[Seq[Post]]
timeline
}
def recurseTimeline(timeline: Seq[Post]): Future[Seq[Post]] = {
if (timeline.length < timeline.head.user.map(user => user.post_count).getOrElse(0)) {
val maxId = timeline.min(Ordering.by((p: Post) => p.id)).id - 1 // API call with maxId is inclusive, subtract 1 to avoid duplicates
for {
nextTimeline <- getTimeline(maxId = Some(maxId))
full <- recurseTimeline(timeline ++ nextTimeline)
} yield full
} else Future.successful(timeline)
}
getTimeline() flatMap recurseTimeline
}
Essentially, you are scheduling things to happen once each future completes (map, flatMap, and for-comprehension), instead of waiting for them to complete.
The code can probably be simplified if i understood a bit better what you're trying to do.
When you say non-blocking that is incompatible with I need to check all of the Posts: you do need to wait for all of the Futures to complete - and thus be able to find the minimum id before proceeding. Correct? or would you like to further explain your intent?
Proceeding on that assumption - you can add each of the Future's to a list and then iterate on the list and calling Await on each one. This is in fact the standard blocking mechanism for obtaining a set of futures.
If there were further considerations and/or assumptions being made then please provide them - to allow tweaking this general approach for your particulars.
I am trying to build a simple RESTful service that performs CRUD operations on a database and returns JSON. I have a service adhering to an API like this
GET mydomain.com/predictions/some%20string
I use a DAO which contains the following method that I have created to retrieve the associated prediction:
def getPrediction(rawText: String): Prediction = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.head
val f = db.run(header)
f.onComplete{case pred => pred}
throw new Exception("Oops")
}
However, this can't be right, so I started reading about Option. I changed my code accordingly:
def getPrediction(rawText: String): Option[Prediction] = {
val predictionAction = predictions.filter{_.rawText === rawText}.result
val header = predictionAction.headOption
val f = db.run(header)
f.onSuccess{case pred => pred}
None
}
This still doesn't feel quite right. What is the best way to invoke these filters, return the results, and handle any uncertainty?
I think the best way to rewrite your code is like this:
def getPrediction(rawText: String): Future[Option[Prediction]] = {
db.run(users.filter(_.rawText === rawText).result.headOption)
}
In other words, return a Future instead of the plain result. This way, the database actions will execute asynchronously, which is the preferred way for both Play and Akka.
The client code will then work with the Future. Per instance, a Play action would be like:
def prediction = Action.async {
predictionDao.getPrediction("some string").map { pred =>
Ok(views.html.predictions.show(pred))
}.recover {
case ex =>
logger.error(ex)
BadRequest()
}
}
I'd like to keep my application as asynchronously as possible. Now I have this repository:
object LanguageRepository extends LanguageRepositoryTrait
{
private val languages = TableQuery[Languages]
private def db:Database = Database.forDataSource(DB.getDataSource())
private def filterQuery(id: Long): Query[Languages, Language, Seq] = languages.filter(_.id === id)
private def filterCode(code: String): Query[Languages, Language, Seq] = languages.filter(_.code === code)
private def all() : Query[Languages, Language, Seq] = languages
override def find(id: Long): Future[Language] =
{
try db.run(filterQuery(id).result.head)
finally db.close()
}
override def find(code: String): Future[Language] =
{
try db.run(filterCode(code).result.head)
finally db.close()
}
override def get(): Future[Seq[Language]] =
{
try db.run(all().result)
finally db.close()
}
}
When I call a url like "domain.tld/{language}" I want to check whether the given language(code) actually exists. Like, if the site isn't available in french (fr) I want to throw an exception or a 404.
Now, my problem is that this whole asynchronously thing is pretty cool and while I do think to understand the theory behind it, I'm rather baffled right now. I mean, I want this to be non-blocking (and asynchronously, which is the reason for me using Future and async ;))
In my controller I want to do something like:
def checkLanguage(language:String) = Action
{
val lang:Future[Language] = languageRepository.find(language)
lang.onComplete
{
case Success(s) = Ok("Yay")
case Failure(f) = 404("Oh no!")
}
}
Of course this can't work, but that's a schema of how I want to have things working. I want to wait or postpone the rendering of the site, until it's confirmed that the given language-code is valid or invalid.
I had a look at the Playframeworks async-documentation for 2.3.6 (https://www.playframework.com/documentation/2.3.6/ScalaAsync) but I couldn't really get this to work as intended.
Any input appreciated!
From your db query don't use .head instead use .headOption. that way ur return type will be a Future [Option [x]]
In ur controller u can do something like this
Lang.map { case Some (x) => Ok (x)
case None => 404 ( "not found ")
}
Try this,
Action.async {
val lang:Future[Option[Language]] = languageRepository.find(language)
lang.map {l => l.map{_ => Ok("Yay") }.getOrElse(NotFound("Oh no!"))
}
First of all, I am assuming that if there is a possibility that a language will not exist then languageRepository.find(language) should return an Option of Language. Change Future[Language] to Future[Result] and use Action.async instead of Action
Now for some explanation, Action takes a block whose result should be Result. However, what you get is Future[Option[Language]]. Play provides async method for Action which needs Future[Result] and it takes cares of completing the request.
So, you need to convert Future[Option[Language]] to Future[Result].
lang.map {l => l.map{_ => Ok("Yay") }.getOrElse(NotFound("Oh no!"))
We map over the lang, if the Option[Language] is not None then we convert it to Ok("yay") else we convert it to NotFound
Even, If you don't get Option[Language], the idea remains the same. Convert Future[Language] to Future[Result] and use Action.async instead of Action
I have an idea how my data access layer with Scala Slick should look like, but I'm not sure if it's really possible.
Let's assume I have a User table which has the usual fields like id, email, password, etc.
object Users extends Table[(String, String, Option[String], Boolean)]("User") {
def id = column[String]("id", O.PrimaryKey)
def email = column[String]("email")
def password = column[String]("password")
def active = column[Boolean]("active")
def * = id ~ email ~ password.? ~ active
}
And I wish to query them in different ways, currently the ugly way is to have a new database session, do the for comprehension and then do different if statements to achieve what I want.
e.g.
def getUser(email: String, password: String): Option[User] = {
database withSession { implicit session: Session =>
val queryUser = (for {
user <- Users
if user.email === email &&
user.password === password &&
user.active === true
} //yield and map to user class, etc...
}
def getUser(identifier: String): Option[User] = {
database withSession { implicit session: Session =>
val queryUser = (for {
user <- Users
if user.id === identifier &&
user.active === true
} //yield and map to user class, etc...
}
What I would prefer is to have a private method for the query and then public methods which define queries along the lines of
type UserQuery = User => Boolean
private def getUserByQuery(whereQuery: UserQuery): Option[User] = {
database withSession { implicit session: Session =>
val queryUser = (for {
user <- Users
somehow run whereQuery here to filter
} // yield and boring stuff
}
def getUserByEmailAndPassword(email, pass){ ... define by query and call getUserByQuery ...}
getUserById(id){….}
getUserByFoo{….}
That way, the query logic is encapsulated in the relevant public functions and the actual querying and mapping to the user object is in a reusable function that other people dont need to be concerned with.
The problem I have is trying to refactor the "where" bit's into functions that I can pass around. Trying to do things like select them in intellij and using the refactoring results in some pretty crazy typing going on.
Does anyone have any examples they could show of doing close to what I am trying to achieve?
1) wrapping queries in a def means the query statement is re-generated on every single request, and, since query params are not bound, no prepared statement is passed to the underlying DBMS.
2) you're not taking advantage of composition
Instead, if you define parameterized query vals that def query wrappers call, you can get the best of both worlds.
val uBase = for{
u <- Users
ur <- UserRoles if u.id is ur.UserID
} yield (u,ur)
// composition: generates prepared statement one time, on startup
val byRole = for{ roleGroup <- Parameters[String]
(u,ur) <- uBase
r <- Roles if(r.roleGroup is roleGroup) && (r.id is ur.roleID)
} yield u
def findByRole(roleGroup: RoleGroup): List[User] = {
db withSession { implicit ss:SS=>
byRole(roleGroup.toString).list
}
}
If you need one-off finders for a single property, use:
val byBar = Foo.createFinderBy(_.bar)
val byBaz = Foo.createFinderBy(_.baz)
Can't remember where, maybe on SO, or Slick user group, but I did see a very creative solution that allowed for multiple bound params, basically a createFinderBy on steroids. Not so useful to me though, as the solution was limited to a single mapper/table object.
At any rate composing for comprehensions seems to do what you're trying to do.
I have recently done something similar, one way to do this could be following, write a general select method which takes a predicate
def select(where: Users.type => Column[Boolean]): Option[User] = {
database withSession { implicit session: Session =>
val queryUser = (for {
user <- Users where(user)
} //yield and map to user class, etc...
}
and then write the method which passes the actual predicate as a higher order function
def getUserByEmail(email:String):Option[User]={
select((u: Users.type) => u.*._2 === email)
}
similarly
def getActiveUserByEmail(email:String):Option[User]={
select((u: Users.type) => u.*._2 === email && u.*._4 === true)
}