In previous versions of Slick, there was insert function defined on a TableQuery that returned an insertion result. I'm migrating to the new API and doing something like:
DBIO.seq(someTable += someValue)
but this has type
dbio.DBIOAction[Unit, NoStream, Write]
How can I get back the rows affected?
edit
The problem here seems to be that the types aren't lining up the way I would expect them to. Here's what Scala sees the types as:
val q: PostgresDriver.DriverAction[PostgresDriver.InsertActionExtensionMethods[(String, String)]#SingleInsertResult, NoStream, Write] = Admins.tableQuery += ((username, grantor))
val seq: dbio.DBIOAction[Unit, NoStream, Write] = DBIO.seq(q)
db.run(seq)
This doesn't make sense to me. If q has type DriverAction[PostgresDriver.InsertActionExtensionMethods[(String, String)]#SingleInsertResult... where SingleInsertResult is a type alias to Int then shouldn't my DBIO.seq return a DBIOAction[Int, NoStream, Write]?
Look at these examples
It's not very different from slick 2.x, but now everything returns Futures.
Maybe something along the lines of (off the top of my head, I can't test it myself now)
val users = TableQuer[Users]
// this is the query
val q = (users += User(None, "Stefan", "Zeiger"))
// now you run it
db.run(q).map{ affectedRows =>
case 1 => println("success")
case _ => println("oops")
}
Related
I want to log in the event that a record doesn't have an adjoining record. Is there a purely functional way to do this? One that separates the side effect from the data transformation?
Here's an example of what I need to do:
val records: Seq[Record] = Seq(record1, record2, ...)
val accountsMap: Map[Long, Account] = Map(record1.id -> account1, ...)
def withAccount(accountsMap: Map[Long, Account])(r: Record): (Record, Option[Account]) = {
(r, accountsMap.get(r.id))
}
def handleNoAccounts(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
tuple
}
def toRichAccount(tuple: (Record, Option[Account]) = {
val (r, a) = tuple
a.map(acct => RichAccount(r, acct))
}
records
.map(withAccount(accountsMap))
.map(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
So there are multiple issues with this approach that I think make it less than optimal.
The tuple return type is clumsy. I have to destructure the tuple in both of the latter two functions.
The logging function has to handle the logging and then return the tuple with no changes. It feels weird that this is passed to .map even though no transformation is taking place -- maybe there is a better way to get this side effect.
Is there a functional way to clean this up?
I could be wrong (I often am) but I think this does everything that's required.
records
.flatMap(r =>
accountsMap.get(r.id).fold{
logger.error(s"no account for ${r.id}")
Option.empty[RichAccount]
}{a => Some(RichAccount(r,a))})
If you're using scala 2.13 or newer you could use tapEach, which takes function A => Unit to apply side effect on every element of function and then passes collection unchanged:
//you no longer need to return tuple in side-effecting function
def handleNoAccounts(tuple: (Record, Option[Account]): Unit = {
val (r, a) = tuple
if (a.isEmpty) logger.error(s"no account for ${record.id}")
}
records
.map(withAccount(accountsMap))
.tapEach(handleNoAccounts) // if no account is found, log
.flatMap(toRichAccount)
In case you're using older Scala, you could provide extension method (updated according to Levi's Ramsey suggestion):
implicit class SeqOps[A](s: Seq[A]) {
def tapEach(f: A => Unit): Seq[A] = {
s.foreach(f)
s
}
}
I am starting to work with slick and scala and it seems that I still do not have the basics iron out:
I am using slick 3.0.0 and scala 2.11.7 and I am connecting to an Oracle database.
I want to get the number of rows in my table so I went and did a search and found the following:
1) This one tells me that .run does not exist:
mytable.length.run
2)This one tells me that there is a type mismatch found: slick.lifted.Rep[Int] and expected String:
var q = for(u <- mytable) yield u.id
print(q.size)
3)This one compiles and runs but prints Rep(Pure $#309962262):
var q = for{row <- mytable} yield row
println(Query(q.length))
So I am not sure if it is because I do not understand how this works but I was imagining that the following should happen:
A) constructQuery
b) "run" query.
So the other queries that I am using are as follow:
val db = Database.forConfig("Oracle")
try{
val f: Future[Unit] = {
val query: StreamingDBIO[Seq[String], String] = participants.map(_.id).result // A)"Construct query"
val stremQuery: DatabasePublisher[String] = db.stream(query) //B) "Run query"
streamQuery.foreach(println)
}
Await.result(f, Duration.Inf)
}
finally db.close
What am I missing? is number 3 not giving me what I want because is not under a db.stream/db.run/db.something command? Or am I just lost =)
Thanks in advance
Tona
Querying a database in slick basically consists of these three steps:
Create a Query
Convert Query to an Action
Execute the Action on a database
So your example would look something like this (types are optional and added for clarity):
val query = mytable.length // length is an aggregation
val action = query.result
val result: Future[Int] = db.run(action)
// Access result in a non blocking way (recommended):
result.map(count: Int => ...)
// or for completeness use Await (not recommended):
val count: Int = Await.result(result, Duration.Inf)
Further reading:
Slick documentation, Queries
Futures and Promises in scala
Is it possible to update variable column list, which number is know only in runtime by slick 3.0?
Below is example what I want to do (won't compile)
var q: Query[UserTable, UserTable#TableElementType, Seq] = userTable
var columns = List[Any]()
var values = List[Any]()
if (updateCommands.name.isDefined) {
columns = q.name :: columns
values = updateCommands.name.get :: values
}
if (updateCommands.surname.isDefined) {
columns = q.surname :: columns
values = updateCommands.surname.get :: values
}
q = q.filter(_.id === updateCommands.id).map(columns).update(values)
Here is what I've done in Slick 3.1. I wasn't sure what worse, editing plain SQL statement or multiple queries. So I decided to go with latter assuming Postgres optimizer would see same WHERE clause in update queries of single transaction. My update method looks like this
def updateUser(user: User, obj: UserUpdate): Future[Unit] = {
val actions = mutable.ArrayBuffer[DBIOAction[Int, NoStream, Write with Transactional]]()
val query = users.withFilter(_.id === user.id)
obj.name.foreach(v => actions += query.map(_.name).update(v))
obj.email.foreach(v => actions += query.map(_.email).update(Option(v)))
obj.password.foreach(v => actions += query.map(_.pwdHash).update(Option(encryptPassword(v))))
slickDb.run(DBIO.seq(actions.map(_.transactionally): _*))
}
In Slick 3.0 they adopted slightly different approach, instead of having updateAll methods, as far as I userstand path of combinators was adopted.
So main idea is to define some actions on the data and then combine them ont he database to make a single run.
Example:
// let's assume that you have some table classes defined somewhere
// then let's define some actions, they might be really different
val action: SqlAction = YourTable.filter(_id === idToAssert)
val anotherAction = AnotherTable.filter(_.pets === "fun")
// and then we can combine them on a db.run
val combinedAction = for {
someResult <- action
anotherResult <- anotherAction
} yeild (someResult,anotherResult)
db.run(combinedAction) // that returns actual Future of the result type
In the same way you can deal with lists and sequences, for that please take a look here: http://slick.typesafe.com/doc/3.1.0-M1/dbio.html
DBIO has some functions that allows you to combine list of actions to one action.
I hope that idea is clear, if you have questions you are wellcome to the comments.
to update a variable number of columns you may use this way as I used for slick 3:
def update(id: Long, schedule: Schedule, fieldNames: Seq[String]): Future[_] = {
val columns = schedules.baseTableRow.create_*.map(_.name).toSeq.filter(fieldNames.map(_.toUpperCase).contains)
val toBeStored = schedule.withDefaults
val actions = mutable.ArrayBuffer[DBIOAction[Int, NoStream, Write with Transactional]]()
val query = schedules.withFilter(_.id === id)
//this is becasue of limitations in slick, multiple columns are not possible to be updated!
columns.find("NAME".equalsIgnoreCase).foreach(x => actions += query.map(_.name).update(toBeStored.name))
columns.find("NAMESPACE".equalsIgnoreCase).foreach(x => actions += query.map(_.namespace).update(toBeStored.namespace))
columns.find("URL".equalsIgnoreCase).foreach(x => actions +=
db.run(DBIO.seq(actions: _ *).transactionally.withPinnedSession)
}
I have two functions which return Futures. I'm trying to feed a modified result from first function into the other using a for-yield comprehension.
This approach works:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get) if sid.isDefined
} yield s
However I'm not happy with having the "if" in there, it seems that I should be able to use a map instead.
But when I try with a map:
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
I get a compile error:
[error] found : Option[scala.concurrent.Future[Option[School]]]
[error] required: scala.concurrent.Future[Option[School]]
[error] s <- sid.map(schoolStore.getSchool(_))
I've played around with a few variations, but haven't found anything attractive that works. Can anyone suggest a nicer comprehension and/or explain what's wrong with my 2nd example?
Here is a minimal but complete runnable example with Scala 2.10:
import concurrent.{Future, Promise}
case class User(userId: Int)
case class UserDetails(userId: Int, schoolId: Option[Int])
case class School(schoolId: Int, name: String)
trait Error
class UserStore {
def getUserDetails(userId: Int): Future[Either[Error, UserDetails]] = Promise.successful(Right(UserDetails(1, Some(1)))).future
}
class SchoolStore {
def getSchool(schoolId: Int): Future[Option[School]] = Promise.successful(Option(School(1, "Big School"))).future
}
object Demo {
import concurrent.ExecutionContext.Implicits.global
val userStore = new UserStore
val schoolStore = new SchoolStore
val user = User(1)
val schoolFuture: Future[Option[School]] = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- sid.map(schoolStore.getSchool(_))
} yield s
}
(Edited to give a correct answer!)
The key here is that Future and Option don't compose inside for because there aren't the correct flatMap signatures. As a reminder, for desugars like so:
for ( x0 <- c0; w1 = d1; x1 <- c1 if p1; ... ; xN <- cN) yield f
c0.flatMap{ x0 =>
val w1 = d1
c1.filter(x1 => p1).flatMap{ x1 =>
... cN.map(xN => f) ...
}
}
(where any if statement throws a filter into the chain--I've given just one example--and the equals statements just set variables before the next part of the chain). Since you can only flatMap other Futures, every statement c0, c1, ... except the last had better produce a Future.
Now, getUserDetails and getSchool both produce Futures, but sid is an Option, so we can't put it on the right-hand side of a <-. Unfortunately, there's no clean out-of-the-box way to do this. If o is an option, we can
o.map(Future.successful).getOrElse(Future.failed(new Exception))
to turn an Option into an already-completed Future. So
for {
ud <- userStore.getUserDetails(user.userId) // RHS is a Future[Either[...]]
sid = ud.right.toOption.flatMap(_.schoolId) // RHS is an Option[Int]
fid <- sid.map(Future.successful).getOrElse(Future.failed(new Exception)) // RHS is Future[Int]
s <- schoolStore.getSchool(fid)
} yield s
will do the trick. Is that better than what you've got? Doubtful. But if you
implicit class OptionIsFuture[A](val option: Option[A]) extends AnyVal {
def future = option.map(Future.successful).getOrElse(Future.failed(new Exception))
}
then suddenly the for-comprehension looks reasonable again:
for {
ud <- userStore.getUserDetails(user.userId)
sid <- ud.right.toOption.flatMap(_.schoolId).future
s <- schoolStore.getSchool(sid)
} yield s
Is this the best way to write this code? Probably not; it relies upon converting a None into an exception simply because you don't know what else to do at that point. This is hard to work around because of the design decisions of Future; I'd suggest that your original code (which invokes a filter) is at least as good of a way to do it.
This answer to a similar question about Promise[Option[A]] might help. Just substitute Future for Promise.
I'm inferring the following types for getUserDetails and getSchool from your question:
getUserDetails: UserID => Future[Either[??, UserDetails]]
getSchool: SchoolID => Future[Option[School]]
Since you ignore the failure value from the Either, transforming it to an Option instead, you effectively have two values of type A => Future[Option[B]].
Once you've got a Monad instance for Future (there may be one in scalaz, or you could write your own as in the answer I linked), applying the OptionT transformer to your problem would look something like this:
for {
ud <- optionT(getUserDetails(user.userID) map (_.right.toOption))
sid <- optionT(Future.successful(ud.schoolID))
s <- optionT(getSchool(sid))
} yield s
Note that, to keep the types compatible, ud.schoolID is wrapped in an (already completed) Future.
The result of this for-comprehension would have type OptionT[Future, SchoolID]. You can extract a value of type Future[Option[SchoolID]] with the transformer's run method.
What behavior would you like to occur in the case that the Option[School] is None? Would you like the Future to fail? With what kind of exception? Would you like it to never complete? (That sounds like a bad idea).
Anyways, the if clause in a for-expression desugars to a call to the filter method. The contract on Future#filteris thus:
If the current future contains a value which satisfies the predicate,
the new future will also hold that value. Otherwise, the resulting
future will fail with a NoSuchElementException.
But wait:
scala> None.get
java.util.NoSuchElementException: None.get
As you can see, None.get returns the exact same thing.
Thus, getting rid of the if sid.isDefined should work, and this should return a reasonable result:
val schoolFuture = for {
ud <- userStore.getUserDetails(user.userId)
sid = ud.right.toOption.flatMap(_.schoolId)
s <- schoolStore.getSchool(sid.get)
} yield s
Keep in mind that the result of schoolFuture can be in instance of scala.util.Failure[NoSuchElementException]. But you haven't described what other behavior you'd like.
We've made small wrapper on Future[Option[T]] which acts like one monad (nobody even checked none of monad laws, but there is map, flatMap, foreach, filter and so on) - MaybeLater. It behaves much more than an async option.
There are a lot of smelly code there, but maybe it will be usefull at least as an example.
BTW: there are a lot of open questions(here for ex.)
It's easier to use https://github.com/qifun/stateless-future or https://github.com/scala/async to do A-Normal-Form transform.
How do you 'un-lift' a value inside a query in Slick when using lifted embedding? I was hoping a 'get', 'toLong' or something like that may do the trick, but no such luck.
The following code does not compile:
val userById = for {
uid <- Parameters[Long]
u <- Users if u.id === uid
} yield u
val userFirstNameById = for {
uid <- Parameters[Long]
u <- userById(uid)
---------------^
// type mismatch; found : scala.slick.lifted.Column[Long] required: Long
} yield u.name
You can't, for 2 reasons:
1) with val this is happening at compile time, there is no Long
value uid. userById(uid) binds a Long uid to the compile time
generated prepared statement, and then .list, .first, etc. invoke
the query.
2) the other issue is as soon as you Parameterize a query,
composition is no longer possible -- it's a limitation dating back to
ScalaQuery.
Your best bet is to delay Parameterization until the final composed query:
val forFooBars = for{
f <- Foos
b <- Bars if f.id is b.fooID
} yield(f,b)
val allByStatus = for{ id ~ active <- Parameters[(Long,Boolean)]
(f,b) <- forFooBars if (f.id is id) && (b.active is active)
} yield(f,b)
def findAllByActive(id: Long, isActive: Boolean) = allByStatus(id, isActive).list
At any rate, in your example you could just as well do:
val byID = Users.createFinderBy(_.id)
The only way that I know to get this kind of thing to work is wrap the query val in a def and pass in a runtime variable, which means Slick has to re-generate the sql on every request, and no prepared statement is sent to underlying DBMS. In some cases you have to do this, like passing in a List(1,2,3) for inList.
def whenNothingElseWorks(id: Long) = {
val userFirstNameById = for {u <- userById(id.bind)} yield u.name
}