How to catch slick postgres exceptions for duplicate key value violations - scala

My table has a unique index on a pair of columns in my postgresql database.
I want to know how I can catch a duplicate key exception when I am inserting:
def save(user: User)(implicit session: Session): User = {
val newId = (users returning users.map(_id) += user
user.copy(id = newId)
}
My logs show this exception:
Execution exception[[PSQLException: ERROR: duplicate key value violates unique constraint "...."
I haven't really used exceptions much in scala either yet also.

Your save method should probably return something different than just User, to indicate the possibility of failure. If the only exception that will be thrown is by unique key, and you really only care about success or failure (and not the type of failure), one way to go would be to return Option[User].
You could use a simple try/catch block, mapping successful saves to Some[User] and PSQLException to None:
def save(user: User)(implicit session: Session): Option[User] = {
try {
val newId = (users returning users.map(_id) += user
Some(user.copy(id = newId))
} catch {
case PSQLException => None
}
}
Personally not the way I'd go, as try/catch isn't really idiomatic Scala, and your error type is discarded. The next option is to use scala.util.Try.
def save(user: User)(implicit session: Session): Try[User] = Try {
val newId = (users returning users.map(_id) += user
user.copy(id = newId)
}
The code here is simpler. If the body of Try is successful, then save will return Success[User], and if not it will return the exception wrapped in Failure. This will allow you to do many things with Try.
You could pattern match:
save(user) match {
case Success(user) => Ok(user)
case Failure(t: PSQLException) if(e.getSQLState == "23505") => InternalServerError("Some sort of unique key violation..")
case Failure(t: PSQLException) => InternalServerError("Some sort of psql error..")
case Failure(_) => InternalServerError("Something else happened.. it was bad..")
}
You could use it like Option:
save(user) map { user =>
Ok(user)
} getOrElse {
InternalServerError("Something terrible happened..")
}
You can compose many together at once, and stop on the first failure:
(for {
u1 <- save(user1)
u2 <- save(user2)
u3 <- save(user3)
} yield {
(u1, u2, u3)
}) match {
case Success((u1, u2, u3)) => Ok(...)
case Failure(...) => ...
}

In Slick 3.x you can use asTry.
I'm using MySQL, but the same code can be used for PostgreSQL, only the exceptions are different.
import scala.util.Try
import scala.util.Success
import scala.util.Failure
db.run(myAction.asTry).map {result =>
result match {
case Success(res) =>
println("success")
// ...
case Failure(e: MySQLIntegrityConstraintViolationException) => {
//code: 1062, status: 23000, e: Duplicate entry 'foo' for key 'name'
println(s"MySQLIntegrityConstraintViolationException, code: ${e.getErrorCode}, sql status: ${e.getSQLState}, message: ${e.getMessage}")
// ...
}
case Failure(e) => {
println(s"Exception in insertOrUpdateListItem, ${e.getMessage}")
// ...
}
}
}
Note: It's also possible to map the action (myAction.asTry.map ...) instead of the future returned by db.run.

Related

Propagate errors through a chain of Scala futures

Considering a sequence of futures each returning Either[Status, Resp].
How would you propagate error status codes through a for comprehension which is using Future and not Either?
The code bellow does not work, since the parsing exception is not caught by .recover of the last future
The use case is Scala Play ActionRefiners which returns Future[Either[Status, TRequest[A]]].
def parseId(id: String):Future[Int] = {
Future.successful(Integer.parseInt(id))
}
def getItem(id: Int)(implicit ec: ExecutionContext): Future[Either[Status, String]] =
Future(Some("dummy res from db " + id)).transformWith {
case Success(opt) => opt match {
case Some(item) => Future.successful(Right(item))
case _ => Future.successful(Left(NotFound))
}
case Failure(_) => Future.successful(Left(InternalServerError))
}
(for {
id <- parseId("bad request")
resp <- getItem(id)
} yield resp).recover {
case _:NumberFormatException => Left(BadRequest)
}
I could move the .recover to parseId, but this makes the for comprehension very ugly - having to treat the Either[Status, id] in the middle
def parseId(id: String):Future[Either[Status, Int]] = {
Future.successful(Right(Integer.parseInt(id))).recover {
case _:NumberFormatException => Left(BadRequest)
}
}
Your exception is not caught because you are not throwing it inside the Future: Future.successful is immediately satisfied with the result of the expression you give it, if it throws an exception, it is executed on the current thread.
Try removing the .successful: Future(id.toInt) will do what you want.
Also, I would recommend to get rid of all the Eithers: these are highly overrated/overused, especially in the context of Future (that already wrap their result into Try anyhow), and just make the code more complicated and less readable without offering much benefit.
case class FailureReason(status: Status)
extends Exception(status.toString)
def notFound() = throw FailureReason(NotFound)
def internalError() = throw FailureReason(InternalError)
def badRequest() = throw FailureReason(BadRequest)
def parseId(id: String):Future[Int] = Future(id.toInt)
def getItem(id: Int): Future[String] = Future(Some("dummy"))
.map { _.getOrElse(notFound) }
.recover { _ => internalError }
// this is the same as your for-comprehension, just looking less ugly imo :)
parseId("foo").flatMap(getItem).recover {
case _: NumberFormatException => badRequest()
}
// if you still want `Either` in the end for some reason:
.map(Right.apply[Status, String])
.recover {
case _: NumberFormatException => Left(BadRequest) // no need for the first recover above if you do this
case FailureReason(status) => Left(status)
}

Scala Playframework not all DB queries get executed

I'm sending via HTTP Post Request a Json to my Playframework backend.
In my backend I validate the Json to a Model. After that, I want to save the entries in my Model to my DB.
def parseJSON: Action[AnyContent] = Action.async {
request =>
Future {
request.body.asJson.map(_.validate[MyModel] match {
case JsSuccess(items, _) =>
itemsToDBController.saveItems(items)
Ok("Success")
case JsError(err) =>
println(err)
BadRequest("Json Parse Error")
}).getOrElse(BadRequest("Error"))
}
}
One Item consists out of several objects. To save all objects to my DB, I need to get some values. Therefore I'm using a for(..) yield(...):
def saveItems(items: MyModel) = {
items.SomeObject.map(obj => {
if (obj.value1.isDefined &&
obj.value2.isDefined ) {
val result = for (
value1Exists <- value1DTO.checkExists(obj.value1.name);
value1Entry <- getOrCreateValue1(value1Exists, obj);
value2Exists <- value2DTO.checkExists(obj.value2.name);
value2Entry <- getOrCreateValue2(value1Exists, obj)
) yield(value1Entry, value2Entry)
result.map({
case (value1Entry, value2Entry) => {
insertAllValue3(value1Entry, value2Entry)
Future.successful()
}
case _ => Future.failed(new Exception("Not all entries defined"))
})
}
else {
Future.successful("Not all objects defined - skipping")
}
})
}
My problem is, after all result.map({...}) have started, my parseJSON Action returns 200 - OK. But not all relevant items get stored to my DB. It seems like after the 200 - OK everything is stopped and it doesn't even throw an error.
I don't want to use Await.result or anything blocking in my Action.
Thanks in Advance
You are starting computations by calling itemsToDBController.saveItems(items) and then immediately return result with Ok("Success"). So exception may be thrown after request if completed.
To fix this issue you need to transform result of itemsToDBController.saveItems from List[Future[T]] to Future[List[T]] with help of Future.sequence. Then call map method on returned future. Call recover on this Future to find which error is thrown:
def parseJSON: Action[AnyContent] = Action.async { request =>
request.body.asJson
.map(_.validate[MyModel] match {
case JsSuccess(items, _) =>
Future
.sequence(itemsToDBController.saveItems(items))
.map(_ => Ok("Success"))
.recover {
case e: Exception => BadRequest(e.getMessage())
}
case JsError(err) =>
println(err)
Future.successful(BadRequest("Json Parse Error"))
})
.getOrElse(Future.successful(BadRequest("Error")))
}
Update
For running all inserts in one transaction you should combine DBIOAction instead of Future. For example you rewrite checkExists(name) as:
def checkExists(name: String): DBIO[Boolean] = {
Objects.filter(obj => obj.name === name).exists
}
getOrCreateValue(exists, obj) as:
def getOrCreateValue(exists: boolean, obj: Object): DBIO[Object] = {
if (exists) {
Objects.filter(o => o.name === name).result.head
} else {
(Objects returning Objects.map(_.id) into ((o, id) => o.copy(id = Some(id)))) += obj
}
}
Now you can run it in single transaction in the following way:
def saveItems(items: MyModel) = {
val insertActions = items.SomeObject.map(obj => {
if (obj.value1.isDefined && obj.value2.isDefined) {
val result = for {
value1Exists <- value1DTO.checkExists(obj.value1.name);
value1Entry <- getOrCreateValue1(value1Exists, obj);
value2Exists <- value2DTO.checkExists(obj.value2.name);
value2Entry <- getOrCreateValue2(value1Exists, obj)
} yield (value1Entry, value2Entry)
result.flatMap({
case (value1Entry, value2Entry) => {
insertAllValue3(value1Entry, value2Entry) // This also returns instance of `DBIOAction`
}
case _ =>
DBIO.failed(new Exception("Not all entries defined"))
})
} else {
DBIO.successful("Not all objects defined - skipping")
}
})
db.run(DBIO.sequence(inserActions).transactionally)
}
For mo info how to work with DBIO actions check this official docs

Using if-guard in for-comprehension to check for existence

My am making 3 database queries, each return a Future. I am trying to use for comprehension to resolve the Futures but it seems I am not using if correctly in for
Each query depends on result of previous one. I look for a token, if found, I look for user and it found, I update the user. Each database query returns a Future[Option]] and I thought I could considitionally perform the next query depending on whether the previous one returns Some or None. I am using isDefined for this. But when I ran the code for an invalid token, I got error [NoSuchElementException: None.get] for code userOption:Option[User]<-userRepo.findUser(tokenOption.get.loginInfo); if tokenOption.isDefined
def verifyUser(token:String) = Action.async {
implicit request => {
val result:Future[Result] = for{
//generator 1 - get token from database
tokenOption:Option[UserToken] <- userTokenRepo.find(UserTokenKey(UUID.fromString(token)))
//generator2. found token, look for corresponding user to which the token belongs
userOption:Option[User] <- userRepo.findUser(tokenOption.get.loginInfo); if tokenOption.isDefined
//generator 3. found user and token. Update profile
modifiedUser:Option[User] <- confirmSignupforUser(userOption.get); if userOption.isDefined
} yield
{ //check if we have user and token and modified user here. If any is missing, return error else success
if(tokenOption.isDefined && userOption.isDefined && modifiedUser.isDefined)
Redirect("http://localhost:9000/home"+";signup=success")//TODOM - pick from config
else
if(tokenOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(userOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(modifiedUser.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else //this shouldn't happen. Unexpected
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
}
result
}
}
TL;DR
Consider using OptionT
https://typelevel.org/cats/datatypes/optiont.html
Have a look at my toned down implementation:
from https://scastie.scala-lang.org/hsXXtRAFRrGpMO1Jl1Li7A
import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.Await.result
import scala.concurrent.duration._
import scala.language.postfixOps
type UserToken = String
type User = String
def fromToken(token: String): Future[Option[UserToken]] = Future.successful(None)
def findUser(userToken: UserToken): Future[Option[User]] = Future.successful(None)
def modify(user: User): Future[Option[User]] = Future.successful(None)
def verifyUser(token: String) = {
val result = for {
tokenOption: Option[UserToken] <- fromToken(token) //generator 1 - get token from database
userOption: Option[User] <- findUser(tokenOption.get);
if tokenOption.isDefined //generator2. found token, look for corresponding user to which the token belongs
modifiedUser: Option[User] <- modify(userOption.get);
if userOption.isDefined //generator 3. found user and token. Update profile
} yield { //check if we have user and token and modified user here. If any is missing, return error else success
if (tokenOption.isDefined && userOption.isDefined && modifiedUser.isDefined)
println("happy")
else
println("sad")
}
result
}
result(verifyUser("hello"), 1 second)
I used the following compile time flags, the last one is important:
scalacOptions ++= Seq(
"-deprecation",
"-encoding", "UTF-8",
"-feature",
"-unchecked",
"-Xprint:typer"
)
Let's focus on this line of the compile output:
(((tokenOption: Option[Playground.this.UserToken]) => Playground.this.findUser(tokenOption.get).
withFilter(((check$ifrefutable$2: Option[Playground.this.User]) => (check$ifrefutable$2: Option[Playground.this.User] #unchecked) match {
case (userOption # (_: Option[Playground.this.User])) => true
case _ => false
...
You can see that the tokenOption.get is invoked before the withFilter. These gets are the source of the exception you get
The, almost complete output of the compile is:
[[syntax trees at end of typer]] // main.scala
....
import scala.concurrent.Future;
import scala.concurrent.ExecutionContext.Implicits.global;
import scala.concurrent.Await.result;
import scala.concurrent.duration._;
import scala.language.postfixOps;
type UserToken = String;
type User = String;
def fromToken(token: String): scala.concurrent.Future[Option[Playground.this.UserToken]] = scala.concurrent.Future.successful[None.type](scala.None);
def findUser(userToken: Playground.this.UserToken): scala.concurrent.Future[Option[Playground.this.User]] = scala.concurrent.Future.successful[None.type](scala.None);
def modify(user: Playground.this.User): scala.concurrent.Future[Option[Playground.this.User]] = scala.concurrent.Future.successful[None.type](scala.None);
def verifyUser(token: String): scala.concurrent.Future[Unit] = {
val result: scala.concurrent.Future[Unit] = Playground.this.fromToken(token).withFilter(((check$ifrefutable$1: Option[Playground.this.UserToken]) => (check$ifrefutable$1: Option[Playground.this.UserToken] #unchecked) match {
case (tokenOption # (_: Option[Playground.this.UserToken])) => true
case _ => false
}))(scala.concurrent.ExecutionContext.Implicits.global).flatMap[Unit](((tokenOption: Option[Playground.this.UserToken]) => Playground.this.findUser(tokenOption.get).withFilter(((check$ifrefutable$2: Option[Playground.this.User]) => (check$ifrefutable$2: Option[Playground.this.User] #unchecked) match {
case (userOption # (_: Option[Playground.this.User])) => true
case _ => false
}))(scala.concurrent.ExecutionContext.Implicits.global).withFilter(((userOption: Option[Playground.this.User]) => tokenOption.isDefined))(scala.concurrent.ExecutionContext.Implicits.global).flatMap[Unit](((userOption: Option[Playground.this.User]) => Playground.this.modify(userOption.get).withFilter(((check$ifrefutable$3: Option[Playground.this.User]) => (check$ifrefutable$3: Option[Playground.this.User] #unchecked) match {
case (modifiedUser # (_: Option[Playground.this.User])) => true
case _ => false
}))(scala.concurrent.ExecutionContext.Implicits.global).withFilter(((modifiedUser: Option[Playground.this.User]) => userOption.isDefined))(scala.concurrent.ExecutionContext.Implicits.global).map[Unit](((modifiedUser: Option[Playground.this.User]) => if (tokenOption.isDefined.&&(userOption.isDefined).&&(modifiedUser.isDefined))
scala.Predef.println("happy")
else
scala.Predef.println("sad")))(scala.concurrent.ExecutionContext.Implicits.global)))(scala.concurrent.ExecutionContext.Implicits.global)))(scala.concurrent.ExecutionContext.Implicits.global);
result
};
scala.Predef.locally[Unit]({
val $t: Unit = scala.concurrent.Await.result[Unit](Playground.this.verifyUser("hello"), scala.concurrent.duration.`package`.DurationInt(1).second);
Playground.this.instrumentationMap$.update(com.olegych.scastie.api.Position.apply(1199, 1236), com.olegych.scastie.api.runtime.Runtime.render[Unit]($t)((ClassTag.Unit: scala.reflect.ClassTag[Unit])));
$t
})
};
object Main extends scala.AnyRef {
def <init>(): Main.type = {
Main.super.<init>();
()
};
private[this] val playground: Playground = new Playground();
<stable> <accessor> def playground: Playground = Main.this.playground;
def main(args: Array[String]): Unit = scala.Predef.println(com.olegych.scastie.api.runtime.Runtime.write(Main.this.playground.instrumentations$))
}
}
I am not sure why you are surprised you are getting and error for None.get with invalid token: if token is invalid, tokenOption is None, so, the next statement tokenOption.get will fail with exactly this error.
You want the "guard" executed before the statement you want to short circuit, not after it:
for {
foo <- bar if foo.isDefined
baz <- foo.get
} yield baz
But this would fail in the end anyway, because there would be nothing to yield (this trick works with Options or Lists etc., but Future.withFilter will end up failing if predicate is not satisfied, there is no other alternative).
The general rule to avoid this kind of errors is never use .get on an Option (or on a Try). Also, never use .head on a List, .apply on a Map, etc.
Here is one (almost) idiomatic way to write what you want:
case object Error extends RuntimeException("")
userTokenRepo
.find(UserTokenKey(UUID.fromString(token)))
.map { _.getOrElse(throw Error)
.flatMap { userRepo.find(_.loginInfo) }
.map { _.getOrElse(throw Error) }
.flatMap(confirmSignupForUser)
.map { _.getOrElse(throw Error) }
.map { _ => "success") }
.recover { case Error => "error" }
.map { result => Redirect(s"http://localhost:9000/home;signup=$result" }
Note, I said this was "almost" idiomatic, because throwing exceptions in scala is frowned upon. A purist would object to it, and suggest using something like a Try . or a biased Either instead, or to make use of a third party library, like cats or scalaz, that provide additional tools for working with Futures of Option (namely, OptionT).
But I would not recommend getting into that right now. You should get comfortable enough with basic "vanilla" scala before starting with that advanced stuff to avoid ending up with something completely incomprehensible.
You could also write this differently, in a completely idiomatic way (without using exceptions), with something like this:
userTokenRepo.find(UserTokenKey(UUID.fromString(token)))
.flatMap {
case Some(token) => userRepo.find(token.loginInfo)
case None => Future.successful(None)
}.flatMap {
case Some(user) => confirmSignupForUser(user)
case None => Future.successful(None)
}.map {
case Some(_) => "success"
case None => "error"
}.map { result =>
Redirect(s"http://localhost:9000/home;signup=$result"
}
This is more "pure", but a little more repetitive, so my personal preference is the first variant.
Finally, you could do away with my Error thingy, and just handle the NoSuchElement exception directly. This is going to be the shortest, but kinda icky even to my taste (what if some downstream code throws this exception because of a bug?):
userTokenRepo
.find(UserTokenKey(UUID.fromString(token)))
.flatMap { userRepo.find(_.get.loginInfo) }
.flatMap(confirmSignupForUser(_.get))
.map { _.get }
.map { _ => "success") }
.recover { case _: NoSuchElementException => "error" }
.map { result =>
Redirect(s"http://localhost:9000/home;signup=$result"
}
I really don't recommend the last version though, despite it being the shortest, and arguably, the most readable one (you can even rewrite it with a for-comprehension to look even nicer). Using Option.get is commonly considered "code smell", and is almost never a good thing to do.
Motivated by How to best handle Future.filter predicate is not satisfied type errors
I rewrote like the following. While the code works, I am curious to know if I am doing it the right way (functional!). Does it look fine?
def verifyUser(token:String) = Action.async {
implicit request => {
println("verifyUser action called with token: " + token) //TODOM - add proper handling and response
val result:Future[Result] = for{tokenOption:Option[UserToken] <- userTokenRepo.find(UserTokenKey(UUID.fromString(token))) //generator 1 - get token from database
userOption:Option[User] <- if (tokenOption.isDefined) userRepo.findUser(tokenOption.get.loginInfo) else Future.successful(None) //generator2. found token, look for corresponding user to which the token belongs
modifiedUser:Option[User] <- if (userOption.isDefined) confirmSignupforUser(userOption.get) else Future.successful(None) //generator 3. found user and token. Update profile
deletedToken:Option[UserTokenKey] <- if(modifiedUser.isDefined) userTokenRepo.remove(UserTokenKey(UUID.fromString(token))) else Future.successful(None)
}
yield { //check if we have user and token and modified user here. If any is missing, return error else success
println("db query results tokenOption: "+tokenOption+", userOption: "+userOption+" : modifiedUserOption: "+modifiedUser+", deletedToken: "+deletedToken)
if(tokenOption.isDefined && userOption.isDefined && modifiedUser.isDefined && deletedToken.isDefined)
Redirect("http://localhost:9000/home"+";signup=success")//TODOM - pick from config
else
if(tokenOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(userOption.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else if(modifiedUser.isEmpty)
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
else //this shouldn't happen. Unexpected
Redirect("http://localhost:9000/home"+";signup=error")//TODOM - pick from config
}
result
}
}

Catching SQL error on future failure in Slick

The code below uses Slick 3.1.x to read a row from a table, attempting to catch any SQL errors. UserDB is the Slick representation of the table, and User is the related object.
This code does not compile in the failure statement with the following error:
type mismatch; found : Unit required:
scala.concurrent.Future[Option[user.manage.User]]
How to fix this to catch the SQL error?
def read (sk: Int): Future[Option[User]] = {
val users = TableQuery[UserDB]
val action = users.filter(_.sk === sk).result
val future = db.run(action)
future.onSuccess {
case result =>
if (!result.isEmpty)
Some(result(0))
else
None
}
future.onFailure { // <-- compilation error
case e => println (e.getMessage)
None
}
}
You can use asTry method to catch exception ex into a successful result Failure(ex) and the successful value in Success(v). In your case the following should work.
db.run(action.asTry).map{
case Failure(ex) => {
Log.error(s"error : ${ex.getMessage}")
None
}
case Success(x) => x
}
As mentioned in the slick documentation, asTry is used to pipeline the exceptions for recovery handling logic.
You can use the onComplete method of the future.
future.onComplete{
case Success(r) ⇒ ...
case Failure(t) ⇒ log.error("failure in db query " + t.getMessage)
}

Handling errors in ReactiveMongo

I am working on a simple RESTful web service using Play Framework 2.1.5 and ReactiveMongo 0.9 using ReactiveMongo Play plugin. It has been a long time since I used Play Framework for the last time. I am trying to insert a document using:
def create = Action(parse.json) { request =>
Async {
val coll = db.collection[JSONCollection](...)
val obj = Json.obj(
"username" -> ...,
...
)
users.insert(obj).map { err => err match {
case e if !e.ok => InternalServerError(Json.obj("result" -> 0, "error" -> e.message))
case _ => Ok(Json.obj("result" -> 1))
}}
}
}
I have expected that once the query execution fails (e.g. due to the duplicate value in an index), I will handle it without any problem. But it is working differently - in case of failure a DatabaseException is thrown instead of satisfying the Promise[LastError] with an appropriate value. What am I missing please?
When an exception happens in a future any calls to map will be ignored and the exception will be passed along the chain of futures.
Explicitly handling the exceptions in a chain of Futures can be done with recover and recoverWith. You can read more about it in the overview of futures in the scala-lang docs:
http://docs.scala-lang.org/overviews/core/futures.html#exceptions
Try this code-
def insert(coll: BSONCollection, doc: BSONDocument): Future[Unit] = {
val p = Promise[Unit]
val f = coll.insert(doc)
f onComplete {
case Failure(e) => p failure (e)
case Success(lastError) => {
p success ({})
}
}
p.future
}
I hope this simplifies your need...
def create = Action (parse.json) { request =>
Async {
val coll = db.collection[JSONCollection](...)
val obj = Json.obj ("username" -> ...)
users.insert(obj).map {
case ins if ins.ok => OK (...)
case ins => InternalServerError (...)
} recover {
case dex: DatabaseException =>
log.error(..)
InternalServerEror(...)
case e: Throwable =>
log.error (..)
InternalServerError (...)
}
}
}