Slick insert not working while trying to return inserted row - scala

My goal here is to retrieve the Board entity upon insert. If the entity exists then I just want to return the existing object (which coincides with the argument of the add method). Otherwise I'd like to return the new row inserted in the database.
I am using Play 2.7 with Slick 3.2 and MySQL 5.7.
The implementation is based on this answer which is more than insightful.
Also from Essential Slick
exec(messages returning messages +=
Message("Dave", "So... what do we do now?"))
DAO code
#Singleton
class SlickDao #Inject()(db: Database,implicit val playDefaultContext: ExecutionContext) extends MyDao {
override def add(board: Board): Future[Board] = {
val insert = Boards
.filter(b => b.id === board.id && ).exists.result.flatMap { exists =>
if (!exists) Boards returning Boards += board
else DBIO.successful(board) // no-op - return specified board
}.transactionally
db.run(insert)
}
EDIT: also tried replacing the += part with
Boards returning Boards.map(_.id) into { (b, boardId) => sb.copy(id = boardId) } += board
and this does not work either
The table definition is the following:
object Board {
val Boards: TableQuery[BoardTable] = TableQuery[BoardTable]
class BoardTable(tag: Tag) extends Table[BoardRow](tag, "BOARDS") {
// columns
def id = column[String]("ID", O.Length(128))
def x = column[String]("X")
def y = column[Option[Int]]("Y")
// foreign key definitions
.....
// primary key definitions
def pk = primaryKey("PK_BOARDS", (id,y))
// default projection
def * = (boardId, x, y).mapTo[BoardRow]
}
}
I would expect that there would e a new row in the table but although the exists query gets executed
select exists(select `ID`, `X`, `Y`
from `BOARDS`
where ((`ID` = '92f10c23-2087-409a-9c4f-eb2d4d6c841f'));
and the result is false there is no insert.
There is neither any logging in the database that any insert statements are received (I am referring to the general_log file)

So first of all the problem for the query execution was a mishandling of the futures that the DAO produced. I was assigning the insert statement to a future but this future was never submitted to an execution context. My bad even more so that I did not mention it in the description of the problem.
But when this was actually fixed I could see the actual error in the logs of my application. The stack trace was the following:
slick.SlickException: This DBMS allows only a single column to be returned from an INSERT, and that column must be an AutoInc column.
at slick.jdbc.JdbcStatementBuilderComponent$JdbcCompiledInsert.buildReturnColumns(JdbcStatementBuilderComponent.scala:67)
at slick.jdbc.JdbcActionComponent$ReturningInsertActionComposerImpl.x$17$lzycompute(JdbcActionComponent.scala:659)
at slick.jdbc.JdbcActionComponent$ReturningInsertActionComposerImpl.x$17(JdbcActionComponent.scala:659)
at slick.jdbc.JdbcActionComponent$ReturningInsertActionComposerImpl.keyColumns$lzycompute(JdbcActionComponent.scala:659)
at slick.jdbc.JdbcActionComponent$ReturningInsertActionComposerImpl.keyColumns(JdbcActionComponent.scala:659)
So this is a MySQL thing in its core. I had to redesign my schema in order to make this retrieval after insert possible. This redesign includes an introduction of a dedicated primary key (completely unrelated to the business logic) which is also an AutoInc column as the stack trace prescribes.
In the end the solution becomes too involved and instead decided to use the actual argument of the add method to return if the insert was actually successful. So the implementation of the add method ended up being something like this
override def add(board: Board): Future[Board] = {
db.run(Boards.insertOrUpdate(board).map(_ => board))
}
while there was some appropriate Future error handling in the controller which was invoking the underlying repo.
If you're lucky enough and not using MySQL with Slick I suppose you might have been able to do this without a dedicated AutoInc primary key. If not then I suppose this is a one way road.

Related

Sqlalchemy + Postgres: synthetic/artificial id mixin with sequence

I've found the mixin pattern to be really handy for staying DRY, but I am having trouble with sequences. Note, I'm using postgres.
We use alembic migrations, and I'd really like the --autogeneration to work with this sequence, though I understand this might not be possible right now. However, it looks like setting up the sequence without an ORM identifier, prevents the sequence from being dropped later if I wanted to perform a downgrade.
Through googling, I found some explanation on how to properly setup a sequence. Essentially: separate the id and its sequence.
Current Code looks a bit like this:
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declared_attr
class AutoIdMixin(object):
"""Generates an synthetic identifier primary key.
"""
# See: http://docs.sqlalchemy.org/en/latest/core/defaults.html#associating-a-sequence-as-the-server-side-default
#declared_attr
def id_seq(cls):
bases = cls.__bases__
Base = bases[0]
sequence_prefix = 'seq'
schema = cls._schema_name
sequence_id = '_'.join((sequence_prefix, schema, cls.__tablename__, 'id'))
sequence = sa.Sequence(sequence_id, 1, 1, metadata=Base.metadata)
return sequence
#declared_attr
def id(cls):
column_id = sa.Column(sa.types.Integer, cls.id_seq.next_value(), primary_key=True)
return column_id
With the code above, I end up with a non-helpful error:
AttributeError: Neither 'next_value' object nor 'Comparator' object has an attribute '_set_parent_with_dispatch'
In an RTM moment, it looks like I missed a keyword: server_default.
#declared_attr
def id(cls):
sequence = cls.id_seq
column_id = sa.Column(sa.types.Integer, server_default=sequence.next_value(), primary_key=True)
return column_id

Is calling a Future method parameter "by-name" necessary?

I have something like the following for caching JSON responses in Redis for a Scala application using http4s, Argonaut, and Slick, and want to confirm that it is working how I expect is. The idea is if a Redis key was not found, then call the given fallback method to get the data from the original source (MySQL) and cache it for future requests, otherwise skip MySQL:
/** Get from MySQL */
def getThingsFromDatabase(matchingId: Int): Future[Seq[Thing]] = {
println("getThingsFromDatabase")
val query = things.filter(_.fieldId === matchingId)
db.run(query.result)
}
/** Get from Redis, else MySQL via `fallback` */
def getThingsFromRedisOrDatabase(key: String,
fallback: Future[Seq[Thing]]):
Future[argonaut.Json] = {
val stored = redis.get(key)
stored match {
// Data exists, return from redis
case Some(s) => {
Parse.parse(s) match { // convert string to Json
case Right(r) => Future { r } // Json => Future[argonaut.Json]
case Left(l) => println(l) // error
}
}
// Data does not exist, get from database and store
case None() => {
val data = fallback.map(_.toList.asJson)
data map { redis.set(key, _) }
data // Future[argonaut.Json]
}
}
}
// GET /things/12
Ok(getThingsFromRedisOrDatabase("things:12", getThingsFromDatabase(12)))
This works, however the above code will always print "getThingsFromDatabase" regardless of whether or not there is data in Redis because getThingsFromDatabase(12) executes when called as a parameter. The original database does not seem to be hit (no errors if it is shutdown) with data in Redis, as intended. I think this is because the fallback Future is not being used in this scenario so it does not complete even though the method was executed.
If fallback: Future[Seq[Thing]] is changed to be call-by-name (i.e. fallback: => Future[Seq[Thing]]), "getThingsFromDatabase" is only printed the first time when the cache is empty, as expected, since fallback is called only in the None() condition and not executed as a parameter.
While the latter is the intended functionality, would there be a difference between the original and the call-by-name version if there wasn't a println in the getThingsFromDatabase method? Both appear to meet the need of not going to MySQL if Redis has the desired data, with neither actually completing the Future even though the former executes the method.
There would be significant difference. As written, the db.run() will be invoked, and the database will execute the query; the results may be discarded but typically the server will do all the work.
If things is a big, unindexed table or if this code is frequently invoked, then yes, you could see significant performance degradation from the unnecessary calls. This example is a poster-child for the usefulness of call-by-name.

Slick codegen not giving any output

I am trying to use the current version of slick and slick-codegen (3.2.0) with a sqlite database. When I try listing the table, I get the names properly. However, when I try to generate classes corresponding to the tables, I do not get any output.
This works:
object TableCodeGenerator extends App
{
val db = Database.forURL("jdbc:sqlite:/home/samik/db/mydb.db", driver = "org.sqlite.JDBC")
val tables = Await.result(db.run(MTable.getTables), 1 second).toList
tables.foreach(println)
}
I get the output below:
MTable(MQName(models),TABLE,null,None,None,None)
MTable(MQName(users),TABLE,null,None,None,None)
However, the following code, run directly in the same way, doesn't work:
object TableCodeGenerator extends App
{
val db = Database.forURL("jdbc:sqlite:/home/samik/db/mydb.db", driver = "org.sqlite.JDBC")
val dbio = SQLiteProfile.createModel(Some(MTable.getTables))
val model = db.run(dbio)
val codegenFuture: Future[SourceCodeGenerator] = model.map(model => new SourceCodeGenerator(model))
codegenFuture.onSuccess
{
case codegen => codegen.writeToFile(
"org.sqlite.JDBC",
"/tmp",
"my.package.dao",
"Tables",
"Tables.scala")
}
}
Meaning, the code runs successfully, but I do not see any output file. Is there anything I am missing?
The above was happening because the underlying code was silently throwing an exception. The reason for this exception was that I was using a "feature" of sqlite where if you don't mention the datatype in schema, sqlite assumes it to be text type. However that creates a problem for the slick code.
More details here. The immediate solution was to fix the schema, but I think this has now been fixed in slick as well.

Composing `Future` result in Play Framework with Scala

I am trying to write a Play Framework asynchronous Action for the following URL:
POST /users/:userId/items
My database calls all return Future[...], where ... is Option[A] for find methods and Option[Id] for create methods.
I would like to check for the existence of the userId before trying to create the new item. I have a method Users.findById(userId) that returns a Future[Option[User]]. The result is Some(User) if the user exists and None if not. Items.create() also returns a Future[Option[itemId]].
I am trying to compose something using for:
for {
user <- Users.findById(userId)
if user.isDefined
} yield {
Items.create(...) map { itemId => Ok(itemId) } getOrElse NotFound
}
I would like to return Ok(itemId) if the item is successfully created. I'm not sure how to handle the error case. I would like to return NotFound if either the userId is invalid or the item cannot be created (maybe a field conflicts with a unique value already in the database).
I'm not sure what to put after the for structure. I tried getOrElse, but that does not compile, since Future does not have a getOrElse method.
Ideally, I can handle URLs containing several ids to check, e.g.:
PUT /users/:userId/foo/:fooId/bar/:barId
and confirm that userId, fooId, and barId are all valid before doing the update. All of those calls (Users.findById, Foo.findById, and Bar.findById) will return Future[Option[A]].
It's that double-nesting (Future of Option) that seems to get people every time. Things become a lot easier if you can flatten stuff out first.
In this case, Future already has a way of representing an error condition, it can wrap an Exception as well as a success value, that's something you can use...
// making this a Singleton avoids the cost of building a stack trace,
// which only happens when an Exception is constructed (not when it's thrown)
object NotFoundException extends RuntimeException("Empty Option")
// The map operation will trap any thrown exception and fail the Future
def squish[T](x: Future[Option[T]]) =
x map { _.getOrElse(throw NotFoundException) }
It's now a lot easier to use those squished results in a comprehension:
val result = for {
user <- squish(Users findById userId)
itemId <- squish(Items.create(user, ...))
} yield {
Ok(itemId)
} recover {
case NotFoundException => NotFound
}
Which will, of course, evaluate to a Future. This is async programming, after all :)
Any exceptions other than NotFoundException will still be exposed.

How to generate an unique ID for an class instance in Scala?

I have a class that needs to write to a file to interface with some legacy C++ application.
Since it will be instantiated several times in a concurrent manner,
it is a good idea to give the file an unique name.
I could use System.currentTimemili or hashcode, but there exists the possibility of collisions.
Another solution is to put a var field inside a companion object.
As an example, the code below shows one such class with the last solution, but I am not sure it is the best way to do it (at least it seems thread-safe):
case class Test(id:Int, data: Seq[Double]) {
//several methods writing files...
}
object Test {
var counter = 0
def new_Test(data: Seq[Double]) = {
counter += 1
new Test(counter, data)
}
}
Did you try this :
def uuid = java.util.UUID.randomUUID.toString
See UUID javadoc, and also How unique is UUID? for a discussion of uniqueness guarantee.
it is a good idea to give the file an unique name
Since all you want is a file, not id, the best solution is to create a file with unique name, not a class with unique id.
You could use File.createTempFile:
val uniqFile = File.createTempFile("myFile", ".txt", "/home/user/my_dir")
Vladimir Matveev mentioned that there is a better solution in Java 7 and later - Paths.createTempFile:
val uniqPath = Paths.createTempFile(Paths.get("/home/user/my_dir"), "myFile", ".txt"),