How to apply manually evolutions in tests with Slick and Play! 2.4 - scala

I would like to manually run my evolution script at the beginning of each test file. I'm working with Play! 2.4 and Slick 3.
According to the documentation, the way to go seems to be:
Evolutions.applyEvolutions(database)
but I don't manage to get an instance of my database. In the documentation play.api.db.Databases is imported in order to get a database instance but if I try to import it, I get this error: object Databases is not a member of package play.api.db
How can I get an instance of my database in order to run the evolution script?
Edit: as asked in the comments, here is the entire source code giving the error:
import models._
import org.scalatest.concurrent.ScalaFutures._
import org.scalatest.time.{Seconds, Span}
import org.scalatestplus.play._
import play.api.db.evolutions.Evolutions
import play.api.db.Databases
class TestAddressModel extends PlaySpec with OneAppPerSuite {
lazy val appBuilder = new GuiceApplicationBuilder()
lazy val injector = appBuilder.injector()
lazy val dbConfProvider = injector.instanceOf[DatabaseConfigProvider]
def beforeAll() = {
//val database: Database = ???
//Evolutions.applyEvolutions(database)
}
"test" must {
"test" in { }
}
}

I finally found this solution. I inject with Guice:
lazy val appBuilder = new GuiceApplicationBuilder()
lazy val injector = appBuilder.injector()
lazy val databaseApi = injector.instanceOf[DBApi] //here is the important line
(You have to import play.api.db.DBApi.)
And in my tests, I simply do the following (actually I use an other database for my tests):
override def beforeAll() = {
Evolutions.applyEvolutions(databaseApi.database("default"))
}
override def afterAll() = {
Evolutions.cleanupEvolutions(databaseApi.database("default"))
}

Considering that you are using Play 2.4, where evolutions were moved into a separate module, you have to add evolutions to your project dependencies.
libraryDependencies += evolutions
Source: Evolutions
Relevant commit: Split play-jdbc into three different modules

To have access to play.api.db.Databases, you must add jdbc to your dependencies :
libraryDependencies += jdbc
Hope it helps some people passing here.
EDIT: the code would then look like this :
import play.api.db.Databases
val database = Databases(
driver = "com.mysql.jdbc.Driver",
url = "jdbc:mysql://localhost/test",
name = "mydatabase",
config = Map(
"user" -> "test",
"password" -> "secret"
)
)
You now have an instance of the DB, and can execute queries on it :
val statement = database.getConnection().createStatement()
val resultSet = statement.executeQuery("some_sql_query")
You can see more from the docs
EDIT: typo

I find the easiest way to run tests with evolutions applied is to use FakeApplication, and input the connection info for the DB manually.
def withDB[T](code: => T): T =
// Create application to run database evolutions
running(FakeApplication(additionalConfiguration = Map(
"db.default.driver" -> "<my-driver-class>",
"db.default.url" -> "<my-db-url>",
"db.default.user" -> "<my-db>",
"db.default.password" -> "<my-password>",
"evolutionplugin" -> "enabled"
))) {
// Start a db session
withSession(code)
}
Use it like this:
"test" in withDB { }
This allows you, for example, to use an in-memory database for speeding up your unit tests.
You can access the DB instance as play.api.db.DB if you need it. You'll also need to import play.api.Play.current.

Use FakeApplication to read your DB configuration and provide a DB instance.
def withDB[T](code: => T): T =
// Create application to run database evolutions
running(FakeApplication(additionalConfiguration = Map(
"evolutionplugin" -> "disabled"))) {
import play.api.Play.current
val database = play.api.db.DB
Evolutions.applyEvolutions(database)
withSession(code)
Evolutions.cleanupEvolutions(database)
}
Use it like this:
"test" in withDB { }

Related

How does slick profile work with respect to slick DB

I am having trouble to understand how to use slick profile.
My problem:
I am trying to use Slick with Akka-stream via the Alpakka JDBC plugin. The example given online is as follows:
#Load using SlickSession.forConfig("slick-h2")
slick-h2 {
profile = "slick.jdbc.H2Profile$"
db {
connectionPool = disabled
dataSourceClass = "slick.jdbc.DriverDataSource"
properties = {
driver = "org.h2.Driver"
url = "jdbc:h2:/tmp/alpakka-slick-h2-test"
}
}
}
import scala.concurrent.Future
import akka.Done
import akka.actor.ActorSystem
import akka.stream.ActorMaterializer
import akka.stream.scaladsl._
import akka.stream.alpakka.slick.scaladsl._
import slick.jdbc.GetResult
object SlickSourceWithPlainSQLQueryExample extends App {
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val ec = system.dispatcher
implicit val session = SlickSession.forConfig("slick-h2")
// The example domain
case class User(id: Int, name: String)
// We need this to automatically transform result rows
// into instances of the User class.
// Please import slick.jdbc.GetResult
// See also: "http://slick.lightbend.com/doc/3.2.1/sql.html#result-sets"
implicit val getUserResult = GetResult(r => User(r.nextInt, r.nextString))
// This import enables the use of the Slick sql"...",
// sqlu"...", and sqlt"..." String interpolators.
// See also: "http://slick.lightbend.com/doc/3.2.1/sql.html#string-interpolation"
import session.profile.api._
// Stream the results of a query
val done: Future[Done] =
Slick
.source(sql"SELECT ID, NAME FROM ALPAKKA_SLICK_SCALADSL_TEST_USERS".as[User])
.log("user")
.runWith(Sink.ignore)
done.onComplete {
case _ =>
session.close()
system.terminate()
}
}
The issue is that it works with
implicit val session = SlickSession.forConfig("slick-h2")
I try to use slick session as follows:
object Main extends App {
implicit val system = ActorSystem()
implicit val mat = ActorMaterializer()
implicit val ec = system.dispatcher
implicit val session = SlickSession.forConfig("pp")
}
Where my pp config is as such:
pp = {
url = "jdbc:oracle:thin:#52.4.90.244:1521:pp"
driver = oracle.jdbc.OracleDriver
keepAliveConnection = true
connectionPool = disabled
user = "xxxxx"
password = "xxxxx"
}
This code breaks at runtime.
Exception in thread "main" slick.SlickException: Configured profile oracle.jdbc.OracleDriver does not conform to requested profile slick.jdbc.JdbcProfile
at slick.basic.DatabaseConfig$.forConfig(DatabaseConfig.scala:99)
at akka.stream.alpakka.slick.javadsl.SlickSession$.forConfig(package.scala:47)
at akka.stream.alpakka.slick.javadsl.SlickSession$.forConfig(package.scala:44)
However in another code, where I do not use Akka-Stream and therefore do not use slickSession as such
object Main extends App {
val db = Database.forConfig("pp")
.....}
the code works perfectly.
I concluded that this has to do with Database.forconfig("pp") and SlickSession.forConfig("slick-h2") that require 2 different things.
There is an explanation about profile in Slick website but that is not very easy to understand and provides little instruction. It does not list the available profile and their syntax.
Hence my question is, what's the difference between the two forConfig.
How the profile works, where are they needed. Why the two configuration files for the database are not dealt with the same way.
Finally and formost, what is the profile for oracle. In slick 3.2.3 Slick is now free. I could not find the profile for it as in profile = "slick.jdbc.H2Profile$"
Can someone help clarify the configuration file difference, what is expected, what the profile does, and what is the profile for Oracle ?
The upgrade guide in the Slick documentation explains the distinction between drivers and profiles:
Slick’s driver concept has been renamed to profile to end the confusion over Slick drivers vs JDBC drivers....
As for the reason that Alpakka's SlickSession doesn't accept your configuration, take a look at the source code:
object SlickSession {
private final class SlickSessionImpl(val slick: DatabaseConfig[JdbcProfile]) extends SlickSession {
val db: JdbcBackend#Database = slick.db
val profile: JdbcProfile = slick.profile // <-- expects a profile
}
...
}
SlickSession specifically looks for a profile in the configuration, which is why defining a driver won't work.
To define a profile for Oracle, use OracleProfile.
As a follow up of #Jeffrey Chung answer. Here is the solution.
implicit val session = SlickSession.forConfig("pp")
import session.profile.api._
and the configuration file should be as such:
pp {
profile = "slick.jdbc.OracleProfile$"
db {
url = "..."
driver = oracle.jdbc.OracleDriver
keepAliveConnection = true
connectionPool = disabled
user = "...."
password = "...."
}
}

How to apply play-evolutions when running tests in play-framework?

I have problems with evolutions when running tests in play framework using
playframework v2.6.6 for scala
play-slick v3.0.2
play-slick-evolutions v3.0.2
The test looks like this:
class TestFooController extends PlaySpec with GuiceOneServerPerSuite {
"foo endpoint should store some data" in {
val wsClient = app.injector.instanceOf[WSClient]
val url = s"http://localhost:$port/foo"
val requestData = Json.obj("foo" -> "bar")
val response = await(wsClient.url(url).post(requestData))
response.status mustBe OK
}
}
The database configuration looks like this:
slick.dbs.default.driver="slick.driver.H2Driver$"
slick.dbs.default.db.driver="org.h2.Driver"
slick.dbs.default.db.url="jdbc:h2:mem:play"
Asume there is an evolution script which creates the table foos and this script is working fine in dev mode.
When running the test the following error is thrown:
play.api.http.HttpErrorHandlerExceptions$$anon$1: Execution exception[[JdbcSQLException: Table "foos" not found;
The table foos could not be found so I assume that database evolutions have not been applied.
Then I changed the database configuration to postgresql which is used in dev mode.
slick.dbs.default.driver = "slick.driver.PostgresDriver$"
slick.dbs.default.db.driver = "org.postgresql.Driver"
slick.dbs.default.db.url = "jdbc:postgresql://localhost:5432/foo-test"
slick.dbs.default.db.user = "user"
slick.dbs.default.db.password = "password"
With this configuration the test work fine and data are stored in the database, so database evolutions ran just fine.
Now the problem is, that the database is not cleaned up after tests. I'd like to run each test suite with a clean database.
To sum up. With H2Db evolutions are not applied, with postgresql evolutions are applied but not cleaned up.
Even if this explicitly defined in application.test.conf
play.evolutions.autoApply=true
play.evolutions.autoApplyDowns=true
I also tried
play.evolutions.db.default.autoApply=true
play.evolutions.db.default.autoApplyDowns=true
no effect.
Then I tried to do this manually via:
def withManagedDatabase[T](block: Database => T): Unit = {
val dbapi = app.injector.instanceOf[DBApi]
val database = dbapi.database("default")
Evolutions.applyEvolutions(database)
block(database)
Evolutions.cleanupEvolutions(database)
}
and then changing the test to:
"foo endpoint should store some data" in withManagedDatabase { _ =>
...
}
For the H2 database configuration it has no effect, the same error that table foos can not be found is thrown. For the postgresql database configuration an evolution exceptions is thrown
play.api.db.evolutions.InconsistentDatabase: Database 'default' is in an inconsistent state![An evolution has not been applied properly. Please check the problem and resolve it manually before marking it as resolved.]
I want evolution ups running before and evolution downs running after each test suite. How can this be achieved?
You can use the following to apply evolutions before each suite and clean up afterwards:
trait DatabaseSupport extends BeforeAndAfterAll {
this: Suite with ServerProvider =>
private lazy val db = app.injector.instanceOf[DBApi]
override protected def beforeAll(): Unit = {
super.beforeAll()
initializeEvolutions(db.database("default"))
}
override protected def afterAll(): Unit = {
cleanupEvolutions(db.database("default"))
super.afterAll()
}
private def initializeEvolutions(database: Database):Unit = {
Evolutions.cleanupEvolutions(database)
Evolutions.applyEvolutions(database)
}
private def cleanupEvolutions(database: Database):Unit = {
Evolutions.cleanupEvolutions(database)
}
}
This is working for me:
class DAOSpec extends PlaySpec with GuiceOneAppPerSuite {
val dbUrl = sys.env.getOrElse("DATABASE_URL", "postgres://foo:password#localhost:5432/foo")
val testConfig = Map("db.default.url" -> dbUrl)
implicit override def fakeApplication() = new GuiceApplicationBuilder().configure(testConfig).build()
lazy val database = app.injector.instanceOf[Database]
lazy val dao = app.injector.instanceOf[DAO]
"create" must {
"work" in Evolutions.withEvolutions(database) {
val foo = await(dao.create("foo"))
foo.id must not be null
}
}
}

How can Slick use different Database driver based on Application environment (e.g. test, prod, etc.)

Slick 3.0.0
play 2.6.2
I am experimenting with Slick and run into an interesting issue. I hope the solution is just trivial and I am thinking too much about this
I have implemented the following simple code.
case class Content(content:String)
class ContentTable(tag: Tag) extends Table[Content](tag, "content"){
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def content = column[String]("content")
override def * : ProvenShape[Content] = (content).mapTo[Content]
}
object ContentDb {
val db = Database.forConfig("databaseConfiguration")
lazy val contents = TableQuery[ContentTable]
def all: Seq[Content] = Await.result(db.run(contents.result), 2 seconds)
}
So for this code to work, it requires the following import among others of course.
import slick.jdbc.H2Profile.api._
or alternatively
import slick.jdbc.PostgresProfile.api._
Now, I think, and please correct me if I am wrong, that the database driver should be a configuration detail. That is, I'd choose to run H2 in memory database while developing. Run against test PostgreSQL instance when testing and then run against another instance when in production. I mean the whole idea of abstracting the driver is to have this flexibility... I think.
Now, I have done some research and found that I could do something like this:
trait DbComponent {
val driver: JdbcProfile
import driver.api._
val db: Database
}
trait H2DbComponent extends DbComponent {
val driver: JdbcProfile = slick.jdbc.H2Profile
import driver.api._
val db = Database.forConfig("databaseConfiguration")
}
trait Contents {
def all: Seq[Content]
}
object Contents {
def apply: Contents = new ContentsDb with H2DbComponent
}
trait ContentsDb extends Contents {
this: DbComponent =>
import driver.api._
class ContentTable(tag: Tag) extends Table[Content](tag, "content") {
def id = column[Long]("id", O.PrimaryKey, O.AutoInc)
def content = column[String]("content")
override def * : ProvenShape[Content] = content.mapTo[Content]
}
lazy val contents = TableQuery[ContentTable]
def all: Seq[Content] = Await.result(db.run(contents.result), 2 seconds)
}
Then I can use dependency injection to inject the right instance for each entity that I have. Not ideal, but possible. So, I start digging on how to have conditional dependency injection based on which environment is running in Play Framework.
I was expecting something similar to the following:
#Component(env=("prod","test")
class ProductionContentsDb extends ContentsDb with PostgresDbComponent
#Component(env="dev")
class ProductionContentsDb extends ContentsDb with H2DbComponent
But, no luck...
EDIT
Just after I have finished writing this and started reading it again, I am curious if we can just have the something similar to:
class DbComponent #Inject (driver: JdbcProfile) {
import driver.api._
val db = Database.forConfig("databaseConfiguration")
}
You can create separate configuration files for each environment. like
application.conf --local
application.prod.conf -- prod with contents below
include "aplication.conf"
###update slick configurations
Then while running application in different stages feed in with -Dconfig.resource=

Testing Play + Slick app

I've a simple CRUD app built with Scala Play 2.4.3 and Play-slick 1.1.0 (slick 3.1.0) that uses a MySQL database for persistent storage.
I was trying to create the tests for my app and I saw 2 main options:
mocking database access, that as far as I've seen, requires some code changes
make tests use an alternative database (probably, in memory H2).
What's the best approach (vantages and desavantages)?
I prefer the second approach, but I'm finding some difficulties in setting up the tests.
What do I need to do? First, I think that I need to do the tests run with a FakeApplication, right? Do I need any sbt dependency to be able to do that?
After that, how do I specify to use the H2 database?
I had the same struggle and I came up with a solution like this(using second approach):
Create a context for DAO to use:
trait BaseContext{
def dbName: String
val dbConfig = DatabaseConfigProvider.get[JdbcProfile](dbName)
val db = dbConfig.db
val profile = dbConfig.driver
val tables = new Tables { // this is generated by Schema Code Generator
override val profile: JdbcProfile = dbConfig.driver
}
}
#Singleton
class AppContext extends BaseContext{
def dbName = "mysql" // name in your conf right after "slick.dbs"
}
#Singleton
class TestingContext extends BaseContext{
def dbName = "h2"
}
Then create a module to bind the injection, and don't forget to enable it in conf using play.modules.enabled += "your.Module":
class ContextModule(environment: Environment, configuration: Configuration) extends AbstractModule {
override def configure(): Unit = {
if (configuration.getString("app.mode").contains("test")) {
bind(classOf[BaseContext])
.to(classOf[TestingContext])
} else {
bind(classOf[BaseContext])
.to(classOf[AppContext])
}
}
}
And inject it to every DAO you've created:
class SomeDAO #Inject()(context: BaseContext){
val dbConfig = context.dbConfig
val db = context.db
val tables = context.tables
import tables.profile.api._
def otherStuff....
// you can call db.run(...), tables.WhateverYourTableIs, tables.TableRowCaseClass, ...
}
And final step, your configuration file. In my case I used app.mode to mark the environment, and I use separate .conf for different environment. Of cause, in these conf you must have the correct DB configuration. Here's the sample:
app.mode = "test"
# Database configuration
slick.dbs = {
# for unit test
h2 {
driver = "slick.driver.H2Driver$"
db = {
url = "jdbc:h2:mem:test;MODE=MYSQL"
driver = "org.h2.Driver"
keepAliveConnection = true
}
}
}
I'm pretty sure my solution is not a elegant one, but it deliver the goods. :)
Any better solution is welcomed!
my solution was to add step(Play.start(fakeApp)) in the beginning of each spec, and step(Play.stop(fakeApp)) in the end of each spec.
Also:
def fakeApp: FakeApplication = {
FakeApplication(additionalConfiguration =
Map(
"slick.dbs.default.driver" -> "slick.driver.H2Driver$",
"slick.dbs.default.db.driver" -> "org.h2.Driver",
"slick.dbs.default.db.url" -> "jdbc:h2:mem:play"
))
}
This was needed because I'm using play-slick, which requires configurations like:
slick.dbs.default.driver = "slick.driver.MySQLDriver$"
slick.dbs.default.db.driver = "com.mysql.jdbc.Driver"
slick.dbs.default.db.url = "jdbc:mysql://localhost/database"
slick.dbs.default.db.user = "user"
slick.dbs.default.db.password = "password"
more info on the docs

Best Practise of Using Connection Pool in Slick 3.0.0 Together with Play Framework

I followed the documentation of Slick 3.0.0-RC1, using Typesafe Config as database connection configuration. Here is my conf:
database = {
driver = "org.postgresql.Driver"
url = "jdbc:postgresql://localhost:5432/postgre"
user = "postgre"
}
I established a file Locale.scala as:
package models
import slick.driver.PostgresDriver.api._
import scala.concurrent.Future
case class Locale(id: String, name: String)
class Locales(tag: Tag) extends Table[Locale](tag, "LOCALES") {
def id = column[String]("ID", O.PrimaryKey)
def name = column[String]("NAME")
def * = (id, name) <> (Locale.tupled, Locale.unapply)
}
object Locales {
private val locales = TableQuery[Locales]
val db = Database.forConfig("database")
def count: Future[Int] =
try db.run(locales.length.result)
finally db.close
}
Then I got confused that when and where the proper time is to create Database object using
val db = Database.forConfig("database")
If I create db like this, there will be as many Database objects as my models. So what is the best practice to get this work?
You can create an Object DBLocator and load it using lazy operator so that its loaded only on demand.
You can always invoke the method defined in DBLocator class to get an instance of Session.