Scala, Slick :too many clients already - scala

ive come across this problem :
Caused by: org.postgresql.util.PSQLException: FATAL: sorry, too many clients already
The app works, but i can only make like 3 or 4 requests and then im getting this error, so whats happening (im guessing) is im creating new connections per request and id like to keep one connection per app lifecycle, any idea how to modify the code to do so?
Tried injecting UsersDao to my controller instead of using it as an object but that changes nothing
Im really new to scala so any help is appreciated
Dao
import config.DatabaseConfig
import domain.{User, UsersTable}
import slick.jdbc.PostgresProfile.api._
import slick.sql.SqlAction
import scala.concurrent.Future
trait BaseDao extends DatabaseConfig {
val usersTable = TableQuery[UsersTable]
protected implicit def executeFromDb[A](action: SqlAction[A, NoStream, _ <: slick.dbio.Effect]): Future[A] = {
println(db)
db.run(action)
}
}
object UsersDao extends BaseDao {
def findAll: Future[Seq[User]] = usersTable.result
def create(user: User): Future[Long] = usersTable.returning(usersTable.map(_.id)) += user
def findByFirstName(firstName: String): Future[Seq[User]] = usersTable.filter(_.firstName === firstName).result
def findById(userId: Long): Future[User] = usersTable.filter(_.id === userId).result.head
def delete(userId: Long): Future[Int] = usersTable.filter(_.id === userId).delete
}
DatabaseConfig
trait DatabaseConfig extends Config {
val driver = slick.jdbc.PostgresProfile
import driver.api._
def db = Database.forConfig("dbname")
implicit val session: Session = db.createSession()
}
Controller
import domain.User
import javax.inject.{Inject, Singleton}
import play.api.libs.json.Json
import play.api.mvc._
import repository.UsersDao
import scala.concurrent.{ExecutionContext, Future}
#Singleton
class UserController #Inject() ()(cc: ControllerComponents, parsers: PlayBodyParsers)(implicit exec: ExecutionContext) extends AbstractController(cc) {
def addUser = Action.async(parse.json[User]) { req => {
UsersDao.create(req.body).map({ user =>
Ok(Json.toJson(user))
})
}
}

Related

Scala Generic Repository Class For Reactive Mongo Repository(alpakka) - Needed Class Found T

Im trying to create a Generic Class in Scala so I can create a repository for different collection without repeating myself.
The problem is that if I do it as a Generic Class(as in this example) I get a problem in this line:
val codecRegistry = fromRegistries(fromProviders(classOf[T]), DEFAULT_CODEC_REGISTRY)
Expected Class but Found [T]
But if I change T for any other class (lets say User) in all the code it works.
This is my class:
package persistence.repository.impl
import akka.stream.Materializer
import akka.stream.alpakka.mongodb.scaladsl.{MongoSink, MongoSource}
import akka.stream.scaladsl.{Sink, Source}
import akka.{Done, NotUsed}
import com.mongodb.reactivestreams.client.MongoClients
import constants.MongoConstants._
import org.bson.codecs.configuration.CodecRegistries.{fromProviders, fromRegistries}
import org.mongodb.scala.MongoClient.DEFAULT_CODEC_REGISTRY
import org.mongodb.scala.bson.codecs.Macros._
import org.mongodb.scala.model.Filters
import persistence.entity.{ProductItem}
import persistence.repository.Repository
import scala.concurrent.{ExecutionContext, Future}
class UserMongoDatabase[T](implicit materializer: Materializer,
executionContext: ExecutionContext)
extends Repository[T] {
val codecRegistry = fromRegistries(fromProviders(classOf[T]), DEFAULT_CODEC_REGISTRY)
val client = MongoClients.create(HOST)
val db = client.getDatabase(DATABASE)
val requestedCollection = db
.getCollection(USER_COLLECTION, classOf[T])
.withCodecRegistry(codecRegistry)
val source: Source[T, NotUsed] =
MongoSource(requestedCollection.find(classOf[T]))
val rows: Future[Seq[T]] = source.runWith(Sink.seq)
override def getAll: Future[Seq[T]] = rows
override def getById(id: AnyVal): Future[Option[T]] = rows.map {
list =>
list.filter {
user => user.asInstanceOf[ {def _id: AnyVal}]._id == id
}.headOption
}
override def getByEmail(email: String): Future[Option[T]] = rows.map {
list =>
list.filter {
user => user.asInstanceOf[ {def email: AnyVal}].email == email
}.headOption
}
override def save(obj: T): Future[T] = {
val source = Source.single(obj)
source.runWith(MongoSink.insertOne(requestedCollection)).map(_ => obj)
}
override def delete(id: AnyVal): Future[Done] = {
val source = Source.single(id).map(i => Filters.eq("_id", id))
source.runWith(MongoSink.deleteOne(requestedCollection))
}
}
This is my repository trait:
package persistence.repository
import akka.Done
import scala.concurrent.Future
trait Repository[T]{
def getAll: Future[Seq[T]]
def getById(id: AnyVal): Future[Option[T]]
def save(user: T): Future[T]
def delete(id: AnyVal): Future[Done]
def getByEmail(email:String): Future[Option[T]]
}
As said in the comments, this is the perfect example of usage of ClassTag in Scala. It allow to retain the actual class of a generic/parameterized class.
class DefaultMongoDatabase[T](implicit ..., ct: ClassTag[T])
extends Repository[T] {
val codecRegistry = fromRegistries(fromProviders(ev.runtimeClass), ...)
(You can move the classtag logic in the trait if you want.)

Is it possible to override an implicit object contained within a trait that is being extended?

I have a wrapper trait that extends BasicFormats from spray json https://github.com/spray/spray-json/blob/release/1.3.x/src/main/scala/spray/json/BasicFormats.scala
but I would like to override the behavior of implicit object BigDecimalJsonFormat extends JsonFormat[BigDecimal] to add rounding logic. Example being
import spray.json._
import scala.math.BigDecimal.RoundingMode
case class Foo(i: BigDecimal)
object Foo extends DefaultJsonProtocol {
implicit val roundedBigDecimalProtocol: JsonFormat[BigDecimal] =
new JsonFormat[BigDecimal] {
def write(f: BigDecimal) = JsNumber(f.setScale(2, RoundingMode.HALF_UP))
def read(json: JsValue): BigDecimal =
DefaultJsonProtocol.BigDecimalJsonFormat.read(json)
}
implicit val fooFormatter = jsonFormat1(this.apply)
}
scastie snippet:
https://scastie.scala-lang.org/9RNhajzGRDGMX5QsuAohVA
Great question, I wasn't even aware this may cause so many problems. The solution I am about to propose is probably not the cleanest, but it does the job... First of all, you don't have to extend DefaultJsonProtocol, you might also import it's members:
object Foo {
import DefaultJsonProtocol._
implicit val roundedBigDecimalProtocol: JsonFormat[BigDecimal] = ...
implicit val fooFormatter = jsonFormat1(this.apply)
}
This doesn't solve the problem but that way you can exclude some of the members so that they are not imported. Here's the syntax for that
import DefaultJsonProtocol.{BigDecimalJsonFormat => _, _}
It basically says: don't import BigDecimalJsonFormat but import the rest.
And the following code sums it up.
import spray.json._
import scala.math.BigDecimal.RoundingMode
case class Foo(i: BigDecimal)
object Foo {
import DefaultJsonProtocol.{BigDecimalJsonFormat => _, _}
implicit val roundedBigDecimalProtocol: JsonFormat[BigDecimal] =
new JsonFormat[BigDecimal] {
def write(f: BigDecimal) = JsNumber(f.setScale(2, RoundingMode.HALF_UP))
def read(json: JsValue): BigDecimal =
DefaultJsonProtocol.BigDecimalJsonFormat.read(json)
}
implicit val fooFormatter = jsonFormat1(this.apply)
}

Play Controllers functional test with PlaySpec

I'm using Play Scala 2.5, I would like to write functional tests for a controller class.
This is the signature of my controller class with a action I would like to test for example.
class MyController #Inject()(implicit context: ExecutionContext, val messagesApi: MessagesApi, val dao: Dao) extends Controller with I18nSupport {
def getMyData = Action.async { implicit request =>
dao.getMyData map { myData =>
Ok(views.html.render(myData))
}.recover {
// recover.
}
}
}
Here is my Dao class that is used in my controller's action :
import play.api.Configuration
class Dao #Inject()(val ws: WSClient, val configuration: Configuration) {
val myConfigData: List[MyConfigData] = { ... } // Get data from configuration
def getMyData(): Future[List[MyData]] = {
// use myConfigData and call web service (ws) to get some data
}
}
I would like to test my controller's action using different configuration in my Dao class.
I manage to do it and my test class is working but I would like some advices to improve the code.
package controllers
import play.core.server.Server
import org.scalatest._
import Inspectors._
import org.scalatestplus.play._
import play.api.routing.sird._
import play.api.libs.json._
import play.api.mvc._
import play.api.Play
import play.api.test._
import play.api.i18n.MessagesApi
import play.api.Configuration
import play.api.test.Helpers.{ OK, status, contentAsString }
import org.scalatest.concurrent.ScalaFutures
import scala.concurrent.Future
import scala.concurrent.Await
import scala.concurrent.duration._
import play.api.libs.json._
import play.api.mvc._
import scala.concurrent.ExecutionContext
import com.typesafe.config.ConfigFactory
import dao.{ Dao }
import akka.util.Timeout
class MyControllerFunctionalSpec extends PlaySpec with OneAppPerSuite with ScalaFutures {
implicit val messagesApi = app.injector.instanceOf[MessagesApi]
implicit val ec = app.injector.instanceOf[ExecutionContext]
implicit val timeout = Timeout(5 seconds)
trait OneConfig {
val config = ConfigFactory.parseString(""" my config """)
}
trait AnotherConfig {
val config = ConfigFactory.parseString(""" my other config """)
}
"MyController" should {
"get some data" in new OneConfig {
Server.withRouter() {
case GET(p"/url1/some/data") => Action {
Results.Ok(Json.arr(Json.obj("name" -> "data1")))
}
} { implicit port =>
WsTestClient.withClient { client =>
implicit val dao = new Dao(client, Configuration(config))
val myController = new MyController()
val result = myController.getMyData(FakeRequest())
status(result) must equal(OK)
//println(contentAsString(result))
}
}
}
}
}
It is working like this but I was guided by compilation error and by fixing them to get it to work.
I would like help or advices to do it the right way.
Any advice welcome.

Database autocreate -> could not find implicit value for parameter session

I am new to scala (long time java developer). I tried to understand implicit and I think I understand the basics however I do not understand why it does not find the implicit value session. I tried to describe my problem with as much information as possible.
I followed the following blog post: http://blog.websudos.com/2015/04/04/a-series-on-phantom-part-1-getting-started-with-phantom/
Everything went/compiled fine until I wanted to test it. Getting the following error:
Error:(15, 46) could not find implicit value for parameter session: com.datastax.driver.core.Session
Await.result(Database.autocreate().future(), 5.seconds)
^
Error:(20, 48) could not find implicit value for parameter session: com.datastax.driver.core.Session
Await.result(Database.autotruncate().future(), 5.seconds)
^
when I execute the following test class:
import org.joda.time.DateTime
import org.scalatest.{BeforeAndAfterAll, FlatSpec}
import scala.concurrent.Await
import scala.concurrent.duration._
class DatabaseTest extends FlatSpec with BeforeAndAfterAll{
override def beforeAll(): Unit = {
super.beforeAll()
Await.result(Database.autocreate().future(), 5.seconds)
}
override def afterAll(): Unit = {
super.afterAll()
Await.result(Database.autotruncate().future(), 5.seconds)
}
"A TwitterMessage" should "be stored in cassandra" in {
val twitterMessageBefore = TwitterMessage(1L, DateTime.now, "This is a message", "me", "none")
Database.twitterMessages.store(
twitterMessageBefore
)
val twitterMessageAfter:Option[TwitterMessage] = Await.result(Database.twitterMessages.getById(1L), 5.seconds)
assert(twitterMessageAfter.isDefined, "No entry was found regarding the id.")
assert(twitterMessageAfter.get equals twitterMessageBefore)
}
}
I also copied the other classes I wrote below:
TwitterMessages.scala
import com.websudos.phantom.dsl._
import scala.concurrent.Future
case class TwitterMessage (
id: Long,
timestamp: DateTime,
msg: String,
user: String,
category: String
)
sealed class TwitterMessages extends CassandraTable[ConcreteTwitterMessages, TwitterMessage]{
object id extends LongColumn(this) with PartitionKey[Long]
object timestamp extends DateTimeColumn(this)
object msg extends StringColumn(this)
object user extends StringColumn(this)
object category extends StringColumn(this)
def fromRow(row: Row): TwitterMessage = {
TwitterMessage(
id(row),
timestamp(row),
msg(row),
user(row),
category(row)
)
}
}
abstract class ConcreteTwitterMessages extends TwitterMessages with RootConnector{
def store(twitterMessage: TwitterMessage): Future[ResultSet] = {
insert.value(_.id, twitterMessage.id)
.value(_.timestamp, twitterMessage.timestamp)
.value(_.msg, twitterMessage.msg)
.value(_.user, twitterMessage.user)
.value(_.category, twitterMessage.category)
.consistencyLevel_=(ConsistencyLevel.ALL)
.future()
}
def getById(id: Long): Future[Option[TwitterMessage]] = {
select.where(_.id eqs id).one()
}
}
Database.scala
import com.websudos.phantom.connectors.{ContactPoint, KeySpaceDef}
object Defaults {
val connector = ContactPoint.local.keySpace("twitter")
}
class Database(val keyspace:KeySpaceDef) extends com.websudos.phantom.db.DatabaseImpl(keyspace){
object twitterMessages extends ConcreteTwitterMessages with keyspace.Connector
}
object Database extends Database(Defaults.connector)
To specifically address your problem, all you have to do is to mix in the connector into the the test suite. This one is on me as I forgot to update the blog post with this information.
class DatabaseTest extends FlatSpec with BeforeAndAfterAll
with Defaults.connector.Connector

Access to ExecutionContext

I have this trait
trait NonBlockingGoodness extends DataStore {
import akka.dispatch.{ Future, ExecutionContext }
import akka.util.duration._
import akka.util.Timeout
implicit val ec = ExecutionContext.fromExecutorService(yourExecutorServiceGoesHere)
implicit lazy val timeout = Timeout(5 seconds)
}
I would like to access the ExecutionContext in another trait like such
trait AsyncGoodness extends NonBlockingGoodness {
import akka.dispatch.Future
def doSomething = {
Future { "Future is the bomb." }
}
However, I am getting the error
Could not find implicit value for parameter executor: akka.dispatch.ExecutionContext
UPDATED:
I figured out how to get the ExecutionContext in scope
trait AsyncGoodness extends NonBlockingGoodness {
import akka.dispatch.ExecutionContext
import akka.dispatch.Future
def doSomething()(implicit executor: ExecutionContext) = {
Future { "Future is the bomb." }
}
However, I have a follow-up question. Since I may have more than 1 method in AsyncGoodness that uses ExecutionContext, is there a way to pass it in at the trait level instead of at each method like I did above.
I know you'd rather not have to import anything extra, but something like this should work for you.
trait NonBlockingGoodness {
import scala.concurrent.{ Future, ExecutionContext }
import scala.concurrent.util.duration._
import akka.util.Timeout
object Implicit {
implicit val ec = ExecutionContext.Implicits.global
implicit lazy val timeout = Timeout(5 seconds)
}
}
trait AsyncGoodness extends NonBlockingGoodness {
import scala.concurrent.Future
import Implicit._
def doSomething = {
Future { "Future is the bomb." }
}
}
As it turns out, all I need to do is explicitly specify ec return type for the compiler to use it. Here's the working code
trait NonBlockingGoodness extends DataStore {
import akka.dispatch.{ Future, ExecutionContext }
import akka.util.duration._
import akka.util.Timeout
implicit val ec: ExecutionContext = ExecutionContext.fromExecutorService(yourExecutorServiceGoesHere)
implicit lazy val timeout = Timeout(5 seconds)
}
trait AsyncGoodness extends NonBlockingGoodness {
import akka.dispatch.Future
def doSomething = {
Future { "Future is the bomb." }
}