I just try to write a simple spec like this:
"saves the record on create" in {
val request = FakeRequest(POST, "/countries").withJsonBody(Json.parse("""{ "country": {"title":"Germany", "abbreviation":"GER"} }"""))
val create = route(app, request).get
status(create) mustBe OK
contentType(create) mustBe Some("application/json")
contentAsString(create) must include("country")
}
But on execution it throws such an error:
java.util.concurrent.RejectedExecutionException: Task slick.backend.DatabaseComponent$DatabaseDef$$anon$2#f456097 rejected from java.util.concurrent.ThreadPoolExecutor#6265d40c[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 1]
It works good for get request test for index page, any ideas how to workaround this ?
The problem was OneAppPerTest since problems with DB connections: just replacing it to OneAppPerSuitesolves the problem
Related
val makeReport =
feed(randomNumberFeeder)
.exec( session => {
http("post_report")
.post("/api/path/reports")
.body(StringBody(JsonFactory.report(id = 1, number= session("number").asOption[String].get))).asJson
.check(jsonPath("$.reportId").saveAs("reportId"))
session
})
val scn = scenario("ReportCreation").exec(makeReport)
But When I run the Gatling tests the request is not being sent, and the whole HTTP block is ignored. Where am I going wrong?
So I am end up with the following exception.
Exception in thread "main" java.lang.UnsupportedOperationException: There were no requests sent during the simulation, reports won't be generated
at io.gatling.charts.report.ReportsGenerator.generateFor(ReportsGenerator.scala:50)
at io.gatling.app.RunResultProcessor.generateReports(RunResultProcessor.scala:65)
at io.gatling.app.RunResultProcessor.processRunResult(RunResultProcessor.scala:40)
at io.gatling.app.Gatling$.start(Gatling.scala:89)
at io.gatling.app.Gatling$.fromArgs(Gatling.scala:45)
at io.gatling.app.Gatling$.main(Gatling.scala:37)
at io.gatling.app.Gatling.main(Gatling.scala)
I fixed it myself.
val makeReport =
feed(randomNumberFeeder)
.exec(http("post_report")
.post("/api/path/reports")
.body(StringBody( session => JsonFactory.report(id = 1, number= session("number").as[String]))).asJson
.check(jsonPath("$.reportId").saveAs("reportId"))
})
I have two services: one that sends stream data and the second one receives it using akka-grpc for communication. When source data is provided Service one is called to process and send it to service two via grpc client. It's possible that multiple instances of server one runs at the same time when multiple source data are provided at the same time.In long running test of my application. I see below error in service one:
ERROR i.a.g.application.actors.DbActor - GraphStage [akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1#59d40805] terminated abruptly, caused by for example materializer or act
akka.stream.AbruptStageTerminationException: GraphStage [akka.grpc.internal.AkkaNettyGrpcClientGraphStage$$anon$1#59d40805] terminated abruptly, caused by for example materializer or actor system termination.
I have never shutdown actor systems but only kill actors after doing their job. Also I used proto3 and http2 for request binding. Here is a piece of my code in service one:
////////////////////server http binding /////////
val service: HttpRequest => Future[HttpResponse] =
ServiceOneServiceHandler(new ServiceOneServiceImpl(system))
val bound = Http().bindAndHandleAsync(
service,
interface = config.getString("akka.grpc.server.interface"),
port = config.getString("akka.grpc.server.default-http-port").toInt,
connectionContext = HttpConnectionContext(http2 = Always))
bound.foreach { binding =>
logger.info(s"gRPC server bound to: ${binding.localAddress}")
}
////////////////////client /////////
def send2Server[A](data: ListBuffer[A]): Future[ResponseDTO] = {
val reply = {
val thisClient = interface.initialize()
interface.call(client = thisClient, req = data.asInstanceOf[ListBuffer[StoreRequest]].toList)
}
reply
}
///////////////// grpc communication //////////
def send2GrpcServer[A](data: ListBuffer[A]): Unit = {
val reply = send2Server(data)
Await.ready(reply, Duration.Inf) onComplete {
case util.Success(response: ResponseDTO) =>
logger.info(s"got reply message: ${res.description}")
//////check response content and stop application if desired result not found in response
}
case util.Failure(exp) =>
//////stop application
throw exp.getCause
}
}
Error occurred exactly after waiting for service 2 response :
Await.ready(reply, Duration.Inf)
I can't catch the cause of error.
UPDATE
I found that some stream is missed such that service one sends an stream an indefinitely wait for the response and service two does not receive any thing to reply to service one but still don't know why stream is missed
I also updated akka grpc plugin but has no sense:
addSbtPlugin("com.lightbend.akka.grpc" % "sbt-akka-grpc" % "0.6.1")
addSbtPlugin("com.lightbend.sbt" % "sbt-javaagent" % "0.1.4")
Is there a way to handle sessions explicitly in Slick 3? I currently have some code that looks like
def findUserByEmail(email: String): Option[User] = {
val users = TableQuery[Users]
val action = users.filter(_.email === email).result.headOption
val result = db.run(action.transactionally)
Await.result(result, Duration.Inf)
}
It works fine the first few times I run it, but then I start running into issues where it looks like connections/sessions are being left open (see below). This code is running inside aws lambda functions and I'm thinking I need to handle sessions more explicitly. How would I do this in Slick 3?
"errorMessage": "Timeout after 5000ms of waiting for a connection.",
"errorType": "java.sql.SQLTimeoutException",
"stackTrace": [
"com.zaxxer.hikari.pool.BaseHikariPool.getConnection(BaseHikariPool.java:233)",
"com.zaxxer.hikari.pool.BaseHikariPool.getConnection(BaseHikariPool.java:183)",
"com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:93)",
"slick.jdbc.hikaricp.HikariCPJdbcDataSource.createConnection(HikariCPJdbcDataSource.scala:18)",
"slick.jdbc.JdbcBackend$BaseSession.<init>(JdbcBackend.scala:424)",
"slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:47)",
"slick.jdbc.JdbcBackend$DatabaseDef.createSession(JdbcBackend.scala:38)",
"slick.basic.BasicBackend$DatabaseDef.acquireSession(BasicBackend.scala:218)",
"slick.basic.BasicBackend$DatabaseDef.acquireSession$(BasicBackend.scala:217)",
"slick.jdbc.JdbcBackend$DatabaseDef.acquireSession(JdbcBackend.scala:38)",
"slick.basic.BasicBackend$DatabaseDef$$anon$2.run(BasicBackend.scala:239)",
"java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)",
"java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)",
"java.lang.Thread.run(Thread.java:745)"
],
"cause": {
"errorMessage": "FATAL: remaining connection slots are reserved for non-replication superuser connections",
"errorType": "org.postgresql.util.PSQLException",
"stackTrace": [
"org.postgresql.core.v3.QueryExecutorImpl.receiveErrorResponse(QueryExecutorImpl.java:2455)",
"org.postgresql.core.v3.QueryExecutorImpl.readStartupMessages(QueryExecutorImpl.java:2586)",
"org.postgresql.core.v3.QueryExecutorImpl.<init>(QueryExecutorImpl.java:113)",
"org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:222)",
"org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:52)",
"org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:216)",
"org.postgresql.Driver.makeConnection(Driver.java:404)",
"org.postgresql.Driver.connect(Driver.java:272)",
You could try to set query timeout. Like this:
db.run(action.transactionally.withStatementParameters(statementInit = st => st.setQueryTimeout(100)))
You can also set different properties on Hikari connection pool as below:
slick {
// https://github.com/slick/slick/blob/master/slick-hikaricp/src/main/scala/slick/jdbc/hikaricp/HikariCPJdbcDataSource.scala
dataSourceClass = "slick.jdbc.DriverDataSource"
user = ${database.user}
password = ${database.password}
url = ${database.url}
connectionPool = HikariCP
maxConnections = 50
numThreads = 10
queueSize = 5000
connectionInitSql = "SELECT 1;"
connectionTestQuery = "SELECT 1;"
registerMbeans = true
properties = {
driver = ${database.driver}
url = ${database.url}
}
}
i am losing messages in my tornado chat and i do not known how to detect when the message wasn't sent and to send the message again
there is any way to detect when the conexion get lost? and when the conexión restart send the message
this is my code
def get(self):
try:
json.dumps(MessageMixin.cache)
except KeyError:
raise tornado.web.HTTPError(404)
class MessageMixin(object):
waiters = {}
cache = {}
cache_size = 200
def wait_for_messages(self,cursor=None):
t = self.section_slug
waiters = self.waiters.setdefault(t, [])
result_future = Future()
waiters.append(result_future)
return result_future
def cancel_wait(self, future):
t = self.section_slug
waiters = self.waiters.setdefault(t, [])
waiters.remove(future)
# Set an empty result to unblock any coroutines waiting.
future.set_result([])
def new_messages(self, message):
t = self.section_slug
#cache = self.cache.setdefault(t, [])
#print t
#print self.waiters.setdefault(t, [])
waiters = self.waiters.setdefault(t, [])
for future in waiters:
try:
if message is not None:
future.set_result(message)
except Exception:
logging.error("Error in waiter callback", exc_info=True)
waiters = []
#self.cache.extend(message)
#if len(self.cache) > self.cache_size:
#self.cache = self.cache[-self.cache_size:]
class MessageNewHandler(MainHandler, MessageMixin):
def post(self, section_slug):
self.section_slug = section_slug
post = self.get_argument("html")
idThread = self.get_argument("idThread")
isOpPost = self.get_argument("isOpPost")
arg_not = self.get_argument("arg")
type_not = self.get_argument("type")
redirect_to = self.get_argument("next", None)
message= {"posts": [post],"idThread": idThread,"isOpPost": isOpPost,
"type": type_not,"arg_not": arg_not}
if redirect_to:
self.redirect(redirect_to)
else:
self.write(post)
self.new_messages(message)
class MessageUpdatesHandler(MainHandler, MessageMixin):
#gen.coroutine
def post(self, section_slug):
self.section_slug = section_slug
try:
self.future = self.wait_for_messages(cursor=self.get_argument("cursor", None))
data = yield self.future
if self.request.connection.stream.closed():
return
self.write(data)
except Exception:
raise tornado.web.HTTPError(404)
def on_connection_close(self):
self.cancel_wait(self.future)
class Application(tornado.web.Application):
def __init__(self):
handlers = [
(r"/api/1\.0/stream/(\w+)", MessageUpdatesHandler),
(r"/api/1\.0/streamp/(\w+)", MessageNewHandler)
]
tornado.web.Application.__init__(self, handlers)
def main():
tornado.options.parse_command_line()
app = Application()
port = int(os.environ.get("PORT", 5000))
app.listen(port)
tornado.ioloop.IOLoop.instance().start()
if __name__ == "__main__":
main()
In the original chatdemo, this is what the cursor parameter to wait_for_messages is for: the browser tells you the last message it got, so you can send it every message since then. You need to buffer messages and potentially re-send them in wait_for_messages. The code you've quoted here will only send messages to those clients that are connected at the time the message came in (and remember that in long-polling, sending a message puts the client out of the "waiting" state for the duration of the network round-trip, so even when things are working normally clients will constantly enter and leave the waiting state)
I have been working on this issue for quite a while now and I cannot find a solution...
A web app built with play framework 2.2.1 using h2 db (for dev) and a simple Model package.
I am trying to implement a REST JSON endpoint and the code works... but only once per server instance.
def createOtherModel() = Action(parse.json) {
request =>
request.body \ "name" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match name =(")).as("application/json")
case name: JsValue =>
request.body \ "value" match {
case _: JsUndefined => BadRequest(Json.obj("error" -> true,
"message" -> "Could not match value =(")).as("application/json")
case value: JsValue =>
// this breaks the secod time
val session = ThinkingSession.dummy
val json = Json.obj(
"content" -> value,
"thinkingSession" -> session.id,
)
)
Ok(Json.obj("content" -> json)).as("application/json")
}
} else {
BadRequest(Json.obj("error" -> true,
"message" -> "Name was not content =(")).as("application/json")
}
}
}
so basically I read the JSON, echo the "value" value, create a model obj and send it's id.
the ThinkingSession.dummy function does this:
def all(): List[ThinkingSession] = {
// Tried explicitly closing connection, no difference
//val conn = DB.getConnection()
//try {
// DB.withConnection { implicit conn =>
// SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
// }
//} finally {
// conn.close()
//}
DB.withConnection { implicit conn =>
SQL("select * from thinking_session").as(ThinkingSession.DBParser *)
}
}
def dummy: ThinkingSession = {
(all() head)
}
So this should do a SELECT * FROM thinking_session, create a model obj list from the result and return the first out of the list.
This works fine the first time after server start but the second time I get a
play.api.Application$$anon$1: Execution exception[[SQLException: Timed out waiting for a free available connection.]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10.jar:2.2.1]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2$$anonfun$applyOrElse$3.apply(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
at scala.Option.map(Option.scala:145) [scala-library.jar:na]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$2.applyOrElse(PlayDefaultUpstreamHandler.scala:261) [play_2.10.jar:2.2.1]
Caused by: java.sql.SQLException: Timed out waiting for a free available connection.
at com.jolbox.bonecp.DefaultConnectionStrategy.getConnectionInternal(DefaultConnectionStrategy.java:88) ~[bonecp.jar:na]
at com.jolbox.bonecp.AbstractConnectionStrategy.getConnection(AbstractConnectionStrategy.java:90) ~[bonecp.jar:na]
at com.jolbox.bonecp.BoneCP.getConnection(BoneCP.java:553) ~[bonecp.jar:na]
at com.jolbox.bonecp.BoneCPDataSource.getConnection(BoneCPDataSource.java:131) ~[bonecp.jar:na]
at play.api.db.DBApi$class.getConnection(DB.scala:67) ~[play-jdbc_2.10.jar:2.2.1]
at play.api.db.BoneCPApi.getConnection(DB.scala:276) ~[play-jdbc_2.10.jar:2.2.1]
My application.conf (db section)
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:file:database/[my_db]"
db.default.logStatements=true
db.default.idleConnectionTestPeriod=5 minutes
db.default.connectionTestStatement="SELECT 1"
db.default.maxConnectionAge=0
db.default.connectionTimeout=10000
Initially the only thing set in my config was the connection and the error occurred. I added all the other stuff while reading up on the issue on the web.
What is interesting is that when I use the h2 in memory db it works once after server start and after that it fails. when I use the h2 file system db it only works once, regardless of the server instances.
Can anyone give me some insight on this issue? Have found some stuff on bonecp problem and tried upgrading to 0.8.0-rc1 but nothing changed... I am at a loss =(
Try to set a maxConnectionAge and idle timeout
turns out the error was quite somewhere else... it was a good ol' stack overflow... have not seen one in a long time. I tried down-voting my question but it's not possible^^