Scala Jetty webapp on Heroku 404 - scala

I'm testing around with a Scala web framework (Udash) and trying to run a toy-example in Heroku. I have it running without issues in local following the instructions in the Heroku docs:
sbt compile stage
heroku local web
However, once deployed, any URL I type goes to 404, even the landing page of the app. These are the objects I am using:
object Launcher extends CrossLogging {
def main(args: Array[String]): Unit = {
val port = Properties.envOrElse("PORT", "5000").toInt
val server = new ApplicationServer(port, "frontend/target/UdashStatics/WebContent")
server.start()
logger.info(s"Application started...")
}
}
class ApplicationServer(val port: Int, resourceBase: String) {
private val server = new Server(port)
private val contextHandler = new ServletContextHandler
private val appHolder = createAppHolder()
contextHandler.setSessionHandler(new SessionHandler)
contextHandler.setGzipHandler(new GzipHandler)
contextHandler.getSessionHandler.addEventListener(new org.atmosphere.cpr.SessionSupport())
contextHandler.addServlet(appHolder, "/*")
server.setHandler(contextHandler)
def start(): Unit = server.start()
def stop(): Unit = server.stop()
private def createAppHolder() = {
val appHolder = new ServletHolder(new DefaultServlet)
appHolder.setAsyncSupported(true)
appHolder.setInitParameter("resourceBase", resourceBase)
appHolder
}
}
Is there any Heroku configuration/characteristic that I am missing?
EDIT
Tried to apply the changes suggested and ended up with the following ApplicationContext:
class ApplicationServer(val port: Int, val resourceBase: String) {
val server = new Server()
val connector = new ServerConnector(server)
connector.setPort(port)
server.addConnector(connector)
private val appHolder = createAppHolder()
val context = new ServletContextHandler(ServletContextHandler.SESSIONS)
context.setBaseResource(Resource.newResource(resourceBase))
context.setContextPath("/")
context.addServlet(appHolder, "/")
server.setHandler(context)
private def createAppHolder() = {
val appHolder = new ServletHolder("default", classOf[DefaultServlet])
appHolder.setInitParameter("dirAllowed", "true")
appHolder.setInitParameter("resourceBase", resourceBase)
appHolder
}
def start(): Unit = server.start()
def stop(): Unit = server.stop()
}
However, I still get Error 404 even on landing page after deploying to Heroku:
HTTP ERROR 404
Problem accessing /. Reason:
Not Found
When running the app on local I get to the landing page correctly.
Thank you!
Thanks!

A few things to adjust that might help you.
resourceBase as a init-parameter on DefaultServlet is for alternate static file serving.
Use ServletContextHandler.setBaseResource(Resource) instead.
Use Resource.newResource(String) to create a new Resource reference. This should be an absolute path on the filesystem, or an absolute URI reference. no relative paths or URI fragments.
The DefaultServlet must be on the url-pattern of "/", not "/*" (this is a servlet spec requirement)
The DefaultServlet must be named, and must have the name "default" (this is a servlet spec requirement, see link on point 1 for example)
set the ServletContextHandler.setContextPath("/") to be explicit about what base context-path you want to use.
Some observations:
Your example code will only serve static content out of the resourceBase.
Since you have no welcomeFiles configured it would serve <resourceBase>/index.html by default (if you don't specify a specific static resource you want to access)
You have a SessionListener setup (org.atmosphere.cpr.SessionSupport), but since there's nothing that would access a Session, that code is pretty much a no-op.
There's no dynamic results from a custom Servlet or Filter present in your example codebase.

Related

Akka gRPC/HTTP Interop giving 404

I have an application running a gRPC service alongside a simple Akka HTTP endpoint. I am following this guide: https://doc.akka.io/docs/akka-grpc/current/server/akka-http.html. The problem: when curling the HTTP endpoint I get 404 not found. I know it found the server because Akka-HTTP/10.2.5 is the server of the response header.
Some code:
object Server extends App {
val conf = ConfigFactory
.parseString("akka.http.server.preview.enable-http2 = on")
.withFallback(ConfigFactory.defaultApplication())
val system = ActorSystem("Interop", conf)
new Server(system).run()
}
class Server(system: ActorSystem) {
def run() = {
// implicit sys, ec...
val grpcService: HttpRequest => Future[HttpResponse] = ServiceHandler(new Service())
val greeter = get {
pathEndOrSingleSlash {
complete("I am alive")
}
}
// lifted this line straight out of the guide
val grpcRoute = { ctx => grpcService(ctx.request).map(RouteResult.Complete) }
val route = concat(greeter, grpcRoute)
val binding = Http().newServerAt("127.0.0.1", 8080).bind(route)
binding
}
}
When I take out the gRPC route, the greeter endpoint works as intended. Otherwise, when I curl http://localhost:8080, I get
*Mark bundle as not supporting multiuser
<HTTP/1.1 404 Not Found
<Server: akka-http/10.2.5
<other-stuff
I am using akka-gRPC 2.0.0
What should I do to ensure interop between the two routes?
gRPC uses HTTP/2 so try
curl --http2 http://localhost:8080
It should also get rid of the initial "Mark bundle..." message.

http4s - how create blaze client with limited count of threads?

I trying to create blaze client with limited number of threads like this:
object ReactiveCats extends IOApp {
private val PORT = 8083
private val DELAY_SERVICE_URL = "http://localhost:8080"
// trying create client with limited number of threads
val clientPool: ExecutorService = Executors.newFixedThreadPool(64)
val clientExecutor: ExecutionContextExecutor = ExecutionContext.fromExecutor(clientPool)
private val httpClient = BlazeClientBuilder[IO](clientExecutor).resource
private val httpApp = HttpRoutes.of[IO] {
case GET -> Root / delayMillis =>
httpClient.use { client =>
client
.expect[String](s"$DELAY_SERVICE_URL/$delayMillis")
.flatMap(response => Ok(s"ReactiveCats: $response"))
}
}.orNotFound
// trying to create server on fixed thread pool
val serverPool: ExecutorService = Executors.newFixedThreadPool(64)
val serverExecutor: ExecutionContextExecutor = ExecutionContext.fromExecutor(serverPool)
// start server
override def run(args: List[String]): IO[ExitCode] =
BlazeServerBuilder[IO](serverExecutor)
.bindHttp(port = PORT, host = "localhost")
.withHttpApp(httpApp)
.serve
.compile
.drain
.as(ExitCode.Success)
}
full code and load-tests
 
But load-test results looks like one thread by one request:
How I make restrict numbers of threads for my blaze client?
There are two obvious things that are wrong with your code:
you're creating an Executor without shutting it down when you're done.
you're using the use method of the httpClient Resource inside the HTTP route, meaning that every time the route is called, it will create, use and destroy the http client. You should instead create it once during startup.
Executors, like any other resource (e. g. file handles etc.) should always be allocated using Resource.make like so:
val clientPool: Resource[IO, ExecutorService] = Resource.make(IO(Executors.newFixedThreadPool(64)))(ex => IO(ex.shutdown()))
val clientExecutor: Resource[IO, ExecutionContextExecutor] = clientPool.map(ExecutionContext.fromExecutor)
private val httpClient = clientExecutor.flatMap(ex => BlazeClientBuilder[IO](ex).resource)
The second problem can easily be fixed by allocating the httpClient before building the HTTP app:
private def httpApp(client: Client[IO]): Kleisli[IO, Request[IO], Response[IO]] = HttpRoutes.of[IO] {
case GET -> Root / delayMillis =>
client
.expect[String](s"$DELAY_SERVICE_URL/$delayMillis")
.flatMap(response => Ok(s"ReactiveCats: $response"))
}.orNotFound
…
override def run(args: List[String]): IO[ExitCode] =
httpClient.use { client =>
BlazeServerBuilder[IO](serverExecutor)
.bindHttp(port = PORT, host = "localhost")
.withHttpApp(httpApp(client))
.serve
.compile
.drain
.as(ExitCode.Success)
}
Another potential problem is that you're using IOApp, and it comes with its own thread pool. The best way to fix that is probably to mix in the IOApp.WithContext trait and implement this method:
override protected def executionContextResource: Resource[SyncIO, ExecutionContext] = ???
Copy from my commment.
Answer for performance issue is properly setup for Blaze client - for me this is .withMaxWaitQueueLimit(1024) parameter.

Slick Connection Pool on per requests

How can use slick connection pool ?
For example :
with this config :
database {
dataSourceClass = org.postgresql.ds.PGSimpleDataSource
driver = org.postgresql.Driver
properties = {
url = "jdbc:postgresql://172.17.0.2/sampleDB"
user = "user"
password = "userpass"
}
minConnections = 10
maxConnections = 20
numThreads = 10
}
I have only one client and this client with web browser request to get all persons from API .
now slick generate 10 connection to database .
second step client refresh browser and slick generate new 10 connection to database without using previous connections .
and then new refresh in browser and slick generate another 10 connection to database . (Now I have about 30 connection on DB with only one client!)
Why ? This is normal ?
Why maxConnections not work ?
I must close connection after requests ?
Or forget some configuration about that ?
Update
This is my sample API :
trait PersonsApi extends DatabaseConfig with JsonMapper {
val getAllPersons = (path("persons") & get) {
complete(db.run(PersonDao.findAll))
}
val getPersonById = (path("persons" / IntNumber) & get) {
num => complete(db.run(PersonDao.findById(num)))
}
val personsApi =
getAllPersons ~
getPersonById
}
This is my example entity class (DAO Pattern) :
class PersonTable(tag: Tag) extends Table[Person](tag, "persons") {
def id = column[Long]("id", O.AutoInc, O.PrimaryKey)
def name = column[String]("name")
def family = column[String]("family")
override def * : ProvenShape[Person] = (id.?, name, family) <> (Person.tupled, Person.unapply)
}
object PersonDao extends BaseDao {
def findAll = personTable.result
def findById(id: Long) = personTable.filter(_.id === id).result
}
This DatabaseConfig interface :
trait DatabaseConfig extends Config {
val driver = slick.driver.PostgresDriver
import driver.api._
def db = Database.forConfig("database")
}
Note : I don't use play framework .
Your configuration seems to be fine. It's impossible to say without further code samples from your application but my bet is you are creating your db on each and every request to your application.
Just make sure that this code:
Database.forConfig("database")
is executed once perhaps by:
putting it as a Singleton injected dependency or
by using play-slick and it's way of dealing with Slick configuration (if you are using Play which is, again, not possible to say from your question, though I assumed it as you mentioned web requests).
EDIT (after question update):
And we have an answer. Each time you call db method new Database object (together with connection pool is created). Just move it as I suggested above (created once per application lifecycle). The easiest way possible (not necessarily the best one) would be to change this line:
def db = Database.forConfig("database")
to this:
lazy val db = Database.forConfig("database")
Above would immediately solve your problem (assuming that there is only one instance of PersonsApi created in your application.
Other solution (better perhaps) would be to create something like this:
object DatabaseConfig extends Config {
val driver = slick.driver.PostgresDriver
import driver.api._
lazy val db = Database.forConfig("database")
}
and then change your API to this:
trait PersonsApi extends JsonMapper {
val getAllPersons = (path("persons") & get) {
complete(DatabaseConfig.db.run(PersonDao.findAll))
}
val getPersonById = (path("persons" / IntNumber) & get) {
num => complete(DatabaseConfig.db.run(PersonDao.findById(num)))
}
val personsApi =
getAllPersons ~
getPersonById
}

Testing Play + Slick app

I've a simple CRUD app built with Scala Play 2.4.3 and Play-slick 1.1.0 (slick 3.1.0) that uses a MySQL database for persistent storage.
I was trying to create the tests for my app and I saw 2 main options:
mocking database access, that as far as I've seen, requires some code changes
make tests use an alternative database (probably, in memory H2).
What's the best approach (vantages and desavantages)?
I prefer the second approach, but I'm finding some difficulties in setting up the tests.
What do I need to do? First, I think that I need to do the tests run with a FakeApplication, right? Do I need any sbt dependency to be able to do that?
After that, how do I specify to use the H2 database?
I had the same struggle and I came up with a solution like this(using second approach):
Create a context for DAO to use:
trait BaseContext{
def dbName: String
val dbConfig = DatabaseConfigProvider.get[JdbcProfile](dbName)
val db = dbConfig.db
val profile = dbConfig.driver
val tables = new Tables { // this is generated by Schema Code Generator
override val profile: JdbcProfile = dbConfig.driver
}
}
#Singleton
class AppContext extends BaseContext{
def dbName = "mysql" // name in your conf right after "slick.dbs"
}
#Singleton
class TestingContext extends BaseContext{
def dbName = "h2"
}
Then create a module to bind the injection, and don't forget to enable it in conf using play.modules.enabled += "your.Module":
class ContextModule(environment: Environment, configuration: Configuration) extends AbstractModule {
override def configure(): Unit = {
if (configuration.getString("app.mode").contains("test")) {
bind(classOf[BaseContext])
.to(classOf[TestingContext])
} else {
bind(classOf[BaseContext])
.to(classOf[AppContext])
}
}
}
And inject it to every DAO you've created:
class SomeDAO #Inject()(context: BaseContext){
val dbConfig = context.dbConfig
val db = context.db
val tables = context.tables
import tables.profile.api._
def otherStuff....
// you can call db.run(...), tables.WhateverYourTableIs, tables.TableRowCaseClass, ...
}
And final step, your configuration file. In my case I used app.mode to mark the environment, and I use separate .conf for different environment. Of cause, in these conf you must have the correct DB configuration. Here's the sample:
app.mode = "test"
# Database configuration
slick.dbs = {
# for unit test
h2 {
driver = "slick.driver.H2Driver$"
db = {
url = "jdbc:h2:mem:test;MODE=MYSQL"
driver = "org.h2.Driver"
keepAliveConnection = true
}
}
}
I'm pretty sure my solution is not a elegant one, but it deliver the goods. :)
Any better solution is welcomed!
my solution was to add step(Play.start(fakeApp)) in the beginning of each spec, and step(Play.stop(fakeApp)) in the end of each spec.
Also:
def fakeApp: FakeApplication = {
FakeApplication(additionalConfiguration =
Map(
"slick.dbs.default.driver" -> "slick.driver.H2Driver$",
"slick.dbs.default.db.driver" -> "org.h2.Driver",
"slick.dbs.default.db.url" -> "jdbc:h2:mem:play"
))
}
This was needed because I'm using play-slick, which requires configurations like:
slick.dbs.default.driver = "slick.driver.MySQLDriver$"
slick.dbs.default.db.driver = "com.mysql.jdbc.Driver"
slick.dbs.default.db.url = "jdbc:mysql://localhost/database"
slick.dbs.default.db.user = "user"
slick.dbs.default.db.password = "password"
more info on the docs

Scala/Akka WSResponse recursively call

Im trying to parse some data from an API
I have a recursion method that calling to this method
def getJsonValue( url: (String)): JsValue = {
val builder = new com.ning.http.client.AsyncHttpClientConfig.Builder()
val client = new play.api.libs.ws.ning.NingWSClient(builder.build())
val newUrl = url.replace("\"", "").replace("|", "%7C").trim
val response: Future[WSResponse] = client.url(newUrl).get()
Await.result(response, Duration.create(10, "seconds")).json
}
Everything is working well but after 128 method calls i'm getting this warning
WARNING: You are creating too many HashedWheelTimer instances. HashedWheelTimer is a shared resource that must be reused across the application, so that only a few instances are created.
After about 20 More calls im getting this exception
23:24:57.425 [main] ERROR com.ning.http.client.AsyncHttpClient - Unable to instantiate provider com.ning.http.client.providers.netty.NettyAsyncHttpProvider. Trying other providers.
23:24:57.438 [main] ERROR com.ning.http.client.AsyncHttpClient - org.jboss.netty.channel.ChannelException: Failed to create a selector.
Questions
1.Im assuming that the connections didnt closed ?? and therefore i can't create new connections.
2.What will be the correct and the safe way to create those HTTP calls
Had the same problem.
Found 2 interesting solutions:
make sure you are not creating tons of clients with closing them
the threadPool you are using may be causing this.
My piece of code (commenting that line of code solved, I'm now testing several configurations):
private[this] def withClient(block: NingWSClient => WSResponse): Try[WSResponse] = {
val config = new NingAsyncHttpClientConfigBuilder().build()
val clientConfig = new AsyncHttpClientConfig.Builder(config)
// .setExecutorService(new ThreadPoolExecutor(5, 15, 30L, TimeUnit.SECONDS, new SynchronousQueue[Runnable]))
.build()
val client = new NingWSClient(clientConfig)
val result = Try(block(client))
client.close()
result
}
for avoiding this you can use different provider.
private AsyncHttpProvider httpProvider =new ApacheAsyncHttpProvider(config);
private AsyncHttpClient asyncHttpClient = new AsyncHttpClient(httpProvider,config);
I ran into this same problem. Before you call your recursive method, you should create builder and client and pass client to the recursive method, as well as getJsonValue. This is what getJsonValue should look like:
def getJsonValue(url: String, client: NingWSClient): JsValue = {
val builder = new com.ning.http.client.AsyncHttpClientConfig.Builder()
val client = new play.api.libs.ws.ning.NingWSClient(builder.build())
val newUrl = url.replace("\"", "").replace("|", "%7C").trim
val response: Future[WSResponse] = client.url(newUrl).get()
Await.result(response, Duration.create(10, "seconds")).json
}