I have a simple akka Http Server:
import akka.http.scaladsl.model.{ContentTypes, HttpEntity}
import akka.http.scaladsl.server.HttpApp
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.settings.ServerSettings
import com.typesafe.config.ConfigFactory
object MinimalHttpServer extends HttpApp {
def route =
pathPrefix("v1") {
path("subscribe" / Segment) { id =>
get {
complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<h1>Hello $id from Akka Http!</h1>"))
} ~
post {
entity(as[String]) { entity =>
complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<b>Thanks $id for posting your message <i>$entity</i></b>"))
}
}
}
}
}
object MinimalHttpServerApplication extends App {
MinimalHttpServer.startServer("localhost", 8088, ServerSettings(ConfigFactory.load))
}
I use Sbt native Packager to build an universal zip. When I deploy my application to Aws Elastic Beanstalk, I receive this error:
[Instance: i-0a846978718d54d76] Command failed on instance. Return code: 1 Output: (TRUNCATED)...xml_2.11-1.0.5.jar Unable to launch application as the source bundle does not contain either a file named application.jar or a Procfile. Unable to launch application as the source bundle does not contain either a file named application.jar or a Procfile. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/01_configure_application.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Any Ideas? Thank You!
It appears AWS Elastic Beanstalk expects your .zip to contain either a file named application.jar or a Procfile, and the zip created by sbt-native-packager does not look like that.
It appears sbt-native-packager does not have support for the format Elastic Beanstalk expects, though GitHub issue 632 shows some work done in that direction.
Related
I am running SBT 1.2.8 and my project needs to download packages from a repo on a privately hosted Artifactory instance. My repo is protected by basic auth. After reading a multitude of examples and instructions, I created a credentials.properties file in my repo.
realm=Artifactory Realm
host=artifactory.mycompany.com
username=my_username
password=my_password
I then added the following to my build.sbt file
credentials += Credentials(new File("credentials.properties"))
Then I added the repository to my list of resolvers in resolvers.sbt
"My Company Artifactory" at "https://artifactory.mycompany.com/artifactory/my_private_repo/",
I built my application and was able to download the protected packages just fine.
However, a system administrator at my company requested I turn on the “Hide Existence of Unauthorized Resources” setting in Artifactory. This setting forces Artifactory to return 404 errors when an unauthenticated user tries to access protected resources. Usually in this case, Artifactory returns 401s with a WWW-Authenticate header.
Suddenly, my application was unable to resolve its dependencies. I turned the Artifactory setting off and then back on again and verified this setting was, in fact, the cause of my problems.
It appears as though SBT will not send credentials unless it is challenged with a 401 and a WWW-Authenticate header (with the proper realm). Looking at the docs and GitHub issues for SBT, Ivy, and Coursier, it seems this “preemptive authentication” is not a supported feature.
I spend many hours trying to resolve this in various ways, but I cannot find a solution. Here is what I have tried:
Adding my Artifactory username and password to the repository url, so it looks like https://my_username:my_password#artifactory.mycompany.com/artifactory/my_private_repo/. This worked in my browser and a REST client, but not with SBT.
Omitting the “realm” from my credentials file
Switching to SBT 1.3.9 and trying everything above with the new version.
Does anyone know how I can get SBT to use preemptive HTTP basic auth? It appears both Maven and Gradle support this (see links below), but I cannot find anything in the SBT docs.
Maven support for preemptive auth: https://jfrog.com/knowledge-base/why-does-my-maven-builds-are-failing-with-a-404-error-when-hide-existence-of-unauthorized-resources-is-enabled/
Gradle support for preemptive auth:
https://github.com/gradle/gradle/pull/386/files
I'm almost thinking of setting up a local proxy to send the proper headers Artifactory, and point SBT to use the local proxy as a resolver. However, that seems needlessly cumbersome for developers to use.
you are correct.
You can setup an AbstractRepository. See https://github.com/SupraFii/sbt-google-artifact-registry/blob/master/src/main/scala/ch/firsts/sbt/gar/ArtifactRegistryRepository.scala#L21 for example:
package ch.firsts.sbt.gar
import java.io.File
import java.util
import com.google.cloud.artifactregistry.wagon.ArtifactRegistryWagon
import org.apache.ivy.core.module.descriptor.Artifact
import org.apache.ivy.plugins.repository.AbstractRepository
import org.apache.maven.wagon.repository.Repository
class ArtifactRegistryRepository(repositoryUrl: String) extends AbstractRepository {
val repo = new Repository("google-artifact-registry", repositoryUrl)
val wagon = new ArtifactRegistryWagon()
override def getResource(source: String): ArtifactRegistryResource = {
val plainSource = stripRepository(source)
wagon.connect(repo)
ArtifactRegistryResource(repositoryUrl, plainSource, wagon.resourceExists(plainSource))
}
override def get(source: String, destination: File): Unit = {
val adjustedSource = if (destination.toString.endsWith("sha1"))
source + ".sha1"
else if (destination.toString.endsWith("md5"))
source + ".md5"
else
source
wagon.connect(repo)
wagon.get(adjustedSource, destination)
}
override def list(parent: String): util.List[String] = sys.error("Listing repository contents is not supported")
override def put(artifact: Artifact, source: File, destination: String, overwrite: Boolean): Unit = {
val plainDestination = stripRepository(destination)
wagon.connect(repo)
wagon.put(source, plainDestination)
}
private def stripRepository(fullName: String): String = fullName.substring(repositoryUrl.length + 1)
}
Compiler error when using example provided in Flink documentation. The Flink documentation provides sample Scala code to set the REST client factory parameters when talking to Elasticsearch, https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html.
When trying out this code i get a compiler error in IntelliJ which says "Cannot resolve symbol restClientBuilder".
I found the following SO which is EXACTLY my problem except that it is in Java and i am doing this in Scala.
Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
I tried copy pasting the solution code provided in the above SO into IntelliJ, the auto-converted code also has compiler errors.
// provide a RestClientFactory for custom configuration on the internally created REST client
// i only show the setMaxRetryTimeoutMillis for illustration purposes, the actual code will use HTTP cutom callback
esSinkBuilder.setRestClientFactory(
restClientBuilder -> {
restClientBuilder.setMaxRetryTimeoutMillis(10)
}
)
Then i tried (auto generated Java to Scala code by IntelliJ)
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
import org.apache.http.auth.AuthScope
import org.apache.http.auth.UsernamePasswordCredentials
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClientBuilder
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
esSinkBuilder.setRestClientFactory((restClientBuilder) => {
def foo(restClientBuilder) = restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = { // elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}
})
foo(restClientBuilder)
})
The original code snippet produces the error "cannot resolve RestClientFactory" and then Java to Scala shows several other errors.
So basically i need to find a Scala version of the solution described in Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
Update 1: I was able to make some progress with some help from IntelliJ. The following code compiles and runs but there is another problem.
esSinkBuilder.setRestClientFactory(
new RestClientFactory {
override def configureRestClientBuilder(restClientBuilder: RestClientBuilder): Unit = {
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
// elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
httpClientBuilder.setSSLContext(trustfulSslContext)
}
})
}
}
The problem is that i am not sure if i should be doing a new of the RestClientFactory object. What happens is that the application connects to the elasticsearch cluster but then discovers that the SSL CERT is not valid, so i had to put the trustfullSslContext (as described here https://gist.github.com/iRevive/4a3c7cb96374da5da80d4538f3da17cb), this got me past the SSL issue but now the ES REST Client does a ping test and the ping fails, it throws an exception and the app shutsdown. I am suspecting that the ping fails because of the SSL error and maybe it is not using the trustfulSslContext i setup as part of new RestClientFactory and this makes me suspect that i should not have done the new, there should be a simple way to update the existing RestclientFactory object and basically this is all happening because of my lack of Scala knowledge.
Happy to report that this is resolved. The code i posted in Update 1 is correct. The ping to ECE was not working for two reasons:
The certificate needs to include the complete chain including the root CA, the intermediate CA and the cert for the ECE. This helped get rid of the whole trustfulSslContext stuff.
The ECE was sitting behind an ha-proxy and the proxy did the mapping for the hostname in the HTTP request to the actual deployment cluster name in ECE. this mapping logic did not take into account that the Java REST High Level client uses the org.apache.httphost class which creates the hostname as hostname:port_number even when the port number is 443. Since it did not find the mapping because of the 443 therefore the ECE returned a 404 error instead of 200 ok (only way to find this was to look at unencrypted packets at the ha-proxy). Once the mapping logic in ha-proxy was fixed, the mapping was found and the pings are now successfull.
I'm using the Play framework 2.5. I created a new project and tried testing web service.
For the moment I just copied the example at https://www.playframework.com/documentation/2.5.x/ScalaTestingWebServiceClients#testing-a-github-client .
On my machine I'm using a proxy, and when running GitHubClientSpec
testOnly GitHubClientSpec
I get an error from the proxy as response.
What I also tried from the scala console is the piece of code :
import play.core.server._
import play.api.routing.sird._
import play.api.mvc._
val server = NettyServer.fromRouter(ServerConfig(
port = Some(19000),
address = "127.0.0.1"
)) {
case GET(p"/hello") => Action {
Results.Ok(s"Hello")
}
}
And curl http://localhost:19000/hello, returned "Hello".
So this works fine, but when running testOnly GitHubClientSpec it is not.
How to prevent Play from using the proxy ?
I have a Play framework app in Scala. It uses database and I need to load data into database before first app start. I thought I could add a class with a main class in the app and start it like play -main loadDataClass. It seems to be working but once I access Play.current.configuration I need this to access database credentials. I get this error:
Exception in thread "main" java.lang.RuntimeException: There is no started application
Please advice how to implement this. Should I somehow start the application?
You need to create a module for loading data on start up. In the production mode (activator start) it would run after application start, in the development mode (activator run) it would run after first request to the server.
Create module
app/modules/PrepareDatabaseModule.scala:
inside this module
package modules
import com.google.inject.AbstractModule
trait PrepareDatabase {}
class PrepareDatabaseClass extends PrepareDatabase {
initialize()
def initialize() = {
// Load your data here
}
}
class PrepareDatabaseModule extends AbstractModule {
def configure() = {
bind(classOf[PrepareDatabase])
.to(classOf[PrepareDatabaseClass]).asEagerSingleton
}
}
in conf/application.conf:
play.modules.enabled += "modules.PrepareDatabaseModule"
I have installed eclipse, the cloudfoundry plugin, the scala plugin,the vaadin plugin(for web developments) and the mongodb libraries.
I created a class like this :
import vaadin.scala.Application
import vaadin.scala.VerticalLayout
import com.mongodb.casbah.MongoConnection
import com.mongodb.casbah.commons.MongoDBObject
import vaadin.scala.Label
import vaadin.scala.Button
class Launcher extends Application {
val label=new Label
override def main = new VerticalLayout() {
val coll=MongoConnection()("mybd")("somecollection")
val builder=MongoDBObject.newBuilder
builder+="foo1" -> "bar"
var newobj=builder.result()
coll.save(newobj)
val mongoColl=MongoConnection()("mybd")("somecollection")
val withFoo=mongoColl.findOne()
label.value=withFoo
add(label)
//bouton pour faire joli
add(new Button{
caption_=("click me!")
})
}
}
the error (the access to the mongodb database is denied) comes from the parameters, which are the default ones.
do you know how to set up the good parameters in scala or in java?
Looks like you got some help on the vcap-dev mailing list
package com.example.vaadin_1
import vaadin.scala.Application
import org.cloudfoundry.runtime.env.CloudEnvironment
import org.cloudfoundry.runtime.env.MongoServiceInfo
import com.mongodb.casbah.MongoConnection
class Launcher extends Application {
val cloudEnvironment = new CloudEnvironment()
val mongoServices = cloudEnvironment.getServiceInfos(classOf[MongoServiceInfo])
val mongo = mongoServices.get(0)
val mongodb = MongoConnection(mongo.getHost(), mongo.getPort())("abc")
mongodb.authenticate(mongo.getUserName(),mongo.getPassword())
}
I would suggest to do it using Spring Data for MongoDB there is a sample application for Cloudfoundry in particular put together by the Spring guys. With a bit of xml configuration you have ready to inject the mongoTemplate similar to the familiar Spring xxxTemplate paradigm.
when deploying to CloudFoundry, the information relative to connecting to a service (i.e mongo in your case) is made available to the app through environment variable VCAP_SERVICES. It is a json document with one entry per service. You could of course parse it yourself, but you will find the class http://cf-runtime-api.cloudfoundry.com/org/cloudfoundry/runtime/env/CloudEnvironment.html useful. You will need to add the org.cloudfoundry/cloudfoundry-runtime/0.8.1 jar to your project. You can use it without Spring.
Have a look at http://docs.cloudfoundry.com/services.html for explanation of the VCAP_SERVICES underlying var