I'm using the Play framework 2.5. I created a new project and tried testing web service.
For the moment I just copied the example at https://www.playframework.com/documentation/2.5.x/ScalaTestingWebServiceClients#testing-a-github-client .
On my machine I'm using a proxy, and when running GitHubClientSpec
testOnly GitHubClientSpec
I get an error from the proxy as response.
What I also tried from the scala console is the piece of code :
import play.core.server._
import play.api.routing.sird._
import play.api.mvc._
val server = NettyServer.fromRouter(ServerConfig(
port = Some(19000),
address = "127.0.0.1"
)) {
case GET(p"/hello") => Action {
Results.Ok(s"Hello")
}
}
And curl http://localhost:19000/hello, returned "Hello".
So this works fine, but when running testOnly GitHubClientSpec it is not.
How to prevent Play from using the proxy ?
Related
I have gone thru' multiple blogs and official documentation but couldn't resolve my issue. I am using testContainers-scala version 0.38.1 and scala version 2.11.
I am trying to create a simple test using testContainer-scala as below:
class MyServiceITSpec extends AnyFlatSpec with ForAllTestContainer {
override val container = GenericContainer(dockerImage="my-service",
exposedPorts = Seq(8080),
env=(HashMap[String, String]("PARAM1" -> "value1", "PARAM2" -> "value2", "PARAM3" -> "value3")),
waitStrategy = Wait.forHttp("/")
)
"GenericContainer" should "start my service and say Hello! Wassupp" in {
assert(Source.fromInputStream(
new URL(s"http://${container.containerIpAddress}:${container.mappedPort(8080)}/").openConnection().getInputStream
).mkString.contains("Hello! Wassupp"))
}
}
On the basis of the above snippet, my understanding is this (please correct if wrong):
Port 8155 is exposed by the docker container and a random host port against the same would be assigned.
We can get that assigned port as container.mappedPort
Here I am trying to assert that http:// localhost:mappedPort/ return Hello! Wassupp.
But, I get the below error:
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:498)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
... 18 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:32869/ should return HTTP 200)
at org.testcontainers.containers.wait.strategy.HttpWaitStrategy.waitUntilReady(HttpWaitStrategy.java:214)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:890)
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:441)
... 20 more
The same image runs just fine with:
docker run -p 8081:8080 -e PARAM1=value1 -e PARAM2=value2 -e PARAM3=VALUE3 my-service
So after juggling with the errors, I found my issue. It is to do with the required Request Headers missing from the request. I am adding the reference code for anyone who runs into similar issue.
import com.dimafeng.testcontainers.{ForAllTestContainer, GenericContainer}
import org.scalatest.flatspec.AnyFlatSpec
import org.testcontainers.containers.wait.strategy.Wait
import scala.collection.immutable.HashMap
import scalaj.http.Http
class MyServiceITSpec extends AnyFlatSpec with ForAllTestContainer {
override val container = GenericContainer(dockerImage="my-service-img:tag12345",
exposedPorts = Seq(8080),
env=(HashMap[String, String]("PARAM1" -> "value1", "PARAM2" -> "value2")),
waitStrategy = Wait.forHttp("/") // or "/health" based on ur implementation
)
"My Service" should "successfully fetch the msg" in {
assert(Http(s"http://${container.containerIpAddress}:${container.mappedPort(8080)}/products/product1")
.header("HEADER1", "value1")
.header("HEADER2", "value2")
.asString.code==200)
}
}
Some explanations that I found after a lot of reading:
You give the port number that your docker application exposes as exposedPorts.
TestContainers then does a mapping of this port against a random port (this is by design to avoid port number conflicts). If you were to run this docker image directly on your machine you would write:
docker run -p 8081:8080 -e PARAM1=value1 -e PARAM2=value2 my-service-img:tag12345
Here, your exposed port is 8080 and the mapped port is 8081.
TestContainers runs the docker image by exposing the port 8080 and then mapping it against a random port. The mapped port can be container.mappedPort().
Another important thing to notice is the wait strategy. This tells the code to wait unless the / endpoint gets up. This is kind of a health check that your application exposes. You can have a better endpoint for the same - like /health. By default, it waits for 60 seconds for this endpoint to become active. Post that it would anyway run the tests and if the application has not started by then, it would cause an error. I am not sure how to override the default timeout but I think there should be a way to do that.
Lastly, I am using scalaj.http.Http to make a HTTP request(it is a pretty easy one to use - you can ).
Compiler error when using example provided in Flink documentation. The Flink documentation provides sample Scala code to set the REST client factory parameters when talking to Elasticsearch, https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/elasticsearch.html.
When trying out this code i get a compiler error in IntelliJ which says "Cannot resolve symbol restClientBuilder".
I found the following SO which is EXACTLY my problem except that it is in Java and i am doing this in Scala.
Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
I tried copy pasting the solution code provided in the above SO into IntelliJ, the auto-converted code also has compiler errors.
// provide a RestClientFactory for custom configuration on the internally created REST client
// i only show the setMaxRetryTimeoutMillis for illustration purposes, the actual code will use HTTP cutom callback
esSinkBuilder.setRestClientFactory(
restClientBuilder -> {
restClientBuilder.setMaxRetryTimeoutMillis(10)
}
)
Then i tried (auto generated Java to Scala code by IntelliJ)
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
import org.apache.http.auth.AuthScope
import org.apache.http.auth.UsernamePasswordCredentials
import org.apache.http.client.CredentialsProvider
import org.apache.http.impl.client.BasicCredentialsProvider
import org.apache.http.impl.nio.client.HttpAsyncClientBuilder
import org.elasticsearch.client.RestClientBuilder
// provide a RestClientFactory for custom configuration on the internally created REST client// provide a RestClientFactory for custom configuration on the internally created REST client
esSinkBuilder.setRestClientFactory((restClientBuilder) => {
def foo(restClientBuilder) = restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = { // elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
}
})
foo(restClientBuilder)
})
The original code snippet produces the error "cannot resolve RestClientFactory" and then Java to Scala shows several other errors.
So basically i need to find a Scala version of the solution described in Apache Flink (v1.6.0) authenticate Elasticsearch Sink (v6.4)
Update 1: I was able to make some progress with some help from IntelliJ. The following code compiles and runs but there is another problem.
esSinkBuilder.setRestClientFactory(
new RestClientFactory {
override def configureRestClientBuilder(restClientBuilder: RestClientBuilder): Unit = {
restClientBuilder.setHttpClientConfigCallback(new RestClientBuilder.HttpClientConfigCallback() {
override def customizeHttpClient(httpClientBuilder: HttpAsyncClientBuilder): HttpAsyncClientBuilder = {
// elasticsearch username and password
val credentialsProvider = new BasicCredentialsProvider
credentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(es_user, es_password))
httpClientBuilder.setDefaultCredentialsProvider(credentialsProvider)
httpClientBuilder.setSSLContext(trustfulSslContext)
}
})
}
}
The problem is that i am not sure if i should be doing a new of the RestClientFactory object. What happens is that the application connects to the elasticsearch cluster but then discovers that the SSL CERT is not valid, so i had to put the trustfullSslContext (as described here https://gist.github.com/iRevive/4a3c7cb96374da5da80d4538f3da17cb), this got me past the SSL issue but now the ES REST Client does a ping test and the ping fails, it throws an exception and the app shutsdown. I am suspecting that the ping fails because of the SSL error and maybe it is not using the trustfulSslContext i setup as part of new RestClientFactory and this makes me suspect that i should not have done the new, there should be a simple way to update the existing RestclientFactory object and basically this is all happening because of my lack of Scala knowledge.
Happy to report that this is resolved. The code i posted in Update 1 is correct. The ping to ECE was not working for two reasons:
The certificate needs to include the complete chain including the root CA, the intermediate CA and the cert for the ECE. This helped get rid of the whole trustfulSslContext stuff.
The ECE was sitting behind an ha-proxy and the proxy did the mapping for the hostname in the HTTP request to the actual deployment cluster name in ECE. this mapping logic did not take into account that the Java REST High Level client uses the org.apache.httphost class which creates the hostname as hostname:port_number even when the port number is 443. Since it did not find the mapping because of the 443 therefore the ECE returned a 404 error instead of 200 ok (only way to find this was to look at unencrypted packets at the ha-proxy). Once the mapping logic in ha-proxy was fixed, the mapping was found and the pings are now successfull.
I have a simple akka Http Server:
import akka.http.scaladsl.model.{ContentTypes, HttpEntity}
import akka.http.scaladsl.server.HttpApp
import akka.http.scaladsl.server.Directives._
import akka.http.scaladsl.settings.ServerSettings
import com.typesafe.config.ConfigFactory
object MinimalHttpServer extends HttpApp {
def route =
pathPrefix("v1") {
path("subscribe" / Segment) { id =>
get {
complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<h1>Hello $id from Akka Http!</h1>"))
} ~
post {
entity(as[String]) { entity =>
complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<b>Thanks $id for posting your message <i>$entity</i></b>"))
}
}
}
}
}
object MinimalHttpServerApplication extends App {
MinimalHttpServer.startServer("localhost", 8088, ServerSettings(ConfigFactory.load))
}
I use Sbt native Packager to build an universal zip. When I deploy my application to Aws Elastic Beanstalk, I receive this error:
[Instance: i-0a846978718d54d76] Command failed on instance. Return code: 1 Output: (TRUNCATED)...xml_2.11-1.0.5.jar Unable to launch application as the source bundle does not contain either a file named application.jar or a Procfile. Unable to launch application as the source bundle does not contain either a file named application.jar or a Procfile. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/01_configure_application.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
Any Ideas? Thank You!
It appears AWS Elastic Beanstalk expects your .zip to contain either a file named application.jar or a Procfile, and the zip created by sbt-native-packager does not look like that.
It appears sbt-native-packager does not have support for the format Elastic Beanstalk expects, though GitHub issue 632 shows some work done in that direction.
I'm using Akka HTTP 10.0.9, but struggling to get my unit tests to have a working Remote Address.
eg unit test:
Get("/").withHeaders(
RawHeader("Remote-Address", "192.168.1.1"), RawHeader("X-Forwarded-For", "192.168.1.1")
) ~> route ~> check {
status must_== StatusCodes.OK
}
And in the web server code:
extractClientIP{ clientAddr =>
complete(s"$clientAddr")
}
When running the app via the command line, the client address is returned correctly. But when run via unit tests, the client address always comes back as "Unknown"
What am I doing wrong?
This is caused by akka http can't handle the RawHeader in test. You can solve it by use the Remote-Address object for set the IP for test:
import akka.http.scaladsl.model.headers.`Remote-Address`
Get().withHeaders(
`Remote-Address`(RemoteAddress(new InetSocketAddress("192.168.1.1", 23)))
)
When attempting to run integration tests, I've run into a baffling problem where the JVM will hang, using 100% of the CPU. The test that comes with the new Play application works correctly, but as soon as it requires database interaction, it will hang indefinitely. For all other unit tests, everything runs smoothly connecting to a mysql database on localhost. I'd like to be able to use that same setup with my integration tests.
Here is an example of a test that will hang upon calling browser.goTo("/")
import org.specs2.mutable._
import play.api.test._
import play.api.test.Helpers._
class TestSpec extends Specification {
"Application" should {
"work from within a browser" in new WithBrowser(webDriver = HTMLUNIT, app = FakeApplication()) {
browser.goTo("/")
println(browser.pageSource)
browser.$("#email").text("test#fakeemail.com")
browser.$("#password").text("password")
browser.$("#loginbutton").click()
browser.pageSource must not contain("Sign in")
browser.pageSource must contain("Logout")
}
}
}
The issue in my case was the selenium version. Adding this line to appDependencies in Build.scala will upgrade selenium:
"org.seleniumhq.selenium" % "selenium-java" % "2.35.0" % "test"
From there I was able to use both HTMLUNIT and FIREFOX for web drivers without any issues.
Have you tried setting a port such as 3333 then using your localhost?
browser.goTo("http://localhost:3333/")
Have you solved this? I have the same problem, it also hangs with simple route(FakeRequest) if there is any db connection.
I solved this by setting (Build.scala) :
.settings( parallelExecution in Test := false)
It helped me with FakeRequest, but selenium tests still hang.