How to run a custom docker image testContainer - scala

I have gone thru' multiple blogs and official documentation but couldn't resolve my issue. I am using testContainers-scala version 0.38.1 and scala version 2.11.
I am trying to create a simple test using testContainer-scala as below:
class MyServiceITSpec extends AnyFlatSpec with ForAllTestContainer {
override val container = GenericContainer(dockerImage="my-service",
exposedPorts = Seq(8080),
env=(HashMap[String, String]("PARAM1" -> "value1", "PARAM2" -> "value2", "PARAM3" -> "value3")),
waitStrategy = Wait.forHttp("/")
)
"GenericContainer" should "start my service and say Hello! Wassupp" in {
assert(Source.fromInputStream(
new URL(s"http://${container.containerIpAddress}:${container.mappedPort(8080)}/").openConnection().getInputStream
).mkString.contains("Hello! Wassupp"))
}
}
On the basis of the above snippet, my understanding is this (please correct if wrong):
Port 8155 is exposed by the docker container and a random host port against the same would be assigned.
We can get that assigned port as container.mappedPort
Here I am trying to assert that http:// localhost:mappedPort/ return Hello! Wassupp.
But, I get the below error:
Caused by: org.testcontainers.containers.ContainerLaunchException: Could not create/start container
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:498)
at org.testcontainers.containers.GenericContainer.lambda$doStart$0(GenericContainer.java:325)
at org.rnorth.ducttape.unreliables.Unreliables.retryUntilSuccess(Unreliables.java:81)
... 18 more
Caused by: org.testcontainers.containers.ContainerLaunchException: Timed out waiting for URL to be accessible (http://localhost:32869/ should return HTTP 200)
at org.testcontainers.containers.wait.strategy.HttpWaitStrategy.waitUntilReady(HttpWaitStrategy.java:214)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:890)
at org.testcontainers.containers.GenericContainer.tryStart(GenericContainer.java:441)
... 20 more
The same image runs just fine with:
docker run -p 8081:8080 -e PARAM1=value1 -e PARAM2=value2 -e PARAM3=VALUE3 my-service

So after juggling with the errors, I found my issue. It is to do with the required Request Headers missing from the request. I am adding the reference code for anyone who runs into similar issue.
import com.dimafeng.testcontainers.{ForAllTestContainer, GenericContainer}
import org.scalatest.flatspec.AnyFlatSpec
import org.testcontainers.containers.wait.strategy.Wait
import scala.collection.immutable.HashMap
import scalaj.http.Http
class MyServiceITSpec extends AnyFlatSpec with ForAllTestContainer {
override val container = GenericContainer(dockerImage="my-service-img:tag12345",
exposedPorts = Seq(8080),
env=(HashMap[String, String]("PARAM1" -> "value1", "PARAM2" -> "value2")),
waitStrategy = Wait.forHttp("/") // or "/health" based on ur implementation
)
"My Service" should "successfully fetch the msg" in {
assert(Http(s"http://${container.containerIpAddress}:${container.mappedPort(8080)}/products/product1")
.header("HEADER1", "value1")
.header("HEADER2", "value2")
.asString.code==200)
}
}
Some explanations that I found after a lot of reading:
You give the port number that your docker application exposes as exposedPorts.
TestContainers then does a mapping of this port against a random port (this is by design to avoid port number conflicts). If you were to run this docker image directly on your machine you would write:
docker run -p 8081:8080 -e PARAM1=value1 -e PARAM2=value2 my-service-img:tag12345
Here, your exposed port is 8080 and the mapped port is 8081.
TestContainers runs the docker image by exposing the port 8080 and then mapping it against a random port. The mapped port can be container.mappedPort().
Another important thing to notice is the wait strategy. This tells the code to wait unless the / endpoint gets up. This is kind of a health check that your application exposes. You can have a better endpoint for the same - like /health. By default, it waits for 60 seconds for this endpoint to become active. Post that it would anyway run the tests and if the application has not started by then, it would cause an error. I am not sure how to override the default timeout but I think there should be a way to do that.
Lastly, I am using scalaj.http.Http to make a HTTP request(it is a pretty easy one to use - you can ).

Related

Allow all requests to the AKKA-HTTP service

Comrades!
I have a small service from the AKKA-HTTP example.
import ch.megard.akka.http.cors.scaladsl.CorsDirectives._
object Server extends App{
val route = cors() {
path("hello") {
get {
complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, "<h1>Привет ёпта</h1>"))
}
}
}
val routes = cors() {
concat(route, getUser, createUser, addMessage, getQueue, test, test2)
}
val bindingFuture = Http().newServerAt("localhost", 8080).bind(routes)
}
For CORS i create file resourses/application.conf:
akka-http-cors {
allowed-origins = "*"
allowed-methods = ["GET", "POST", "PUT", "DELETE", "HEAD", "OPTIONS"]
}
When I run a project in intellij idea, the route works fine:
But if I run the project in Docker, the route doesn't want to work.
Error in chrome:
Error in Postman:
Below are the errors when the project is turned off everywhere:
How do I properly configure the application.conf file so that the application accepts third-party requests? Or maybe the error is hidden in something else?
Please tell me!
I've been thinking for two days.
UPD: File DockerFile:
FROM openjdk:8-jre-alpine
WORKDIR /opt/docker
ADD --chown=daemon:daemon opt /opt
USER daemon
ENTRYPOINT ["/opt/docker/bin/servertelesupp"]
CMD []
Project on GitHub: https://github.com/MinorityMeaning/ServerTeleSupp
Change
Http().newServerAt("localhost", 8080).bind(routes)
to
Http().newServerAt("0.0.0.0", 8080).bind(routes)
By binding to localhost inside docker, you will not be able route traffic from outside to it.
What is the value of -p flag that you are using? It should be 8080:8080 if the server is using 8080 port inside the docker, or 8080:80 if the server is using 80 (and so on). Also please do verify that the port is indeed free inside docker.
PS: I have low reputation and can't add a comment.

Proxy authentication issue using Jsoupbrowser in container

I'm having an issue trying to authenticate my proxy inside a Docker.
That's what I did :
Authenticator.setDefault(new Authenticator() {
override def getPasswordAuthentication = new PasswordAuthentication(<USERNAME>, <PASSWORD>.toCharArray)
})
val browser = new JsoupBrowser(ua,proxy) {
override def requestSettings(conn: Connection) = conn.timeout(5000)
}
// Step 1: We __scrape__ the current page.
val doc = browser.get(baseUrl)
It works locally, but when I deploy it on my server I get an Error 407
java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 407 Proxy Authentication Required"
I also tried upgrading the configuration to container level but it didn't work.
I found a solution.
The problem was only at the deploy, so I concluded that the problem came with the build of the docker.
I add this JVM parameter in my Dockerfile :
-Djdk.http.auth.tunneling.disabledSchemes=
To the CMD
CMD java -Dhttp.port=${port} -Djdk.http.auth.tunneling.disabledSchemes= -Dplay.crypto.secret=secret $* -jar ./app-assembly.jar
It works now.

Changing hostname and port with Vapor 3

The Config/server.json file doesn't seem to be read by Vapor 3, and as such I can't configure the hostname and port that a Vapor 3 app binds to.
Does Vapor 3 have a different method for this?
Official Solution (as sanctioned by the maintainers)
You can use NIOServerConfig.
let serverConfiure = NIOServerConfig.default(hostname: "0.0.0.0", port: 9090)
services.register(serverConfiure)
Vapor version is 3.0.3
Currently, you can set the port and hostname when running your server:
swift run Run --hostname 0.0.0.0 --port 9000
There appears to be struct-based configuration for EngineServer but I don’t think it is configurable at run time just yet. The last time the Vapor developer answered this question (on their Slack) the command-line argument method was the suggested one.
in Vapor 4
app.http.server.configuration.hostname = "127.0.0.1"
app.http.server.configuration.port = 8000
in Vapor 3
services.register(NIOServerConfig.default(hostname: "127.0.0.1", port: 8000))
Be sure you are using the Vapor 3 version then use this:
vapor run --hostname=0.0.0.0 --port=8080
If you don't add = after the parameter, you are going to receive the follow complaint:
CommandError: Unknown command 8080
If you do as I recommended above, you are going to receive:
[Deprecated] --option=value syntax is deprecated.
Please use --option value (with no =) instead but the command will run and works fine.
I was unable to find a way to run this command without = after the parameter.
Editing the Run scheme's "Arguments Passed on Launch" also worked for me
My $0.02
import Vapor
/// Called before your application initializes.
///
/// [Learn More →](https://docs.vapor.codes/3.0/getting-started/structure/#configureswift)
public func configure(
_ config: inout Config,
_ env: inout Environment,
_ services: inout Services
) throws {
if env == .development {
services.register(Server.self) { container -> EngineServer in
var serverConfig = try container.make() as EngineServerConfig
serverConfig.port = 8989
serverConfig.hostname = "192.168.31.215"
let server = EngineServer(
config: serverConfig,
container: container
)
return server
}
}
//Other configure code
}
It works perfectly on Vapor 3.0.0 RC 2.4.1
You can set the hostname and port with command line flags:
--hostname localhost --port 8080
You could also register your EngineServerConfig in services.
In configure.swift, insert the code below:
let myServerConfig = try EngineServerConfig.detect(from: &env, port: 8081)
services.register(myServerConfig)
This should work for 3.0.0-rc.2.2
In vapor: stable 3.1.10
open: configure.swift
In: public func configure()
Add the following:
// Define Hostname & Port to listen to ...
let myServerConfig = NIOServerConfig.default(hostname: "servers-hostname.local", port: 8080)
services.register(myServerConfig)
What iOS Guy had written needs a bit of modification for Vapor 3.3.1
// Define Hostname & Port to listen to ...
let myServerConfig = NIOServerConfig.default(hostname: "localhost", port: 8081)
services.register(myServerConfig)
So NIOServerConfig.default can be used only with two parameters hostname & port and it can be used if one just wants to change the port number.

AWS-ECS - Communication between containers - Unknown host error

I have two Docker containers.
TestWeb (Expose: 80)
TestAPI (Expose: 80)
Testweb container calls TestApi container. Host can communicate with TestWeb container from port 8080. Host can communicate with TestApi using 8081.
I can get TestWeb to call TestApi in my dev box (Windows 10) but when I deploy the code to AWS (ECS) I get "unknown host" exception. Both the containers work just fine and I can call them individually. But when I call a method that internally makes a Rest call using HttpClient to a method in Container2, it gives the error:
An error occurred while sending the request. ---> System.Net.Http.CurlException: Couldn't resolve host name.
Code:
using (var client = new HttpClient())
{
try
{
string url = "http://testapi/api/Tenant/?i=" + id;
var response = client.GetAsync(url).Result;
if (response.IsSuccessStatusCode)
{
var responseContent = response.Content;
string responseString = responseContent.ReadAsStringAsync().Result;
return responseString;
}
return response.StatusCode.ToString();
}
catch (HttpRequestException httpRequestException)
{
return httpRequestException.Message;
}
}
The following are the things I have tried:
The two containers (TestWeb, TestAPI) are in the same Task definition in AWS ECS. When I inspect the containers, I get the IP address of each of the containers. I can ping container2 from container1 with their IP address. But I can't ping using container2 name. It gives me "unknown host" error.
It appears ECS doesn't use legit docker-compose under the hood, however, their implementation does support the Compose V2 "links" feature.
Here is a portion of my compose file I just ran on ECS that needed this same functionality AND had the same "could not resolve host" error you were getting. The "links" I added fixed my hostname resolution issue on Elastic Container Service!
version: '3'
services:
appserver:
links:
- database:database
- socks-proxy:socks-proxy
This allowed my appserver to communicate TO the database and socks-proxy hostnames. The format is "SERVICE:ALIAS" and it is fine to keep them both the same as a default practice.
In your example it would be:
version: '3'
services:
testapi:
links:
- testweb:testweb
testweb:
links:
- testapi:testapi
AWS does not use Docker compose but provides a interface to add Task Definitions.
Containers that need to communicate together can be put on the same Task definition. Then we can also specify in the links section the containers that will be called from the current container. Each container can be given its container name on the "Host" section of Task Definition. Once I added the container name to the "Host" field, Container1 (TestWeb) was able to communicate with Container2 (TestAPI).

Docker Akka-Http application endpoint not reachable

I have a very basic Akka-http app that is basically not much more than a Hello-world setup - I have defined an endpoint and simply bound it to "localhost" and port "8080":
object Main extends App with Routes {
private implicit val system = ActorSystem()
protected implicit val executor: ExecutionContext = system.dispatcher
protected implicit val materializer: ActorMaterializer = ActorMaterializer()
protected val log: LoggingAdapter = Logging( system, getClass )
log.info( "starting server" )
Http().bindAndHandle( logRequestResult("log",Logging.InfoLevel)( allRoutes ), "localhost", 8080 )
log.info( "server started, awaiting requests.." )
}
(allRoutes is mixed in via Routes, but is just a dummy endpoint that serialises a simple case class to a JSON response)
If I start it up using sbt then the endpoints works fine (http://localhost:8080/colour/red for example).
I am now trying to package it into a Docker container to run it - I have read things like http://yeghishe.github.io/2015/06/24/running-akka-applications.html and have added the sbt-native-package plugin (http://www.scala-sbt.org/sbt-native-packager/formats/docker.html#customize).
Now I run sbt docker:publishLocal
And I can see that the docker image has been created:
REPOSITORY TAG IMAGE ID CREATED SIZE
sample-rest-api 0.0.1 3c6ee44985b4 9 hours ago 714.4 MB
If I now start my image, mapping the 8080 port as follows:
docker run -p 8080:8080 sample-rest-api:0.0.1
I see the log output I normally see on startup, so it looks like it has started ok, however, if I then attempt to access the same URL as previously I now get the response
"Problem loading the page: The connection was reset"
If I check docker ps I see that the image is running, and the ports mapped as expected:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
27848729a425 sample-rest-api:0.0.1 "bin/sample-rest-api" About a minute ago Up About a minute 0.0.0.0:8080->8080/tcp furious_heisenberg
I am running on Ubuntu 16.04 - anyone have any ideas?
Try changing 'localhost' to 0.0.0.0
In the http.bindAndHandle