I´m implementing my test framework using scalatest and I think I made a mistake using this framework instead of Cucumber
I´m trying to use some sort of features as Scenario outline of cucumber to avoid break DRY
here my problem
feature("Features of mus client") {
scenario("GET message with mus client") {
Given("a Musin message")
val config: Properties = new Properties
config.put("method", "POST")
config.put("encoding", "UTF-8")
config.put("uri", "http://localhost:9083/musClient")
When("I make a request to f2e")
val response = HttpClientTest.request(config, createJSON(READ))
Then("The message it´s returned successfully")
assert(response != null)
}
scenario("POST message with mus client") {
Given("a Musin message")
val config: Properties = new Properties
config.put("method", "POST")
config.put("encoding", "UTF-8")
config.put("uri", "http://localhost:9083/musClient")
When("I make a request to f2e")
val response = HttpClientTest.request(config, createJSON(CREATE))
Then("The message it´s returned successfully")
assert(response != null)
}
As you can see I have two scenarios where the 99% it´s the same steps but a variable that change the request.
Any idea how to do this elegant and efficient in scalatest
And I'm one of those too who chose scalatest over cucumber, cucumber was too much for me(ME) to write the feature file, and then come back to scala/java file and change accordingly. Maintain two files. I actually played cucumber java, scala cucumber might be more fluent. Anyway, I am liking scalatest so far for all my unit testing, component testing and the flow testing.
In case like yours if the properties are common for multiple scenarios and you won't mutate inside scenarios then defining as common property would be fine, as below.
class E2E extends FeatureSpec with GivenWhenThen {
feature("Features of mus client") {
Given("http config")
val config: Properties = new Properties(){{
put("method", "POST") //you are doing POST in both case by the way
put("encoding", "UTF-8")
put("uri", "http://localhost:9083/musClient")
}}
scenario("GET message with mus client") {
When("I make a request to f2e")
val response = HttpClientTest.request(config, createJSON(READ))
Then("The message it´s returned successfully")
assert(response != null)
}
scenario("POST message with mus client") {
When("I make a request to f2e")
val response = HttpClientTest.request(config, createJSON(CREATE))
Then("The message it´s returned successfully")
assert(response != null)
}
}
}
But, You might also want to use property based testing for the only part that changes, property based check was very fluent and readable in spock framework.
property based check in scalatest would look like below where I am testing for two different input parameters. (you neeed import org.scalatest.prop.TableDrivenPropertyChecks._)
class TestE2E extends FeatureSpec with GivenWhenThen {
val requestResponse =
Table(
("request", "response"),
( "GET", "GET-something"),
( "POST", "POST-something")
)
feature("testMe") {
forAll (requestResponse) { (givenRequestFromTable: String, expectedResponseFromTable: String) =>
scenario("for input " + givenRequestFromTable) {
When("input is " + givenRequestFromTable)
val output = testMe(input = givenRequestFromTable)
Then("responseFromTable has something appended to it")
assert(output == expectedResponseFromTable)
}
}
}
def testMe(input: String) : String = {
input + "-something"
}
}
There would be two scenarios based on two given properties,
And for you, the tests would be something as below with property based, hope there's no compile error :)
import org.scalatest.prop.TableDrivenPropertyChecks._
import org.scalatest.prop.Tables.Table
import org.scalatest.{FeatureSpec, GivenWhenThen}
class PaulWritesSpecs extends FeatureSpec with GivenWhenThen {
val requestResponse =
Table(
("httpMethod", "requestType"),
("GET", READ),
("POST", CREATE))
feature("Features of mus client") {
forAll(requestResponse) { (httpMethod: String, requestType: String) => {
scenario(s"$httpMethod message with mus client") {
Given("http config")
val config: Properties = new Properties() {{
put("method", httpMethod)
put("encoding", "UTF-8")
put("uri", "http://localhost:9083/musClient")
}}
When("I make a request to f2e")
val response = HttpClientTest.request(config, createJSON(requestType))
Then("The message it´s returned successfully")
assert(response != null)
}
}
}
}
}
Related
I'm using akka http and streams to fulfill API requests.
When the request is invalid I want to return a 400 and if it's valid, i want to proceed with the computation and return the result afterwards.
The Problem I'm facing is, that the Payload I'm receiving from the POST request is a Source and I cannot convert it into 2 Streams (one for valid and one for invalid input data) and complete the request correct.
path("alarms")(
post(entity(asSourceOf[String]) { message =>
val flow = message.via(Flow[String].map((it) =>
Try(if valid(it) then it else throw Exception("Wrong input"))
))
complete(repository.create(flow).run) // <-- here I only want to pass all events that are valid. For the other events complete(HttpResponse(NotFound, entity = "Invalid input")) should be used
})
)
/// The signature of the repository.create looks like that
def create(message: Source[String, NotUsed]): RunnableGraph[Future[Done]]
You may use the akka-http handleExceptions directive, sth like this:
val exceptionHandler = ExceptionHandler {
case ex: RuntimeException =>
complete(HttpResponse(NotFound, entity = "Invalid input"))
}
path("alarms")(
handleExceptions(exceptionHandler) {
post(entity(asSourceOf[String]) { message =>
val flow = message.via(Flow[String].map((it) =>
Try(if valid(it) then it else throw new RuntimeException("Invalid input"))
))
complete(repository.create(flow).run)
})
}
)
Doc:
https://doc.akka.io/docs/akka-http/current/routing-dsl/directives/execution-directives/handleExceptions.html
https://doc.akka.io/docs/akka-http/current/routing-dsl/exception-handling.html
There is also handleRejections directive for even more control - see https://doc.akka.io/docs/akka-http/current/routing-dsl/directives/execution-directives/handleRejections.html
I've been trying to perform a test that uses a mock Http server to respond and a function that returns a Future[String] or an Exception if the Http server response isn't 200.
I'm trying to achieve a test without using Awaits, but instead AsyncFunSuiteLike.
However the following test seems impossible to resolve without doing it synchronously:
test("Error responses") {
Future.sequence {
NanoHTTPD.Response.Status.values().toList.filter(status => status.getRequestStatus >= 400).map {
status => {
httpService.setStatusCode(status)
val responseBody = s"Request failed with status $status"
httpService.setResponseContent(responseBody)
val errorMessage = s"Error response (${status.getRequestStatus}) from http service: $responseBody"
recoverToExceptionIf[ServiceException] {
myObject.httpCall("123456")
}.map {
ex => assert(ex.getMessage === errorMessage)
}
}
}
}.map(assertions => assert(assertions.forall(_ == Succeeded)))
}
Basically the problem is that when the Futures are tested, the NanoHTTPD is set to the last valued set in the map, so all ex.getMessage are the same. If I run those status codes one by one I do get the desired results, but, is there a way to perform all this in one single Async test?
From the looks of it, NanoHTTPD is stateful, so you have a race between the .set... calls and the .httpCall.
If you can spin up a new httpService within each Future, then you should be able to parallelize the tests (unless the state in question would be shared across instances, in which case you're likely out of luck).
So you'd have something like (replace Status with the type of status in your code and HTTPService with the type of httpService):
// following code composed on the fly and not run through the compiler...
def spinUpHTTPService(status: Status, body: String): Future[HTTPService] = {
// insert the code outside of the test which creates httpService
httpService.setStatusCode(status)
httpService.setResponseContent(body)
httpService
}
test("Error responses") {
Future.sequence(
NanoHTTPD.Response.Status.values().toList.filter(status => status.getRequestStatus >= 400).map { status =>
spinUpHTTPService(status, s"Request failed with status $status")
.flatMap { httpService =>
val errorMessage = s"Error response (${status.getRequestStatus}) from http service: $responseBody"
recoverToExceptionIf[ServiceException] {
myObject.httpCall("123456")
} map {
ex => assert(ex.getMessage === errorMessage)
}
} // Future.flatMap
} // List.map
).map { assertions => assertions.forAll(_ == Succeeded) }
}
I wanted to create a SourceFunction which reads a http stream.
I used ScalaJ which does what I want (it splits the incoming text by \n-s).
Obviously the code works outside Flink, but I get a NullPointerExcetion every time I start it as a Flink job (sometimes immediately sometimes after 1-2 seconds after it transmitted 1-2 elements). It kind of looks like the Http object has some problems.
import org.apache.flink.streaming.api.functions.source.SourceFunction
import scala.io.Source.fromInputStream
import scalaj.http._
class HttpSource(url: String) extends SourceFunction[String] {
#volatile var isRunning = true
override def cancel(): Unit = isRunning = false
override def run(ctx: SourceFunction.SourceContext[String]): Unit =
httpStream(ctx.collect)
private def httpStream(f: String => Unit) = {
val request = Http(url)
request
.execute { inputStream =>
fromInputStream(inputStream)
.getLines()
.takeWhile(_ => isRunning)
.foreach(f)
}
}
}
Here's the exception I usually get:
(Sometimes it's a bit different, for example I tried to make the request value transient, then it's already null when it tries to refer to request)
Caused by: java.lang.NullPointerException
at java.io.Reader.<init>(Reader.java:78)
at java.io.InputStreamReader.<init>(InputStreamReader.java:129)
at scala.io.BufferedSource.reader(BufferedSource.scala:24)
at scala.io.BufferedSource.bufferedReader(BufferedSource.scala:25)
at scala.io.BufferedSource.scala$io$BufferedSource$$charReader$lzycompute(BufferedSource.scala:35)
at scala.io.BufferedSource.scala$io$BufferedSource$$charReader(BufferedSource.scala:33)
at scala.io.BufferedSource.scala$io$BufferedSource$$decachedReader(BufferedSource.scala:62)
at scala.io.BufferedSource$BufferedLineIterator.<init>(BufferedSource.scala:67)
at scala.io.BufferedSource.getLines(BufferedSource.scala:86)
at flinkextension.HttpSource$$anonfun$httpStream$1.apply(HttpSource.scala:21)
at flinkextension.HttpSource$$anonfun$httpStream$1.apply(HttpSource.scala:19)
at scalaj.http.HttpRequest$$anonfun$execute$1.apply(Http.scala:323)
at scalaj.http.HttpRequest$$anonfun$execute$1.apply(Http.scala:323)
at scalaj.http.HttpRequest$$anonfun$toResponse$3.apply(Http.scala:388)
at scalaj.http.HttpRequest$$anonfun$toResponse$3.apply(Http.scala:380)
at scala.Option.getOrElse(Option.scala:121)
at scalaj.http.HttpRequest.toResponse(Http.scala:380)
at scalaj.http.HttpRequest.scalaj$http$HttpRequest$$doConnection(Http.scala:360)
at scalaj.http.HttpRequest.exec(Http.scala:335)
at scalaj.http.HttpRequest.execute(Http.scala:323)
at flinkextension.HttpSource.httpStream(HttpSource.scala:19)
at flinkextension.HttpSource.run(HttpSource.scala:14)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:87)
at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:55)
at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:95)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:263)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:702)
at java.lang.Thread.run(Thread.java:748)
Everything else seems to be working fine, when I don't use a http request, but something else like file read with the same InputStream type, just a plain while loop with strings or even when I use single http requests, which aren't streaming.
I feel like I'm missing some theoretical background, maybe flink does something in the background which destroys the Http object or the InputStream, but I didn't find anything in the documentation.
UPDATE #1:
If I put a null check into the lambda, the job usually exits immediately, sometimes processes a few elements, sometimes timeouts after hanging for a minute. Here's this version of the httpStream function:
private def httpStream(f: String => Unit) = {
val request = Http(url)
request
.execute { inputStream =>
if (inputStream == null) println("null inputstream")
else {
println("not null inputstream")
fromInputStream(inputStream)
.getLines()
.takeWhile(_ => isRunning)
.foreach(f)
}
}
}
UPDATE #2:
The code actually works in distributed mode and with StreamExecutionEnvironment.createLocalEnvironment()
I only experience the issue if I use start-local.sh and submit the jar to it.
Helo,
at the beginning i wold like to apologize for my english :)
akka=2.3.6
spray=1.3.2
scalatest=2.2.1
I encountered strange behavior of teting routes, which asks actors in handleWith directive,
I've route with handleWith directive
pathPrefix("firstPath") {
pathEnd {
get(complete("Hello from this api")) ~
post(handleWith { (data: Data) =>{ println("receiving data")
(dataCalculator ? data).collect {
case Success(_) =>
Right(Created -> "")
case throwable: MyInternalValidatationException =>
Left(BadRequest -> s"""{"${throwable.subject}" : "${throwable.cause}"}""")
}
}})
}
}
and simple actor wchich always responds when receive object Data and has own receive block wrapped in LoggingReceive, so I should see logs when message is receiving by actor
and i test it using (I think simple code)
class SampleStarngeTest extends WordSpec with ThisAppTestBase with OneInstancePerTest
with routeTestingSugar {
val url = "/firstPath/"
implicit val routeTestTimeout = RouteTestTimeout(5 seconds)
def postTest(data: String) = Post(url).withJson(data) ~> routes
"posting" should {
"pass" when {
"data is valid and comes from the identified user" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis()).asJson) ~> check {
print(entity)
status shouldBe Created
}
}
"report is valid and comes from the anonymous" in {
postTest(correctData.copy(createdAt = System.currentTimeMillis(), adid = "anonymous").asJson) ~> check {
status shouldBe Created
}
}
}
}
}
and behavior:
When I run either all tests in package (using Intellij Idea 14 Ultimate) or sbt test I encounter the same results
one execution -> all tests pass
and next one -> not all pass, this which not pass I can see:
1. fail becouse Request was neither completed nor rejected within X seconds ( X up tp 60)
2. system console output from route from line post(handleWith { (data: Data) =>{ println("receiving data"), so code in handleWith was executed
3. ask timeout exception from route code, but not always (among failed tests)
4. no logs from actor LoggingReceive, so actor hasn't chance to respond
5. when I rerun teststhe results are even different from the previous
Is there problem with threading? or test modules, thread blocking inside libraries? or sth else? I've no idea why it isn't work :(
I'm trying to implement "request based" sessions in scalaquery in the play framework. I create a session using scalaquery and attempt to store it in the current http context, as follows:
def withTransaction[A](bp: BodyParser[A])(f: Request[A] => Result): Action[A] = {
Action(bp) {
request =>
val context = Http.Context.current()
val session = createSession()
session.conn.setAutoCommit(false)
context.args.put("scalaquery.session", session)
try {
val result = f(request)
session.conn.commit()
result
}
catch {
case t: Throwable =>
session.conn.rollback()
throw t
}
finally {
session.close()
context.args.remove("scalaquery.session")
}
}
}
then i wrap my actions in my controllers like:
withTransaction(parse.anyContent) {
Action {
//code that produces a result here
}
}
However, it crashes in the following line saying:
val context = Http.Context.current()
[RuntimeException: There is no HTTP Context available from here.]
So, why is the context not available? This code is called directly by the framework, so shouldn't the context be set by the time this code executes? Or am i using the wrong way for accessing the context?
EDIT: The "session" is of type org.scalaquery.session.Session. The reason why i want to set it in the HttpContext is so that the wrapped actions can access it in an "http scoped" fashion, i.e. That each request stores their session separately, yet all services that need a session can find it in a public scope that is separated per request.
I think the problem is you're using the Java API with the Scala controller. Http.Context is only set if you're using the Java controller. Have you considered using the Scala Session API?
Also, another question is, why do you need to store the session in the context? I see you just remove it at the end anyway. If what you need is for the sub-actions to be able to access the session, you could just pass it in the function.
I'm just going to assume session is of type Session
def withTransaction[A](bp: BodyParser[A])(f: Session => Request[A] => Result): Action[A] = {
Action(bp) {
request =>
val session = createSession()
session.conn.setAutoCommit(false)
try {
val result = f(session)(request)
session.conn.commit()
result
}
catch {
case t: Throwable =>
session.conn.rollback()
throw t
}
finally {
session.close()
}
}
}
and your sub-actions would be
withTransaction(parse.anyContent) { session => request =>
//code that produces a result here
}
you don't need to wrap this in Action anymore since it's already wrapped by withTransaction