How to fail a Gatling test from within "exec"? - scala

A Gatling scenario with an exec chain. After a request, returned data is saved. Later it's processed and depending on the processing result, it should either fail or pass the test.
This seems like the simplest possible scenario, yet I can't find any reliable info how to fail a test from within an exec block. assert breaks the scenario and seemingly Gatling (as in: the exception throw doesn't just fail the test).
Example:
// The scenario consists of a single test with two exec creating the execChain
val scn = scenario("MyAwesomeScenario").exec(reportableTest(
// Send the request
exec(http("127.0.0.1/Request").get(requestUrl).check(status.is(200)).check(bodyString.saveAs("MyData")
// Process the data
.exec(session => {
assert(processData(session.attributes("MyData")) == true, "Invalid data");
})
))
Above the scenario somewhere along the line "guardian failed, shutting down system".
Now this seems a useful, often-used thing to do - I'm possibly missing something simple. How to do it?

You have to abide by Gatling APIs.
With checks, you don't "fail" the test, but the request. If you're looking for failing the whole test, you should have a look at the Assertions API and the Jenkins plugin.
You can only perform a Check at the request site, not later. One of the very good reasons is that if you store the bodyString in the Sessions like you're doing, you'll end using a lot of memory and maybe crashing (still referenced, so not garbage collectable). You have to perform your processData in the check, typically in the transform optional step.

were you looking for something like
.exec(http("getRequest")
.get("/request/123")
.headers(headers)
.check(status.is(200))
.check(jsonPath("$.request_id").is("123")))

Since the edit queue is already full.
This is already resolved in the new version of Gatling. Release 3.4.0
They added
exitHereIf
exitHereIf("${myBoolean}")
exitHereIf(session => true)
Make the user exit the scenario from this point if the condition holds. Condition parameter is an Expression[Boolean].

I implemented something using exitHereIfFailed that sounds like exactly what you were trying to accomplish. I normally use this after a virtual user attempts to sign in.
exitHereIfFailed is used this way
val scn = scenario("MyAwesomeScenario")
.exec(http("Get data from endpoint 1")
.get(request1Url)
.check(status.is(200))
.check(bodyString.saveAs("MyData"))
.check(processData(session.attributes("MyData")).is(true)))
.exitHereIfFailed // If we weren't able to get the data, don't continue
.exec(http("Send the data to endpoint 2")
.post(request2Url)
.body(StringBody("${MyData}"))
This scenario will abort gracefully at exitHereIfFailed if any of the checks prior to exitHereIfFailed have failed.

Related

Gatling: Producer and consumer users

I have a load test where three sets of users create something and a different set of users perform some actions on them.
What is the recommended way to co-ordinate this behaviour in Gatling?
I'm currently using an object which contains a LinkedBlockingQueue which the "producers" put the ID and consumers take, see below.
However, it causes the test to hang after ~20s (targeting 1tps).
I've also tried using poll with a timeout, but instead of hanging the poll almost always fails (after 30s) or causes a hang if the timeout is larger (1m+).
This seems to be because all the threads are blocked waiting for something from the queue so isn't compatible with the way Gatling tests run (i.e. not 1 thread per user). Is there a non-blocking way to wait in the Gatling DSL?
Producer.scala
// ...
scenario("Produce stuff")
.exec(/* HTTP call which extracts an ID*/)
.exec(session => Queue.ids.put(session("my-id").as[String])
// ...
Consumer.scala
// ...
scenario("Consume stuff")
.exec(session => session.set("my-id", Queue.ids.take()))
.exec(/* HTTP call which users ID*/)
// ...
Queue.scala
object Queue {
val ids = new LinkedBlockingQueue[String]()
}
As an alternative I've tried to use the application functionality but it seems a harder problem to ensure that each user picks a unique item from the app.
Acknowledging this is all a hack, my current solution in Consumer.scala is:
doIf(_ => Queue.ids.size() < MIN_COUNT)(
pause(30) // wait for 30s if queue is initially too small
)
.doWhile(_ => Queue.ids.size() >= MIN_COUNT)(
exec(session => session.set("my-id", Queue.ids.take()))
.exec(...)
.pause(30)
)

Play ws scala server hangs up on request after 120seconds - which options to use?

I am pretty sure that this is a config problem, so I'll post my code and the relevant application.conf options of my play app.
I have a play server that needs to interact with another server "B" (basically multi-file upload to B). The interaction happens inside an async -Action which should result in an OK with B's response on the upload. This is the reduced code:
def authenticateAndUpload( url: String) = Action.async( parse.multipartFormData) { implicit request =>
val form = authForm.bindFromRequest.get
val (user, pass) = (form.user, form.pass)
//the whole following interaction with the other server happens in a future, i.e. login returns a Future[Option[WSCookie]] which is then used
login(user, pass, url).flatMap {
case Some(cookie) => //use the cookie to upload the files and collect the result, i.e. server responses
//this may take a few minutes and happens in yet another future, which eventually produces the result
result.map(cc => Ok(s"The server under url $url responded with $cc"))
case None =>
Future.successful(Forbidden(s"Unable to log into $url, please go back and try again with other credentials."))
}
}
I am pretty sure that the code itself works since I can see my server log which nicely prints B's responses every few seconds and proceeds until everything is correctly uploaded. The only problem is that the browser hangs up with a server overloaded message after 120s which should be a play default value - but for which config parameter?
I tried to get rid of it by setting every play.server.http. timeout option I could get my hands on and even decided to use play.ws, specific akka, and other options of which I am quite sure that they are not necessary... however the problem remains, here is my current application.config part:
ws.timeout.idle="3600s"
ws.timeout.request ="3600s"
ws.timeout.response="3600s"
play.ws.timeout.idle="3600s"
play.ws.timeout.request="3600s"
play.ws.timeout.response="3600s"
play.server.http.connectionTimeout="3600s"
play.server.http.idleTimeout="3600s"
play.server.http.requestTimeout="3600s"
play.server.http.responseTimeout="3600s"
play.server.http.keepAlive="true"
akka.http.host-connection-pool.idle-timeout="3600s"
akka.http.host-connection-pool.client.idle-timeout= "3600s"
The browser hang up happened both on Safari and Chrome, where Chrome additionally started a second communication with B after about 120 seconds - also both of these communications succeeded and produced the expected logs, only the browsers had both hang up.
I am using Scala 2.12.2 with play 2.6.2 in an SBT environment, the server is under development, pre-compiled but then started via run - I read that it may not pick up the application.conf options - but it did on some file size customizing. Can someone tell me the correct config options or my mistake on the run process?

Spray.io test response not matching actual output

I'm trying to set up some tests for an API made by a coworker with spray.io, and I'm encountering some odd behavior. When a request results in an error for any reason, we want to return a JSON value along the lines of:
{"status":false,"message":"useful message here"}
This happens just fine in the actual browser. I have navigated to an unhandled route in the web browser, and I get the desired JSON value. So, I want to test this. Now, since I'm new to spray.io, I started off with the very simple test:
"leave GET requests to root path unhandled" in {
Get() ~> myRoute ~> check {
handled must beFalse
}
}
This went fine, no problems. Since it's my first time playing with spray.io, I looked at some of the sample tests for testing false routes, and wrapped myRoute with sealRoute() so I could check the response without failing tests:
"leave GET requests to root path unhandled" in {
Get() ~> sealRoute(myRoute) ~> check {
handled must beTrue
}
}
This also works fine. So, I decided to just make sure the text of the response was usable with this, before I went to the trouble of parsing JSON and verifying individual values:
"leave GET requests to root path unhandled" in {
Get() ~> sealRoute(myRoute) ~> check {
responseAs[String] contains "false"
}
}
This is failing. To investigate, I threw a simple line of code in to log the actual value of responseAs[String] to a file, and I got this:
The requested resource could not be found.
Can anyone tell me what I'm doing wrong? I'm thinking that one of the following is occurring:
responseAs[String] is doing more than taking the exact response and giving it back to me, applying some type of filter along the way
The framework itself is not fully evaluating the query, but rather making a mockup object for the test framework to evaluate, and therefore not executing the desired 'turn errors to json' methods that my co-worker has implemented
I have tried searching google and stack overflow specifically for similar issues, but I'm either not putting in the right queries, or most other people are content to have the default error messages and aren't trying to test them beyond checking handled must beFalse.
Edit - This is the relevant part of the RejectionHandler:
case MissingQueryParamRejection(paramName) :: _=>
respondWithMediaType(`application/json`) {
complete(BadRequest, toJson(Map("status" -> false, "message" -> s"Missing parameter $paramName, request denied")))
}
Okay, so with insight from here and a coworker, the problem has been found:
Basically, the custom RejectionHandler was defined within our custom Actor object, and it wasn't coming into scope in the tests. To resolve this, the following steps were taken:
Moved the definition for the custom RejectionHandler into its own object in a separate file (as it had to do its own imports, it was causing a "encountered unrecoverable cycle resolving import" error)
Imported this new object into both the original file and the test spec.
(fun fact - http://spray.io/documentation/1.2.2/spray-routing/key-concepts/rejections/#rejectionhandler seems to demonstrate the RejectionHandler as a top-level object but you can't have top-level implicit vals in Scala, hence the need for an object)

store and setRequest

I have a jobque mechanism in ZF.
The jobque simlpy stores the the function call (Class, Method and params) and later executes it as CLI daemon. The daemon works, however at places the application looks for information from the request object, and when called from the CLI these places fail, or get no info.
I would like to store the original request object together with the job and when the job is processed set the request object back as if the job was done by the originall request, somethin along the line of the following pseudo code:
$ser_request = serialize(Zend_Controller_Front::getInstance ()->getRequest ());
-->save to db
-->retrive from db
$ZCF= new Zend_Controller_Front;
$ZCF::getInstance ()->setRequest (unserialize($ser_request))
The aim is to store and replay the jobs later withouth having to change the rest of the application.
Any suggestions how to do that?
I am not sure if this works, but here's an idea. Try to implement _sleep and _wakeup magic methods for the request object. Haven't tried it out, but maybe it's at least a starting solution.

Rack Warden: how can I access the message passed to the fail! method?

When failing an authentication strategy in Warden, a message can be passed to the fail! method. How can I access this message in my default failure application action such that I can display it in a flash message? I tried request.env['warden'].message, but after looking at the code for Warden::Proxy this variable is only set for winning strategies. Thoughts?
When using the fail or fail! methods, you can obtain the failure message from the environment warden object.
Example:
def unauthenticated
flash[:alert] = env["warden"].message unless env["warden"].message.blank?
end
http://www.rubydoc.info/github/hassox/warden/Warden/Proxy#message-instance_method
Perhaps a different way of failing strategies (from recent documentation), is to use throw(:warden, :foo => 'bar'). Anything you throw with will be available in your failure application through request.env['warden.options'], including a string of the desired url as well. HTH