I'm trying to validate whether error messages returned by API are proper or not.
So, I stored all local error message strings in HashMap errorMessage
.doIf(errorMessages.size()>1) {
exec(session => {
assert(ResponseJSON.contains(errorMessages.get("errorMessage1")))
for ((k,v)<- errorMessages){
assert(ResponseJSON.contains(v))
}
}
I could see the error on console as
hook-3' crashed with 'java.lang.AssertionError: assertion failed', forwarding to the next one
But, the Gatling scenarios are not failing here, what is I'm missing ?
Try using an exitHereIfFailed to exit the scenario.
Related
I have an application using React and axios, and I want to this the following behvior:
Fill a form
submit form
Success message shows
Fill the form again
submit form
While the request is made, the success message should not be showing**
request finish, success message shows again
**the problem is to test this step (6)
I'm using axios-mock-adapter to mock axios. I tried the following approach:
axiosMock
.onPost('/api/auth/alteracaoSenha').replyOnce(204)
.onPost('/api/auth/alteracaoSenha').replyOnce(() => {
expect(screen.queryByText('Success message')).not.toBeInTheDocument()
return [204]
})
When I try this, it works fine. But, if I remove the code that reset the success message to see if the test broke, the test continues to pass.
What I found is that the expect throws an error, but the axios mock catch this error and just return it to the API :/ So the expect line that should file, did not break the test.
Is there any other option to do that test?
I am using Katalon studio with the Navigate to Url action.
It would be useful to me to be able to retry this action until there is no error loading the page, either the 'connection refused' or 404 type error.
After the web page loads succesfully, it is OK to go ahead and execute the rest of my script.
Is there any example of a way to do this?
Try something like this, just change the css selector:
TestObject errorMessage = new TestObject().addProperty('css', ConditionType.EQUALS, 'span.error-message-example')
while (!WebUI.verifyElementVisible(errorMessage, 3, FailureHandling.OPTIONAL)){
WebUI.navigateToUrl('https://example.com')
}
I also have this issue, I found that I kept having 404 or connection reset issue if CPU is over-utilized. However I need to run a number of test cases so it is normal my CPU is overloaded.
I writen the following Chrome retry logic and use this for a year or so, been serving me well for :
for (times in 1..5 ) {
try {
WebUI.openBrowser('')
WebUI.navigateToUrl(url)
//Chrome
WebUI.verifyTextNotPresent("ERR_CONNECTION_RESET", false)
break
} catch ( Exception e) {
println "Cannot navigate to URL, try again in " + times
WebUI.closeBrowser()
WebUI.delay(1)
if(times >= 5){
//False error
WebUI.verifyMatch('1', '0', false)
}
}
}
Basically, if anything failed during initial startup it will try again for 5 time before give up.
See if this is something that helps you as well.
I would like to uses jmeter for api functional testing, the jmeter dashboard reporting is not ideal for functional testing.
I have attempted to integrate extent 2.41.2 reporting with groovy script that validates responses (http and expected response code).
I have attempted to use the idea given in Using extentreports for jmeter test results
However that has failed. I used a js2322 assertion to check for valid responses, but then I get errors whenever attempt to run.
I'm not sure whether it should be setup as post processor step instead of an assertion.
Has anyone got any ideas on how this can be achieved?
You can assert result by using prev which is SampleResult:
prev - (SampleResult) - gives access to the previous SampleResult (if any)
Here's example of checking token exists in response and if not return relevant assertion:
import org.apache.jmeter.assertions.AssertionResult;
boolean assertToken = prev.getResponseDataAsString().contains("token");
prev.setSuccessful(assertToken);
if (!assertToken) {
AssertionResult assertionResult = new AssertionResult("Assertion expected to contain token")
assertionResult.setFailureMessage("Assertion failure message: Test failed: text expected to contain /token/");
assertionResult.setFailure(true);
prev.addAssertionResult(assertionResult);
}
I need to customise the error that shows up when someone did not send the CSRF code with the POST request.
So that no one will know what happened with the error and they will not even try to hack in to the CSRF mechanism.
Hope this is clear
For now Sails.js CSRF hook uses res.forbidden() function to handle wrong CSRF token.
It uses it with this message:
return res.forbidden("CSRF mismatch");
So you could rewrite this response by placing a new file with name forbidden.js into /api/responses
Actually you cound copy this one: https://github.com/balderdashy/sails/blob/master/lib/hooks/responses/defaults/forbidden.js
And add condition to check data before production mode check:
...
else sails.log.verbose('Sending 403 ("Forbidden") response');
if (data == 'CSRF mismatch') {
//Return another response for example:
return res.jsonx(500, {/* data here */});
}
// Only include errors in response if application environment
// is not set to 'production'. In production, we shouldn't
// send back any identifying information about errors.
if (sails.config.environment === 'production') {
...
Anyway as long as you will use development mode for sails. You will see all errors when getting 500 or any other error from sails. But in production mode all error messages will be hidden. And your users wouldn't get any error details. Except of error code.
So in production mode without any changes you will get only HTTP 403 code.
A Gatling scenario with an exec chain. After a request, returned data is saved. Later it's processed and depending on the processing result, it should either fail or pass the test.
This seems like the simplest possible scenario, yet I can't find any reliable info how to fail a test from within an exec block. assert breaks the scenario and seemingly Gatling (as in: the exception throw doesn't just fail the test).
Example:
// The scenario consists of a single test with two exec creating the execChain
val scn = scenario("MyAwesomeScenario").exec(reportableTest(
// Send the request
exec(http("127.0.0.1/Request").get(requestUrl).check(status.is(200)).check(bodyString.saveAs("MyData")
// Process the data
.exec(session => {
assert(processData(session.attributes("MyData")) == true, "Invalid data");
})
))
Above the scenario somewhere along the line "guardian failed, shutting down system".
Now this seems a useful, often-used thing to do - I'm possibly missing something simple. How to do it?
You have to abide by Gatling APIs.
With checks, you don't "fail" the test, but the request. If you're looking for failing the whole test, you should have a look at the Assertions API and the Jenkins plugin.
You can only perform a Check at the request site, not later. One of the very good reasons is that if you store the bodyString in the Sessions like you're doing, you'll end using a lot of memory and maybe crashing (still referenced, so not garbage collectable). You have to perform your processData in the check, typically in the transform optional step.
were you looking for something like
.exec(http("getRequest")
.get("/request/123")
.headers(headers)
.check(status.is(200))
.check(jsonPath("$.request_id").is("123")))
Since the edit queue is already full.
This is already resolved in the new version of Gatling. Release 3.4.0
They added
exitHereIf
exitHereIf("${myBoolean}")
exitHereIf(session => true)
Make the user exit the scenario from this point if the condition holds. Condition parameter is an Expression[Boolean].
I implemented something using exitHereIfFailed that sounds like exactly what you were trying to accomplish. I normally use this after a virtual user attempts to sign in.
exitHereIfFailed is used this way
val scn = scenario("MyAwesomeScenario")
.exec(http("Get data from endpoint 1")
.get(request1Url)
.check(status.is(200))
.check(bodyString.saveAs("MyData"))
.check(processData(session.attributes("MyData")).is(true)))
.exitHereIfFailed // If we weren't able to get the data, don't continue
.exec(http("Send the data to endpoint 2")
.post(request2Url)
.body(StringBody("${MyData}"))
This scenario will abort gracefully at exitHereIfFailed if any of the checks prior to exitHereIfFailed have failed.