I am developing an API, using apiary.io and Swagger. I want to test the API using dredd, and for normal API responses (HTTP status code 200) this is fine. However, when the API returns different status code based on parameters (e.g. 403), dredd reports the test as failed. I would like to mark the test as successful, with an optional logging message.
I've tried this:
hooks.afterEach(function(transaction){
if (transaction.real){
switch (transaction.real.statusCode){
case "400":
case "403":
case "406":
transaction.fail=false;
hooks.log('Failed, but expected by the API');
}
}
});
However this doesn't work, the test is still marked as failed. What am I missing?
Is your API description document in OAS3? You might be experiencing this issue. Dredd should be marking the non-2xx cases as skipped automatically in case of OAS2, but it is not doing this yet for OAS3, support for that format is still experimental.
A workaround would be to set the transaction as skipped yourself in a before hook:
hooks.beforeEach(function(transaction){
if (transaction.real){
switch (transaction.real.statusCode){
case "400":
case "403":
case "406":
transaction.skip = true;
}
}
});
Related
I would like to uses jmeter for api functional testing, the jmeter dashboard reporting is not ideal for functional testing.
I have attempted to integrate extent 2.41.2 reporting with groovy script that validates responses (http and expected response code).
I have attempted to use the idea given in Using extentreports for jmeter test results
However that has failed. I used a js2322 assertion to check for valid responses, but then I get errors whenever attempt to run.
I'm not sure whether it should be setup as post processor step instead of an assertion.
Has anyone got any ideas on how this can be achieved?
You can assert result by using prev which is SampleResult:
prev - (SampleResult) - gives access to the previous SampleResult (if any)
Here's example of checking token exists in response and if not return relevant assertion:
import org.apache.jmeter.assertions.AssertionResult;
boolean assertToken = prev.getResponseDataAsString().contains("token");
prev.setSuccessful(assertToken);
if (!assertToken) {
AssertionResult assertionResult = new AssertionResult("Assertion expected to contain token")
assertionResult.setFailureMessage("Assertion failure message: Test failed: text expected to contain /token/");
assertionResult.setFailure(true);
prev.addAssertionResult(assertionResult);
}
I am using Spring Cloud Contract to create stubs for a REST service so I can test with a REST client. I have the stub runner working within a Spring Boot application, and it all works as expected. The problem I am having is that I'd like to see elements of the requests in the responses, to better simulate the eventual behavior of the REST service. For example, in this contract, I'd like what is passed in the "code" field in the request to appear regurgitated in the response:
package contracts
org.springframework.cloud.contract.spec.Contract.make {
request {
method('POST')
url $("/resource")
body ([
code : $(client(regex('[a-zA-Z0-9]{5,32}')))
])
}
response {
status 200
body([
code: ???
])
}
}
Obviously the input "code" can be anything that matches the regular expression, and so the actual value is unknown until runtime. Is there anything i can put in place of "???" to return the code submitted in the request ? I tried accessing, for example:
request.body.serverValue['code']
but that value it seems is generated at compile time, perhaps to enable the auto-generation of tests in ContractVerifierTest.java under generated-test-sources.
Can this be done ? Is this an appropriate use of Spring Cloud Contract ?
Currently, it's not supported. We prefer an approach where you have simpler contracts. If you need in a response a value from the request just hard code both the request and the response parts of the contract.
You can, however, file an issue and we can try to think of something in the future releases.
UPDATE:
With version 1.1.0 that's already possible. Check out the docs for more info - http://cloud.spring.io/spring-cloud-static/spring-cloud-contract/1.1.0.RELEASE/#_referencing_request_from_response
I'm currently running SailsJS on a Raspberry Pi and all is working well however when I execute a sails.models.nameofmodel.count() when I attempt to respond with the result I end up getting a empty response.
getListCount: function(req,res)
{
var mainsource = req.param("source");
if(mainsource)
{
sails.models.gatherer.find({source: mainsource}).exec(
function(error, found)
{
if(error)
{
return res.serverError("Error in call");
}
else
{
sails.log("Number found "+found.length);
return res.ok({count: found.length});
}
}
);
}
else
{
return res.ok("Error in parameter");
}
},
I am able to see in the logs the number that was found (73689). However when responding I still get an empty response. I am using the default stock ok.js file, however I did stick in additional logging to try to debug and make sure it is going through the correct paths. I was able to confirm that the ok.js was going through this path
if (req.wantsJSON) {
return res.jsonx(data);
}
I also tried adding .populate() to the call before the .exec(), res.status(200) before I sent out a res.send() instead of res.ok(). I've also updated Sails to 11.5 and still getting the same empty response. I've also used a sails.models.gatherer.count() call with the same result.
You can try to add some logging to the beginning of your method to capture the value of mainsource. I do not believe you need to use an explicit return for any response object calls.
If all looks normal there, try to eliminate the model's find method and just evaluate the request parameter and return a simple response:
getListCount: function(req, res) {
var mainsource = req.param("source");
sails.log("Value of mainsource:" + mainsource);
if (mainsource) {
res.send("Hello!");
} else {
res.badRequest("Sorry, missing source.");
}
}
If that does not work, then your model data may not actually be matching on the criteria that you are providing and the problem may lie there; in which case, your response would be null. You mentioned that you do see the resulting count of the query within the log statement. If the res.badRequest is also null, then you may have a problem with the version of express that is installed within sailsjs. You mention that you have 11.5 of sailsjs. I will assume you mean 0.11.5.
This is what is found in package.json of 0.11.5
"express": "^3.21.0",
Check for any possible bugs within the GitHub issues for sailsjs regarding express and response object handling and the above version of express.
It may be worthwhile to perform a clean install using the latest sailsjs version (0.12.0) and see if that fixes your issue.
Another issue may be in how you are handling the response. In this case .exec should execute the query immediately (i.e. a synchronous call) and return the response when complete. So there should be no asynchronous processing there.
If you can show the code that is consuming the response, that would be helpful. I am assuming that there is a view that is showing the response via AJAX or some kind of form POST that is being performed. If that is where you are seeing the null response, then perhaps the problem lies in the view layer rather than the controller/model.
If you are experiencing a true timeout error via HTTP even though your query returns with a result just in time, then you may need to consider using async processing with sailjs. Take a look at this post on using a Promise instead.
I need to customise the error that shows up when someone did not send the CSRF code with the POST request.
So that no one will know what happened with the error and they will not even try to hack in to the CSRF mechanism.
Hope this is clear
For now Sails.js CSRF hook uses res.forbidden() function to handle wrong CSRF token.
It uses it with this message:
return res.forbidden("CSRF mismatch");
So you could rewrite this response by placing a new file with name forbidden.js into /api/responses
Actually you cound copy this one: https://github.com/balderdashy/sails/blob/master/lib/hooks/responses/defaults/forbidden.js
And add condition to check data before production mode check:
...
else sails.log.verbose('Sending 403 ("Forbidden") response');
if (data == 'CSRF mismatch') {
//Return another response for example:
return res.jsonx(500, {/* data here */});
}
// Only include errors in response if application environment
// is not set to 'production'. In production, we shouldn't
// send back any identifying information about errors.
if (sails.config.environment === 'production') {
...
Anyway as long as you will use development mode for sails. You will see all errors when getting 500 or any other error from sails. But in production mode all error messages will be hidden. And your users wouldn't get any error details. Except of error code.
So in production mode without any changes you will get only HTTP 403 code.
The mojolicious application that I use is JSON based, that is the interaction between the client and the server is more of an exchange of JSON structured data.
I am trying to implement a standard way of handling errors with proper HTTP response code when an error occurs during one of the REST calls. What is the best way of implementing such a standard and where do I do it?
I see a couple of ways of doing it
Create a class and list all the error response and its associated content, a call could be made to this class with the response code, which would return the JSON structure(combination of hashes and arrays) containing all the associated entry, then use the render_json() method in controller and return this as a response to the client
I can create a table in the Database with entry for all the fields that are required for the response, use the filed to access the JSONstructure, create the appropriate response and use render_json() in controller and return this as a response to the client.
Example of error response might be like
{
"Message": "The requested resource is not found"
"Type" : "http://this.is.an.error.com/error/resource_not_found",
"ErrorCode" : 404,
"Created" : "2012-11-05T11:59:29-05:00",
"Request" : "GET /types/Foo/instances"
}
What is the right way of standardizing such a response?
As titanofold mentioned, I'd go for option 2.
Regarding error codes, try to stick with standard HTTP Response Status Codes.
Besides setting the ErrorCode property in your JSON, you should send the status code in the response header because:
you can treat errors in a single place - the error callback of your javascript function
in the future you might have other consumers of your backend (mobile apps for example)
this is why they have been invented
You can achieve that extremely simple with Mojolicious:
$self->render_json( {
Message => "The requested resource is not found",
Type => "http://this.is.an.error.com/error/resource_not_found",
ErrorCode => 404,
Created => "2012-11-05T11:59:29-05:00",
Request => "GET /types/Foo/instances",
},
status => 404);
The wonderful things about standards are that there are so many to choose from, and if you don't like any of them you can make your own.
As to the REST structure, that's up to you. I would go for the generic 'code' rather than 'ErrorCode' as you should return a code on success, too.
For your method options, I'd go with option 2.
I would also opt for option 2. But I do not understand the need for the error details to be part of the database. I would rather suggest you use a the OO concept of base class holding all the error details and the inheriting it to other classes, making sure you have access to it.