I am currently writing REST API automation test scripts. As most of the research article suggests that we should write DAMP (Descriptive And Meaningful Phrases) tests that promote readability. However, I feel that there are a lot of duplicate codes in my tests and in an attempt to remove the duplicates I end up with 'DRY' (Don't repeat yourself) code which tends to dependency tests. So I am a bit confused about which approach to use? I would really appreciate it If anyone can give me some suggestions on this?
A general rule is to keep the code related to the Test Objective DAMP, having everything else DRY.
To simplify the rule, code related to the Test Objective may reference to:
Actions that DIRECTLY impact the expected result
Data parameters that DIRECTLY affect the expected result
Code NOT related to the Test Objective may reference to:
Actions that do not impact the expected result DIRECTLY
(Example: authorization for the tests not related to login)
Configuration data and data parameters that does not affect the expected results DIRECTLY
(Example: Base URL, login and password for the tests not related to authorization)
My recommendations are:
re-use payloads (JSON or XML) from files where possible
sign-in flows that set an Authorization header should be re-usable
do not combine API requests to different end-points into a re-usable Scenario
even for the same end-point, for very different payloads (e.g. boundary / error conditions) use a separate Scenario for each
use Scenario Outlines for data-driven tests
Also please refer this answer for a good example of what NOT to do: https://stackoverflow.com/a/54126724/143475
Related
Does anybody know how to get list of categories (provided with 'where' filter to nunit-console) at runtime?
Depending on this, I need to differently initialize the test assembly.
Is there something static like TestExecutionContext that may contain such information?
The engine doesn't pass information on to the framework about "why" it's running a particular test... i.e. if it's running all tests or if it was selected by name or category. That's deliberately kept as something the test doesn't know about with the underlying philosophy being that tests should just run based on the data provided to them.
On some platforms, it's possible to get the command-line, which ran the test. With that info you could decode the various options and make some conclusions but it seems as if it would be easier to restructure the tests so they didn't need this information.
As a secondary reason, it would also be somewhat complicated to supply the info you want and to use it. A test may have multiple categories. Imagine a test selected because two categories matched, for example!
Is it possible that what you really want to do is to pass some parameters to your tests? There is a facility for doing that of course.
I think this is a bit of an XY problem. Depending on what you are actually trying to accomplish, the best approach is likely to be different. Can you edit to tell us what you are trying to do?
UPDATE:
Based on your comment, I gather that some of your initialization is both time-consuming and not needed unless certain tests are run.
Two approaches to this (or combine them):
Do less work in all your initialization (i.e. TestCase, TestCaseSource, SetUpFixture. It's generally best not to create your classes under test or initialize databases. Instead, simply leave strings, ints, etc., which allow the actual test to do the work IFF it is run.
2.Use a SetUpFixture in some namespace containing all the tests, which require that particular initialization. If you dont' run any tests from that namespace, then the initialization won't be done.
Of course both of the above may entail a large refactoring of your tests, but the legacy app won't have to be changed.
I need some advice on how to best structure the requests and the responses for my Rest API.
I'm mostly trying to limit myself to CRUD operations on one resources and I work with one object: for example if the ressource is "book" I end up with the following actions in the controller
[HttpPost("books")] Book Create(Book book)
[HttpGet("books")] Book Get(int id)
This is relatively strait forward.
Now for a more complex example for the creation of a resource, I need to receive a complexe object different from my ressource and return an object containing the resource and extra data
For example for the Order resource I have a the following action in the controller:
[HttpPost("/order")] CreateOrderResponse CreateOrder(CreateOrderRequest createOrderRequest)
Here my action will use the "CreateOrderRequest" object to create to build an Order.
Then I would like to return a "createOrderResponse" object which contains the Order but also extra information that the client needs.
I'm not sure this is the best way to go, any advice ?
Thanks in advance for your help
I prefer the following:
[HttpPost("/order")] CreateOrderResponse CreateOrder(CreateOrderRequest createOrderRequest)
And here is why:
By this method, you are able to protect your public API from implementation details. If you expose your model to your API then you cannot make the same guarantee.
You can also make your validations specific to the request format. In some cases, you might require one subset of your model when creating a record and another subset when editing data. This approach will allow you to handle that scenario as well.
Security. Were you going to add that Book right to a DbContext and save it? Or attach it and update directly? Those would be potential issues from security and data quality perspectives.
But there are downsides:
This approach is time consuming. It may not be worth the time invested if you are writing something as a learning exercise or a quick implementation. And it adds complexity. But then, you might find complexity when you realize your Book object is insufficent in all cases.
You will feel like there is duplicate code in different places. The code may appear to be the same, but the use cases are actually different and may diverge over time. Having a Book parameter will be a liability at that point.
Very general issue in large integration of verification environment.
Our verification development involves large group across different time zone.
Group has preference to use $plusargs instead factory mechanism.
Probably main reason it is hard to set factory from command line processor,
we have more layers of scripts to start simulation.
Recently i found that same string been used in different environment to control behavior of environment. In this case two different score board used same string to disable some checking and test pass. Both those environment some time created at run time. Also some time it is OK to re-use same string, and it will require owner to be involved.
Is there any way to find duplication like this from final elaborated model, and provide locations in code as a warning?
I thought create our own wrapper, but problem that we are integrating some code that we are not owners as in this case was.
Thanks,
This is a perfect example of how people think they can get things done quicker by not following the recommended UVM methodology and instead create time consuming complexity later on.
I see at least two possible options.
Write a script that searches the source code for $plusargs and hopefully they have used string literals for you to trace for duplicates.
You can override $plusargs with PLI code and have it trace duplicates.
The choice depends on wether you are better at writing Perl/Python or C code.
As a general rule for any RESTful CRUD operation, I follow these steps:
Validating information on client-side
Sending required information in JSON format to the server (possibly a web service)
Validating information on the server
Doing the operation
Returning JSON as the result of operation
Updating DOM based on server's response
Though this list is general, I think it's the most complete list. The only problem is that, I do it for any and every operation. I mean, DRY (don't repeat yourself) tells us to stop repeating things. Is it considered a repetition? Or should we follow these steps always?
Well, you can skip validating the data client-side if you wish…
Seriously, those are the necessary minimum for doing a lot of things; you must validate server side to prevent a whole host of potential problems and the other parts are just fundamental. OK, you can skip the sending to the server, but then you're not interacting with a REST service in the first place. You could also skip updating the DOM, but then you're not showing the results. In other words, every step of that sequence serves its own purpose that is independent from the other ones: they're not redundant.
But that doesn't mean that you should ignore DRY. Not at all. Instead, you should factor out as much of that code as possible into a single place so as to keep the number of repetitions to a minimum. (Maybe even find a framework to do some of that for you.)
I am hoping that Combinator parsers, (http://debasishg.blogspot.com/2008/04/external-dsls-made-easy-with-scala.html), will work for a design to process the routing rules for a REST service that is implemented with Scalatra,(http://tutorialbin.com/tutorials/80408/infoq-scalatra-a-sinatra-like-web-framework-for-scala).
This REST service is to serve as a proxy so external applications can get access to services within the firewall, as it will have additional layers of security that can be customized for the business requirements of each REST service.
So, if a person wants to access their class schedule there will be less security than if you want to look at the transcript of someone.
I would like the rules for where to go to actually get the information, and how to return it, as well as what security is needed, in a DSL.
But, the first problem is how to dynamically change the routing rules for the REST service based on a DSL, as I am trying to create a framework that doesn't require a great deal of recompiling to add new rules, but just write the appropriate scripts and just let it be processed.
So, can a DSL be implemented using the Combinator Parser, in Scala, that will allow JAX-RS (http://download.oracle.com/javaee/6/tutorial/doc/giepu.html) to have dynamically changed routing?
UPDATE:
I haven't designed the language yet, but this is what I am trying to do:
route /transcript using action GET to
http://inside.com/transcript/{firstparam}/2011/{secondparam}
return json encrypt with public key from /mnt/publickey.txt
for /education_cost using action GET combine http://combine.com/SOAP/costeducate with
http://combine.com/education_benefit/2010 with
http://combine.com/education_benefit/2011 return html
These are two possible ideas where rules for a request for a transcript is sent to a different site, such as within a firewall, and the data is encrypted and returned.
The second would be more complicated in that the results of a SOAP and two REST requests will be combined, and there would need to be additional commands on how this is combined, but the idea is to put all of this in files that can be parsed on the fly.
If I used Groovy then some new classes could be generated for the routing, which would remove some performance hits, but I think using Scala would be the best bet, even if I took a performance hit.
My hope is to make a framework that is more maintainable so new routing rules can be written by people that don't know any OOP or functional languages, but the specifications could be written using Specs (http://code.google.com/p/specs/) so that the functional side could be certain that their requirements are tested on a regular basis.
UPDATE 2:
When I start working on a design I may intuitively understand some options, but not know why. Today I realized that the reason that Groovy may be a better fix for this is that I could then generate the classes for routing, using the metaprogramming (http://www.justinspradlin.com/programming/groovy-metaprogramming-adding-behavior-dynamically/), then I would be able to use Scala or Groovy to dynamically use the routing that was generated. I am not certain how to get Scala to generate the classes if they don't already exist.
In Groovy, as well as some other languages, as shown here (http://langexplr.blogspot.com/2008/02/handling-call-to-missing-method-in.html) if a method is missing you can dynamically generate the method and it will henceforth exist, so it will be missing one time.
It almost seems that I should be mixing Groovy with Java to make this work, but then the result may be that some of the code is in Scala and some in Java, for the routing of REST services.
Splitting the question in two parts:
can a DSL be implemented using the Combinator Parser
Yes. There are things that cannot be implemented using a combinator parser, or even other kinds of parser. For instance, Perl itself cannot be parsed (it must be evaluated). And combinator parsers are also not particularly good for complex languages (such as Scala -- its compiler is not based on combinator parsers), or if you demand top performance (such as the compilers used to compile hundreds of thousands of lines of code).
If, however, you plan to go to such extremes, choosing the parser is not going to be your main problem. For DSLs of average complexity, they'll do just fine.
that will allow JAX-RS to have dynamically changed routing
Well, I don't know JAX-RS, but if dynamically changed routing can be done with it, then combinators parsers will be able to provide whatever input is needed.
EDIT
Seeing your example, I think parser combinators are certainly enough. From their results, I expect you could dynamically create BlueEyes binders -- I haven't used BlueEyes, so I'm not sure how dynamic they are.
Another alternative would be go with Lift. Lift's binders are partial functions, and they can be combined in all the usual ways -- f1 orElse f2, f1 andThen f2, etc. I didn't suggest it at first because it is most often used with sessions, but it has a RESTful model which, I think, is stateless.
I don't know Scalatra, so I don't know if it would be adaptable to this or not.