The test cases in Selenium IDE, look similar to the test cases written in files in keyword-driven test frameworks.
So is it correct to say, Selenium IDE is a tool built on the concepts of keyword-driven testing
Though it seems like a keyword (Click,type,sendkeys) driven framework it is not a keyword driven framework. Logically in a keyword driven framework common tasks with respect to a specific functionality are grouped under basic keywords and these keywords are then combined to achieve a complex functionality or business flow.
According to me Selenium IDE which provides us a record and playback option is a type of linear script / structured framework that allows even a non-technical user to easily perform his tests in sequentially ordered flow. Of course , these scripts can be customized in future if required but the ultimate goal here is to automate the tests with less/no technical knowledge.
Related
I am working on JPA project and I want to have unit tests (although as a database is required, in this case it will be more as integration tests.)
What is the best way to test JPA project? jUnit can do that ? Is there other better way ?
Thank you very much
You have given limited information on the tools/frameworks you are using and a very general question, but I will give a quick answer on the points you raise. These are just pointers however as I believe you need to do a good bit more leg-work in order for you to figure out what is best for your particular project.
Junit allows you to target your class methods with specific parameters and to examine the return values. The returned values maybe an entity that should have certain field at certain values, a list of entities with certain expected field values, exceptions etc., etc. (Whatever you methods are). You can run your test as you introduce new functionality, and re-run them to test for regression as development proceeds. You can easily test edge cases and non-nominal stuff. Getting Junit up and running in Java SE/EE is quite straight forward so that could be a good option for you to get stick-in with testing. It is one of the quicker ways I use to test new functionality.
Spring/MVC – Using an MVC framework can certainly be useful. I have used JSF/Primefaces. But that is principally because the application was to be a JSF application and such development tests gave confidence that the ‘Model’ layer provided what was needed to the rest of the framework. So this provides some confidence in the model/JPA/DB layers (it is certainly nice to see the data that is delivered) but does not provide for flexible, nimble and targeted testing you might expect from Junit.
I think Dbunit might be something to look at when you’ve made some progress with JUnit.
See http://dbunit.sourceforge.net/
DbUnit is a JUnit extension (also usable with Ant) targeted at
database-driven projects that, among other things, puts your database
into a known state between test runs. This is an excellent way to
avoid the myriad of problems that can occur when one test case
corrupts the database and causes subsequent tests to fail or
exacerbate the damage.
This is regarding an issue I have been facing for sometime. Though I have found a solution, I really would like to get some opinion about the approach taken.
We have an application which receives messages from a host, does some processing and then pass that message on to an external system. This application is developed in Java and has to run on Linux/Oracle and HP-NonS top Tandem/SQLMX OS/DB combination.
I have developed a test automation framework which is written in Perl.This script traverses directories (specified as an argument to this script) and executes test cases specified under those directories. Test cases could be organized into directories as per functionality. This approach was taken to ensure that a specific functionality can also checked in addition to entire regression suite.For verification of the test results, script read test case specific input files which has sql queries mentioned in them.
In Linux/Oracle, Perl DBD/DBI interface is used to query Oracle database.
When this automation tool was run in Tandem, I came to know that there was no DBD/DBI interface for SQLMX. When we contacted HP, they informed us that it would be a while before they develop DBD/DBI interfaces for SQLMX DB.
To circumvent this issue, I developed a small Java application which accepts DB connection string, user name, password and various other parameters. This Java app is now responsible for test case verification functionality.
I must say it meets our current needs, but something tells me (do not know what) that approach taken is not a good one, though now I have the flexibility of running this automation with any DB which has a JDBC interface.
Can you please provide feedback on the above approach and suggest a better solution?
Thanks in advance
The question is a bit too broad to comment usefully on except for one part.
If the project is in Java, write the tests in Java. Writing the tests in a different language adds all sorts of complications.
You have to maintain another programming language and attendant libraries. They can have different caveats and bugs for the same actions, such as you ran into with a lack of a database driver in a certain environment.
Having the tests done in a different language than the project is developed in drives a wedge between testing and development. Developers will not feel responsible for participating in the testing process because they don't even know the language.
With the tests written in a different language, they cannot leverage any work which has already been done. They have to write all over again basic code to access and work with the data and services, doubling the work and doubling the bugs. If the project code changes APIs or data structures, the test code can easily fall out of sync requiring extra maintenance hassles.
Java already has well developed testing tools to do what you want. The whole structure of running specific tests vs the whole test suite is built into test suites like jUnit.
So I can underscore the point, I wrote Test::More and I'm recommending you not use it here.
As many of you are aware as of the release of MatLab 2013a, xUnit a popular unit testing framework for MatLab is canceling further development.
Is MatLab's new and native unit testing framework comparable to xUnit? what features is it lacking when compared to xUnit? Is it better or worse than xUnit?
MATLAB xUnit has been an excellent contribution to the test focused development efforts of those writing MATLAB code. It has a solid implementation, it follows the xUnit paradigm very well, and has been invaluable as a file exchange contribution.
The MATLAB Unit Test framework has indeed learned from this submission as well as decades of requirements and test focused development for the MathWorks' internal code base. We have also learned and extended upon frameworks in other languages such as JUnit, NUnit, and python's unittest framework. As such there certainly are many more features in the R2013a-beyond framework, and it is designed to scale and extend.
There are too many other features to go into in a simple answer, but perhaps one way to describe some of the differences are that the 13a framework is what I loosely call an "xUnit 2.0" and the file exchange submissions is an "xUnit 1.0" framework. If you are familair with JUnit, this is like the difference between JUnit 3 and JUnit 4.
There are also other intangible or as yet unrealized benefits, such as:
The framework is included directly in MATLAB so you can share tests with others and know that they can run the tests even if they are not familiar with testing and do not want to download the file exchange framework.
The framework is under active development with a pipeline of additional features and capabilities in the works for future releases.
Hope that helps. I would be happy to go over any questions you have about specific functionality or features.
I don't believe MathWorks are planning at all to stop making xUnit available, so you can continue using it if you like. xUnit had not seen any large changes for quite a while in any case, and even though it won't be developed further in terms of features, it may receive an occasional fix if any are needed.
I have tried out the new framework quite a bit, but have not used it on any large projects yet. Previously I have used xUnit on large projects. However, I'm no expert on unit testing - so please read the following opinions in that context.
I'm pretty sure there's nothing you can do in xUnit that you can't do in the new framework. In general it's much more flexible and powerful than xUnit, providing additional features and a better way to organise and structure your tests. It's a lot easier to set up and tear down suites of tests, to manage and close resources (files, figure windows, database connections etc), and to carry out tricky tests such as checking that the right number of arguments are returned.
However, whereas a typical xUnit test was implemented as a fairly simple MATLAB function, tests in the new framework are typically implemented (in 13a, but see below for 13b) as classes using MATLAB's OO syntax, and if you're not comfortable with that it may seem like a big leap.
I should also add that although the documentation for the testing framework is excellent as reference material, I haven't found it to be great as a tutorial.
In 13b, the need to use classes has been offset a bit with the introduction of the functiontests command, which creates a test suite for you from a file containing tests implemented as local functions. That will make things much easier if you're not comfortable with class syntax. But I would think that if you want to take advantage of everything, you'd probably still want to use the main framework.
Hope my experience is of help - if you're lucky, perhaps #AndyCampbell will chime in...
I usually made php forms and "try" to use "good practices" in them.
I'm concerned about the real safety and error-free of that forms and I want to do some tests simulating the customer behavior, and I do it manually, but I find that is a hard work specially when the form is large and I know that there are a lot of combinations that I can't test, so usually I find bugs in the production phase.
Is there a tool that do this? I listened about Selenium, did somebody use it in the way I need? Or how can I create my own test tools that simulate user inputs at random?
User inputs implies: not filling/checking all the fields, putting in invalid data, using differents setups (no javascript, browser versions, ...), SQL injections, and I don't know so more...
You'll need to consider a combination of approaches here: good test case design, data driving those tests with various input combinations, and an automation tool such as Selenium, WebDriver, Telerik's Test Studio (commercial tool I help promote), or some other automation tool.
Design your test cases such that you're focusing on groups of behavior (a successful path case, a case validating invalid input, a case validating protection against SQL injection, etc.). Then you can look to (perhaps) data drive those test cases with sets of inputs and expected results. You can randomize that as needed through your test case code.
Most good functional automation tools support multiple browsers running the same test script, so that's a good help for hitting multi-browser testing.
Above all, start your automation efforts with small steps and focus first on high-value tests. Don't spend time trying to automate everything because that costs you a lot of time.
Selenium is used to automate browsers in exactly the way you described.
Its used for what is called Functional Testing. Where you test the external aspects of an application to ensure that they meet the specifications.
Its is most often combined with unit tests that test the internal aspects. For example to test that your application is safe against different forms of SQL injection.
Each programming language usually has several different frameworks for writing unit tests.
This are often used together this with an approach called test driven development (TDD) where you write the tests before the application code.
How can I create a wrapper around a testing framework? We still doesn't know which testing framework are going to use, but I need to start writing unit tests. With this question I want to know how can I switch from NUnit to mbUnit, xUnit or even MSTest.
You could create a wrapper - but I think you can utilise your time much better. I'd say pick the simplest one (My personal favourite would be the war-horse NUnit) that fits your needs - the newer frameworks add functionality that help you write more complex test fixtures.
However I value simplicity over "flexibility". In the future, if you find yourself wanting that "cool feature X in Y testing framework", you could either write that particular test fixture using Y. (you could also migrate the entire test fixture to use Y for consistency - but time is always scarce) - Switch between 2 unit testing framework is usually monotonous work (rename attributes) however some might be more work (disclaimer: no flying time with MbUnit)
Your comment however worries me a bit.
Why is the customer deciding the testing framework that you'd use for development - it should be a choice of the development team. The customer wouldn't want you to define product requirements - would he/she ? The quality of tests doesn't depend on the framework used so I don't see how this affects the customer.
You could use an existing wrapper that allows you to run multiple unit test frameworks, so even if you switch frameworks you can still use the old unit tests. For the unit test frameworks you listed, I would recommend taking a look at Gallio.
From http://www.gallio.org/...
At present Gallio can run tests from MbUnit versions 2 and 3, MSTest, NBehave, NUnit, xUnit.Net, csUnit, and RSpec. Gallio provides tool support and integration with AutoCAD, CCNet, MSBuild, NAnt, NCover, Pex, Powershell, Resharper, TestDriven.Net, TypeMock, and Visual Studio Team System.