I am wondering if tests can be separated so that my whole test suite will not run on every build but runs on nightly builds.
Is it possible to separate it this way?
I know to separate the tests on multi browser but unsure in this case.
I did it by passing two seperate sets of tests. It worked! Thanks
Related
the company where I work is evaluating different CI/CD systems, we tried GoCD (v17.4), Jenkins 2 (v2.7) and Concourse (v3.2.1).
We liked Concourse, but a big downside was the fact that the test reports were not displayed in a usable way. I asked in the slack chat, I was told Concourse shows the output of the console, respecting the ANSI colors, if any...
...but the thing is, XML test reports contain a lot more information than just a red color for failing tests and we need to use that information.
I created a failing test and Jenkins has a nice plugin to group all tests, show extra info/metrics and group the failing tests to spot them at once. It also keeps the history of test results.
In Concourse, without a tests reporter one has to scroll down a log to see all failing tests... my colleagues are concerned about this.
Is there a way in Concourse to parse a junit XML test report and show it in the UI in a usable/interactive (clickable) way, as jenkins does?
As I learnt is that Concourse has no plugins and simplicity by design, it seems that the answer is: "NO, there isn't: you can just see the console logs as is". but if I'm wrong, please let me know... Thanks
Concourse doesn't discriminate against types of outputs on purpose.
Concourse is made to be generic. That way there isn't highly specialize, unrepeatable deployments of itself.
Jenkins is specialized to solve these types of issues. To the extent it has deep integration for having UIs display custom output.
It sounds like Jenkins solves all your use cases. I wouldn't try to hammer concourse into this use case.
Concourse is minimal in that way. Concourse is made to run tasks in a pipeline configuration, and do so in an atomic container setup. That is also way it does not store build artifacts and so on. It forces you to do the right thing and save everything you need elsewhere, like buckets etc. Push the XML to a service or store it in a bucket for a tool to use later on.
Is there any way to make NUnit abort running any more tests in that class only when it encounters the first error? I'm running integration tests, using the [Order] attribute. These tests can get quite lengthy, and in certain cases there's no need to continue running tests in the class if one of the tests fail. I want NUnit to continue on to other classes, but I want it to abort calling any more classes in that specified class.
Is there any way to do this?
I'm using NUnit 3.2
Thanks!
Buzz
There isn't currently any way to stop execution of tests within a class on first error. There is only the command line option --stoponerror which stops running all tests on the first error.
The Order attribute is new in 3.2. Unlike other frameworks, it was only intended to order your tests, not setup dependencies for your tests. There is an open enhancement on GitHub for a Test Dependency Attribute, but it hasn't been worked on because people cannot come to a consensus on the design. I think a good first step would be a variation of the Order attribute with dependencies, but some people want a full dependency graph. I would recommend that you head over to GitHub and comment on the issue with your requirements.
Unit testing some projects can take some time, and with multiple projects you have to start them one at-a-time sequentially. In NetBeans is it possible to run all unit tests for a single project in one go.
Is it also possible execute all unit tests for all projects in one go?
In NetBeans is it possible to run all unit tests for a single project in one go.
In NetBeans 7.1, yes, it is. See my previous answer to this same question.
Is it also possible execute all unit tests for all projects in one go?
I don't believe that this is possible.
In NetBeans is it possible to run all unit tests for a single project in one go.
This is possible by right clicking the project node in the Projects explorer and choosing Test from the context menu. You can also use the key board shortcut Alt+F6 to achieve the same result.
Yes in Netbeans also it is possible go to current project Properties by right clicking it, and go to Libraries, there u can add your other projects to run in a single go.
I have a bunch of NUNIT tests in several TestFixtures. Currently, I just display all the tests for everyone. Is there a way to hide some tests and/or test fixtures. I have various "customers" and they don't all need to see every test. For example, I have engineers using low level tests, and I have a QA department that is using higher level tests. If I could have a configuration (XML?) file that I distributed with the dll that would be ideal. Can someone point me to the documentation and example? I did search the NUNIT site and did not see anything.
I am aware of the [IGNORE] attribute and I suppose a somewhat acceptable solution would be to have a configuration file that can apply IGNORE to various tests or testfixtures. I'd hand out a different version of the configuration file to each customer. At least that way certain customers would not be able run certain tests.
I'm using version 2.5.5
Ideas?
Thanks,
Dave
Yes - if the tests are in seperate assemblies, this can be accomplished by proper configuration of your NUnit projects. However, this is not an option if the tests are in one large test assembly. If this is the case, you may wish to break up the test assembly. Here is the documentation on the NUnit ProjectEditor: http://www.nunit.org/index.php?p=projectEditor&r=2.2.10
I am testing the web application using Selenium RC. All things works fine and I have written many test cases and executing these test cases using Nunit.
Now the hurdle that I am facing is how to keep track of failures or how to generate the Reports?
Please advice which could be best way to capture this.
Because you're using NUnit you'll want to use NUnits reporting facilities. If the GUI runner is enough, that's great. But if you're running from the command line, or NAnt, you'll want to use the XML output take a look at the NAnt documentation for more information.
If you're using Nant you'll want to look at NUnit2Report. It's nolonger maintained, but it may suit your needs. Alternatively, you could extract it's XSLT files and apply it against the XML output.
Selenium itself doesn't have report because it is only a library used by many different languages.
For anyone else happening randomly into this question, the 'nunit2report' task is now available in NAntContrib.
NantContrib Nunit2report task