Is it possible to run TesNG tests based on priority? For example, say I want to run only the tests which have priority=1.
<testng outputDir="${report.dir}" haltOnFailure="true" groups="${groups}">
<!-- Extra project classpath-->
<!-- Tell Ant where is the project and test classes -->
<classpath refid="selenium.classpath" />
<classpath refid="dynamicreports.classpath" />
<!-- Tell Ant what test classes need to run -->
<classfileset dir="${bin.dir}" includes="**/*.class" />
</testng>
Yes, you can do it using BeanShell script in TestNG XML suite definition. Something like:
<method-selector>
<script language="beanshell">
<![CDATA[ testngMethod.getPriority() > 1 ]]>
</script>
</method-selector>
See http://testng.org/doc/documentation-main.html#beanshell for more details.
You can have Groups to do this for you . Assign a group to the testcase according to the priority and run only that group
http://testng.org/doc/documentation-main.html#test-groups
Related
I want to execute an arbitrary selection of tasks in a Phing build.
I'm passing in a list of modules for building. Each module is of a particular type. The type is specified in the name, as {type}_{unitname}. I started with a build file that took a single module name and built it, that works fine. I now want to pass in a list of modules and have it build all of them. (What I'd really like to do is load the list of modules from an XML manifest file, but perhaps one thing at a time).
I've tried multiple ways and have found a problem with each.
I seem to need an ad-hoc task to derive my properties (task and related directory settings) from the module name. This seems to cause problems, but not at first.
At first I tried to use the loop variable as the target
<foreach list="${mylist}" param="item" target="${item"} />
but it doesn't seem to allow a variable as a target name. So I split it up into two tasks.
<foreach list="${parts}" param="dTarg" target="doIt" >
<task name="DoIt">
<phingcall target="build">
<property name="extension" value="${dTarg}" />
</phingcall -->
</task>
My problem here is (I think) "extension" is a constant and thus can't be overwritten. I tried using "var", which the docs say is a thing, but my setup complains it doesn't exist. Is it a 3.0 feature? I'm running 2.17.
So I tried changing the "phingcall" to "phing" and put my main functionality in a separate file. Here I run into problems with the ad-hoc task again. If I put it in the "subordinate" file, it complains that's it's re-declared (I think, the message isn't very helpful) when the file is called a second time. If I leave it in the main file, the subordinate file can't find it, even with inheritrefs and inheritall set.
How can I execute tasks whose names are in list?
At first I tried to use the loop variable as the target
<foreach list="${mylist}" param="item" target="${item"} />
but it doesn't seem to allow a variable as a target name
The target attribute of the foreach task is able to use variables as a value, but at this point param="item" is not yet available but in the target it is.
So I split it up into two tasks.
<foreach list="${parts}" param="dTarg" target="doIt" >
<task name="DoIt">
<phingcall target="build">
<property name="extension" value="${dTarg}" />
</phingcall>
</task>
Here you try to use a task task which is simply not a valid target.
What you want to do instead is to have targets to iterate over. The following example demonstrates the usage:
Input build.xml
<?xml version="1.0" encoding="utf-8" ?>
<project name="test" default="main">
<property name="mylist" value="A,B,C" />
<target name="main">
<foreach list="${mylist}" param="item" target="DoIt"/>
</target>
<target name="DoIt">
<echo>${item}</echo>
</target>
</project>
Output
test > main:
test > DoIt:
[echo] A
test > DoIt:
[echo] B
test > DoIt:
[echo] C
BUILD FINISHED
Complex Example (with property override and inheritAll)
<?xml version="1.0" encoding="utf-8" ?>
<project name="test" default="main">
<property name="mylist" value="A,B,C" />
<target name="main">
<foreach list="${mylist}" param="item" target="DoIt"/>
</target>
<target name="DoIt">
<phingcall target="${item}" inheritAll="true">
<property name="extension" override="true" value="${item}-ext" />
</phingcall>
</target>
<target name="A">
<echo>Inside target ${item} with ${extension} extension</echo>
</target>
<target name="B">
<echo>Inside target ${item} with ${extension} extension</echo>
</target>
<target name="C">
<echo>Inside target ${item} with ${extension} extension</echo>
</target>
</project>
Output
test > main:
test > DoIt:
test > A:
[echo] Inside target A with A-ext extension
test > DoIt:
test > B:
[echo] Inside target B with B-ext extension
test > DoIt:
test > C:
[echo] Inside target C with C-ext extension
BUILD FINISHED
Example execute as one task with changed values from the list
<?xml version="1.0" encoding="utf-8" ?>
<project name="test" default="main">
<property name="mylist" value="A,B,C" />
<target name="main">
<foreach list="${mylist}" param="item" target="DoIt"/>
</target>
<target name="DoIt">
<phingcall target="build">
<property name="extension" override="true" value="${item}-ext" />
</phingcall>
</target>
<target name="build">
<echo>Inside target build with ${extension}</echo>
</target>
</project>
Output
test > main:
test > DoIt:
test > build:
[echo] Inside target build with A-ext
test > DoIt:
test > build:
[echo] Inside target build with B-ext
test > DoIt:
test > build:
[echo] Inside target build with C-ext
BUILD FINISHED
Simplified and final build
<?xml version="1.0" encoding="utf-8" ?>
<project name="test" default="main">
<property name="mylist" value="A,B,C" />
<target name="main">
<foreach list="${mylist}" param="item" target="build">
<property name="extension" override="true" value="${item}-ext" />
</foreach>
</target>
<target name="build">
<echo>Inside target build with ${extension}</echo>
</target>
</project>
Output
test > main:
test > build:
[echo] Inside target build with A-ext
test > build:
[echo] Inside target build with B-ext
test > build:
[echo] Inside target build with C-ext
BUILD FINISHED
For my Zend Framework (v1.12) project deployment Im using Phing. It deploys front-end and back-end applications. Everything goes fine, untill Phing starts phpunit testing:
<!-- Testing backend -->
<target name="backend-test" description="Testing backend system">
<phpunit codecoverage="false" haltonerror="true" haltonfailure="true" printsummary="true" bootstrap="${path.build}/${DSTAMP}${TSTAMP}/admin/tests/bootstrap.php">
<batchtest>
<fileset dir="${path.build}/${DSTAMP}${TSTAMP}/admin/tests">
<include name="**/*Test.php" />
</fileset>
</batchtest>
<formatter type="summary" usefile="false" />
</phpunit>
</target>
<!-- Testing frontend environment -->
<target name="site-test" description="Testing frontend system">
<phpunit codecoverage="false" haltonerror="true" haltonfailure="true" printsummary="true" bootstrap="${path.build}/${DSTAMP}${TSTAMP}/site/tests/bootstrap.php">
<batchtest>
<fileset dir="${path.build}/${DSTAMP}${TSTAMP}/site/tests">
<include name="**/*Test.php" />
</fileset>
</batchtest>
<formatter type="summary" usefile="false" />
</phpunit>
</target>
Back-end part phpunit tests succeed, but when it starts testing front-end - it fails it error:
[phingcall] Plugin with stackIndex "1" already registered
[phingcall] #0 /var/www/zend/Controller/Front.php(733): Zend_Controller_Plugin_Broker->registerPlugin(Object(Application_Plugin_ACL), 1)
[phingcall] #1 /var/www/_qa/build/201311181711/admin/application/Bootstrap.php(199): Zend_Controller_Front->registerPlugin(Object(Application_Plugin_ACL), 1).......
Any ideas? Your help would be appreciated.
Your bootstrap is being called twice (or at least the resource [_init function]).
You get this error because you can't define an other plugin instance for the same stack index.
You should try to find a way to mock all periferral classes loaded in your bootstrap and test each of these classes individually.
An other way would be to bootstrap your application in the phpUnit setup method and make sure it's destroyed in the teardown. This will be more ressource instensive and will most likely slow down your tests as the whole Zend stack will be instanciated, then killed, then re-instantiated for each of your test cases.
In our continuous integration setup, I would like to set up CruisControl.NET to automatically run all our unittests. However, I don't want to have to specify every unittest dll seperately in the configuration.
All the unittest projects are all postfixed with .Test (and all non-unittest projects are not). How can I configure CruiseControl.NET to run all the unittests from these projects (I am using v1.5.7256.1 of CruiseControl.NET)?
My current config attempt:
<nunit>
<path>$(nunit.path)</path>
<assemblies>
<assembly>$(working.dir)\**\*.Test.dll</assembly>
</assemblies>
</nunit>
I'm finding it very difficult to find documentation on this specific nunit element. Most pages I can find talk about using exec, nunit2 or another nunit element or the nunit-console commandline options.
I don't have much experience with managing the build environment and am working on an existing configuration where every assembly was specified separately in the following manner.
<nunit>
<path>$(nunit.path)</path>
<assemblies>
<assembly>$(artifact.dir)\test1.dll</assembly>
<assembly>$(artifact.dir)\test2.dll</assembly>
</assemblies>
</nunit>
Hence my failed attempt using wild cards.
EDIT:
Here is some extra xml of my configuration file to show the context a little bit:
<cruisecontrol xmlns:cb="urn:ccnet.config.builder">
<project name="MyProject">
<!-- whole bunch of other elements -->
<tasks>
<nunit>
<!-- see above -->
</nunit>
</tasks>
</project>
</cruiscontrol>
After Mightmuke's suggestion, I tried replacing the <nunit> element with his suggestion, but got the following exception: Unable to instantiate CruiseControl projects from configuration document. Configuration document is likely missing Xml nodes required for properly populating CruiseControl configuration. Unable to load array item 'property' - Cannot convert from type System.String to ThoughtWorks.CruiseControl.Core.ITask for object with value: ""
Then I tried to move the <property> and <foreach> element outside the element. Then I get the exception: Unused node detected: <property name="nunit.filelist" value="" />
I'm now trying to find out more about the <foreach> element and where I can put that, but somehow I find it hard to find any documentation about it.
EDIT2:
I found the documentation of the nunit task I'm using: http://ccnet.sourceforge.net/CCNET/NUnit%20Task.html
I specifies the element to be of type String[]. I'm not sure what that means... but it seems from the example that it just means that it must contain a list of child elements of the same name in Singular form.
PS: I realize this question is getting a bit out of hand... When the whole thing is solved, I'll try to edit it in such a format so that it might be useful to someone else later.
This is an example configuration if you were to use the nunit console.
<property name="nunit.filelist" value="" />
<foreach item="File" property="testfile" verbose="true">
<in>
<items basedir=".">
<include name="${working.dir}\**\*.Test.dll" />
</items>
</in>
<do>
<property name="nunit.filelist" value="${nunitfile.list + ' ' + testfile}" />
</do>
</foreach>
<exec program="nunit-console-x86.exe" failonerror="true" verbose="true">
<arg value="${nunit.filelist}" />
<arg value="/xml=nunit-results.xml" />
<arg value="/nologo" />
<arg value="/nodots" />
</exec>
This hasn't been tested, and there are likely better ways to skin it, but it will hopefully provide a starting point for you.
I've been following this tutorial on 3. Getting Started (ANT), and it says <taskdef resource="emma_ant.properties" classpathref="emma.lib" /> but does not give any reference to the contents of emma_ant.properties. Any ideas?
Other websites such as 3.4. How do I change an EMMA property default setting? also leave things to be desired (and is based on the command prompt and not a digital file). I've found another website Using EMMA with ANT for JUnit test coverage reporting, but again it leaves the properties file to the imagination (doesn't even provide an example file).
Any ideas on how to manipulate the emma_ant.properties to Load and custom tasks for ANT?
but does not give any reference to the contents of emma_ant.properties. Any ideas?
Check for emma.jar & emma_ant.jar which you have placed in the path specified you will find emma_ant.properties
Any ideas on how to manipulate the emma_ant.properties to Load and custom tasks for ANT?
You need not to manipulate the properties file to use the tasks.
To use emma tasks you should
<!-- directory that contains emma.jar and emma_ant.jar -->
<property name="emma.dir" value="${YOUR_BASE_DIR}/lib/emma" />
<!-- Set emma.lib to refer to the list of EMMA jar files -->
<path id="emma.lib">
<fileset dir="${emma.dir}">
<include name="*.jar" />
</fileset>
</path>
and
<!-- Load <emma> custom tasks so that they can be used in ANT -->
<taskdef resource="emma_ant.properties" classpathref="emma.lib" />
and you should be able to use emma tasks.
Here are the contents of emma_ant.properties inside emma.jar
# -------------------------------------------------------------
emma: com.vladium.emma.emmaTask
emmajava: com.vladium.emma.emmajavaTask
# -------------------------------------------------------------
# end of file
Also Check out Emma Property Summary if it helps you...
I have a class named "ATestClass" and it has some methods such as "launchSite", "loginToSite" etc.
Now, I want to control the order of execution of these methods. So, I created an xml file "testng.xml" as below:
<!DOCTYPE suite SYSTEM "http://testng.org/testng-1.0.dtd" >
<suite name="Classes" verbose="10">
<test name="ATestClass" preserve-order="true">
<classes>
<class name="ATestClass" >
<methods>
<include name="launchSite" />
<include name="searchAndExport" />
<include name="loginToSite" />
</methods>
</class>
</classes>
</test>
</suite>
Now, my class doesn't care this xml file and executes the methods in alphabetical order. I created both class and xml file in the same package.
Somehow, the class is not able to recognize its corresponding xml config file.
Please help me!
Are you using the latest version of TestNG? Make sure you are.