I've inherited a java testing application written in java/eclipse/testng, I'm a .net developer. I can run the test suite just fine, but I'm having difficulty in figuring out how to rerun the tests immediately after the suite has run. Does anyone have anything to point me in the right direction?
Thanks!
rerun the tests? what do you mean by here?
Do you want to run the multiple suites one after the other?
if so
<suite name="automated suites">
<suite-files>
<suite-file path="suitea.xml" />
<suite-file path="suiteb.xml" />
...
</suite-files>
</suite>
or the same suite?
<suite name="automated suites">
<suite-files>
<suite-file path="suitea.xml" />
<suite-file path="suitea.xml" />
...
</suite-files>
</suite>
Related
Is it possible to run TesNG tests based on priority? For example, say I want to run only the tests which have priority=1.
<testng outputDir="${report.dir}" haltOnFailure="true" groups="${groups}">
<!-- Extra project classpath-->
<!-- Tell Ant where is the project and test classes -->
<classpath refid="selenium.classpath" />
<classpath refid="dynamicreports.classpath" />
<!-- Tell Ant what test classes need to run -->
<classfileset dir="${bin.dir}" includes="**/*.class" />
</testng>
Yes, you can do it using BeanShell script in TestNG XML suite definition. Something like:
<method-selector>
<script language="beanshell">
<![CDATA[ testngMethod.getPriority() > 1 ]]>
</script>
</method-selector>
See http://testng.org/doc/documentation-main.html#beanshell for more details.
You can have Groups to do this for you . Assign a group to the testcase according to the priority and run only that group
http://testng.org/doc/documentation-main.html#test-groups
I have a standard CruiseControl.Net setup that pulls code from svn, builds, tests and produces the test results. I have a intermittent problem where the build reports success (green) but test results show 1 or 2 failures.
Running the task step manually, to run just the NUnit step, succeeds but output and test results show the test failing. I've tried ContinueOnFailure both true and false (it blows up on other options and indicates only true or false are valid values which I assume because of an older version).
Is the task supposed to error out (Errors != 0)? Or is the failure procured later perhaps in the merge test results step of ccnet.config?
Does ccnet depend on the NUnit task to fail to indicate a failed build?
Thank you.
My understanding is that ("default") it will only generate the test-results.xml, and put them in the "uber.xml" file to display the results.
http://build.sharpdevelop.net/ccnet/doc/CCNET/Using%20CruiseControl.NET%20with%20NUnit.html
And that it is overrideable with the following:
<!-- Acceptance Test Assembly -->
<exec program="nunit-console.exe" failonerror="false" resultproperty="testresult.acceptancetestassembly">
<arg value="AcceptanceTestAssembly.dll" />
<arg value="/xml=AcceptanceTestAssembly-Results.xml" />
</exec>
<!-- Check the results and fail if necessary -->
<fail message="Failures reported in unit tests." unless="${int::parse(testresult.unittestassembly)==0}" />
<fail message="Failures reported in acceptance tests." unless="${int::parse(testresult.acceptancetestassembly)==0}" />
I've search all over the web and this problem is driving me crazy.
I'm building a mobile application (J2ME, BB) using ANT and Proguard.
In my javac task, I've added the debuglevel paramater, like so:
<javac srcdir="${src.dir}" destdir="${tmp.dir}${application.name}" failonerror="true" source="1.4" target="1.4" debug="true" debuglevel="lines,vars,source">
In my proguard task, I have:
<proguard defaultpackage="" usemixedcaseclassnames="false" note="false" microedition="true" verbose="true" printmapping="true" optimizationpasses="10" renamesourcefileattribute="SourceFile">
...
...
<keepattribute name="LineNumberTable" />
<keepattribute name="SourceFile" />
</proguard>
However, when I look at the output console in Eclipse (after building my application and launching it on an emulator), I don't see lines number, just stuff like that:
at com.mycomp.myapp.Main.a(+47)
at gd.a(+24)
at fq.a(+368)
at fq.b(+188)
at com.mycomp.myapp.Main.run(+29)
I'm using Proguard 4.8.
Thanks a lot for your help,
You are using the correct setting in Proguard:
<keepattribute name="LineNumberTable" />
Do you see the line numbers on the console?, that worked for me.
I am using Spring, TestNG, Eclipse, Maven, Mac OS
Scenario is
I am doing some changes in my test cases ( TestNG test cases ),
Then i do Project -> Clean in eclipse
Now i am running the test file, but the changes not updated.
I suspects that the test files are not compiled.
I can run the server and i can run my webservices
But if i run maven test ( mvn test ), the entire code is compiling including test cases.
So to run the test cases i am running mvn test command only.
Not able to run through eclipse.
Eclipse config:
Auto build on
In Java build path the test package is included
If you need any more info i'll provide.
org.testng.TestNGException:
Cannot find class in classpath: com.***.***.model.***.case.CaseModelImplTest
at org.testng.xml.XmlClass.loadClass(XmlClass.java:76)
at org.testng.xml.XmlClass.init(XmlClass.java:68)
at org.testng.xml.XmlClass.<init>(XmlClass.java:54)
at org.testng.xml.TestNGContentHandler.startElement(TestNGContentHandler.java:512)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.startElement(AbstractSAXParser.java:501)
at com.sun.org.apache.xerces.internal.parsers.AbstractXMLDocumentParser.emptyElement(AbstractXMLDocumentParser.java:179)
at com.sun.org.apache.xerces.internal.impl.dtd.XMLDTDValidator.emptyElement(XMLDTDValidator.java:788)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanStartElement(XMLDocumentFragmentScannerImpl.java:1343)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl$FragmentContentDriver.next(XMLDocumentFragmentScannerImpl.java:2755)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentScannerImpl.next(XMLDocumentScannerImpl.java:648)
at com.sun.org.apache.xerces.internal.impl.XMLDocumentFragmentScannerImpl.scanDocument(XMLDocumentFragmentScannerImpl.java:511)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:808)
at com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:737)
at com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:119)
at com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1205)
at com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:522)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:395)
at javax.xml.parsers.SAXParser.parse(SAXParser.java:198)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:17)
at org.testng.xml.SuiteXmlParser.parse(SuiteXmlParser.java:10)
at org.testng.xml.Parser.parse(Parser.java:170)
at org.testng.TestNG.initializeSuitesAndJarFile(TestNG.java:304)
at org.testng.remote.RemoteTestNG.run(RemoteTestNG.java:86)
at org.testng.remote.RemoteTestNG.initAndRun(RemoteTestNG.java:199)
at org.testng.remote.RemoteTestNG.main(RemoteTestNG.java:170)
It seems , you are not included your Test case class into the build path. Right click on your Project folder, go to the project properties->src
include your Testcase src folder by checking it.
Now it should run.
Open the Problems view. In Eclipse go to Window->Show View->Problems. It will list out all the failures while compiling the code.
Are the test cases being compiled, but to the wrong directory? They need to be compiled to test-classes not classes. In your build path, you need to set your Output folder to be
PROJECT/target/test-classes
This is true for all test resources (including src/test/java & src/test/resources).
Edit your project properties in Eclipse, select Java Build Path, and then in the Source tab, you'll see all of your source directories. Each one has an output folder. This output folder needs to be as above.
Go to Project --> Clean
That worked for me with similar problem.
Please check your testNG.xml file for correct test because I got the same error and it was because the wrong test name was entered.
is your test folder added as a source folder in Eclipse? If it isn't it won't be included in the build.
I solved this by:
Changing the build path / deleting some of the wrong ones
Cleaning the project
I think some time we get this error if your test class is in default package and in Testng.xml, you specify as below -
<suite name="Suite" parallel="none">
<listeners>
<listener class-name="org.uncommons.reportng.HTMLReporter" />
<listener class-name="org.uncommons.reportng.JUnitXMLReporter" />
</listeners>
<test name="Test">
<classes>
<class name=".Test1" />
</classes>
</test> <!-- Test -->
Here in above, I moved my testclass from default package to some package and also updated testng.xml as below - and everything started working.
<suite name="Suite" parallel="none">
<listeners>
<listener class-name="org.uncommons.reportng.HTMLReporter" />
<listener class-name="org.uncommons.reportng.JUnitXMLReporter" />
</listeners>
<test name="Test">
<classes>
<class name="com.sigma.rest.api.Test1" />
</classes>
</test> <!-- Test -->
I hope this helps you... if issue still exists, try above options.
thanks!
Check if your project artifacts like excel, property files, xml files etc are open in text editors etc. ( if yes please close them and do rebuild )
When you clean and rebuild make sure all project dependent files are closed and you do build.
Is there any chance to get this work? I want my tests to be run by nunit2 task in NAnt. In addition I want to run NCover without running tests again.
I figured it out. You change the path of the NUnit launcher to that of TeamCity's own. Here is an example:
<mkdir dir="${build}/coverage" failonerror="false"/>
<!-- run the unit tests and generate code coverage -->
<property name="tools.dir.tmp" value="${tools.dir}"/>
<if test="${not path::is-path-rooted(tools.dir)}">
<property name="tools.dir.tmp" value="../../${tools.dir}"/>
</if>
<property name="nunitpath" value="${lib.dir}/${lib.nunit.basedir}/bin/nunit-console.exe"/>
<property name="nunitargs" value=""/>
<if test="${property::exists('teamcity.dotnet.nunitlauncher')}">
<property name="nunitpath" value="${teamcity.dotnet.nunitlauncher}"/>
<property name="nunitargs" value="v2.0 x86 NUnit-2.4.8"/>
</if>
<ncover program="${tools.dir.tmp}/${tools.ncover.basedir}/ncover.console.exe"
commandLineExe="${nunitpath}"
commandLineArgs="${nunitargs} ${proj.name.unix}.dll"
workingDirectory="${build}"
assemblyList="${proj.srcproj.name.unix}"
logFile="${build}/coverage/coverage.log"
excludeAttributes="System.CodeDom.Compiler.GeneratedCodeAttribute"
typeExclusionPatterns=".*?\{.*?\}.*?"
methodExclusionPatterns="get_.*?; set_.*?"
coverageFile="${build}/coverage/coverage.xml"
coverageHtmlDirectory="${build}/coverage/html/"
/>
As you can see, I have some of my own variables in there, but you should be able to figure out what is going on. The property you are concerned with is teamcity.dotnet.nunitlauncher. You can read more about it here at http://www.jetbrains.net/confluence/display/TCD4/TeamCity+NUnit+Test+Launcher.
Why not have NCover run NUnit? You get the exact same test results. Also, what exactly are you trying to measure when running NCover outside of the tests? There's other ways to find stale or unreferenced code.
I am having to do the same thing. I think the best we can hope for is to break open the NUnit jar file that comes with TeamCity and writing a custom task that integrates NUnit2 and NCover. I wish this wasn't so, but the NUnit2 task does not produce any visible output, so TeamCity is obviously not reading StdOut for the test results.