I'm currently moving one of our projects to DNX (.NET Core now) and I was forced to update to nunit3. Because of other considerations, we run compile the test project as a console app with its own entry point, basically self-hosting the NUnit runner.
I now need to report the results to TeamCity via the XML Reporter, which doesn't seem to parse Nunit3 TestResults.xml files.
Any advice on how to work around this?
The NUnit 3 console has the option to produce results formatted in the NUnit 2 style.
Use the option:
--result=[filename];format=nunit2
Docs: https://github.com/nunit/nunit/wiki/Console-Command-Line
To add to the answer above:
NUnitLite inherits the --result CLI parameter which seems to do the trick.
Another option, which I went for in the end is using the --teamcity CLI parameter:
dotnetbuild --project:<path to project directory> -- --teamcity
which will integrate with TC's service messages. This will also do real-time updates.
Related
So my nunit 2.6 tests run in team city, the results appear in a 'tests' tab. I just select the built in nunit build step, type the name of the file with the tests in "Runs Tests from:" and the name of the category(s) in "NUnit categories include:"
However, If I upgrade my tests and select NUnit3 from the "NUnit runner" drop down things start to go wrong.
After much googling and mucking round for command lines and console runners I can't get this working with the full 'results in a tab' level of compatibility.
There is a lot of advice out there on how to make this work, but the best articles are at least a year old and I'm not at all clear that they apply to 3.5.0
Has anyone got this fully working or is it no longer supported? Can you explain how you managed it?
NUnit 3.5.0 does work with TeamCity, but it requires that you use the NUnit TeamCity Event Listener Extension. You can install it along with the NUnit Console Runner package or use the NUnit Console Runner with Extensions which includes the TeamCity extension along with other commonly used NUnit extensions.
Once you have the extension, TeamCity should automatically add the --teamcity command line option to the nunit3-console.exe.
Check here more information on the various NUnit Runner NuGet packages and what is included in each.
You add these packages to one of your test projects. That will cause them to be installed in the package directory of your solution root. From there, the built in NUnit 3 step will work, just update the executable location to point to 3.5.0. See the Getting Started With NUnit and TeamCity document. I would use the Case 4, NUnit Build Step.
If you only want to test certain categories, you will need to add your --where clause as an additional command line parameter.
I am using an NUnit 3 runner in a TeamCity 9.1.6 step. I've chosen "JetBrains dotCover" as the .NET Coverage tool, and now I'd like this step to use the xxx.sln.DotSettings file that we've put in source control and that we're sharing across devs in Visual Studio, rather than to duplicate settings to TeamCity Filters, Attribute Filters etc. Is this possible in TeamCity?
It is not possible from the box now. It is a great idea, could you create an issue here https://youtrack.jetbrains.com/
But there is a simple workaround:
you could parse dotSettins manually on the first step
publish configuration parameters using ##teamcity[setParameter name='ddd' value='fff'] TeamCity service message (see for details https://confluence.jetbrains.com/display/TCD9/Build+Script+Interaction+with+TeamCity)
use those configuration parameters in the appropriate fields for dotCover like %ddd%
I tried to find this answer but hardly found it anywhere. I am doing the API testing, In process I need to call the rest API from my local machine. local machine contains the maven project and a framework to call respective rest API.
I need to check the code coverage of remote Rest API and form a report based on the code coverage. please help, how to do that?
Note: I found this link useful but it does not elaborate clearly on what to do?
http://eclemma.org/jacoco/trunk/doc/agent.html
you will probably do a bit of file copying around - depending on the way you run the tests.
JaCoCo runs as a java agent. So you usually add the javaagent parameter as mentioned in the docs you linked to the start script of you application server.
-javaagent:[yourpath/]jacocoagent.jar=[option1]=[value1],[option2]=[value2]
so it would look like:
java -javaagent: -jar myjar.jar
Using tomcat you can add the "-javaagent" part into JAVA_OPTS or CATALINA_OPTS environment variables. Should be similar for other servers.
this will create the jacoco*.exec files. you need to copy those back to your build or CI server to show its results (for ex if you use sonar you need those files before running the sonar reporter). Its important to just include the packages you're interested in.
You can also create one jacoco.exec file per test flavour (jacoco.exec for unit tests, jacoco-it.exec for integration tests, jacoco-at.exec for application tests).
And I would not mix coverage with performance testing - just to mention that too.
There are some examples on stackoverflow for JBoss
I remember doing this in an older version of Team City in which the NUnit Runner failed, so I tried running NUnit using the Command Line Runner instead.
I think there was an option in the Command Line Runner settings which let you specify the kind of output file: NUnit result.xml was one of them. Doing this, you could see the test result breakdown in the Web GUI, just like when the NUnit Runner worked.
But I don't see this in the latest Team City (6.5.5). Can it still be done?
PS: Before anyone asks, I can't use Team City's NUnit Runner because NUnit has a bug which will be fixed in the soon to come 2.6.0 version. Said bug does not occurr when running nunit-console
This could be done using importdata TeamCity message, this is really helpful when you are running NUnit tests manually executing nunit-console.exe from an MSBuild script, for instance using NUnit MSBuild Community Task, but keep in mind that TeamCity provides built in NUnit tests runner so you do not need this low level TeamCity scripting. Anyway you can import report in this way:
<Message
Text="##teamcity[importData type='nunit' path='...\TestResults.xml']"
Importance="High"/>
See Importing XML Reports
If you prefer to run code coverage, code inspection, test tools or
duplicate finders directly from build script, not as a build runner,
you can use the importData service messages to import generated xml
reports into TeamCity.
The functionality was still there, found it by accident when looking at another build configuration. Importing an XML is now a "Build Feature", see:
We recently upgraded CruiseControl.Net 1.4.2 to 1.5.7256.1. To our dismay the nice nunit results page that had previously been there had disappeared. All we get now is a build log with the verbose output from command line of our build.
After some research we discovered you can log into "Dashboard Administrator" from the main ccnet dashboard page and install the NUnit package. However, this did not change the output of our builds to include a NUnit result page.
Is there something extra we need to do here to have our output displayed? Any help would be greatly appreciated. I don't goto stack overflow until I've exhaustively searched on Google. I'm surprised no one else is confused or complaining about this.
FYI, we are merging our nunit XML output in the project configuration.
I'm not used to the Dashboard Administrator stuff, but you can edit the dashboard.config manually. Opposite to earlier versions CCNET version 1.5 comes with a naked configuration file.
Alternatively you may simply replace the configuration file with your 1.4 dashboard.config.
Possible duplicate of Cruise Control .net: Using packages and showing NUnit results
Don't merge manually if you use the nunit task
Edit the dashboard.config and the ccservice.exe.config files to add the nunit xsl files. Restart IIs.
Check that the Iis user has been granted access to the dashboard dir.