So my nunit 2.6 tests run in team city, the results appear in a 'tests' tab. I just select the built in nunit build step, type the name of the file with the tests in "Runs Tests from:" and the name of the category(s) in "NUnit categories include:"
However, If I upgrade my tests and select NUnit3 from the "NUnit runner" drop down things start to go wrong.
After much googling and mucking round for command lines and console runners I can't get this working with the full 'results in a tab' level of compatibility.
There is a lot of advice out there on how to make this work, but the best articles are at least a year old and I'm not at all clear that they apply to 3.5.0
Has anyone got this fully working or is it no longer supported? Can you explain how you managed it?
NUnit 3.5.0 does work with TeamCity, but it requires that you use the NUnit TeamCity Event Listener Extension. You can install it along with the NUnit Console Runner package or use the NUnit Console Runner with Extensions which includes the TeamCity extension along with other commonly used NUnit extensions.
Once you have the extension, TeamCity should automatically add the --teamcity command line option to the nunit3-console.exe.
Check here more information on the various NUnit Runner NuGet packages and what is included in each.
You add these packages to one of your test projects. That will cause them to be installed in the package directory of your solution root. From there, the built in NUnit 3 step will work, just update the executable location to point to 3.5.0. See the Getting Started With NUnit and TeamCity document. I would use the Case 4, NUnit Build Step.
If you only want to test certain categories, you will need to add your --where clause as an additional command line parameter.
Related
On a new VS solution, I've started using NUnit 3.2. Older versions (2.6) had a external NUnit GUI, that made it possible to watch assemblies and automatically run tests on modifications. But I cannot find anything similar to this for 3.2 - neither in the docs nor through Google.
I've installed NUnit.3.2.0.msi, I've also installed the NUnit3 Test Adapter in VS + NUnit3.2 nuget package for my project.
I can easily run all my tests through VS' Test Explorer. But I miss some way to run them automatically. Anyone know how?
In older versions of Visual Studio, there used to be an option to run tests after every build, but it was removed. It was always buggy and tended to lock files and prevent you from rebuilding.
You could set a post build command on your test project to automatically run NUnit console whenever your test project recompiles. You have NUnit console installed, so you could point to that, or use the NUnit.Runners package to install it into the packages folder of your solution.
Open your test project settings and go to the Build Events tab. Click on Edit Post-Build. Enter the following;
"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "$(TargetPath)"
Now, whenever you build, your tests will be run and the results will appear in the output window.
Maybe not ideal if there is a lot of build output after your tests but it works.
FYI, the colour in the build output is a side-effect of the VSColorOutput Visual Studio extension, it is not from NUnit.
Visual Studio captures STDOUT, so I haven't been able to get it to open a CMD window and run the tests. If anyone knows how to do that, add a comment and I will update.
There is a GUI for NUnit 3 under developement on GitHub - but it's not advised for production use yet.
We set our tests up using the NUnitLite runner. This allows you to turn your test assembly into an executable - and on run, will launch the console and run all tests. [Documentation]
I'm currently moving one of our projects to DNX (.NET Core now) and I was forced to update to nunit3. Because of other considerations, we run compile the test project as a console app with its own entry point, basically self-hosting the NUnit runner.
I now need to report the results to TeamCity via the XML Reporter, which doesn't seem to parse Nunit3 TestResults.xml files.
Any advice on how to work around this?
The NUnit 3 console has the option to produce results formatted in the NUnit 2 style.
Use the option:
--result=[filename];format=nunit2
Docs: https://github.com/nunit/nunit/wiki/Console-Command-Line
To add to the answer above:
NUnitLite inherits the --result CLI parameter which seems to do the trick.
Another option, which I went for in the end is using the --teamcity CLI parameter:
dotnetbuild --project:<path to project directory> -- --teamcity
which will integrate with TC's service messages. This will also do real-time updates.
Steps taken:
I downloaded and install NUnit.
I downloaded Concordion.NET.
I downloaded the Concordion.NET kickstarter (zip)
I verified that the specification (HelloWorld.html) is marked as an embedded resource.
I verified that the RequiredAddin("ConcordionNUnitAddin") assembly attribute was specified.
I built the solution
I copied the Concordion.NUnit.dll into the addin directory of my NUnit installation (C:\Program Files (x86)\NUnit 2.6.4\bin\addins).
I loaded the kickstarter solution in NUnit GUI runner.
Issue:
NUnit does not recognize the Spec project containing the specifications, i.e. the Spec assembly is marked as non-runnable. (It runs the actual (NUnit) unit tests fine.)
My expectation is that NUnit will recognize the Kickstart.Spec project and that when I run the tests, the results of those test will be indicated in the passed/failed information. Currently, as the project is not recognized, only the success of the NUnit unit test (ShouldGreet) is present.
Are my expectations incorrect, or is there some additional set-up step I missed necessary to get the process running?
You need to use NUnit 2.6.3 for the current release of Concordion.NET.
Unfortunatelly, NUnit addins are version dependent (http://nunit.org/index.php?p=extensionTips&r=2.6.4). This is also stated in the documentation of NUnit: "Most of the add-in samples provided with NUnit are currently version dependent."
NUnit 2.6.4 was released on December 16th, 2014. Future releases of Concordion.NET will support this version as well.
Thank you for providing your feedback - the documentation of Concordion.NET needs to be updated as well to include the information which version of NUnit is supported.
Additionally, you could use the NuGet package of Concordion.NET (https://www.nuget.org/packages/Concordion.NET/) for your Visual Studio projects, which would ensure that you use always the right references of NUnit in your projects.
I remember doing this in an older version of Team City in which the NUnit Runner failed, so I tried running NUnit using the Command Line Runner instead.
I think there was an option in the Command Line Runner settings which let you specify the kind of output file: NUnit result.xml was one of them. Doing this, you could see the test result breakdown in the Web GUI, just like when the NUnit Runner worked.
But I don't see this in the latest Team City (6.5.5). Can it still be done?
PS: Before anyone asks, I can't use Team City's NUnit Runner because NUnit has a bug which will be fixed in the soon to come 2.6.0 version. Said bug does not occurr when running nunit-console
This could be done using importdata TeamCity message, this is really helpful when you are running NUnit tests manually executing nunit-console.exe from an MSBuild script, for instance using NUnit MSBuild Community Task, but keep in mind that TeamCity provides built in NUnit tests runner so you do not need this low level TeamCity scripting. Anyway you can import report in this way:
<Message
Text="##teamcity[importData type='nunit' path='...\TestResults.xml']"
Importance="High"/>
See Importing XML Reports
If you prefer to run code coverage, code inspection, test tools or
duplicate finders directly from build script, not as a build runner,
you can use the importData service messages to import generated xml
reports into TeamCity.
The functionality was still there, found it by accident when looking at another build configuration. Importing an XML is now a "Build Feature", see:
We recently upgraded CruiseControl.Net 1.4.2 to 1.5.7256.1. To our dismay the nice nunit results page that had previously been there had disappeared. All we get now is a build log with the verbose output from command line of our build.
After some research we discovered you can log into "Dashboard Administrator" from the main ccnet dashboard page and install the NUnit package. However, this did not change the output of our builds to include a NUnit result page.
Is there something extra we need to do here to have our output displayed? Any help would be greatly appreciated. I don't goto stack overflow until I've exhaustively searched on Google. I'm surprised no one else is confused or complaining about this.
FYI, we are merging our nunit XML output in the project configuration.
I'm not used to the Dashboard Administrator stuff, but you can edit the dashboard.config manually. Opposite to earlier versions CCNET version 1.5 comes with a naked configuration file.
Alternatively you may simply replace the configuration file with your 1.4 dashboard.config.
Possible duplicate of Cruise Control .net: Using packages and showing NUnit results
Don't merge manually if you use the nunit task
Edit the dashboard.config and the ccservice.exe.config files to add the nunit xsl files. Restart IIs.
Check that the Iis user has been granted access to the dashboard dir.