On the TFS 2013 BuildServer, i run Post-Test PowerShellScript to Execute JavaScript Unit Test using Grunt. This produces the .xml file on the build server.
I want to write the results from .xml to the TestResult of TFS Server.
Is there a way to do this ?
Looking for a way to push the Pass and Faile results of JavaScript Test into TestResult(.trx) of TFS 2013.
You can use vstest.console.exe to publish test results. See this blog post on publishing test results using vstest: http://blogs.msdn.com/b/visualstudioalm/archive/2012/12/06/publishing-test-results-through-command-line-test-runner.aspx You should be able to invoke that exe from an InvokeProcess activity.
vstest.console.exe with TfsPublisher switch would allow you to run tests results from a unit test run executed by vstest.console.exe. Since, you are using JavaScript unit test with Grunt, that won't work.
Your best bet is to create an activity that reads information from the xml file and publish results to you team build. Have a look at this project
https://github.com/hamidshahid/TfsBuildResultPublisher
There are bits of code that you an use from this project specially around adding your own test results information.
Related
I have a TeamCity build configuration that builds a C# project, runs some unit tests, and then does some extra things. My question is: Can I get information about my unit test run stored into build configuration variables (i.e. how many tests were run, how many were successful, how many failed, how many were skipped) so that I can then check these variables in a PowerShell script in later build steps and perform different actions depending on how many tests have passed?
AFAIK the best way is to ask these information directly to teamcity server using its REST API (pay attention, maybe the build locator could be a little be tricly to be found, if the build is still running).
By other hand, you can parse your NUnit test result file (or files if you run more than one NUnit test runner step in your build) inside your build agent machine.
I did some research and I always found that the tests are located in some dll. However in my project there are no .dlls created and the tests are located in the .exe file.
Do I have to set some specific parameters?
This is the log from vsts after I pointed the path for test assemblies to the exe. Before I pointed it to **\*.test*.dll but it could never find any tests. When running the tests or build locally there is also never a test dll created.
2017-08-15T08:55:15.1386003Z Starting test execution, please wait...
2017-08-15T08:55:15.4276084Z Information: NUnit Adapter 3.8.0.0: Test execution started
2017-08-15T08:55:15.4276084Z
2017-08-15T08:55:15.4276084Z Information: Running all tests in d:\a\1\s\Workshop\Workshop\bin\Release\Workshop.exe
2017-08-15T08:55:15.4286085Z
2017-08-15T08:55:16.8199687Z Information: NUnit3TestExecutor converted 1 of 1 NUnit test cases
2017-08-15T08:55:16.8199687Z
2017-08-15T08:55:16.9559755Z Information: NUnit Adapter 3.8.0.0: Test execution complete
2017-08-15T08:55:16.9559755Z
2017-08-15T08:55:17.2181003Z Passed TwoIntegers_Add_Sum
How do I make the tests show up as .dlls instead of being inside the .exe?
As your example shows, NUnit as well as VS Test Explorer are very happy to have tests in an .exe along with the production code. Most people prefer to have the tests separate from the production code, which is distributed to users, so they create a separate project to contain them.
So, your answer is that there is no way to make the tests "show up" as a .dll if they are actually in the .exe. OTOH, assuming you are the developer of the application, you can create a separate project, which creates a class library, and move the tests into that project.
Im using TeamCity with Gallio/XUnit/Specflow and trying to display our unit/acceptance test results. I believe the output is NUnit xml test result format.
Our unit tests auotmatically display using the command runner with Gallio. The acceptance tests are run through a large Powershell script which calls Gallio (Run-Gallio).
Results are output to acceptance-test-results.xml. Is there a way to display the acceptance-test-results.xml in TeamCity (6.5.5)?
To do this in a PowerShell script add the following
Write-Output "##teamcity[importData type='nunit' path='C:\SomeDirectory\YourResults.xml']"
You can use built in TeamCity feature importdata service message:
<!-- Send to TeamCity a service message using MSBuild -->
<Message Text="##teamcity[importData
type='nunit'
path='$(OutputPath)\UnitTestsReport.xml']"
Importance="High" />
For more details see: Build Script Interaction with TeamCity - Importing XML Reports
Thought this might be useful also for other people that came to this SO post with slightly different requirements. It's the public wiki for TC that covers this particular area.
http://confluence.jetbrains.com/display/TCD8/Build+Script+Interaction+with+TeamCity#BuildScriptInteractionwithTeamCity-ImportingXMLReports
I remember doing this in an older version of Team City in which the NUnit Runner failed, so I tried running NUnit using the Command Line Runner instead.
I think there was an option in the Command Line Runner settings which let you specify the kind of output file: NUnit result.xml was one of them. Doing this, you could see the test result breakdown in the Web GUI, just like when the NUnit Runner worked.
But I don't see this in the latest Team City (6.5.5). Can it still be done?
PS: Before anyone asks, I can't use Team City's NUnit Runner because NUnit has a bug which will be fixed in the soon to come 2.6.0 version. Said bug does not occurr when running nunit-console
This could be done using importdata TeamCity message, this is really helpful when you are running NUnit tests manually executing nunit-console.exe from an MSBuild script, for instance using NUnit MSBuild Community Task, but keep in mind that TeamCity provides built in NUnit tests runner so you do not need this low level TeamCity scripting. Anyway you can import report in this way:
<Message
Text="##teamcity[importData type='nunit' path='...\TestResults.xml']"
Importance="High"/>
See Importing XML Reports
If you prefer to run code coverage, code inspection, test tools or
duplicate finders directly from build script, not as a build runner,
you can use the importData service messages to import generated xml
reports into TeamCity.
The functionality was still there, found it by accident when looking at another build configuration. Importing an XML is now a "Build Feature", see:
Following a MSDN web page, I am trying to manually run mstest within my tfsbuild.proj and put the results into the pass/fail logic so the build will fail if this particular test fails. It's kind of like running a FxCop or something else from CMD and capturing a "0" or "1" and force-fail the build.
MSTest /testcontainer:test.dll /publish:http://ourtfsmachine:8080 /teamproject:ProjectName /publishbuild:BuildNumber01 /platform:AnyCpu /flavor:Release
I could understand running this inside an Exec task, butI don't know what the BuildNumber is, for example.
Help?
Instructions for getting the Build Number from http://msdn.microsoft.com/en-us/library/ms243151%28VS.100%29.aspx:
Open Visual Studio and connect to a Team Foundation Server.
Open Team Explorer.
Open your team project and expand the team project node.
Under the build, double-click All Build Types or a specific build type to see its builds. Build names that you can use are in the Name column.