My bamboo build NUnit Runner task is creating the following result file:
${bamboo.build.working.directory}\bld-output\${bamboo.ACM.AssemblyInformationalVersion}\ACCCMApplication\TestResult.xml
But my NUnit Parser bamboo task (which is the next running task) is failing logging
build 06-Nov-2018 06:48:46 Tests run: 1, Errors: 0, Failures: 0, Inconclusive: 0, Time: 2.6861236 seconds
build 06-Nov-2018 06:48:46 Not run: 0, Invalid: 0, Ignored: 0, Skipped: 0
build 06-Nov-2018 06:48:46
simple 06-Nov-2018 06:48:46 Parsing test results under D:\build-dir\ACM-NUNITINT-JOB1...
simple 06-Nov-2018 06:48:46 Failing task since test cases were expected but none were found.
I have tried the following options in the NUnit Parser task for NUnit Test Results File/Directory with no success. What is the correct way to format the path to this xml? :
${bamboo.build.working.directory}\bld-output\${bamboo.ACM.AssemblyInformationalVersion}\ACCCMApplication\TestResult.xml
**/bld-output/${bamboo.ACM.AssemblyInformationalVersion}/ACCCMApplication/TestResult.xml
**/test-reports/*.xml
${bamboo.build.working.directory}/bld-output/${bamboo.ACM.AssemblyInformationalVersion}/ACCCMApplication/TestResult.xml (with outside build checked)
In the end made it as simple as possible.
The NUnit Runner configuration Result Filename was left at the root directory (i.e TestResult.xml)
And the NUnit Parser configuration NUnit Test Results File was set to **/TestResult.xml
Related
I'm having a silly problem, with Nunit3TestAdapter version 3 under dotnet 5, I could see tests while they were passing, with the execution time detailed, the "Passed Test1" in the following transcript, as long as verbosity was set to at least normal:
$ dotnet test -v normal
[...]
NUnit Adapter 3.17.0.0: Test execution complete
Passed Test1 [21 ms]
Passed Test2 [< 1 ms]
Test Run Successful.
Total tests: 2
Passed: 2
I recently upgraded to dotnet 6 and nunit adapter 4.2.0, and now I'm unable to display the detailed output, even with the higher (detailed) verbosity:
$ dotnet test -v detailed
[...]
Test run for /tmp/nunit-repro/bin/Debug/net6.0/nunit-repro.dll (.NETCoreApp,Version=v6.0)
Microsoft (R) Test Execution Command Line Tool Version 17.0.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Passed! - Failed: 0, Passed: 2, Skipped: 0, Total: 2, Duration: 24 ms - /tmp/nunit-repro/bin/Debug/net6.0/nunit-repro.dll (net6.0)
I've been looking around for some time now and cannot find a relevant configuration option. Am I missing something?
Having integration test suites made of hundreds of tests and taking several minutes to pass, it's quite frustrating to have no visual progress whatsoever, not knowing if things are running or hanging.
Someone found the solution for me, add -l "console;verbosity=detailed":
dotnet test -l "console;verbosity=detailed"
Our Jenkins server has been successfully building projects for years using Powershell to run a cake script. Out of the blue today, we started getting the following error on multiple projects:
12:10:46 Running build script...
12:10:46
12:10:46 Error: Unknown command 'Default'.
12:10:46
12:10:46 .\scripts\build.cake -target=Default -configuration=Release
12:10:46 -verbosity=Verbose -environment=dev -certificatePath=C:/Program Files
12:10:46 (x86)/xxxxx/Verisign Certificate -certificateName=xxxxxxxx_cert.pfx
12:10:46 -certificatePassword=xxxxxxxxxxx
12:10:46 ^^^^^^^ No such command
12:10:46
12:10:47 Build step 'Windows PowerShell' marked build as failure
The build.ps1 script has not changed. The build.cake script has not changed. Jenkins was not updated since the last successful builds nor any of the add-ins.
If I remove "-target=Default" from the argument list, I just get the same error on "-configuration=Release" indicating unknown command: "Release".
Has anyone else experienced this?
The possible root cause of this failure is the cake version update (0.38.x --> 1.0)
You have two ways to fix this issue
Change the Argument Syntex - arguments should always be called with multi-dash syntax (e.g. --target=Foo) instead of (e.g. -target=Foo)
Refer: https://cakebuild.net/docs/getting-started/upgrade
Pinning the cake version
Specified the version of Cake in the tools/packages.config file
Refer: https://cakebuild.net/docs/writing-builds/reproducible-builds/pinning-cake-version
The VSTS task: Generate Allure Report is failed.
The error is :
Unable to process command '##vso[results.publish type=Allure;mergeResults=true;publishRunAttachments=true;resultFiles=D:\a\1\s\allure-report\34;]' successfully. Please reference documentation (http://go.microsoft.com/fwlink/?LinkId=817296)
Unknown Test Runner.
Question: can it be a bug or problem connected to configs?
Configurations:
.Net version: 4.6.1, Nunit3, Nunit3 allure adapter
Task to run tests is a Command Line task where I used nunit3-console.exe to run test.
Configurations for Allure plugin:
Results directory : allure-results
Target directory : allure-report/$(Build.BuildNumber)
The output logs:
****************************************************************************** Starting: Generate Allure Report
============================================================================== Task : Generate Allure Report Description : Generates Allure
report based on the test results Version : 1.0.0 Author :
Molecula Help : More
Information
============================================================================== C:\Program Files\nodejs\node.exe
D:\a_tasks\AllureGenerate_5c975f9d-1c3a-469f-b7c2-8907bf3eacfb\1.0.0\node_modules\allure-commandline\bin\allure
generate --output D:\a\1\s\allure-report\34 D:\a\1\s\allure-results
Command aborted due to exception {}.
org.apache.commons.exec.ExecuteException: Process exited with an
error: 1 (Exit value: 1) at
org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
at
ru.yandex.qatools.allure.command.ReportGenerate.runUnsafe(ReportGenerate.java:48)
at
ru.yandex.qatools.allure.command.AbstractCommand.run(AbstractCommand.java:52)
at ru.yandex.qatools.allure.CommandLine.main(CommandLine.java:46)
org.apache.commons.exec.ExecuteException: Process exited with an
error: 1 (Exit value: 1) at
org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:166)
at
org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:153)
at
ru.yandex.qatools.allure.command.ReportGenerate.runUnsafe(ReportGenerate.java:48)
at
ru.yandex.qatools.allure.command.AbstractCommand.run(AbstractCommand.java:52)
at ru.yandex.qatools.allure.CommandLine.main(CommandLine.java:46) at
ru.yandex.qatools.allure.data.AllureReportGenerator.generate(AllureReportGenerator.java:58)
at
ru.yandex.qatools.allure.data.AllureReportGenerator.generate(AllureReportGenerator.java:53)
at ru.yandex.qatools.allure.AllureMain.main(AllureMain.java:48)
Exception in thread "main"
ru.yandex.qatools.allure.data.ReportGenerationException: Could not
find any allure results at
ru.yandex.qatools.allure.data.AllureReportGenerator.generate(AllureReportGenerator.java:58)
at
ru.yandex.qatools.allure.data.AllureReportGenerator.generate(AllureReportGenerator.java:53)
at ru.yandex.qatools.allure.AllureMain.main(AllureMain.java:48)
Unable to process command '##vso[results.publish
type=Allure;mergeResults=true;publishRunAttachments=true;resultFiles=D:\a\1\s\allure-report\34;]'
successfully. Please reference documentation
(http://go.microsoft.com/fwlink/?LinkId=817296) Unknown Test Runner.
The default setting directory "allure-results" in the Allure task usually does not point to the actual path to result directory correctly since the path is different in different projects and environment. You need to update the setting to configure it to the actual path to the result directory manually.
According to the information you provided, the path to the result could be like this:
$(Build.SourcesDirectory)\SolutionFolder\ProjectFolder\bin\$(BuildConfiguration)\allure-results
I use NUnit and TeamCity to run my tests.
Some tests (not all) has actions made in test class constructor. I call these actions "pre-actioins" for validations. So in one test class I have for example 5 validations (tests) and a set of pre-actions.
I noticed that if a suite of tests if failed on the stage of pre-actions executing then TeamCity doesn't display these tests in its report at all (not under any status).
In build log I see error like:
SetUp Error : {test_name} + error code.
What I expect from TeamCity is to report these tests at least as Ignored.
To compare running tests using TeamCity with running tests using Visual Studio in Visual Studio the result of the same failure condition will be failure for all the test suite. Failure error will be the same for all the tests.
So what I want is just to know if some of my tests were not run at all because if TeamCity doesn't include then in test results then I don't even know about problems!
Configs: TeamCity 10.0, NUnit 3.0.
Command line params: --result=TestResult.xml --workers=4 --teamcity
Update: results of tests executing in log looks like:
[13:03:48][Step 1/1] Test Run Summary
[13:03:48][Step 1/1] Overall result: Failed
[13:03:48][Step 1/1] Tests run: 82, Passed: 0, Errors: 82, Failures: 0, Inconclusive: 0
[13:03:48][Step 1/1] Not run: 0, Invalid: 0, Ignored: 0, Explicit: 0, Skipped: 0
[13:03:48][Step 1/1] Start time: 2016-09-08 09:56:33Z
[13:03:48][Step 1/1] End time: 2016-09-08 10:03:48Z
[13:03:48][Step 1/1] Duration: 434,948 seconds
So NUnit marks such tests even not as failed but as "erros". Still I want them in test results.
Your tests are errors because you are throwing an exception in the constructor. Since the test fixture can't be constructed, the test is not really being run as far as NUnit is concerned. The fact that it's an NUnit assertion failure causing the exception is irrelevant in the context of constructing the object.
We have always advised people to keep their constructors very simple because NUnit makes no guarantees about when and how often your object will be constructed. Using assertions in the constructor is an extreme violation of that principal and, in fact, I've never seen anyone do it before.
The OneTimeSetUp attribute is there if you want some thing to happen every time your test is run as opposed to constructed. NUnit does make guarantees about when that method will be executed. :-)
None of this tells me for sure why TC is not recognizing the error but I'm guessing it's because once the constructor fails, the tests are never actually run. NUnit itself compensates for that by reporting the tests as errors but TC would not necessarily do the same.
We have a Scala project and we use SBT as its build tool.
our CI tool is TeamCity, and we build the project using the command line custom script option with the following command:
call %system.SBT_HOME%\bin\sbt clean package
The build process works fine when the build succeeds, however, when compilation fails - TeamCity thinks that the script exited with exitCode 0 and not 1 as expected, this cause TeamCity build to succeed although the compilation failed.
when we run the same commands on local cmd we see that the errorLevel is 1.
the relevant part of the build log:
[11:33:44][Step 1/3] [error] trait ConfigurationDomain JsonSupport extends CommonFormats {
[11:33:44][Step 1/3] [error] ^
[11:33:44][Step 1/3] [error] one error found
[11:33:45][Step 1/3] [error] (compile:compile) Compilation failed
[11:33:45][Step 1/3] [error] Total time: 12 s, completed Jan 9, 2014 11:33:45 AM
[11:33:45][Step 1/3] Process exited with code 0
how can we make TeamCity recognize the failure of the build?
Try explicitly exit with:
call %system.SBT_HOME%\bin\sbt clean package
echo the exit code is %errorlevel%
exit /b
If you can't get the process to output a non-zero exit code then you could use a build failure condition based on specific text in the build log. See this page for the documentation but in essence you can get the build to fail if it finds the text error found in the build log.