I have a Program.dll which I execute with Nunit GUI Runner on windows 7. Works perfect. The [Test] method is static.
When I execute it through bash in linux, in the XML it says :
signature is not correct: it must be an instance method.
here is the method declaration
[Test]
static public void NUnitWriter()
why does it work in windows with the Nunit GUI runner but not with it's linux console counterpart?
It's possible you're using an older version of the console application; static test methods weren't supported until version 2.5. You can check the version by running
> nunit-console.exe /?
NUnit-Console version 2.6.4.14350
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
...
If it's 2.4 or earlier, that's your problem.
Related
In test run log I have Run UnitTests step, which uses Visual Studio Test. However I see log message:
No test is available in C:\stuff\Debug\x64\bin\x86\ilc\MyApp.Test.dll some_other_dlls Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Test are visible in visual studio, and I can run all of them (I have over 50 tests).
I have xunit.runner.visualstudio package installed. What I am missing, why Azure ignores my tests?
**************** Starting test execution *********************
\vstest.console.exe "#C:\path\fadisfjla.tmp"
Microsoft (R) Test Execution Command Line Tool Version 16.4.0
Copyright (c) Microsoft Corporation. All rights reserved.
vstest.console.exe "C:\dir\Release\x64\bin\arm\MyApp.Test\testhost.dll"
"C:\dir\Release\x64\bin\arm\MyApp.Test\xunit.runner.visualstudio.uwp.testadapter.dll"
"C:\dir\Release\x64\bin\arm\ilc\MyApp.Test.dll"
"C:\dir\Release\x64\bin\x86\MyApp.Test\testhost.dll"
"C:\dir\Release\x64\bin\x86\MyApp.Test\xunit.runner.visualstudio.uwp.testadapter.dll"
"C:\dir\Release\x64\bin\x86\ilc\MyApp.Test.dll"
/Settings:"C:\tmp\fadslfj.tmp.runsettings"
/EnableCodeCoverage
/Logger:"trx"
/TestAdapterPath:"C:\dir\Release\x64"
Starting test execution, please wait...
Test run will use DLL(s) built for framework .NETFramework,Version=v4.0 and platform X86. Following DLL(s) do not match framework/platform settings.
testhost.dll is built for Framework .NETStandard,Version=v1.4 and Platform AnyCPU.
xunit.runner.visualstudio.uwp.testadapter.dll is built for Framework .NETCore,Version=v5.0 and Platform AnyCPU.
MyApp.Test.dll is built for Framework .NETFramework,Version=v4.0 and Platform ARM.
testhost.dll is built for Framework .NETStandard,Version=v1.4 and Platform AnyCPU.
xunit.runner.visualstudio.uwp.testadapter.dll is built for Framework .NETCore,Version=v5.0 and Platform AnyCPU.
Go to more details on managing these settings.
Microsoft (R) Coverage Collection Tool Version 16.0.30319.200
Copyright (c) Microsoft Corporation. All rights reserved.
1.5873
A total of 6 test files matched the specified pattern.
No test is available in C:\dir\Release\x64\bin\arm\MyApp.Test\testhost.dll C:\dir\Release\x64\bin\arm\MyApp.Test\xunit.runner.visualstudio.uwp.testadapter.dll C:\dir\Release\x64\bin\arm\ilc\MyApp.Test.dll C:\dir\Release\x64\bin\x86\MyApp.Test\testhost.dll C:\dir\Release\x64\bin\x86\MyApp.Test\xunit.runner.visualstudio.uwp.testadapter.dll C:\dir\Release\x64\bin\x86\ilc\MyApp.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Microsoft (R) Coverage Collection Tool Version 16.0.30319.200
Copyright (c) Microsoft Corporation. All rights reserved.
Results File: C:\path\fdsfdsfd.trx
Attachments:
C:\path\file-dfsfdsfe.coverage
Vstest.console.exe exited with code 0.
In my project file
<PackageReference Include="xunit.runner.visualstudio">
<Version>2.4.1</Version>
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
<PrivateAssets>all</PrivateAssets>
</PackageReference>
First, make sure you have added the xunit.runner.visualstudio package. And then make sure you have pointed to your generated test assemblies under Test selection/Test filesin Visual Studio Test task, for example:
**\bin\$(BuildConfiguration)\**\*test*.dll
!**\obj\**
!**\xunit.runner.visualstudio.testadapter.dll
!**\xunit.runner.visualstudio.dotnetcore.testadapter.dll
Check the following link:
https://xunit.net/docs/getting-test-results-in-azure-devops
It turns out that if You have UWP project for Windows version 1803, and refrence it from test project in this version it ignores all tests in test project. I think this is issue in xunit or windows sdk. If just test project is updated to 1903 it works flawlessly.
I've configured a VSTS build job to build my MockingFrameworkExamples .NET 4.7.1 (C#) solution from GitHub. This solution includes five assemblies with NUnit v3.10 unit tests.
I included a Visual Studio Test step in the build definition, but when it runs, the console output says no tests were available in the test assemblies:
No test is available in D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
What do I need to change to have the test runner to find and run my tests?
My local Visual Studio 2017 instance has no trouble finding and running the tests. The documentation for the Visual Studio Test step says it will run NUnit tests:
Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run.
I've made sure the assemblies are actually building and I've also utilized the Typemock SmartRunner build step to run the tests, which finds and executes them (though, it fails with the MS Fakes assembly, so I can't just use it).
Below are the full build step configuration and the console output from the "Visual Studio Test" step. The build process is configured to run on a "Hosted VS2017" agent.
2018-07-19T18:28:28.1197107Z ##[section]Starting: VsTest - testAssemblies
2018-07-19T18:28:28.1205564Z ==============================================================================
2018-07-19T18:28:28.1205798Z Task : Visual Studio Test
2018-07-19T18:28:28.1206209Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test runner. Test frameworks that have a Visual Studio test adapter such as xUnit, NUnit, Chutzpah, etc. can also be run. Tests can be distributed on multiple agents using this task (version 2).
2018-07-19T18:28:28.1206596Z Version : 2.136.10
2018-07-19T18:28:28.1206775Z Author : Microsoft Corporation
2018-07-19T18:28:28.1207003Z Help : [More Information](https://go.microsoft.com/fwlink/?LinkId=835764)
2018-07-19T18:28:28.1207231Z ==============================================================================
2018-07-19T18:28:28.6970694Z Run the tests locally using vstest.console.exe
2018-07-19T18:28:28.6971006Z ========================================================
2018-07-19T18:28:28.6971725Z Test selector : Test assemblies
2018-07-19T18:28:28.6972103Z Test assemblies : **\release\*examples*.dll,!**\release\typemock*examples*.dll,!**\obj\**
2018-07-19T18:28:28.6972430Z Test filter criteria : null
2018-07-19T18:28:28.6972706Z Search folder : D:\a\1\s
2018-07-19T18:28:28.6972990Z Run settings file : D:\a\1\s
2018-07-19T18:28:28.6973481Z Run in parallel : false
2018-07-19T18:28:28.6973748Z Run in isolation : false
2018-07-19T18:28:28.6975710Z Path to custom adapters : null
2018-07-19T18:28:28.6975982Z Other console options : null
2018-07-19T18:28:28.6976254Z Code coverage enabled : true
2018-07-19T18:28:28.6976788Z Rerun failed tests: false
2018-07-19T18:28:28.6977166Z VisualStudio version selected for test execution : latest
2018-07-19T18:28:29.7035629Z ========================================================
2018-07-19T18:28:30.0191054Z [command]"C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" #D:\a\_temp\88c64f61-8b81-11e8-a282-258e7155540f.txt
2018-07-19T18:28:33.5193511Z Microsoft (R) Test Execution Command Line Tool Version 15.7.2
2018-07-19T18:28:33.5198868Z Copyright (c) Microsoft Corporation. All rights reserved.
2018-07-19T18:28:33.5199025Z
2018-07-19T18:28:33.5440200Z vstest.console.exe
2018-07-19T18:28:33.5441525Z "D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll"
2018-07-19T18:28:33.5442179Z "D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll"
2018-07-19T18:28:33.5442549Z "D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll"
2018-07-19T18:28:33.5442909Z "D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll"
2018-07-19T18:28:33.5443312Z /EnableCodeCoverage
2018-07-19T18:28:33.5443575Z /logger:"trx"
2018-07-19T18:28:35.6891585Z Starting test execution, please wait...
2018-07-19T18:28:42.3946126Z Microsoft (R) Coverage Collection Tool Version 15.0.30319.1
2018-07-19T18:28:42.3947458Z
2018-07-19T18:28:42.3947721Z
2018-07-19T18:28:42.3948539Z Copyright (c) Microsoft Corporation. All rights reserved.
2018-07-19T18:28:42.3949162Z
2018-07-19T18:28:42.3949300Z
2018-07-19T18:28:42.3949487Z
2018-07-19T18:28:42.3949624Z
2018-07-19T18:28:42.6811429Z 2.7184
2018-07-19T18:28:53.8206723Z No test is available in D:\a\1\s\MSFakesExamples\bin\Release\MSFakesExamples.dll D:\a\1\s\MoqExamples\bin\Release\MoqExamples.dll D:\a\1\s\NSubstituteExamples\bin\Release\NSubstituteExamples.dll D:\a\1\s\RhinoMocksExamples\bin\Release\RhinoMocksExamples.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
2018-07-19T18:28:53.9442220Z Microsoft (R) Coverage Collection Tool Version 15.0.30319.1
2018-07-19T18:28:53.9443798Z
2018-07-19T18:28:53.9445199Z
2018-07-19T18:28:53.9465185Z Copyright (c) Microsoft Corporation. All rights reserved.
2018-07-19T18:28:53.9465983Z
2018-07-19T18:28:53.9466184Z
2018-07-19T18:28:53.9466360Z
2018-07-19T18:28:53.9466554Z
2018-07-19T18:28:54.6483114Z
2018-07-19T18:28:54.6491201Z Attachments:
2018-07-19T18:28:54.6491635Z D:\a\1\s\TestResults\324c9712-62bd-4959-b3b6-40bebf7c05ec\VssAdministrator_factoryvm-az243 2018-07-19 18_28_41.coverage
2018-07-19T18:28:54.6491944Z
2018-07-19T18:28:54.6574103Z Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:<pathToCustomAdapters>.
2018-07-19T18:28:54.6717566Z ##[warning]No results found to publish.
2018-07-19T18:28:54.7119869Z ##[section]Finishing: VsTest - testAssemblies
It turns out I had to add a NuGet reference to each of the unit test assemblies to the NUnit3TestAdapter NuGet package for this to work. No other config changes were needed.
You need to specify the path to the NUnit adapter using /TestAdapterPath.
I tried compiling .c files to .mex , everything goes well and mex files are created but still it gives me the following errors. can anybody explain what is .exp file and why its giving me error?
Could Not Find E:\research data\libsvm-3.21\libsvm-3.21\matlab\svmtrain.exp
Could Not Find E:\research data\libsvm-3.21\libsvm-3.21\matlab\svmtrain.exp
MEX completed successfully.
Building with 'Microsoft Windows SDK 7.1 (C++)'.
Could Not Find E:\research data\libsvm-3.21\libsvm-3.21\matlab\svmpredict.exp
Could Not Find E:\research data\libsvm-3.21\libsvm-3.21\matlab\svmpredict.exp
MEX completed successfully.
This problem may be caused by the different version of the C++ compiler.
I solved this problem by chooseing the C++ 2010 compiler.
To setup the compiler, type "mex -setup" first then type "mex -setup:I:\matlab2014b_win7\installation(root of your matlab installation)\bin\win64\mexopts\msvc2010.xml"
MEX configured to use 'Microsoft Visual C++ 2012 (C)' for C language compilation.
Warning: The MATLAB C and Fortran API has changed to support MATLAB
variables with more than 2^32-1 elements. In the near future
you will be required to update your code to utilize the
new API. You can find more information about this at:
http://www.mathworks.com/help/matlab/matlab_external/upgrading-mex-files-to-use-64-bit-api.html.
To choose a different C compiler, select one from the following:
Microsoft Visual C++ 2010 (C)
mex -setup:I:\matlab2014b_win7\installation\bin\win64\mexopts\msvc2010.xml C
Microsoft Visual C++ 2012 (C)
mex -setup:C:\Users\Administrator\AppData\Roaming\MathWorks\MATLAB\R2014b\mex_C_win64.xml C
Microsoft Visual C++ 2013 Professional (C)
mex -setup:I:\matlab2014b_win7\installation\bin\win64\mexopts\msvc2013.xml C
I have some parametrized tests that use a fixture defined in conftest. My tests are failing when pytest-xdist is used but pass with standard pytest. The fixture creates a directory which is then used by the test and I'm getting IOErrors.
From the output, it looks like the two failing parametrized tests are either being collected and/or being run multiple times. Note in the output below that test_save_load[obj2] and test_save_load[obj0] are failing and these appear twice so I assume they are being run multiple times. I can write the test differently to not parametrize; but was curious to understand this problem. Any ideas on what is going on or how pytest-xdist works to help understand the issue would be greatly appreciated.
Thanks!
Here's a snippet of my output:
platform darwin -- Python 2.7.6 -- pytest-2.5.1 --
plugins: xdist
[gw0] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw1] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw2] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw3] darwin Python 2.7.6 cwd: /tests/unit_tests
[gw0] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw2] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw1] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
[gw3] Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54) -- [GCC 4.2.1 (Apple Inc. build 5666) (dot 3)]
gw0 [30] / gw1 [30] / gw2 [30] / gw3 [30]
scheduling tests via LoadScheduling
test_save_load.py:17: test_exceptions
test_save_load.py:135: test_save_load[obj0]
test_save_load.py:45: test_gnome_obj_reference
test_save_load.py:135: test_save_load[obj2]
[gw3] PASSED test_save_load.py:17: test_exceptions
[gw1] PASSED test_save_load.py:45: test_gnome_obj_reference
test_save_load.py:31: test_reference_object
[gw3] PASSED test_save_load.py:31: test_reference_object
test_save_load.py:73: test_savloc_created
test_save_load.py:135: test_save_load[obj4]
[gw1] PASSED test_save_load.py:73: test_savloc_created
test_save_load.py:135: test_save_load[obj5]
[gw3] PASSED test_save_load.py:135: test_save_load[obj4]
test_save_load.py:135: test_save_load[obj6]
[gw2] FAILED test_save_load.py:135: test_save_load[obj2]
[gw0] FAILED test_save_load.py:135: test_save_load[obj0]
xdist output is a little different than standard's pytest because the tests are running in parallel. When you see:
test_save_load.py:135: test_save_load[obj0]
it means that the test has been sent to one of the nodes for execution.
When you see:
[gw0] FAILED test_save_load.py:135: test_save_load[obj0]
it just means that the test has finished in node 0.
That being said, your tests are being executed only once. Your description of the test failure seems to indicate that the tests are sharing some disk resource (for instance, they are all trying to write to the same file name).
You should make sure that your tests that deal with files work in separate directories so each test is completely isolated from the others; you can accomplish that with the help of the tmpdir fixture.
If you post more information about the test themselves perhaps people can give more specific advice.
Cheers,
please help me:
I run nunit in in the windows console.
After more than 300 tests the programm hangs. I don't konw why.
C:\Projekte\Sochi2014_TP_testing\reporting>"NUnit 2.6"\bin\nunit-console.exe /la
bels /out=NUnitTestResult.txt /xml=NUnitTestResult.xml ..\bin\x86\Debug\Sochi201
4_TP_testing.dll
NUnit-Console version 2.6.0.12051
Copyright (C) 2002-20011 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
CLR Version: 2.0.50727.5456 ( Net 3.5 )
ProcessModel: Default DomainUsage: Single
Execution Runtime: net-3.5
Log4Net not configured. Looked for file: C:\Projekte\Sochi2014_TP_testing\bin\x8
6\Debug\log4net.config
More details for my issue:
I use selenium and specflow.
Nunit hangs up in a "Scenario Outline". The previous test runs normal. The browser (FF 13) is open and runs normal when I use the mouse.
My guess is that you have a test with a deadlock. You can add a timeout to NUnit:
nunit-console /timeout=10000 (other options as before)
This adds a 10 second timeout to each test. Then you should be able to see which test times out, and work out where the problem is. The number of tests shouldn't be an issue - my Noda Time project has thousands of tests which run without a problem, and I'm sure that's far from the biggest project using NUnit...