What I'm aiming for is closing the loop on test coverage.
My goal would be:
Define test cases with PM and list them out under a Test Plan in Azure DevOps.
Associate Selenium tests in Visual Studio with specific test case(s).
When the release pipeline runs the UI test, I would expect to see on the dashboard the % of test cases that passed/failed or were not even associated.
Is that even possible?
Check the following link:
https://learn.microsoft.com/en-us/azure/devops/test/associate-automated-test-with-test-case?view=azure-devops
Coded UI test, Selenium tests, and unit tests written using Version 1
of the MSTest framework can be associated with a test case.
The process to associate an automated test with a test case is:
Create a test project containing your automated test.
Check your test project into an Azure DevOps repository.
Create a build pipeline for your project, ensuring that it contains the automated test.
Use Visual Studio Enterprise or Professional 2017 or a later version to associate the automated test with a test case. The test case must have been added to a test plan that uses the build you just defined.
Check this extension that I have created https://github.com/JanuszNowak/janono.ado.testcase.associate.cli . It allows associating in automatic manner. So if yours selenium test are using xunit, nunit or mstest you can associate them automatically.
Code sample:
namespace ExampleTestProject
{
[TestClass]
[janono.ado.testcase.associate.Organization("janono-pub")]
public class UnitTest1
{
[TestMethod]
[janono.ado.testcase.associate.TestCase(5)] //<---
public void TestMethod1()
{
//yours test method content
//...
//
}
}
}
I'm using VS 2013 and NUnit.
I'm running my tests in Resharpers's TestRunner GUI - this thing:
I have multiple projects (Solution.Foo, Solution.Bar, etc), and tests for each project in Solution.Foo.Tests, Solution.Bar.Tests, etc.
One of my tests in Foo.Tests is accessing a file, located in the Solution.Foo.Tests Folder.
I use TestContext.CurrentContext.TestDirectory to locate it and when I run that test it all works just fine.
If I run the whole suite of tests in Foo.Tests, (i.e. click on Solution.Foo.Tests in the Runner GUI and run) then it's fine.
But if I select one test from Bar.Tests and my file-reading test from Foo.Tests, then it fails, because suddenly TestDirectory is set to a path in Bar.Tests.
It's as if NUnit set TestDirectory once when it starts running a collection of tests, determined by where the first test is and then never updates it again.
Any ideas what's going on and how to fix it?
ReSharper has an optimisation for speed, by sharing an AppDomain across multiple assemblies. The downside to this is that it will pick an arbitrary assembly to be the main one, which means the directory can be wrong (and it might not pick up the correct app.config, too).
You can disable the optimisation in ReSharper → Options → Unit Testing, by checking the "Use separate AppDomain for each assembly with tests".
Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
I've installed NUnit Test Adapter for VS2012 + 2013. When I first installed the Adapter tests were showing up, but they stopped showing up for some reason today. After building, rebuilding, cleaning, restarting, nothing shows up in Test Explorer. Why would this be happening? I'm using VS2013 Ultimate.
If you're using a NUnit3+ version, there is a new Test Adapter available.
Go to "Tools -> Extensions and Updates -> Online" and search for "NUnit3 Test Adapter" and then install.
If your test project is set to target a 64bit platform, the tests won't show up in the NUnit Test Adapter.
My test assembly is 64-bit. From the menu bar at the top of visual studio 2012, I was able to select 'Test' -> 'Test Settings' -> 'Default Processor Architecture' -> 'X64'. After a 'Rebuild Solution' from the 'Build' menu, I was able to see all of my tests in test explorer. Hopefully this helps someone else in the future =D.
Check for NUnit versions mismatch. The currently available NUnit Test Adapter only works for NUnit version 2.6.4 and below.
To downgrade NUnit from version 3.x go to
Package Manager Console > update-package NUnit -version 2.6.4
http://jeremybytes.blogspot.co.ke/2015/11/review-of-unit-testing-makes-me-faster.html
In my situation the 'NUnit3 Test Adapter' has been disabled.
To re-enable it go to menu
Tools->Extensions and Updates...
On the left side select 'Installed'->'All'.
On the upper right corner search for 'nunit'.
If you have 'NUnit3 Test Adapter' installed, with the found item you can enable/disable it.
This answer seems pretty basic but wasn't completely obvious to me at first. If you (re)build the solution it only builds the projects that are configured to build in the Build -> Configuration Manager
This was my issue, I must have inadvertently changed a build configuration settings or something that caused my test projects not to build (when they previously were). So the Test Explorer window was looking at dlls that were out of date. It became clear to me this was the case after doing a Clean and seeing most of my tests disappear and not come back after a rebuild...further inspection of bin folder showed that these projects weren't being built at all.
I had a working setup (for NUnit2 and NUnit3 depending on the solution, and multiple versions of Visual Studio between 2012 and 2017), and it suddenly stopped working one day: no tests detected in any solution or version of VS.
In my case, it helped to delete %localappdata%\Temp\VisualStudioTestExplorerExtensions. After a restart of VS, everything worked as before.
If you are using VS 2017 and .net core ,as said here, you should add references to the test framework NUnit 3.6.1, to the test runner NUnit3TestAdapter 3.8.0-alpha1 and to the test SDK Microsoft.NET.Test.Sdk 15.0.0.
Tools
NuGet Package Manager
Manage NuGet Packages For Solution
Browse
NUnitTestAdapter.WithFramework
Ctrl+R,A to build/run tests
Using NUnitTestAdapter.WithFramework makes sure there are little/no inconsistencies across versions of NUnit and NUnit Adapter (i.e. "it just works")
Check whether you have stated
[TestFixureSetUp]
and
[Test]
in the test class
sample:
namespace ClassLibrary1
{
public class SimpleCalculator
{
public Calculator _calculator;
[TestFixtureSetUp]
public void initialize()
{
_calculator = new Calculator();
}
[Test]
public void DivideTest()
{
int a = 10;
int b = 2;
int expectedValue = a/b;
int actualValue = _calculator.Divide(a, b);
Assert.AreEqual(expectedValue, actualValue, "Functionality not working properly!");
}
}
}
I had to uninstall then re-install the xunit.runner.visualstudio nuget package. I tried this after trying all the above suggestions, so may be it was a mixture of things.
One other cause to this problem is if you open a project from a mapped drive - Visual Studio handles such projects properly, but apparently Nunit doesn't support them.
Copying the project to a physical fixed the issue.
I also found that when I uninstalled nunit v3.2.1, the nunit framework reference for v3.2.1 was still in my project in solution explorer.
Solution Explorer > ProjectName > References
If you right click it will show the version. Remove this
Then Right click on References > Add Reference.
Search for the version 2.x version and add then rebuild solution.
That worked for me!
I experienced the problem mentioned by op
My case was that I was handed an old project and the tests were actually part of the system under test. I assume they were using the external test runner.
this task chain resolved the issue for me
created a test project,
moved the test files there
added references so the test project would compile
added the Nunit and Nunit adapter NuGet packages to the test project
recompiled
i was able to successfully run the Nunit tests.
If you are using the TestCaseSource attribute, ensure the source exists and respects the documentation, otherwise your tests will not be discovered.
I had a similar issue where the tests where not being discovered. I had the correct version of NUnit, versions matched up between NUnit and adapter, and the tests where tagged correctly. I was running VS 2017 Enterprise not as an administrator. After starting VS as administrator the tests appeared.
I started a new solution with a test project in it, and compared it against my original, problem project. The original, for some reason, had an app.config in it. I excluded that file from the project and saw my tests reappear in the test explorer.
I had this problem too but the cause was different. I'm using VS2017 with F# 4.0.
Firstly, the console in Visual Studio does not give you enough details why the tests could not be found; it will just fail to the load the DLL with the tests. So use NUnit3console.exe on the command line as this gives you more details.
In my case, it was because the test adapter was looking for a newer version of the F# Core DLL (4.4.1.0) (F# 4.1) whereas I'm still using 4.4.0.0 (F# 4.0). So I just added this to the app.config of the test project:-
<dependentAssembly>
<assemblyIdentity name="FSharp.Core" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-65535.65535.65535.65535" newVersion="4.4.0.0" />
</dependentAssembly>
i.e. redirect to the earlier F# core.
I had the same problem, when suddenly any test didn't appeared on Test Explorer window.
I has the updated version of "NUnit3TestAdapter"
and after lots of searches and efforts,
I found that I need set the following values in project properties:
[In Solution Explorer window: right click on Project > Properties]
Under Build tab, set Platform=x64, and set Platform target=x86 or Any CPU
Build the project and all tests will be appear on Test Explorer window.
Important note:
I came to a solution after seeing the next msg in the output window:
"Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run: AutomationTests.dll is built for Framework Framework45 and Platform X64."
I had some msbuild.exe processes that were hung. I don't know if that was my problem or not, but it took me a lot of trail and error with reinstalling various NUnit adaptors before I found the hung processes.
Just to add my $.02 here, I ran into a similar issue just yesterday, where 168 of my tests were missing. I tried most everything in this post - most especially making sure my version(s) of NUnit were the same - all to no avail. I then remembered that I had my tests divided into playlists; and these do not update automatically as you add new tests. So, when I deleted the playlists, BAM!, all of my tests were back once more.
I use NUnit 3 and tried most of the other solutions I saw here. None of those worked for me.
Even though already selected, reselecting the "Playlist: All Tests" option in test explorer showed all hidden tests in my case. I need to do this after every rebuild.
Make sure your test class is public. I often make this mistake then I just look at the code for 5 minutes what is going on.
[Test]
public void YourTest()
{
...
}
Tools
NuGet Package Manager
Manage NuGet Packages For Solution
Browse
NUnit3TestAdapter (NUnit 3 in my case ..)
After installation it should be possible to use the default TestExplorer included with the installation of Visual Studio.
If your test project is not x86, the tests will not be found until you tell the test runner the correct target platform.
Prior to Visual Studio 2022, this was the "Test/Test Settings/Default Processor Architecture" option.
In VS2022 you need to create a 'runsettings' file and set the test project to use it. In that file you can specify the Target Platform, like this:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<TargetPlatform>x64</TargetPlatform>
</RunConfiguration>
</RunSettings>
You still also need to include the Nunit3TestAdapter and Microsoft.NET.Test.Sdk nugets (and make sure the project actually built, and check the various other more obscure mistakes mentioned in other answers)
Make sure your tests are properly marked with the Test attribute. If all of the tests are marked with only the Explicit attribute, the TestAdapter doesn't recognize the fixture.
Hi i created a unit test case using NUnit.When i run the test NUnit it works fine.But i need to run it by using Visual studio.So i referred the NUnit website and followed the below steps
1)Right clicked the test project and clicked properties |Chose Debug option
2)checked the external program option and choosed the nUnit exe file.
3)And i run the test
Error i got:
Cannot start because the test project does not contain any test method.But i included 4 test methods which works in NUnit GUI.
Thanks in advance
Check out TestDriven.NET, a free Visual Studio add-in that allows you to run your unit tests with NUnit directly from the IDE.
Also ReSharper beautifully supports testing with NUnit!
http://www.jetbrains.com/resharper/features/unit_testing.html
On the Debug options tab, make sure to also set a command line argument with the name of your test assembly.
how to assign a test file in resharper?
whenever I select 'run tests' it says no test file found.
I created a test project, how do I link it?
I've only done minimal work with NUnit and Resharper, but the way I've always done it is just write all of the test code in your test project, and link/include the library you want to test using using statements. Make sure to include the [TestFixture] and [Test] attributes in your test classes, and when you select "Run Unit Tests" in VS on your test project, it should go through each [TestFixture] class and run all of the [Test] functions.
All that being said, I have found that running nunit tests in Resharper (v3.1 at least) to be less than ideal. It does not recognize some test conditions such as ExpectedException, so I always run my tests directly in NUnit itself.