I have found an explicable (but frustrating) behavior when working with Moles and MsTest.
Just imagine the following case:
"Test DLL A" is using Moles on mscorlib
"Test DLL B" is using Moles on mscorlib
To improve compilation time, in both cases we are editing the .moles files in order to ask the generation of moles for a single class.
When we do so, our projects will compile perfectly fine.
But when we run the test of our solution the MsTest process will be :
Copy all DLLs in the "Out" folder
Run the tests in the "Out" folder
As a consquence, the copy to the "Out" folder will try to copy two version of mscorlib.Moles.dll (one with type 1, and one with type 2) and of course, the second one will overwrite the first one.
And so my test of "Test DLL A" will fail because my mole assembly is not correct.
There are of course two simple workaround :
either include all needed types (in all projects) in every .moles file
either do not use type filtering
Have you ever faced this "problem" also ? is there any other solution ?
Many thanks !
Pierre-Emmanuel
DotNetHub user group lead
This is a late reply I know, but we did run into the same thing here at my shop.
What we ended up doing was creating a project just for moles. And then having all other unit test projects reference the .dlls created in our MolesProject/Moles folder.
We were able to leverage that and improve build times
Related
I did some research and I always found that the tests are located in some dll. However in my project there are no .dlls created and the tests are located in the .exe file.
Do I have to set some specific parameters?
This is the log from vsts after I pointed the path for test assemblies to the exe. Before I pointed it to **\*.test*.dll but it could never find any tests. When running the tests or build locally there is also never a test dll created.
2017-08-15T08:55:15.1386003Z Starting test execution, please wait...
2017-08-15T08:55:15.4276084Z Information: NUnit Adapter 3.8.0.0: Test execution started
2017-08-15T08:55:15.4276084Z
2017-08-15T08:55:15.4276084Z Information: Running all tests in d:\a\1\s\Workshop\Workshop\bin\Release\Workshop.exe
2017-08-15T08:55:15.4286085Z
2017-08-15T08:55:16.8199687Z Information: NUnit3TestExecutor converted 1 of 1 NUnit test cases
2017-08-15T08:55:16.8199687Z
2017-08-15T08:55:16.9559755Z Information: NUnit Adapter 3.8.0.0: Test execution complete
2017-08-15T08:55:16.9559755Z
2017-08-15T08:55:17.2181003Z Passed TwoIntegers_Add_Sum
How do I make the tests show up as .dlls instead of being inside the .exe?
As your example shows, NUnit as well as VS Test Explorer are very happy to have tests in an .exe along with the production code. Most people prefer to have the tests separate from the production code, which is distributed to users, so they create a separate project to contain them.
So, your answer is that there is no way to make the tests "show up" as a .dll if they are actually in the .exe. OTOH, assuming you are the developer of the application, you can create a separate project, which creates a class library, and move the tests into that project.
I'm to use nunit-console to run all of the tests in my solution.
I did this:
c:\some\path>nunit-console-x86.exe MySolution.sln
NUnit-Console version 2.6.2.12296
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
CLR Version: 2.0.50727.5466 ( Net 3.5 )
ProcessModel: Default DomainUsage: Default
Execution Runtime: net-3.5
Could not load file or assembly 'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
So, I decided to try nunit-x86.exe I did File > Open Project > MySolution.sln and got this:
---------------------------
NUnit
---------------------------
Test load failed!
System.IO.FileNotFoundException : Could not load file or assembly
'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral,
PublicKeyToken=null' or one of its dependencies. The system cannot
find the file specified.
For further information, use the Exception Details menu item.
---------------------------
OK
---------------------------
The exception can be found here
What is happening and how do I fix it? (without having to maintain a MySolution.nunit file)
More information
MyNamespace.Administration is not even one of the dlls that contains tests, which means that nunit fails trying to load it to look for tests to run. Knowing this I edited the file created by nunit-x86.exe (MySolution.nunit) and removed all dlls that did not have tests. Sure enough, the tests work (in both gui and console). This is not acceptable for me because it would mean that I have to keep yet another configuration file. Nunit supporting .sln files was supposed to avoid this.
My tests run fine using TestDriven.Net (but I really need to run them using nunit-console)
I have looked at this answer but I cannot make sense of what the fusion log viewer says. Would posting that log help? Assembly binding Log Viewer, lists 3 files being created:
nunit-agent-x86.exe, this one seems to be trying to find MyNamespace.Administration.dll/EXE inside the nunit directories
Tests_24398275 x2 - one looking for nunit.core in my project folders and another looking for unit.core.interfaces inside my project folders. I would pay little attention to these two since they also appear in my manually edited .nunit project).
(per andreister comment) The problem seems to be with the project/assembly itself and not the creation method. If I create a .nunit project and try to add MyNamespace.Administration to it (using 'Add Assembly...' or 'Add VS project...') it fails.
Calling nunit-console-x86 somepath/bin/Debug/MyNamespace.Administration.dll directly works.
Reposting my reply on nunit-discuss:
The NUnit feature of loading VS solutions is really fairly limited and intended to work with simple projects or as a quick way to create an NUnit project file - i.e. load the solution and save as an NUnit project, then edit the xml file that is created. Since the solution file format doesn't indicate which files are tests, NUnit attempts to load each project to check if it contains any tests. (This is the same thing that Visual Studio 2012 and later does when using the test explorer window, btw.)
As you suggest, I think the particular assembly fails to load because of having a dependency that is one level up. When loading either a VS solution file or an NUnit project file, NUnit sets the application base to the directory containing the solution or project. That's why an NUnit project file one level up works.
The designers' intent in this sort of situation is that you would create an NUnit project file. I recognize that this is somewhat inconvenient, since it gives you another configuration file to maintain. I'm open to suggestions regarding the use of globs either on the command line or within the project file. Any such changes would probably go into the next major upgrade, NUnit 3.0.
Unfortunately, even after posting on nunit-discuss group I was unable to find a proper solution for this problem.
nunit-discuss group confirmed that my tests are failing because of having a dependency that is one level up.
I did however found an acceptable work-around.
Since calling the .dlls directly didn't have the same issues.
I could do this with globs, but I'm on windows... but I have git bash installed.
Taking advantage of my somewhat rigid project structure and naming convention I managed to do this:
"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll'
Please note that I took advantage of my naming convention. This is very important to do in order to reduce the number of arguments.
When I did nunit-console-x86 MysolutionFolder/*/*/bin/Debug/*.dll instead of MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll I got an error from nunit-console-x86 saying Bad file number.
Besides, it's faster if I just provide the right files.
If you have a more recent version of bash (4.0+, I think) you can instead use the following command (note the use of **):
"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/**/bin/Debug/*.Tests.dll'
Which is shorter and more permissive on the project structure.
I am evaluating PostSharp for a new project but cannot seem to get past the following error when I first build the project after changes:
Cannot copy file "C:\SourcePath\Output\Debug\MyApp.vshost.exe to
obj\Debug\Before-PostSharp\MyApp.vshost.exe: the file is locked by
process(es):MYAPP.VSHOST (8064)
The error only occurs in the first build attempt. If I immediately re-build, the error does not occur. I can only guess this is because the project isn't actually being rebuilt the second time.
I've read a few posts in the SharpCrafters forum that indicate this problem existed prior to v2.1 but was reportedly fixed. I am using v2.1.6.14 from NuGet (in VS 2010) and getting this error for every project I reference PostSharp. It is certainly not reasonable to require 2 builds every time, so I'm looking for a possible solution. I'm really pleased with what I've seen thus far but will have to go another direction if that can't be resolved.
UPDATE
Per Gael's request, I generated the diagnostic build log and sent it to him and it looks like he was able to resolve the problem in the latest release (2.1.6.14).
I believe part (or all) of the issue may be due to the fact that the build output for all of my projects is set to a common location (i.e. not the /bin/debug folder under each project). This is because we are using a MEF DirectoryCatalog which will discover Imports and Exports contained in the assemblies located in the output path. The PostSharp.targets file has the vshost.exe file excluded from the copy operation but only when it shares the name of the output assembly. In my case, the vshost.exe file has a different name and was, therefore, not being excluded.
The issue has been re-fixed in PostSharp 2.1.6.15.
Well we are facing a strange problem with JetBrains TeamCity induced unit tests on our main project where tests from few library projects are failing regularly. Apparently, it's not reading the config file (coming from app.config and nicely stored in project -> bin -> debug -> projectName.dll.config).
Hints or tips on what could be the real issue would be highly appreciated.
I've got the same problem and wasted a couple of hours to figure out what the problem is.
In our case, the NUnit plugin was configured to run the tests from:
**\*Tests.dll
Though this sounds to be OK, it has turned out that this pattern will not only match to the MyTests.dll in the bin\Debug folder but also to the obj\Debug\MyTests.dll. The obj folder is used internally for the compilation and does not contain the config file.
Finally the solution was to change the plugin configuration to
**\bin\Debug\*Tests.dll
Actually we use a system variable for the build configuration so we did not have the "Debug" hard-coded. Using bin* might be also dangerous when the workspace is also used for Debug/Release builds and you don't have a full cleanup specified.
You might wonder why I did not realize the test count mismatch (actually it was doubled, because they were running once from bin and once from obj), but this is typical: while everything is green, you don't care about the count. When we have introduced the first test depending on the config, we had only one failure (because the one from bin was passing), so the duplication was not outstanding.
In addition to Gaspar Nagy's accepted answer, check to see if your project has multiple test dlls and one of them is referencing another.
This causes the referenced dll to be run twice, and the copy that was in the other dll's folder to not have the proper app.config entries. The proper fix is to remove any and all references from the other test project.
TeamCity (v6.5.4) has its own NUnit Test Runner and there seems to be an inconsistency between it and the NUnit GUI test runner (2.5.10). The NUnit GUI Test Runner follows the long standing convention of expecting the configuration file name to be of the format .config. You can see this in NUnit by looking at Project -> Edit...
TeamCity on the other hand is looking for an app.config.
Your options are to either:
Set the NUnit GUI to point to app.config and include the resultant
nunit project in your source control.
Have both an app.config and a .config - syncing both
manualy.
Add a step to your build process to copy .config to
app.config (or vice versa).
I had similar woes
This may help; additionally we had issues where this still would not work - we ended up copying the relevant config sections into the highest level config file. (i.e. if it was a web app copy it into the Web.Config) - fairly kludgy but we had wasted a few days on the issue
I learned recently that app.config files are not read for a class library... Maybe this link could help :)
app.config for a class library
If you need a config file for your "unit" tests then you are doing it wrong. Proper unit testing never needs configuration or access to the database, file system etc. You should change your testing strategy.
Good point to start is mark your tests that need configuration with the[Category("Integration")] annotation and set the Teamcity test runner to ignore this category. Then you should focus on refactoring these test.
I have 13 separate but related architecture assemblies, and 13 separate NUnit test assemblies, each one containing all the test fixtures for its matching architecture assembly. I am using NUnit 2.5.2 (latest version currently).
I can run each test assembly separately in the NUnit GUI and all the tests pass. However, when I come to combine them into a single NUnit project file, NUnit insists on applying a single config file to the whole test run. This won't work because each test assembly requires different config. I can't merge them into one "uber-config" file because some of the sections are mutually exclusive. I have tried running each assembly in the project in separate AppDomains, and also separate processes, but in both cases it fails to use the DLL-specific config file, so all the tests crash and burn.
I have done a Google search but so far I have not found any indication that NUnit supports this scenario. Am I right, or have I missed something?
I have tried my hardest to re-architecture the tests so that they could share the same config file, but I've had to admit defeat on that front.
NUnit 2.5 has as setting where you can enable each assembly to run in a separate AppDomain. By doing this, NUnit will load the config for the assembly and not the one for the .nunit project.
For more info, see here:
http://nunit.org/index.php?p=settingsDialog&r=2.5
In the past I've done this with a batch file running each assembly through the nunit console independently. At one point I had something that merged the xml output together. It might be in the CruiseControl.Net code.
I haven't worked on the NUnit project for a while. I only have the older code in my head. But the issue is that you get one config per AppDomain and NUnit loads all the test assemblies into one AppDomain.
You might want to try alternate runners such as Resharper or TestDriven.net