Test projects not reading app.config in TeamCity -> NUnit phase - nunit

Well we are facing a strange problem with JetBrains TeamCity induced unit tests on our main project where tests from few library projects are failing regularly. Apparently, it's not reading the config file (coming from app.config and nicely stored in project -> bin -> debug -> projectName.dll.config).
Hints or tips on what could be the real issue would be highly appreciated.

I've got the same problem and wasted a couple of hours to figure out what the problem is.
In our case, the NUnit plugin was configured to run the tests from:
**\*Tests.dll
Though this sounds to be OK, it has turned out that this pattern will not only match to the MyTests.dll in the bin\Debug folder but also to the obj\Debug\MyTests.dll. The obj folder is used internally for the compilation and does not contain the config file.
Finally the solution was to change the plugin configuration to
**\bin\Debug\*Tests.dll
Actually we use a system variable for the build configuration so we did not have the "Debug" hard-coded. Using bin* might be also dangerous when the workspace is also used for Debug/Release builds and you don't have a full cleanup specified.
You might wonder why I did not realize the test count mismatch (actually it was doubled, because they were running once from bin and once from obj), but this is typical: while everything is green, you don't care about the count. When we have introduced the first test depending on the config, we had only one failure (because the one from bin was passing), so the duplication was not outstanding.

In addition to Gaspar Nagy's accepted answer, check to see if your project has multiple test dlls and one of them is referencing another.
This causes the referenced dll to be run twice, and the copy that was in the other dll's folder to not have the proper app.config entries. The proper fix is to remove any and all references from the other test project.

TeamCity (v6.5.4) has its own NUnit Test Runner and there seems to be an inconsistency between it and the NUnit GUI test runner (2.5.10). The NUnit GUI Test Runner follows the long standing convention of expecting the configuration file name to be of the format .config. You can see this in NUnit by looking at Project -> Edit...
TeamCity on the other hand is looking for an app.config.
Your options are to either:
Set the NUnit GUI to point to app.config and include the resultant
nunit project in your source control.
Have both an app.config and a .config - syncing both
manualy.
Add a step to your build process to copy .config to
app.config (or vice versa).

I had similar woes
This may help; additionally we had issues where this still would not work - we ended up copying the relevant config sections into the highest level config file. (i.e. if it was a web app copy it into the Web.Config) - fairly kludgy but we had wasted a few days on the issue

I learned recently that app.config files are not read for a class library... Maybe this link could help :)
app.config for a class library

If you need a config file for your "unit" tests then you are doing it wrong. Proper unit testing never needs configuration or access to the database, file system etc. You should change your testing strategy.
Good point to start is mark your tests that need configuration with the[Category("Integration")] annotation and set the Teamcity test runner to ignore this category. Then you should focus on refactoring these test.

Related

nunit-console "could not load file or assembly" using MySolution.sln

I'm to use nunit-console to run all of the tests in my solution.
I did this:
c:\some\path>nunit-console-x86.exe MySolution.sln
NUnit-Console version 2.6.2.12296
Copyright (C) 2002-2012 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Microsoft Windows NT 6.1.7601 Service Pack 1
CLR Version: 2.0.50727.5466 ( Net 3.5 )
ProcessModel: Default DomainUsage: Default
Execution Runtime: net-3.5
Could not load file or assembly 'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
So, I decided to try nunit-x86.exe I did File > Open Project > MySolution.sln and got this:
---------------------------
NUnit
---------------------------
Test load failed!
System.IO.FileNotFoundException : Could not load file or assembly
'MyNamespace.Administration, Version=0.0.0.1, Culture=neutral,
PublicKeyToken=null' or one of its dependencies. The system cannot
find the file specified.
For further information, use the Exception Details menu item.
---------------------------
OK
---------------------------
The exception can be found here
What is happening and how do I fix it? (without having to maintain a MySolution.nunit file)
More information
MyNamespace.Administration is not even one of the dlls that contains tests, which means that nunit fails trying to load it to look for tests to run. Knowing this I edited the file created by nunit-x86.exe (MySolution.nunit) and removed all dlls that did not have tests. Sure enough, the tests work (in both gui and console). This is not acceptable for me because it would mean that I have to keep yet another configuration file. Nunit supporting .sln files was supposed to avoid this.
My tests run fine using TestDriven.Net (but I really need to run them using nunit-console)
I have looked at this answer but I cannot make sense of what the fusion log viewer says. Would posting that log help? Assembly binding Log Viewer, lists 3 files being created:
nunit-agent-x86.exe, this one seems to be trying to find MyNamespace.Administration.dll/EXE inside the nunit directories
Tests_24398275 x2 - one looking for nunit.core in my project folders and another looking for unit.core.interfaces inside my project folders. I would pay little attention to these two since they also appear in my manually edited .nunit project).
(per andreister comment) The problem seems to be with the project/assembly itself and not the creation method. If I create a .nunit project and try to add MyNamespace.Administration to it (using 'Add Assembly...' or 'Add VS project...') it fails.
Calling nunit-console-x86 somepath/bin/Debug/MyNamespace.Administration.dll directly works.
Reposting my reply on nunit-discuss:
The NUnit feature of loading VS solutions is really fairly limited and intended to work with simple projects or as a quick way to create an NUnit project file - i.e. load the solution and save as an NUnit project, then edit the xml file that is created. Since the solution file format doesn't indicate which files are tests, NUnit attempts to load each project to check if it contains any tests. (This is the same thing that Visual Studio 2012 and later does when using the test explorer window, btw.)
As you suggest, I think the particular assembly fails to load because of having a dependency that is one level up. When loading either a VS solution file or an NUnit project file, NUnit sets the application base to the directory containing the solution or project. That's why an NUnit project file one level up works.
The designers' intent in this sort of situation is that you would create an NUnit project file. I recognize that this is somewhat inconvenient, since it gives you another configuration file to maintain. I'm open to suggestions regarding the use of globs either on the command line or within the project file. Any such changes would probably go into the next major upgrade, NUnit 3.0.
Unfortunately, even after posting on nunit-discuss group I was unable to find a proper solution for this problem.
nunit-discuss group confirmed that my tests are failing because of having a dependency that is one level up.
I did however found an acceptable work-around.
Since calling the .dlls directly didn't have the same issues.
I could do this with globs, but I'm on windows... but I have git bash installed.
Taking advantage of my somewhat rigid project structure and naming convention I managed to do this:
"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll'
Please note that I took advantage of my naming convention. This is very important to do in order to reduce the number of arguments.
When I did nunit-console-x86 MysolutionFolder/*/*/bin/Debug/*.dll instead of MysolutionFolder/Tests/*/bin/Debug/*.Tests.dll I got an error from nunit-console-x86 saying Bad file number.
Besides, it's faster if I just provide the right files.
If you have a more recent version of bash (4.0+, I think) you can instead use the following command (note the use of **):
"C:\Program Files (x86)\Git\bin\bash.exe" -c 'nunit-console-x86.exe //framework=net-4.5 //xml:nunitresults.xml MysolutionFolder/**/bin/Debug/*.Tests.dll'
Which is shorter and more permissive on the project structure.

System.Resources.MissingManifestResourceException when Updating database

I moved a web application I am working on from one machine to another. It is built using .Net MVC and Entity Framework but when I execute the Update-Database command so that the database is updated, I get this error:
Could not find any resources appropriate for the specified culture or the neutral culture. Make sure "NameofMigration.resources" was correctly embedded or linked into assembly "NameofProject" at compile time, or that all the satellite assemblies required are loadable and fully signed.
Does anyone know how to fix this error?
I had a similar issue when the resx part of the migration was not included in the project file when a fellow developer checked the project in (probably due to a merge issue). You may find that the resx file is there but greyed out. If it's there, try right clicking the "NameofMigration.resx" file and selecting "include in project". If it's not there, you better go find it on the other machine and add it to the project :-)
I think the issue (one issue) is that the .resx file is added as "dependent upon" (nested under) the .cs file, and the way the build engine works, "dependent upon" changes the name that an embedded resource is saved with (something like, it changes from being based on the filename to being based on the type name; I've dealt with this in other scenarios but can't remember for sure).
This leads to problems when using SDK .csproj files, for some reason (I guess that by default SDK .csproj does not change the resource name in this situation, but the migrations system expects it to).
As someone else had posted, SDK .csproj can use the following tag to change the embedded resource naming scheme for "dependent upon" resources, which then allows the migrations system to find the embedded resource:
<EmbeddedResourceUseDependentUponConvention>
true
</EmbeddedResourceUseDependentUponConvention>
This should go in a <PropertyGroup> of your SDK .csproj file.
For VS 2017, the solution is as follows:
Go to the project file, and for all of the migrations, apply the following format:
<Compile Include="Migrations\201804251606403_emailsWithEffort.cs" />
<Compile Include="Migrations\201804251606403_emailsWithEffort.Designer.cs">
<DependentUpon>201804251606403_emailsWithEffort.cs</DependentUpon>
</Compile>
<EmbeddedResource Include="Migrations\201804251606403_emailsWithEffort.resx">
<DependentUpon>201804251606403_emailsWithEffort.cs</DependentUpon>
</EmbeddedResource>
I guess that the problem is when changing version(s) of Visual Studio, old format of describing dependencies stays, and the Visual Studio 2017 can not interpret it correctly.
Hence, applying the format as described above (change your format to this), you can make the Visual Studio get the idea of where it's resources are.
Slightly different situation, where I created a new environment, and database, and received the above error message.
For my fix, I had to right-click on the migration files (initial and resx) and set property to embedded as resource. Update-database command ran fine afterward.
I encountered the same issue (VS 2017) and none of the solutions provided here worked. I fixed the problem by cleaning the solution and manually deleting the bin folder and then building it again.
If anybody wants to look into the source or compiler to know why this is happening; I don't feel like it right now. After an hour of tinkering, my resolution is odd.
Granted, I shouldn't have done this in the first place, but for quick code I temporarily added classes into the same file as my generated DbMigration 201906212110305_initial.cs. The mere existence of those temporary classes in the same file caused this error. As soon as I moved them to their own file (which I was going to do all along anyway) the runtime error vanished.
Unloading and then reloading migration file worked for me!

Build error with PostSharp 2.1.6 (NuGet)

I am evaluating PostSharp for a new project but cannot seem to get past the following error when I first build the project after changes:
Cannot copy file "C:\SourcePath\Output\Debug\MyApp.vshost.exe to
obj\Debug\Before-PostSharp\MyApp.vshost.exe: the file is locked by
process(es):MYAPP.VSHOST (8064)
The error only occurs in the first build attempt. If I immediately re-build, the error does not occur. I can only guess this is because the project isn't actually being rebuilt the second time.
I've read a few posts in the SharpCrafters forum that indicate this problem existed prior to v2.1 but was reportedly fixed. I am using v2.1.6.14 from NuGet (in VS 2010) and getting this error for every project I reference PostSharp. It is certainly not reasonable to require 2 builds every time, so I'm looking for a possible solution. I'm really pleased with what I've seen thus far but will have to go another direction if that can't be resolved.
UPDATE
Per Gael's request, I generated the diagnostic build log and sent it to him and it looks like he was able to resolve the problem in the latest release (2.1.6.14).
I believe part (or all) of the issue may be due to the fact that the build output for all of my projects is set to a common location (i.e. not the /bin/debug folder under each project). This is because we are using a MEF DirectoryCatalog which will discover Imports and Exports contained in the assemblies located in the output path. The PostSharp.targets file has the vshost.exe file excluded from the copy operation but only when it shares the name of the output assembly. In my case, the vshost.exe file has a different name and was, therefore, not being excluded.
The issue has been re-fixed in PostSharp 2.1.6.15.

Visual Studio 2010 Publish Web feature not including all DLLs

I have an ASP.NET MVC 2 application.
Web project contains a reference to SomeProject
SomeProject contains references to ExternalAssembly1 and ExternalAssembly2.
SomeProject explicitly calls into ExternalAssembly1, but NOT ExternalAssembly2.
ExternalAssembly1 calls into ExternalAssembly2
When I perform a local build everything is cool. All DLLs are included in the bin\debug folder. The problem is that when I use the Publish Web command in Visual Studio 2010, it deploys everything except ExternalAssembly2.
It appears to ignore assemblies that aren't directly used (remember, ExternalAssembly2 is only used by ExternalAssembly1).
Is there any way I can tell Visual Studio 2010 to include ExternalAssembly2?
I can write a dummy method that calls into ExternalAssembly2. This does work, but I really don't want to have dummy code for the sole purpose of causing VS2010 to publish the DLL.
None of these answers are sufficient in my mind. This does seem to be a genuine bug. I will update this response if I ever find a non-hack solution, or Microsoft fixes the bug.
Update:
Doesn't seem promising.
https://connect.microsoft.com/VisualStudio/feedback/details/731303/publish-web-feature-not-including-all-dlls
I am having this same problem (different assemblies though). If I reference the assemblies in my web project, then they will get included in the publish output, but they should be included anyway because they are indirect dependencies:
Web Project ---> Assembly A ---> Assembly B
On build, assemblies A and B are outputed to the \bin folder. On publish, only assembly A is outputed to the publish folder.
I have tried changing the publish settings to include all files in the web project, but then I have files in my publish output that shouldn't be deployed.
This seems like a bug to me.
I had the same problem with VS2010 and a WCF Service Application.
It turns out that if your (directly or indirectly) referenced DLL's are deployed to GAC, the VS publishing feature excludes them. Once I removed the assemblies from GAC, publishing feature started working as expected.
I guess VS is assuming that if your assemblies can be located in GAC on the machine you build, they will be located in GAC on the target machine as well. At least in my case this assumption is false.
My tests show that the external assemblies get published when I have a reference on them in the web project. I do not have to write any dummy code to make it work. This seems acceptable to me.
I agree with Nicholas that this seems to be a bug in visual studio. At least it escapes me what the reason for the behavior could be.
I have created this issue as a bug on Microsoft Connect. If anyone experiencing it could vote it up https://connect.microsoft.com/VisualStudio/feedback/details/637071/publish-web-feature-not-including-all-dlls then hopefully we'll get something done about it.
If you go into the ExternalAssembly2 reference property list and change the "Copy Local" to "True" i think that might solve your issue.
I don't know if you are watching this still but I found the solution (I had the exact same issue) via this MSDN article. Under "build action" for the file choose "Content" that should include it in the list of files publish brings over.
I have created a new Connect bug here https://connect.microsoft.com/VisualStudio/feedback/details/731303/publish-web-feature-not-including-all-dlls
I've also attached a solution and detailed steps to reproduce this issue. Lets hope this time they won't close it as Can't Reproduce.
Vote for this connect issue if you experience the missing dll problem.
Copy local did the trick. I had an issue that the Newtonsoft.Json assembly get included in the deploymeny package. Copy local was set to false.
I am experiencing the same type of issue with a web project. I have a web project that references assembly A which references assembly B. It worked fine for some time but today it was broken. I did a rebuild of the solution and this time it deployed everything correctly.
I had this same problem today. I published my web project and realized that not all of the reference DLL's were there. In particular, the indirect DLL references.
It turns out that the directory in which I was publishing to was out of disk space (network share). I had just enough space to publish all the files except for few indirect reference DLL's. The sad part is that VS08 didn't throw any errors. It just published the files are usual. I cleared out some HDD space and everything worked fine.
I didn't find the HDD space issue until I tried to manually move the DLL's over.
in my case it is quite tricky.
Reference to ExternalAssembly2 is not required to Build the project but vital for run-time since we use reflection to configure Unity container.
So, I delete the reference - build the project successfully, but get run-time error.
If I preserve the reference I can Build and Run the application but I cannot Publish it with ExternalAssembly2 - get run-time exception as well.
This is happen because of internal VS2010 assemblies optimization.
So, what we can do here?
1. Put some unrequired peice of code to use any ExternalAssembly2's class.
2. escape from reflection and use static assemblies linking.
Hope this helps to smbd.
I got the same problem and this is a VS2010 bug if there's a reference link like:
Web Project --> custom project --> assembly1 -->(indirectly) assembly2.
For now I find if I reference the Assembly1 in the web project, then assembly2 is included in the bin folder.
So I had to add an additional reference link like:
Web project --> assembly1 -->(indirectly) assembly2.
Then VS can recognize assembly2 and include its dll file in publish action.

Multiple NUnit test assemblies, each requiring different config. How can I get NUnit to run them all at once?

I have 13 separate but related architecture assemblies, and 13 separate NUnit test assemblies, each one containing all the test fixtures for its matching architecture assembly. I am using NUnit 2.5.2 (latest version currently).
I can run each test assembly separately in the NUnit GUI and all the tests pass. However, when I come to combine them into a single NUnit project file, NUnit insists on applying a single config file to the whole test run. This won't work because each test assembly requires different config. I can't merge them into one "uber-config" file because some of the sections are mutually exclusive. I have tried running each assembly in the project in separate AppDomains, and also separate processes, but in both cases it fails to use the DLL-specific config file, so all the tests crash and burn.
I have done a Google search but so far I have not found any indication that NUnit supports this scenario. Am I right, or have I missed something?
I have tried my hardest to re-architecture the tests so that they could share the same config file, but I've had to admit defeat on that front.
NUnit 2.5 has as setting where you can enable each assembly to run in a separate AppDomain. By doing this, NUnit will load the config for the assembly and not the one for the .nunit project.
For more info, see here:
http://nunit.org/index.php?p=settingsDialog&r=2.5
In the past I've done this with a batch file running each assembly through the nunit console independently. At one point I had something that merged the xml output together. It might be in the CruiseControl.Net code.
I haven't worked on the NUnit project for a while. I only have the older code in my head. But the issue is that you get one config per AppDomain and NUnit loads all the test assemblies into one AppDomain.
You might want to try alternate runners such as Resharper or TestDriven.net