I'm setting up a CI server for my company and integrating Unit Testing. I've chosen to use TeamCity and NUnit, as they've worked well for other organizations I've worked for in the past.
I have TeamCity up and running and I've created just a test Unit Test project for now so I can verify that TC is handling my NUnit configuration properly.
My solution file contains 6 C# projects, one VB.NET project and one unmanaged c++ project. For each of those (where required) there is one separate C# project that houses the test cases for the associated project, and contains a reference to that project.
When the NUnit step runs and finds .dlls where I've told it to look, it complains that the .dll files are not the right architecture, not the right .NET version, not the right alignment of the stars etc... This is because it's trying to run NUnit against business logic .dll files, as they're built and output from the test project along with the test .dll. The test .dll file is the correct architecture, .NET version etc... as is required by NUnit.
My question is this:
How can I tell TeamCity to either
Not care if the .dll file is the correct architecture, just skip it if it isn't
Only look for the proper .dlls
I've tried to keep just a folder called Test Dlls at my root, but I can't figure out how to get every project to only put the test .dll file there.
Lastly, I will be having my coworkers generating their own unit tests for their solution files, which are not of the same layout and configuration as I described above. I really don't want to have to tell my coworkers to name their files a certain way, or set their projects to build in a particular fashion etc... if at all avoidable.
Related
I have a project that targets two different operating systems/frameworks:
net461 on Windows and
netcoreapp2.0 on OSX
I'm trying to figure out how to correctly package this for NuGet. According to this post I should be able to package them like this:
/runtimes/win/lib/net461/myassembly.dll
/runtimes/osx/lib/netcoreapp2.0/myassembly.dll
By when I add the NuGet package to another project, the packaged assemblies aren't added as references to the target project.
Then I read somewhere that you also need to add reference libraries to the /ref folder so I tried this:
/runtimes/win/lib/net461/myassembly.dll
/runtimes/osx/lib/netcoreapp2.0/myassembly.dll
/ref/net461/myassembly.dll
/ref/netcoreapp2.0/myassembly.dll
In this case the assemblies get added as a reference to the target project and I can build it, but the required assemblies aren't copied to the output folder.
The documentation on all this is extremely vague and I'm fairly lost.
What am I missing?
Associated NuGet Issue: https://github.com/NuGet/Home/issues/7316
Update: I've put together a sample project that demonstrates what I'm trying to achieve. In particular see the bottom of the readme, titled "NuGet Packaging".
This is what I've finally figured out/guessed (because as best I can tell there's no official documentation for some of this)
Files added to the /runtimes folder aren't automatically added as references to the target project.
The /ref and /runtime folder should be used in conjunction with each other and only for the .NET Core target. As best I can .NET Framework targets apparently don't support these folders.
The /ref folder is for compile time references and anything added here will be added as a reference to the target project.
Assemblies in the /ref folder don't need to have an implementation - every public API could just throw a not implemented exception. In practice however you typically just take a copy of one of the implementation assemblies and declare it as the compile time API.
I've read (but haven't tested myself) that assemblies in the /ref folder must be "Any CPU" builds. You can use CorFlags utility to patch an implementation assembly for this if necessary.
The /runtimes folder is used to provide an implementation assemblies for any references included in the /ref folder. These assemblies are used at runtime and during deployment.
The /runtimes folder can include additional assemblies that are only required at runtime and don't need to be seen by the client project. These additional assemblies won't be included as references in the target project but will be available for run/deployment.
As mentioned by others, the files in the /runtimes folder aren't copied to the output folder of the build. Instead config files are placed there that tell the runtime how to locate the /runtimes files from the NuGet cache.
For .NET Framework targets (ie: net461) just use the /lib folder as there's no other runtimes for .NET aside from Windows anyway.
Putting this all together, my original example, should have looked like this:
/lib/net461/myassembly.dll (net461/Windows Compile and Runtime)
/runtimes/osx/lib/netcoreapp2.0/myassembly.dll (netcore/OSX Runtime)
/runtimes/win/lib/netcoreapp2.0/myassembly.dll (netcore/Win Runtime)
/ref/netcoreapp2.0/myassembly.dll (netcore/* Compile Time)
I spent a fair amount of time trying your project on OSX in both Visual Studio for Mac and VS Code. I'll try to stick with factual observations without getting into "why don't you do X instead".
The runtimes/{rid}/lib/{tfm}/*.dll paths look ok
target="lib/{tfm}/..." assemblies are automatically referenced, runtimes/... are not
Using target framework of netstandard seems like it would make your package work in both netcoreapp and netstandard projects (e.g. use target="lib/netstandard1.6/..."). Compare with this
runtimes/ seems to be intended for platform-dependent assemblies you'll load at runtime. For example, 32/64-bit native assemblies in runtimes/win-x64/native/ and runtimes/win-x86/native/) loaded with AssemblyLoadContext (another post by McMaster)
Using separate slns for Windows and OSX, or separate platform-specific projects that reference platform-agnostic projects (like Xamarin) would obviate some of the configuration wrangling
I found no documentation on target="ref/...", but you can add Explicit Assembly <references> (inside the nuspec <metadata> block)
Packaged assemblies won't appear in the output directory, but when prepared for distribution with dotnet publish they'll be included:
.NET Core and .NETSTANDARD don't copy dependencies to output directory, they are mapped using deps.json which points to relative paths from local NuGet cache.
This has been a very useful thread to get more information and hints on how to create a NuGet package that references native DLLs, and is consumed in both .NET Framework as well as .NET Core / modern .NET libraries / applications.
My experience so far has been that if this library (let's call it library A) only targets .NET Standard, consuming this library in a .NET Core / 5.0 or 6.0 application does lead to the native assemblies being pulled in correctly from the runtimes folder. In a .NET Framework 4.7 application however, this does not appear to be the case. Unless the runtime is explicitly specified when compiling, e.g.:
dotnet build ... --runtime win-x86
When using library A in a .NET Core or .NET 5.0/6.0 application however, this runtime identifier is not required - all runtimes are made available and the right one is selected at runtime.
If you want library A to be consumed in applications that also target .NET Framework, and you don't want the user to have to specify the runtime explicitly, then it seems to be necessary to:
Target both .NET Standard and .NET Framework
Ensure that the native assemblies end up in the following folder structure in the NuGet package:
"lib/{tfm}/..."
While the .NET documentation referenced by tm1 earlier here talks about how to get this to work using nuspec files, it is less clear how to do so in the SDK .csproj format. I managed to do this in the NLoptNet project, see the relevant .csproj file here. Final relevant point (in addition to the two bullets above):
Use "<None Include" rather than "<None Update" to add the native assemblies
So far - this works, but there is one quirk - as you can see here it generates warning MSB3246 when consuming library A in a .NET Framework application. See also this Stack Overflow post. This leads me to believe that maybe the above is not the right approach, and therefore to some questions:
Is this the intended way to consume library A in .NET Core, modern .NET and .NET Framework applications?
Should one always specify the runtime identifier when using dotnet build / dotnet test?
Can you try to target .NET Standard 2.0 instead of net461 and netcoreapp2.0? Libraries built against netstandard2.0 should work with .NET Core 2.0 and .NET Framework 4.6.1: https://learn.microsoft.com/en-us/dotnet/standard/net-standard
Are you using the new csproj format? If so it has built in support for multiple target frameworks.
For example running dotnet pack against a .csproj file with this content:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFrameworks>net461;netcoreapp2.1;netstandard2.0</TargetFrameworks>
</PropertyGroup>
</Project>
will result in a .nupkg that works for .NET Framework 4.6.1, .NET Core 2.1 and .NET Standard 2.0.
Various trick can then be used to include specific parts for each framework depending on what's available.
I'm trying to solve the same problem.
The solution proposed by you works fine, but there is one question ...
The case of Win and net46 is clear. And now I need to add a reference to the assembly in the project for a netcoreapp for the Win and for Linux. The problem is that this is a DIFFERENT assembly with the SAME name. Those my package looks like this:
/lib/net461/myassembly1.dll (net461/Windows Compile and Runtime)
/runtimes/ubuntu/lib/netcoreapp2.0/myassembly2.dll (netcore/Ubuntu Runtime)
/runtimes/win/lib/netcoreapp2.0/myassembly1.dll (netcore/Win Runtime)
/ref/netcoreapp2.0/???
Update: Actually, the myassembly1.dll and myassembly2.dll are both called myassembly.dll. But to show that one is assembled for Windows, and the second one for Linux, I will leave here such a name.
The most interesting thing is that I tried to put any assembly in the folder ref, and it works on both Windows and Linux.
This version works on both systems
/lib/net461/myassembly1.dll
/runtimes/ubuntu/lib/netcoreapp2.0/myassembly2.dll
/runtimes/win/lib/netcoreapp2.0/myassembly1.dll
/ref/netcoreapp2.0/myassembly1.dll
And this too
/lib/net461/myassembly1.dll
/runtimes/ubuntu/lib/netcoreapp2.0/myassembly2.dll
/runtimes/win/lib/netcoreapp2.0/myassembly1.dll
/ref/netcoreapp2.0/myassembly2.dll
But I think this is not right and I was wrong somewhere.
We've recently added the excellent script# to our project. Currently we have it so that our VS build simply copies the compiled .js file from the output directory to the scripts directory of our web app.
We've decided to make it a permanent feature and so would like to make it so that the .js file gets generated as part of our web build NAnt script to ensure that it's always up to date. Is there any way to do this nicely or do I need to call MSBuild from my NAnt script specifying the .csproj file to run the compilation?
Thanks
Stu
This isn't likely the full answer (given I don't have experience with NAnt), but I'll offer it anyway, as it may help.
A script# csproj is very much like any other csproj relying on msbuild. If you've got some way to integrate other msbuild projects into your NAnt build script, the same model should ideally apply to script# projects as well.
In the version of script# that is in the github repository, a web project can add a reference to a script# project (thereby becoming dependent on the script# project), and include an msbuild deploy task, that will copy over scripts from the built script# project into the web project. You can see this in action in the Todo sample (https://github.com/nikhilk/scriptsharp/tree/cc/samples/Todo)
I am using the built in TeamCity NUnit runner to run tests stored in .DLL files.
My .DLLs are referencing other DLLs which are present in the same folder.
When running these, i am getting a FileNotFoundException for the referenced DLLs.
How can i fix this? I am thinking perhaps the TC NUnit runner is using some other dir as the main app execution folder, and thus preventing from files that are actually residing next to the test DLL to not load properly, but cannot prove this claim yet...
Anyone else encountered this behaviour?
OK the problem is solved and it is indeed silly and embarrassing:
I have set the NUnit test runner to execute these tests:
**/*.Tests.dll
This means that it will include tests from /bin/Debug/ and also from other folders, such as obj/
In these folders, no references are included of course, making the same tests run twice and some of these fail of course!
Paths are related to build checkout directory, not to any working directory. If you need another paths, use %reference% syntax to refer to TeamCity property with a directory path. Do you run it under Windows? Do you use TeamCity 6.5?
If that does not help, feed free to contact teamcity-feedback#jetbrains.com
In my case, the missing DLL was a native library and it could not be loaded because there was an incorrect version of VC runtime installed on the build agent. My solution was to add runtime DLLs (located in 'C:\Program Files (x86)\Microsoft Visual Studio 11.0\VC\redist\x86\Microsoft.VC110.CRT') directly into the project (build action = Content, copy to output directory = Copy always). This also makes installation friendlier.
Hope this helps.
I'm trying to convert a Netbeans 6.9.1 project into a scripted build (without netbeans). Of course, it fails (or I wouldn't be asking for help).
In the failure it says that the org.apache.commons.httpclient package does not exist. (Of course, it worked when we ran the build in Netbeans).
Now I know exactly where the commons-httpclient.jar file is located in my project structure, but I can't seem to tell it to the compiler via the ant build files and the netbeans property files.
Perhaps related to this is when I ran "ant -v" to build my software, it said,
Property lib.mystuff.classpath has not been set. This variable is important, I guess, because
the file nbproject/project.properties uses lib.mystuff.classpath in its definition of javac.classpath, which of course tells the Java compiler where to find the JARs.
So...when moving a Netbeans project to a netbeans-independent scripted build, how can the build script set these properties? Also, how can I ensure that the jar file gets included in the ant build?
I appreciate any help I can get, as I am a Java newbie.
UPDATE AFTER ACCEPTING ANSWER FROM vkraemer:
There are a few best practices for build scripts for production software:
Put everything needed for a build under a single directory tree. (Netbeans = fail)
Put everything in source code control. (I did that)
The first line of the build script should clear all environment variables.
The next section of the build script should explicitly set all environment variables to values which are known to work.
The next part of the build should be able to execute using command-line programs such as javac, ant, cc, etc, and must not depend on firing up an IDE such as Eclipse or Netbeans.
It is a shame that Netbeans makes this hard.
I did a quick look in a Java Application project and found the following...
javac.classpath = ${libs.MyStuff.classpath}
libs.MyStuff.classpath is defined in %HOME%/.netbeans/6.9.1/build.properties.
You may be able to get by doing the following...
ant -Dlibs.MyStuff.classpath=c:\a\b\c.jar
You would need to do more if you have multiple jar files in the MyStuff library that you created in NetBeans.
I have a bit of code for a dll that is needed by two or more projects in eclipse. Currently each project has a copy of the code and builds the DLL separately. I want to separate the dll code into a separate eclipse project so there is a common location. But I want to avoid the situation where we have to build the dll in the one project, then copy the dll back to the other projects and check the dll to each respective project. This will create a dll for each project that isn't traceable to the exact code that it was built with.
Is there a way to somehow symbolically link the dlls to another eclipse project that is using CVS as the version control system so that it is possible to tell which version of the code was used to create the dll? Am I making this too complicated or missing something obvious?
I thought about working sets in the package manager for eclipse, but I have to investigate more on how to use them with CVS to avoid making it a nightmare for the next person who checks it out and can't figure out why their project won't compile.
Thanks.
What about creating a new folder in a separate project. In the advanced section of creating a new folder there is an option to link to another location on the file system.
Or you could also create a container project that makes use of a projectset.psf file. Have the projectset file link to the different projects in your repository. When you want to check out that project, check out the container instead and right click on the projectset file and select Import Project Set...
If you are working with one workspace, you end up with three projects, each mirrored in CVS: One is the dll, the others are the projects using the dll (configured as a project dependency of these projects upon the dll project).
With three projects I wouldn't aim for working sets - they are good for managing a lot of projects within one workspace, for three projects, I'd consider them overkill. I usually tend to aim for several workspaces instead of working sets.
Regarding the next person working with these projects: You need to keep some kind of documentation about how to setup your projects. You might say that your eclipse project files do just that (as they define a project dependency upon another project) but this is for the machine - humans tend to like other communication means.
If you are worried about changes to the dll being incompatible to one project (because the person applying these changes doesn't care about the other project), aim for a build server. This will build all projects and dependent projects whenever something under version control changes, run all tests, provide a build number and package it all ready for use. This way you can be sure that - whatever is in your deliverable - can be reproduced, because the buildserver is not able to make local (uncommitted) changes to the code. Also a buildserver will signal failure (either broken API or broken tests) at the moment of the last commit (well - a few minutes later) and place the burden of repairing the damage on the one causing the damage.