Testing a Monotouch app with NUnit in MonoDevelop 2.4 - nunit

I'm new to both Monotouch and Monodevelop. Trying to get started with NUnit and I'm having a lot of trouble -- none of the sketchy references I can find on line seem to match what I'm seeing in the UI (MonoDevelop 2.4 on Mac OS 10.6). I've tried:
Adding an "NUnit assembly test collection" project to my solution.
Adding an "Empty MonoTouch Project" project to my solution, and then adding the NUnit assemblies to it, and adding my main project as a reference.
Adding a C# "Empty Project" to my solution, and adding NUnit, MonoTouch, and my own project as references. This produces a build error complaining that "'[test project name].exe' does not contain a static 'Main' method suitable for an entry point."
(1) produces a strange sort of project to which I can only add assemblies -- no references and certainly no tests.
(2) and (3) behave pretty much identically:
First, a build error complaining that there's no static 'Main' method. I can fix this by changing the compile target to "Library" in the Build -> General project options.
Next, when I try to run the tests (from the Unit Testing tab), it says it's running them using the "Debug|iPhoneSimulator" configuration.
The Test Results panel shows this "running tests" message, and never any other output.
The counts stay at "Tests: 0 Failed: 0 Ignored: 0".
Clearly I'm doing something wrong here, but what am I supposed to be doing?
Just for grins, here's my test.
using System;
using NUnit.Framework;
namespace mynamespace
{
[TestFixture]
public class NavItemTest
{
[Test]
public void TestAll()
{
Assert.AreEqual(4, NavItem.all().Count);
}
}
}

In case you missed it, there is now a NUnitLite runner available for MonoTouch which is designed to work for UI agnostic code and executed on devices (or simulator).
See: .NET Unit test runner for iOS

Have written up a few details on what we have found to be best practice so far. You can find it here: http://ben.phegan.name/index.php/2011/02/28/monotouch-and-unit-testing. Would be happy to hear other ways of doing this.
Short answer:
Add NUnit project to normal solution
Add MonoTouch assemblies as references
Write tests, avoiding testing anything that uses monotouch.dll (design patterns to abstract this stuff).
Win!

I had the same problem:
This produces a build error complaining that "'[test project name].exe' does not contain a static 'Main' method suitable for an entry point."
Fixed by going into Project -> [Project name] Options -> Build General. Changed Compile Target to Library. I hadn't made a Main-method class yet, but will probably later; so then I'll change back.

Related

"Copy swift standard libraries" build step does not include SwiftOnoneSupport.dylib

I have been experiencing a troubling problem lately that has to do with the running of tests on a project which has a cocoapods dependency to a framework which crashed at runtime with error:
Library not loaded: #rpath/libswiftSwiftOnoneSupport.dylib
Referenced from: MyCocoapodsDependency
There are other stackoverflow topics out there about that particular problem but none of them address root cause (for example here), which is that during the "Copy swift standard libraries" step of the test build, the missing library is not copied over.
It looks like this (this is the build log when I build the tests):
When it should look like this (this is the build log for a test app that uses the exact same dependencies):
OnOneSupport seems to have to do with with a project setting that has to do with "Whole module optimization".
But even if I change to "not onOne" it doesn't make a difference. The dylib is referred to in https://github.com/apple/swift/blob/master/cmake/modules/AddSwift.cmake
I cannot for the life of me figure out why it's omitted from the standard libraries build step for one target and not for another, but its obvious that my tests needs it. Anybody know how I can force the compiler to include SwiftOnoneSupport?
Workaround
Add a build phase to the test target (in the project file) that copies libswiftSwiftOnoneSupport.dylib to Frameworks. The dylib can be found at:
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/bitcode_strip /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/lib/swift/iphonesimulator/libswiftSwiftOnoneSupport.dylib
I also had this same issue with two of my frameworks. The strangest thing was that everything was working fine with one of them but not with the other.
While I wasn't able to figure out why this happens (since it seems to be an Xcode bug), I was able to find a pretty neat workaround.
Turns out that using print() anywhere in your code will somehow force libswiftSwiftOnoneSupport.dylib to be loaded. So, by adding something like this the problem should go away:
private func dummy() {
print("Hello world!")
}
I'm using Xcode 10.1, Swift 4.2 and the pod that was giving me this issue was Nimble.
Hope this helps!

C++ Boost Test, package structure, and Eclipse project settings

I'm using C++98, and besides the standard library, I only have access to an old version of Boost (which thankfully has Boost Test). The documentation though is daunting, long, and I just don't know where to begin.
I have some experience doing unit testing in Java (and I'm looking for unit testing in C++), and I've seen test packages that contain the unit test code separate from the src package, and I also saw Where do you put your unit test? as well as Unit Testing with Boost and Eclipse. Their suggestions vary, and present reasoning for different packaging structures for separating test code from production code, or having them together.
Before I even started looking into Boost Test, I created this structure within Eclipse (perhaps erroneously):
-- ProjectName
|-- Debug
|-- src
|-- test
and I wrote another main method to run test functions. Eclipse didn't like that, because I had two main methods in the same project. I fumbled around through Project Properties and didn't find anything useful for separating my production code from test code when building (linking, really). My temporary fix was to just use g++ in the terminal and ad hoc compile just my "test" code.
I found something suggesting on Boost::Test -- generation of Main()? that Boost actually generated its own main method, so this is currently my plan of attack for unit testing, especially for having a library of testing tools already available.
What is the conventional way of organizing unit tests for C++?
How do I get started with Boost Test? (Boost is already installed)
Is there anything I need to change in Eclipse to be able to run my Boost unit tests separate from my production code within the IDE? (One of the nice things about IntelliJ, with Java, is how it'll automatically run any main method you like with a click) -- The goal here to be able to build and run my tests within Eclipse.
Should my tests be in a separate Eclipse project? (this was suggested in an answer to the second SO question I linked)
Edit: I found this article for an introduction to Boost Test, but it doesn't discuss how it can be handled within an IDE setting.
I figured out how to do this on my own, and I'll document my solution for others who are just starting with C++ as well and need to do testing of their code. There currently is no good introduction anywhere that I could find. Here are the resources that I used though (and found useful):
C++ Unit Testing With Boost Test which introduces Boost Test in a much better fashion than Boost's documentation.
Where do you put your unit test? which discusses the most conventional ways of doing tests with C++.
unit test in eclipse g++ which discusses Eclipse build configurations which allow testing
The C++ convention for testing is like that of other coding languages, just write your tests in a directory called test under the project. Using Boost Test requires that you link the unit test framework: -l boost_unit_test_framework which in Eclipse:
Right click on your project, go to Properties, C/C++ Build, Settings, Tool Settings, GCC C++ Linker, Libraries, and add the library name boost_unit_test_framework (add -mt to the name if you require multithreading; additionally, once the testing build configuration exists, you can go back and choose just that configuration to link the library -- it'll reduce the size of your executable for your other builds).
To be able to run the unit tests in Eclipse separate from your main method, we need to establish a new build configuration. That way, Eclipse knows to exclude your source file with a main method when you're executing your tests.
Click on Project, Build Configurations, Manage..., and select New... and call it Test (or something other than test). Choose your existing configuration so that we'll inherit properties from the production build.
Next, we need to differentiate the build configurations from one another so when we build them, they actually correspond to the production and test builds.
Right click on test, Resource Configurations, Exclude from Build..., and select the builds that represent your production build (i.e. Debug and or Release). Once done with that, right click on your source file with your main method, and exclude that from the Test build.
There's still some things we need to change. We don't have any test code yet, but we still can't run our test build, nor would our test build know about resources existing in src because Eclipse wouldn't automatically include those source files. They're practically invisible to your test code in test.
Right click on your project, go to Properties, C/C++ Build, Settings, Tool Settings, GCC C++ Compiler, Includes, and add the path /.../workspace/ProjectName.
Now to be able to run your test build in Eclipse, it needs to know what executable you're expecting the IDE to run.
Click on Run, Run Configurations..., and looking at your current run configuration, consolidate these settings by giving, for example, a debug build the name "Debug Build", the C/C++ Application "Debug/Artifact_Name", and the Build Configuration "Debug". Next, create a new run configuration, and call it something like "Test Build", set C/C++ Application to "Test/Artifact_Name", and ensure the build configuration is Test.
Now you'll be able to switch between running production code and test code by selecting either the "active" build configuration or running the correct run configuration.
Finally, here's an example of a unit test with Boost Test once all of this is set up:
//unit_tests.cpp
#define BOOST_TEST_DYN_LINK
#define BOOST_TEST_MODULE someModuleName
#include <boost/test/unit_test.hpp>
#include <src/some_object.h>
struct template_objects {
some_object x;
template_objects() {
BOOST_TEST_MESSAGE("Setting up testing objects");
}
~template_objects() {
BOOST_TEST_MESSAGE("Tearing down testing objects");
}
}
BOOST_FIXTURE_TEST_SUITE(testSuiteName, template_objects)
BOOST_AUTO_TEST_CASE(testCase1) {
x.update();
BOOST_CHECK(x.is_up_to_date());
}
BOOST_AUTO_TEST_CASE(testCase2) {
BOOST_CHECK(x.is_not_up_to_date());
}
BOOST_AUTO_TEST_SUITE_END()
This demonstrates some critical things about using Boost Test:
defining BOOST_TEST_DYN_LINK is recommended; you need to define a way of linking the testing framework before including any of Boost's libraries
you must name the "module" something, doesn't have to be the file name
to get automated setup up and teardown of objects before entering test cases, Boost Test has fixtures, which allow you to call an object's pre-existing state multiple times
the struct is what groups these fixtures, and it's implied that your object should have a well defined constructor and destructor for automatic scoping (if you didn't call new, you don't need delete in teardown)
test suites are just a way of logically grouping your test cases (I haven't tested it yet, but you can probably break up your suites into multiple files for better logical separation)
Additional tidbit: to make Boost Test more verbose, go to your run configuration for the test build, and add the argument --log_level=test_suite.

NUnit Unit tests not showing in Test Explorer with Test Adapter installed

Want to improve this post? Provide detailed answers to this question, including citations and an explanation of why your answer is correct. Answers without enough detail may be edited or deleted.
I've installed NUnit Test Adapter for VS2012 + 2013. When I first installed the Adapter tests were showing up, but they stopped showing up for some reason today. After building, rebuilding, cleaning, restarting, nothing shows up in Test Explorer. Why would this be happening? I'm using VS2013 Ultimate.
If you're using a NUnit3+ version, there is a new Test Adapter available.
Go to "Tools -> Extensions and Updates -> Online" and search for "NUnit3 Test Adapter" and then install.
If your test project is set to target a 64bit platform, the tests won't show up in the NUnit Test Adapter.
My test assembly is 64-bit. From the menu bar at the top of visual studio 2012, I was able to select 'Test' -> 'Test Settings' -> 'Default Processor Architecture' -> 'X64'. After a 'Rebuild Solution' from the 'Build' menu, I was able to see all of my tests in test explorer. Hopefully this helps someone else in the future =D.
Check for NUnit versions mismatch. The currently available NUnit Test Adapter only works for NUnit version 2.6.4 and below.
To downgrade NUnit from version 3.x go to
Package Manager Console > update-package NUnit -version 2.6.4
http://jeremybytes.blogspot.co.ke/2015/11/review-of-unit-testing-makes-me-faster.html
In my situation the 'NUnit3 Test Adapter' has been disabled.
To re-enable it go to menu
Tools->Extensions and Updates...
On the left side select 'Installed'->'All'.
On the upper right corner search for 'nunit'.
If you have 'NUnit3 Test Adapter' installed, with the found item you can enable/disable it.
This answer seems pretty basic but wasn't completely obvious to me at first. If you (re)build the solution it only builds the projects that are configured to build in the Build -> Configuration Manager
This was my issue, I must have inadvertently changed a build configuration settings or something that caused my test projects not to build (when they previously were). So the Test Explorer window was looking at dlls that were out of date. It became clear to me this was the case after doing a Clean and seeing most of my tests disappear and not come back after a rebuild...further inspection of bin folder showed that these projects weren't being built at all.
I had a working setup (for NUnit2 and NUnit3 depending on the solution, and multiple versions of Visual Studio between 2012 and 2017), and it suddenly stopped working one day: no tests detected in any solution or version of VS.
In my case, it helped to delete %localappdata%\Temp\VisualStudioTestExplorerExtensions. After a restart of VS, everything worked as before.
If you are using VS 2017 and .net core ,as said here, you should add references to the test framework NUnit 3.6.1, to the test runner NUnit3TestAdapter 3.8.0-alpha1 and to the test SDK Microsoft.NET.Test.Sdk 15.0.0.
Tools
NuGet Package Manager
Manage NuGet Packages For Solution
Browse
NUnitTestAdapter.WithFramework
Ctrl+R,A to build/run tests
Using NUnitTestAdapter.WithFramework makes sure there are little/no inconsistencies across versions of NUnit and NUnit Adapter (i.e. "it just works")
Check whether you have stated
[TestFixureSetUp]
and
[Test]
in the test class
sample:
namespace ClassLibrary1
{
public class SimpleCalculator
{
public Calculator _calculator;
[TestFixtureSetUp]
public void initialize()
{
_calculator = new Calculator();
}
[Test]
public void DivideTest()
{
int a = 10;
int b = 2;
int expectedValue = a/b;
int actualValue = _calculator.Divide(a, b);
Assert.AreEqual(expectedValue, actualValue, "Functionality not working properly!");
}
}
}
I had to uninstall then re-install the xunit.runner.visualstudio nuget package. I tried this after trying all the above suggestions, so may be it was a mixture of things.
One other cause to this problem is if you open a project from a mapped drive - Visual Studio handles such projects properly, but apparently Nunit doesn't support them.
Copying the project to a physical fixed the issue.
I also found that when I uninstalled nunit v3.2.1, the nunit framework reference for v3.2.1 was still in my project in solution explorer.
Solution Explorer > ProjectName > References
If you right click it will show the version. Remove this
Then Right click on References > Add Reference.
Search for the version 2.x version and add then rebuild solution.
That worked for me!
I experienced the problem mentioned by op
My case was that I was handed an old project and the tests were actually part of the system under test. I assume they were using the external test runner.
this task chain resolved the issue for me
created a test project,
moved the test files there
added references so the test project would compile
added the Nunit and Nunit adapter NuGet packages to the test project
recompiled
i was able to successfully run the Nunit tests.
If you are using the TestCaseSource attribute, ensure the source exists and respects the documentation, otherwise your tests will not be discovered.
I had a similar issue where the tests where not being discovered. I had the correct version of NUnit, versions matched up between NUnit and adapter, and the tests where tagged correctly. I was running VS 2017 Enterprise not as an administrator. After starting VS as administrator the tests appeared.
I started a new solution with a test project in it, and compared it against my original, problem project. The original, for some reason, had an app.config in it. I excluded that file from the project and saw my tests reappear in the test explorer.
I had this problem too but the cause was different. I'm using VS2017 with F# 4.0.
Firstly, the console in Visual Studio does not give you enough details why the tests could not be found; it will just fail to the load the DLL with the tests. So use NUnit3console.exe on the command line as this gives you more details.
In my case, it was because the test adapter was looking for a newer version of the F# Core DLL (4.4.1.0) (F# 4.1) whereas I'm still using 4.4.0.0 (F# 4.0). So I just added this to the app.config of the test project:-
<dependentAssembly>
<assemblyIdentity name="FSharp.Core" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-65535.65535.65535.65535" newVersion="4.4.0.0" />
</dependentAssembly>
i.e. redirect to the earlier F# core.
I had the same problem, when suddenly any test didn't appeared on Test Explorer window.
I has the updated version of "NUnit3TestAdapter"
and after lots of searches and efforts,
I found that I need set the following values in project properties:
[In Solution Explorer window: right click on Project > Properties]
Under Build tab, set Platform=x64, and set Platform target=x86 or Any CPU
Build the project and all tests will be appear on Test Explorer window.
Important note:
I came to a solution after seeing the next msg in the output window:
"Test run will use DLL(s) built for framework Framework45 and platform X86. Following DLL(s) will not be part of run: AutomationTests.dll is built for Framework Framework45 and Platform X64."
I had some msbuild.exe processes that were hung. I don't know if that was my problem or not, but it took me a lot of trail and error with reinstalling various NUnit adaptors before I found the hung processes.
Just to add my $.02 here, I ran into a similar issue just yesterday, where 168 of my tests were missing. I tried most everything in this post - most especially making sure my version(s) of NUnit were the same - all to no avail. I then remembered that I had my tests divided into playlists; and these do not update automatically as you add new tests. So, when I deleted the playlists, BAM!, all of my tests were back once more.
I use NUnit 3 and tried most of the other solutions I saw here. None of those worked for me.
Even though already selected, reselecting the "Playlist: All Tests" option in test explorer showed all hidden tests in my case. I need to do this after every rebuild.
Make sure your test class is public. I often make this mistake then I just look at the code for 5 minutes what is going on.
[Test]
public void YourTest()
{
...
}
Tools
NuGet Package Manager
Manage NuGet Packages For Solution
Browse
NUnit3TestAdapter (NUnit 3 in my case ..)
After installation it should be possible to use the default TestExplorer included with the installation of Visual Studio.
If your test project is not x86, the tests will not be found until you tell the test runner the correct target platform.
Prior to Visual Studio 2022, this was the "Test/Test Settings/Default Processor Architecture" option.
In VS2022 you need to create a 'runsettings' file and set the test project to use it. In that file you can specify the Target Platform, like this:
<?xml version="1.0" encoding="utf-8" ?>
<RunSettings>
<!-- Configurations that affect the Test Framework -->
<RunConfiguration>
<TargetPlatform>x64</TargetPlatform>
</RunConfiguration>
</RunSettings>
You still also need to include the Nunit3TestAdapter and Microsoft.NET.Test.Sdk nugets (and make sure the project actually built, and check the various other more obscure mistakes mentioned in other answers)
Make sure your tests are properly marked with the Test attribute. If all of the tests are marked with only the Explicit attribute, the TestAdapter doesn't recognize the fixture.

Unit testing iphone app withXcode, SenTestCase and C++ dependencies

I have a iphone project with a embedded (for ease of use) open source C++ project in it (meaning its folders are just a group inside the project). Following Apple's tutorial on unit testing Xcode, I could run the tests no problem, provided the only imported files are from Objective-C classes.
However, whenever I run tests that import C/C++ code, the test target fails on tons of "file not found". Any idea on how to solve this, without turning all C/C++ deps on system files?
Try using the .mm extension for your C files, if you're not using it already...
If we're talking about TONS of files, go ahead and try renaming one and see if that removes it from the "file-not-found-pile".
Actually just configuring the search path on the test target solved this. Which is kinda odd, as the project target is a dependency. However, it lead to other issues, which this SO thread took care of it.

Unit Testing iPhone - Linker Errors

I've followed the Unit Testing Applications guide from the iPhone Development documentation. I followed all the steps and it worked with the TestCase from the documentation. But as soon as I changed the TestCase to test real Code from my project I ended up with linker errors. All classes that are used in the TestCase are reported as missing.
I've already searched the internet and found that the Bundle Loader property must be set to "$(BUILT_PRODUCTS_DIR)/MyApplication.app/Contents/MacOS/MyApplication". But this also fails because the file could not be found.
Any ideas what I have to do to tell the linker where to search for the missing files?
Make sure that you have added your classes to the unit test target. So if you are creating a unit test for class foo right click on foo.m, select info and then from the Targets tab ensure the checkbox for your unit test target is selected.