I have a test like
[Test]
[TestCase("Chrome", TestName = "One")]
[TestCase("Firefox", TestName = "Two")]
[TestCase("IE", TestName = "Three")]
public void MyTest(string Browser)
{
.............
}
I am calling this through programming as
SimpleNameFilter filter = new SimpleNameFilter() { };
Test = "SeleniumTests.Test.One";
filter.Add(Test);
CoreExtensions.Host.InitializeService();
TestPackage testPackage = new TestPackage(#"D:\Test.dll");
RemoteTestRunner remoteTestRunner = new RemoteTestRunner();
remoteTestRunner.Load(testPackage);
TestResult result = remoteTestRunner.Run(new NullListener(), filter, true, LoggingThreshold.All);
when i executed the above code, the test "one" is executing , but calling multiple time, can anybody tell why this is happening like this?
Thanks in advance,
Kishore.
First of in your case you don't need the first [Test], since you already have [TestCase(...)]. Just remove the attribute and see if this is going to make a difference.
Second of all, I believe what happens is, the Test class - One (is what I think you called it) get's picked up and executes all the tests within it the class. For example MyTest will run 3 times (for each TestCase).
I hope I understood you correct.
Related
I am trying to take a screenshot of every Test Case and have it exported to a screenshot directory with its name.
I am using:
testName = RunConfiguration.getExecutionSourceName().toString()
but this only contains the name of the Test Suite and NOT the Test Case name.
WebUI.takeScreenshot('path'+testName+'.png')
How would I reference the Test Case name and not the Test Suite name?
Thank you.
EDIT: The code I am taking a screenshot of currently lives in the "TearDownTestCase" method located in Test Suites.
Okay so I figured it out with the help of #Mate Mrse. Running the .getExecutionSource() method would return me the Test Suite name when running a Test Suite. However I needed to return the Test Case name.
I first created a Test Listener and added to the '#BeforeTestCase':
class TestCaseName {
#BeforeTestCase
def sampleBeforeTestCase(TestCaseContext testCaseContext) {
String testCaseId = testCaseContext.getTestCaseId()
}
}
This returns the path:
../Katalon Studio/Test Cases/Test Case Name
Then I've used the .substring() method to store the Test Case Name as a string
class TestCaseName {
#BeforeTestCase
def sampleBeforeTestCase(TestCaseContext testCaseContext) {
String testCaseId = testCaseContext.getTestCaseId()
GlobalVariable.testCaseName = testCaseId.substring((testCaseId.lastIndexOf("/").toInteger()) + 1)
}
}
Thank you #Mate Mrse
You can use RunConfiguration.getExecutionSource() to get the full path to the test case being run.
And then you can do with that whatever you want. For example, to get the test case name you might do something like
RunConfiguration.getExecutionSource().toString().substring(RunConfiguration.getExecutionSource().toString().lastIndexOf("\\")+1)
Explanation:
The .getExecutionSource() method will give you a full path to your test case, something like C:\users\user.name\Katalon Studio\Test Cases\Test Case Name.tc (you might have something different).
Since you want only the last part, you can use Groovy to cut this string to something you like. So, I cut the string at the place of the last \ (that is what the lastIndexOf is doing) just before the Test Case Name.tc (+1 because I want to cut the backslash as well).
Then the .substring() method will give me just what is leftover after the cut.
I've a application that's run on top of terminal.
This App uses only Scala and SBT and I'm testing it using ScalaTest.
I want to test all components like a integration test, running the app, but, for that, I want to simulate the user sending values via standard input using something like a robot. Most important of all, when I call readLine or readInt, I want to send differents values while testing.
Thanks in advance!
Edit 1
So, I've this code. It basically prints the options for the user. I want, for example, to send 1, and then 3,4, to create a new Cell and check my CellArray to check if a new cell was really created in that position.
do {
println("Select one of the options: \n \n");
println("[1] Make a cell alive");
println("[2] Next generation");
println("[3] Halt");
print("\n \n Option: ");
option = parseOption(readLine)
}while(option == 0)
option match {
case MAKE_CELL_ALIVE => makeCellAlive
case NEXT_GENERATION => nextGeneration
case HALT => halt
}
The general approach will be something along these lines.
// This is the class containing the logic you want to test
class MyInputThing {
def readInput() = readLine
def thingIWantToTest = {
val input = readInput
doStuffWithInput(input)
}
}
// This test class returns a value representing something you want to verify.
class TestMyInputThing extends MyInputThing {
override def readInput = "123"
}
Then, in your test, use TestMyInputThing.thingIWantToTest() and validate the response. You can also pull out readInput into a trait, parameterise the creation of TestMyInputThing, etc, in order to clean this up. I would also recommend looking into ScalaCheck so you don't need to handcode test scenarios.
I have scalatest codes like following:
class myTest extends FlatSpec with ParallelTestExecution {
val testSuiteId: String = GenerateSomeRandomId()
it should "print test id" in {
println(testSuiteId)
}
it should "print test id again" in {
println(testSuiteId)
}
}
The two tests cannot print the testSuiteId I generate before them. Instead they regenerate the ID and print it. I understand that because of ParallelTestExecution which extends OneInstancePerTest, each test here runs on its own instance and have a copy of the variable "testSuiteId".
But I do want a fixed Id for this test suite and each test case in this suite have access to this fixed it without modifying it. I tried to create the fixed id in BeforeAll{ } but still it didn't work.
How should I achieve what I want?
One way to work around it would be to put the shared state in some sort of external object:
object SuiteId {
lazy val id: String = GenerateSomeRandomId()
}
Admittedly this is very much a hack, and I wouldn't be surprised if scalatest has a way to handle this built-in which I am unaware of.
I am writing a unit test for a plugin using Dynamics CRM with RhinoMocks.
After stubbing out the OrganizationService.Retrieve() method, when I invoke the stubbed out method, I am getting null back.
From what I can see (correct me if I'm wrong), is that the stubbed out method signature must the same as the invocation signature.
Here is my code:
TestSetup
var someGuid = Guid.Empty;
var organisationServiceMock = MockRepository.GenerateMock<IOrganizationService>();
organisationServiceMock.Expect(x => x.Retrieve("someCrmEntity", someGuid, SomeCrmEntityColumnSetQuery.ColumnSet))
.Return(new Entity
{
LogicalName = "someCrmEntity",
Id = Guid.NewGuid(),
});
SomeCrmEntityColumnSetQuery Code
public static class SomeCrmEntityColumnSetQuery
{
public static ColumnSet ColumnSet => new ColumnSet("column1", "column2");
}
Invocation Code
var someEntity = organisationServiceMock.Retrieve("someCrmEntity", someGuid, SomeCrmEntityColumnSetQuery.ColumnSet);
//someEntity is null
Things I have tried
Removed the ColumnSet and replaced it with null - this works
Replaced the static class SomeCrmEntityColumnSetQuery with a default instance (new ColumnSet())
I have set the someGuid to Guid.Empty thinking that it was not "joining" on the correct Guid hence the null return value.
I have tried to replace .Expect() with .Stub() - no joy
Edit
In the expectation, I have tried the .WhenCalled(...) and that is how I found out that if I replace the columnSet argument with a null in the expectation and the invocation, it works. So it's go to do with something in my static class that represents a ColumnSet. The code works as I have it running in my DEV environment.
If anyone can share some light on this, that would be magic!
Charles
So I found the answer after watching a PluralSight video on RhinoMocks.
My problem was that when setting up the stub, the stub does not take values but rather the signature of the method that you are stubbing out. For e.g:
var organisationServiceMock = MockRepository.GenerateMock();
//Wrong
organisationServiceMock.Expect(x => x.Retrieve("someCrmEntity", someGuid, SomeCrmEntityColumnSetQuery.ColumnSet)).Return(new Entity());
//The stub does not care about what values are being sent into the method when invoked but rather if the method signature types match.
//Correct
organisationServiceMock.Expect(x => x.Retrieve(Arg.Is.Anything, Arg.Is.Anything, Arg.Is.Anything)).Return(new Entity());
//During the invocation, stubbed method now expects the first argument to be a string, then 2nd to be a Guid, 3rd to be a ColumnSet.
I hope this helps anyone who has also been struggling with this. :)
I am using Xamarin Studio 5.2 on Mac OS X 10.9.4 with NUnit 2.6.3 and FakeItEasy 1.23.0.
When I run tests for this code:
using System;
using ValueSet = System.Collections.Generic.HashSet<uint>;
using NUnit.Framework;
using FakeItEasy;
namespace SetTest
{
[TestFixture]
class TestFixture
{
[Test]
public void CallsUsersWithSetAndReducedSet()
{
var values = new ValueSet { 1, 2, 3 };
var setUser = A.Fake<SetUser>();
ClassUnderTest testInstance = new ClassUnderTest();
using (var scope = Fake.CreateScope())
{
testInstance.RunWith(setUser);
using (scope.OrderedAssertions())
{
A.CallTo(() => setUser.Use(A<ValueSet>.That.IsEqualTo(values))).MustHaveHappened(Repeated.Exactly.Once);
A.CallTo(() => setUser.Use(A<ValueSet>.That.Matches(set =>
set.Count == 2 && set.Contains(1)))).MustHaveHappened(Repeated.Exactly.Once);
}
}
}
}
public class SetUser
{
public virtual void Use(ValueSet set)
{
}
}
class ClassUnderTest
{
public static void Main(string[] arguments)
{
}
public void RunWith(SetUser setUser)
{
var values = new ValueSet { 1, 2, 3 };
setUser.Use(values);
values.Remove(3);
setUser.Use(values);
}
}
}
I get the following error output:
FakeItEasy.ExpectationException: Assertion failed for the following call: SetTest.SetUser.Use(1[System.UInt32]>) Expected to find it exactly once but found it #0 times among the calls:
1. SetTest.SetUser.Use(set: System.Collection.Generic.HashSet1[System.UInt32]) repeated 2 times
I don't understand what is causing this failure and how to fix it.
What is needed to get this type of test to pass?
#Tim Long is on the right track in his comment.
Here's a little more detail, as well as updates to respond to your comments of 2014-08-11 03:25:56:
The first reason the first MustHaveHappened fails:
According to the FakeItEasy argument constraints documentation, That.IsEqualTo tests for "object equality using object.Equals". That's what's causing the unexpected behaviour.
Not passing values into the method isn't necessarily a problem, or wouldn't be if ValueSet.Equals performed a value comparison, but ValueSet is a HashSet<uint>, so you can see from that class's method documentation that it doesn't—it uses object.Equals, which tests for reference equality. Thus, your IsEqualTo assertion fails. If you use a more sophisticated matcher that performed a value-type comparison for HashSet, perhaps something closer to what you use in your second A.CallTo, or maybe something using That.Contains, I think you'll have better success.
You may think to use That.IsSameSequenceAs, but be careful if doing so: the HashSet doesn't guarantee the order of the elements in the enumeration, so even if the set has the same elements, you may get a failure.
The second reason the first MustHaveHappened fails:
RunWith changes the contents of the values set between calls to setUser.Use. So the same set is used in two calls, first with 3 elements, then when it has only 2 elements. This means that by the time the first MustHaveHappened call is made, the set has only 2 elements, so the comparison fails. You could see this more clearly by writing an argument formatter for the ValueSet. That would provide more information.
The cause of the mismatch is that when a call is made to a faked method, FakeItEasy captures the arguments. However, for reference types, such as ValueSet (HashSet), only the reference to the argument is kept. Thus, if the object is modified later, in particular between the execution and the verification stages of the test, the object will look different than it did at the time of the faked call. See #jimmy_keen's answer to MustHaveHappened fails when called twice on the same object. There's a little more discussion over at FakeItEasy Issue 306 - Verifying multiple method calls with reference parameters.
In this case, the usual approach is to do as he suggests—provide code to capture the important state of the incoming argument at call time, and then query that saved state later.
You might be able to use something like this:
[Test]
public void CallsUsersWithSetAndReducedSet()
{
var capturedValueSets = new List<List<uint>>();
var setUser = A.Fake<SetUser>();
A.CallTo(() => setUser.Use(A<ValueSet>._)) // matches any call to setUser.Use
.Invokes((ValueSet theSet) => capturedValueSets.Add(theSet.ToList()));
ClassUnderTest testInstance = new ClassUnderTest();
testInstance.RunWith(setUser);
Assert.That(capturedValueSets, Has.Count.EqualTo(2),
"not enough calls to setUser.Use");
Assert.That(capturedValueSets[0], Is.EquivalentTo(new uint[] {1, 2, 3}),
"bad set passed to first call to setUser.Use");
Assert.That(capturedValueSets[1], Has.Count.EqualTo(2) & Has.Member(1),
"bad set passed to second call to setUser.Use");
}
You can see that each time Use is called, we add the contents of the ValueSet argument to capturedValueSets. Then at the end we
make sure 2 calls were made, by checking the length of capturedValueSets
make sure that the first time Use was called, the set had the elements 1, 2, and 3. Is.EquivalentTo checks the two lists but ignores order
make sure that the second time Use was called, the set had 2 elements, one of which was 1
By checking the two captured value sets in turn, all the bits about the scopes and ordered assertions became unnecessary.